Reduce the SVN repo size

Question: I have a huge SVN repo, 450K+ revisions, am trying to reduce the size of the history to just take the last couple of thousands of revisions, for example 20K and restore it in a new repo, I trI tried many way to achieve that, and non worked, below is the methods I recall that I used
1- normal dump, using the command below, it took two days,and still running, the dump size exceeded the repo size 8 times ( the repo size is 70G and before I killed the dump process the size of the dump reached 500GB)

svnadmin dump demo2/ -r420000:HEAD > /share/svnbkup/split.dump
  WARNING 0x0001: Mergeinfo referencing revision(s) prior to the oldest dumped >revision (r420000). Loading this dump may result in invalid mergeinfo.

2- using the same command but I used --incremental, The dump is not restorable

svnadmin dump demo2/ -r420000:HEAD --incremental > /share/svnbkup/split.dump
  * Dumped revision 420136.
  WARNING 0x0000: Referencing data in revision 419907, which is older than the >oldest dumped revision (r420000).  Loading this dump into an empty >repository will fail.
  WARNING 0x0001: Mergeinfo referencing revision(s) prior to the oldest dumped revision (r419907). Loading this dump may result in invalid mergeinfo.
  * Dumped revision 420137.

3- I used the dump from the step above and tried to use svndumpfilter, and it did nothing,
4- I tried to restore the dump as it is,

<<< Started new transaction, based on original revision 420000
  svnadmin: E160013: File not found: transaction '0-0', path '/trunk/tool/test.xml'
      * editing path : trunk/tool/test.xml ...root@lxc:/user#
  root@lxc:/SVN# du -sh final/
  168K   final/
Answer:
Find LOWER border, which will NOT have references outside range (read warnings carefully - now you have to include at least from r419907)
Don't use --incremental for the first, base dump
Slice range, found in step 1, into small slices, and dump small ranges (second and later - with --incremental option) in turn - less time, in case of any error you have to re-done only one small dump

On the rights on delirium
Forget about historic data, transfer only HEAD in new repo
Comment: I tried the steps you mentioned, I have another dump with the missing references outside the range, but whether I started with the dump with the missing references or the dump with the bulk of the references, I receive the same error, missing paths and files,
and if don't use '--incremental' the dump size will sky high, which beat the purpose of the whole thing
am still new to the SVN, and don't have much experience in it
Comment: @user3179814 - verify|repair original repo before dumping (it seems as damaged), text-dump **MUST** be bigger, than corressponding part of repo, but loaded dump have to produce less repo-size
Comment: Sorry for the late respond, I tried to restore the dump, the 5K rev dump, the size of the repo is still the same, it just reduced from 70G to 61G, I tried to run svnadmin verify, and it showed that the repo is not damaged, that is the only way I know of to check svn repo

Reduce the SVN repo size


Lower bound on variance of sum of random variables

Question: Let $X,Y$ be two positive random variables. Is it true that $Var(X+Y) \geq \dfrac{Var(X)+Var(Y)}{2}$.
Comment: Suppose $X=1+\epsilon$ and $Y=1-\epsilon$ where $\epsilon$ is a small random variable.
Comment: In the first line of your proof attempt, there should be a factor of $2$ in front of the covariance. Then, the resulting work gives $Var(X+Y) \ge 0$.
Comment: You miss the factor $2$ before covariance, and thus you will obtain $Var[X+Y] \geq 0$ in the last line as expected, which is trivial. Have you read about lulu's comment? This trivial lower bound is attained when you are talking about the variance of a constant/degenerated random variable. So when $X+Y$ is a constant, then it attains the lower bound, which cannot be improved unless you have more assumptions to exclude this trivial example.
Comment: Thank you for pointing that out. I am sorry, i missed that.
Answer: Community wiki answer so the question can be marked as answered:
As noted in the comments, you missed a factor of $2$, and the claim is in fact false, since $X+Y$ can be a constant with zero variance with $X$ and $Y$ non-constant with non-zero variance.


Proving that $\log\log x = o((\log x)^{\epsilon})$

Question: I am trying to show that $\log\log x = o((\log x)^{\epsilon})$ for all $\epsilon > 0$.
Attempt:
We wish to show that $\lim_{x \to \infty}\frac{\log \log x}{(\log x)^{\epsilon}} \rightarrow 0$
Let $x = e^{y}$ then we have $\frac{\log y}{y^{\epsilon}} \rightarrow 0$. Hence the result follows. Is this correct?
Thanks.
Comment: Good point. Will edit.
Comment: Your substitution changes the question to proving $\log y =o(y^\epsilon)$.  Is asserting $\frac{\log y}{y^{\epsilon}} \rightarrow 0$ enough of a proof for that?
Comment: $\frac{\log y}{y^{\epsilon}}$ is a fairly standard result, so I just assumed the reader would know it. The proof is fairly simple, pretty much using the same method as above.
Answer: Yes, what you have done is correct.
Answer: As noticed your proof is fine, to justify the latter limit we can use again $y = e^{z}\to \infty$ then
$$\lim_{y\to \infty}\frac{\log y}{y^{\epsilon}} =\lim_{z\to \infty}\frac1\epsilon\frac{z\epsilon}{e^{z\epsilon}} = 0$$
since eventually $$e^{z\epsilon}\ge (z\epsilon)^2 \implies \frac{z\epsilon}{e^{z\epsilon}}\le \frac{z\epsilon}{(z\epsilon)^2}=\frac1{z\epsilon}\to 0$$


Evaluating limits at positive and negative infinity

Question: I am studying limits at infinity, and I have a doubt about evaluating them. From what I know, limits only exist if both sides of the limit exist and are equal. For example, take a look at the following limit:
$$\lim_{ x\to \infty}\frac{x}{e^{-x}}$$
As $x \to +\infty$ the function goes to $+\infty$. However, as $x \to -\infty$ the function goes to $0$.
Does this mean that the limit does not exist or I only have to evaluate the limit as x goes to $+\infty$?
Comment: The limit at positive infinity is a different problem then the limit at negative infinity. They don't have to be the same and very often they aren't the same, like in your example. One limit exist (second one) and the other doesn't (first one). I think you are confused with taking the limit at a single point, where right hand limit has to be the same as left hand limit in order for THE limit to exist.
Comment: That means that I have to evaluate it with x approaching plus infinity? because that problem does not specify if it is plus infinity or minus infinity.
Comment: in that case i believe the problem is not well formulated
Comment: When looking purely at the limit as you presented, you only consider the limit as $x$ goes to infinity. That answer is infinity and so technically that limit does not exist although some people take infinity as an acceptable answer, as Marios does.
Comment: @inframat  a limit which has an infinite value is an existent limit..in another case if  $x$ approaches a point from the left and right and you have different limits then the limit there does not exist..take for instance $\frac{1}{x}$ as $x \to 0$
Comment: @imranfat also  either  case is not wrong i believe..for me an infinite limte is existent.
Comment: You could say that the limit$$\lim_{|x|\to\infty}\frac x{e^{-x}}$$does not exist.
Comment: @MariosGretsas From a graphical stand point, infinity is used as an answer to a limit to clarify curve behavior. However, in more stringent cases, with limit laws are to be applied, you got to be very careful. For example "The product of a limit is the limit of a product" can only be used if the individual limits exists and are finite, otherwise things will absolutely go wrong. (It doesn't apply to the OP's point here.)
Comment: @imranfat of course my argument is from  graphical and intuitive point of view...Now the 'algebra' of limits or limit laws is another thing ...For me personally( for limits only of course) if i can give you an answer about a value of a limit either  finite or infinite then the limit exists..now for instance  i cannot give you an answer for the case $1/x$ as $x \rightarrow 0$ because giving you such an answer would contradict  the property about the side  limits to $0^+,0^-$
Comment: It might be instructive to compare to $\pm 5$. We don't require the limit at $5$ to be the same as the limit at $-5$, so we don't have to require the limit at $\infty$ to be the same as the limit at $-\infty$.
Answer: $$\frac{x}{e^{-x}}=xe^x$$
If you take $x \to +\infty$ then the limit is $+ \infty$
If you take $x \to -\infty$ then the limit is $0$
You can say that the limit exist because the existence of the limit as $x$ goes to $-\infty$ does not depend to the limit as $x$ goes to $+\infty$
In both cases the limit exists,just in the first case is $+\infty$
Comment: So, do I have to evaluate both cases and thus give two answers?
Comment: as i said in the above comment in this case the problem is not well formulated..the author must be more specific
Comment: Also @Nick_17..some authors consider an infinite limit existent and some others not ...i don't think personally that one these two cases is wrong
Answer: This graph might help you see what is going on.
Answer: Although closely related:
i) $\lim_{x\rightarrow a}f(x) = L; a\in \mathbb R$
ii) $\lim_{x\to a^+}f(x) = L$
iii) $\lim_{x\to a^-}f(x) = L$
iv) $\lim_{x\to \infty} f(x) = L$
and v) $\lim_{x\to -\infty} f(x) = L$.
have technically different definitions.
i) means as $x$ gets close to $a$, then $f(x)$ gets close to $L$.  or technically:
i) means For any $\epsilon > 0$ we can find $\delta$ so that whenever $|x -a| < \delta$ it will follow that $|f(x) - L| < \epsilon$.
ii) means as $x$ gets close to $a$ but larger than $a$, then $f(x)$ gets close to $L$.  or technically:
ii) means For any $\epsilon > 0$ we can find $\delta$ so that whenever $a < x < a+ \delta$ it will follow that $|f(x) - L| < \epsilon$.
Notice that if i) is true than ii) must be true but if ii) is true i) is not nescessarily true.
iii) means as $x$ gets close to $a$ but smaller than $a$, then $f(x)$ gets close to $L$.  or technically:
iii) means For any $\epsilon > 0$ we can find $\delta$ so that whenever $a -\delta < x < a$ it will follow that $|f(x) - L| < \epsilon$.
Notice that if i) is true than ii) and iii) must be true.  And if both ii) and iii) is true then i) is true.  But if one or the other of ii) or iii) is true but the other isn't i) will not be true.
So the statement "i) if ii) and iii)" is not a definition but an observation.  Well.... it could be a definition as they are equivalent.
Now...
Limit to infinity is not that as $x$ gets close to infinity $f(x)$ will get close to $L$.  That wouldn't make any sense because we can't get $x$ "close to infinity".  Instead we mean, as $x$ becomes large, $f(x)$ gets close to $L$.  or technically:
iv) For any $\epsilon > 0$ there is some $M \in \mathbb R$ so that whenever $x > M$ it will follow that $|f(x) - L| < \epsilon$.
It makes no sense to talk of $\lim_{x\to \infty^+} f(x)$ because we can't "approach from the right".  And to talk of $\lim_{x \to \infty^-}f(x)$ is the exact same thing as $\lim_{x \to \infty}f(x)$ because we can only "approach from the left".
So it is NOT the case that $\lim_{x\to\infty} f(x) = L$ if $\lim_{x\to \infty^+}f(x) = L$ and $\lim_{x\to \infty^-}f(x) = L$.  Instead it is just $\lim_{x\to\infty} f(x) = L$ if $\lim_{x\to \infty^-}f(x) = L$.
$\lim_{x \to -\infty}f(x)=L$ means we are talking the values of very big negative numbers (very small, or negative with very large magnitude or absolute values).  It should be very clear that $-\infty$ means infinity beyond the negative extreme of the reals, whereas $\infty^-$ means something entirely different; it means approaching the the positive extreme of the real numbers but at some finite value less than infinity.
v) For any $\epsilon > 0$ there is some $M \in \mathbb R$ so that whenever $x < M$ it will follow that $|f(x) - L| < \epsilon$.
Again it makes no sense to talk of $\lim_{x\to -\infty^-}f(x)$ and it is redundant to talk of $\lim_{x\to -\infty^+} f(x)$ for the exact same reasons.


How to search data in object and cast to list

Question: I want to search value from object and convert result to list
I tried to get data object name "data" and i want to search every item that has "querytext".How to convert object data to List
public class SimpleMovieSearchService implements MovieSearchService {
   @Autowired
   private MovieDataService movieDataService;

   @Override
   public List<Movie> search(String queryText) {
   MoviesResponse data =movieDataService.fetchAll();
       List<Movie> result = data.stream() // problem in this line
            .filter(item -> item.getTitle().equals("queryText"))
            .collect(Collectors.toList());

    return result;
   }
}

MovieData.java
public class MovieData {
   private String title;
   private int year;
   private List<String> cast;
   private List<String> genres;

   getter and setter
}

Movie.java
public class Movie {
   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   private Long id;

   private String name;

   @ElementCollection(fetch = FetchType.EAGER)
   private List<String> actors = new ArrayList<>();

 getter and setter
}

edit
MovieResponse.java
public class MoviesResponse extends ArrayList<MovieData> {
}

MovieDataService.java
public interface MovieDataService {
  MoviesResponse fetchAll();
}

Type of data i got is List of MovieData but i want result in List of Movie how to fix this or another way to do.and without change anything in Movie.java or MovieData.java
Update
now i can do with this way
MoviesResponse a = movieDataService.fetchAll();
    List<Movie> result = a.stream()
            .map(movie -> new Movie(movie.getTitle()))
            .filter(movie -> movie.getName().equals(queryText)) //this line
            .collect(Collectors.toList());

if data are "banana" ,"nature" , "red" and then query text is "na" i want result are "banana" and "nature".in sql in situation it can use "like".How to use "like" with this filter ?
Comment: Please post your MoviesResponse and your MovieDataService class too.
Answer: Instead of fetching all movies and filer them by stream, write a method that queries the ones you want.
Something like
SELECT * FROM movies WHERE title = 'queryText'
Depending on your code/framework, you have to implement this or just have to choose another method in you MovieDataService.
Comment: I want to but This subject is use object from method fetchAll() and search value in object with "querytext" and then contain to List for return List
Answer: Try to use stream map
data.stream() // problem in this line
        .filter(item -> item.getTitle().equals("queryText"))
        .map(movie -> {
        //logic to transform MovieResponse in Movie here
        })
        .collect(Collectors.toList());

See more in: https://dzone.com/articles/how-to-use-map-filter-collect-of-stream-in-java-8
Comment: Thank you.But i got a little question.If i want operator like "like" in sql what i should put in stead " .equals "
Comment: You can use the .contains method of String.


outerHTML behavior in a

Question: Given this <template>:
<template id="my-template">

Hello!

</template>

And JS:
var t = document.querySelector("#my-template");
var h = t.content.querySelector("h1");
h.outerHTML = "
AAAAAAAAAAAAAA
";

It's interesting that it works in FireFox and Edge but in Chrome outerHTML requires a parent element, otherwise it throws error in console (https://chromium.googlesource.com/chromium/blink/+/master/Source/core/dom/Element.cpp#2528):
<template id="my-template">

Hello!

</template>

See https://jsfiddle.net/q5fmn186/11/
My question is, is Chrome behavior the correct one? Should setting outerHTML not work in <template> on direct children? Why aren't the other web-browser treat it like an error?
Answer: The other web browsers won't treat it like an error because they are following the DOM Parsing and Serialization W3C Candidate Recommendation (which is not a standard yet):

On setting [outerHTML], the following steps must be run:

Let parent be the context object's parent.
If parent is null, terminate these steps. There would be no way to obtain a reference to the nodes created even if the remaining steps were run.
If parent is a Document, throw a DOMException with name "NoModificationAllowedError" exception.
If parent is a DocumentFragment, let parent be a new Element with body as its local name, the HTML namespace as its namespace, and the context object's node document as its node document.
Let fragment be the result of invoking the fragment parsing algorithm with the new value as markup, and parent as the context element.
Replace the context object with fragment within the context object's parent.

The <template>'s content is of type DocumentFragment (step 4) but it is treated (in this situation) as a Document (step 3) by Chrome and Safari.
Comment: thank you very much. So on longer term, we should expect to see Chrome and Safari follow the recommendation (which eventually hopefully will become a standard)?

outerHTML behavior in a


Show if $\sum_{n=1}^{\infty} \frac{(-1)^n}{1+\sqrt{n}}$ is absolute convergent or divergent

Question: Show if $\sum_{n=1}^{\infty} \frac{(-1)^n}{1+\sqrt{n}}$ is absolute convergent or divergent
First i subbed numbers in
$$\lim_{n \to \infty} \frac{(-1)^n}{1+\sqrt{n}} = \frac{-1}{1+\sqrt{1}} + \frac{1}{1+\sqrt{2}} - \frac{-1}{1+\sqrt{3}}$$
So it's divergent
1 LIMIT
$$\lim_{n \to \infty} \frac{1}{1+\sqrt{n}}=0 \quad \text{hence divergent} $$
2 $a_{n}$ and $a_{n+1}$
$$ \frac{1}{1+\sqrt{n}}>\frac{1}{1+\sqrt{n+1}} $$
$$ \frac{1}{1+\sqrt{n}}-\frac{1}{1+\sqrt{n+1}}> 0 $$
hence increasing
or
$$ \frac{1}{1+\sqrt{n}}-\frac{1}{1+\sqrt{n+1}} =  \frac{\sqrt{n+1}-\sqrt{n}}{\sqrt{n+1}\sqrt{n+1}} $$
$$ \therefore \sqrt{n+1}-\sqrt{n}> 0$$ hence decreasing
$ \sum_{n=1}^{\infty} \left\lvert \frac{-1^n}{1+\sqrt{n}}  \right\rvert = \sum_{n=1}^{\infty}   \frac{1}{1+\sqrt{n}}   $
$$ \sum_{n=1}^{\infty} \frac{1}{1+\sqrt{n}} = \frac{1}{1+\sqrt{1}} + \frac{1}{1+\sqrt{2}} + \frac{1}{1+\sqrt{3}} $$
if this is convergent then the whole thing is absolutely convergent but i don't know how to prove this

UPDATE
I got something out for the second part
Comment: What you tried to do, what you explain is your problem or your issues is absolutely incomprehensible...
Answer: There are so many mistakes (and so severe) in your "solution" that it is a superhuman task to discuss them. Your series is convergent, Leibniz's test solves the problem in a second. The series is not absolutely convergent by an application of the limit comparison test (compare your series with $\sum \frac 1 {\sqrt n}$ which is known to be divergent).
Comment: @Ivan: You misuse the sign "$\therefore$": you must put it in front of the conclusion, not of the hypothesis (and, in general, very few people use it, better avoid it completely). What you want to say is $\sqrt {n+1} > \sqrt n \Rightarrow \frac 1 {1 + \sqrt n} > \frac 1 {1 + \sqrt {n+1}}$, and this is correct (and necessary in order to use Leibniz's test).
Comment: Obviously i can't use liebniz's test i have to use alternating test
Comment: @Ivan: Leibniz's test *is* another name of the alternating test...
Comment: Is this right though

$$ \frac{1}{1+\sqrt{n}}-\frac{1}{1+\sqrt{n+1}} =  \frac{\sqrt{n+1}-\sqrt{n}}{\sqrt{n+1}\sqrt{n+1}} $$

$$ \therefore \sqrt{n+1}-\sqrt{n}> 0$$ hence decreasing
Comment: oh ok so i just did it around the wrong way.... Umm what happens if it's decreasing?
Comment: @Ivan: Being decreasing towards $0$ is what you need in order to use Leibniz's test. Seriously, are you reading what I write? Everything you need is already written above; you just have to put these pieces together.
Comment: @AlexM. NB I fixed a small but major problem in your post. The series is convergent, but not _absolutely_ convergent. I suppose that's what you meant to say anyway ;)
Comment: @AlexR: Thank you for fixing that; yes, that is what I meant, I typed it in a rush. Also, thank you for signalling this edit to me; it displeases me when people correct things silently without the original author's knowledge. What if the author really meant it that way? (Not the case here, though.)
Answer: First, you have some errors I think in your writeup. $-1^n$ likely should have parentheses around the $-1$.  It does change the convergence or divergence of this particular series. There is a different treatment for alternating series. But I'll proceed to answer the problem as written.
Second, your first use of $a_n$ and $a_{n+1}$ ends with a conclusion that the sequence of terms is increasing. But when fractions get bigger denominators, they get smaller... So this is a decreasing sequence.
Third, the series diverges by comparison with the $p-$series. If you are not familiar with this, you should look it up. It's a standard thing in most textbooks.
Now your series is not exactly a $p-$series, but it behaves like one. To show that it diverges, you can take
$$\lim_{n\to\infty} \dfrac{1/\sqrt n} {1/(1+\sqrt n)} $$
to see why. Since this limit is finite and positive, both series behave the same way.
Now if your series is alternating, i.e. the general term contains $(-1)^n,$ then the series converges conditionally.  This is because the sequence of the absolute value of terms strictly decreases to zero, while the series with the absolute value of terms diverges (as we mentioned above).
Either way, your series does not converge absolutely! For this to occur, the alternating series and its associated series with the absolute values of terms must BOTH converge.
Answer: Let $a_n=\frac{(-1)^n}{1+\sqrt{n}}$.
$$ a_{2n}+a_{2n+1} = \frac{1}{1+\sqrt{2n}}-\frac{1}{1+\sqrt{2n+1}} =  \frac{\sqrt{2n+1}-\sqrt{2n}}{(1+\sqrt{2n})(1+\sqrt{2n+1})}  =  \sqrt{2n} \frac{\sqrt{1+\frac{1}{2n}}-1}{(1+\sqrt{2n})(1+\sqrt{2n+1})} \sim \frac{n^{-\frac{3}{2}}}{4\sqrt{2} } = O\left(\frac{1}{n^{\frac{3}{2}}}\right)$$
Therefore the serie $\sum (a_{2n}+a_{2n+1})$ is convergent and so is the serie $\sum a_n$.
Moreover, $|a_n| \sim \frac{1}{\sqrt{n}} $ so your serie is not absolutely convergent.
Answer: $$\sum_{n=1}^{\infty}\left|\frac{(-1)^n}{1+\sqrt{n}}\right|=\sum_{n=1}^{\infty} \frac1{1+\sqrt{n}}>\frac12\sum_{n=1}^{\infty} \frac1{\sqrt{n}}$$ is divergent.
Answer: Absolute convergence

If
  $$ \sum \left|a_n\right| =\mbox{convergent}$$
  Then
  $$ \sum a_n =\mbox{absolutely convergent}$$
  Note that an absolutely convergent series is also convergent.

Conditional convergence

If
  $$ \sum \left|a_n\right| =\mbox{divergent}$$
  And
  $$ \sum a_n =\mbox{convergent}$$
  Then
  $$ \sum a_n =\mbox{conditionally convergent}$$

So now
$$\sum\limits_{n=1}^{\infty} \left|\frac{(-1)^n}{1+\sqrt{n}}\right|=\sum\limits_{n=1}^{\infty} \left|\frac{1}{1+\sqrt{n}}\right|$$
Since
$$ \sum\limits_{n=1}^{\infty}\left|\frac{1}{\sqrt{n}+\sqrt{n}}\right|\leq \sum\limits_{n=1}^{\infty}\left|\frac{1}{1+\sqrt{n}}\right|$$
And
$$ \sum\limits_{n=1}^{\infty}\left|\frac{1}{\sqrt{n}+\sqrt{n}}\right| =\frac12\sum\limits_{n=1}^{\infty}\left|\frac{1}{\sqrt{n}}\right|=\mbox{divergent}$$
Then by the comparison test
$$\sum\limits_{n=1}^{\infty}\left|\frac{1}{1+\sqrt{n}}\right|=\mbox{divergent}$$
Now for the following series
$$\sum\limits_{n=1}^{\infty} \frac{(-1)^n}{1+\sqrt{n}}$$
Since
$$\lim\limits_{n\to\infty} \frac{1}{1+\sqrt{n}}=0$$
And
$$  \frac{1}{1+\sqrt{n}} \gt  \frac{1}{1+\sqrt{n+1}} $$
Then by the alternating series test
$$\sum\limits_{n=1}^{\infty} \frac{(-1)^n}{1+\sqrt{n}} =\mbox{convergent}$$
Putting everything together, we see that
$$\sum\limits_{n=1}^{\infty} \frac{(-1)^n}{1+\sqrt{n}} =\mbox{conditionally convergent}$$


Where best to fetch data from a third-party API service within the architecture of a Phoenix app

Question: I have a question about how best to handle fetching data from a third-party API within the architecture of my phoenix app. Essentially, I have a controller that receives the client's IP address as a parameter. I then need to fetch the geolocation associated with that IP address from an external API and store both in the database (i.e., ip and geolocation).
Right now my approach is to use a plug, and then return the result in the connection struct.
But I could also fetch the data by creating a function directly in the model, and then call it during the changeset operations.
Or perhaps there's another alternative such as making it an OTP app.
Looking for some guidance on the best approach here, or at least the tradeoffs of one vs the other.
Although my scenario is specific, this is probably a common design question that people encounter.
Answer: I can tell you how I'm approaching a somewhat similar problem. My application is a web app for sending SMS texts (I use Twilio as my API).
When a user submits their message (to_phone_number, body_text, etc.) I save it to the database without doing any additional work. I give it a separate status of "Queued."
Then, I have a GenServer in a loop picking up all Message where status == queued and doing the API calls. In the transaction, it changes the status to "Delivered", and that becomes visible to the user in the UI.
This won't be as useful an approach if your users are expecting geolocation data to show on-screen with their response. Anyway, this was the approach I took for my use-case...


Weird regex bug in iis url-rewrite

Question: This is my pattern:
^(\w{2}-\w{2})/questions(?:/(\w+))?(?:/(\d+))?(?:/.*?)?$

these are the what I'm testing:
en-us/questions/ask
en-us/questions/newest/15
en-us/questions/12/just-a-text-to-be-ignored

It works perfectly, here is the demo:
https://regex101.com/r/yC3tI8/1
but the following rewrite rule:
<rule name="en-us questions" enabled="true" stopProcessing="true">
  <match url="^(\w{2}-\w{2})/questions(?:/(\w+))?(?:/(\d+))?(?:/.*?)?$" />
  <action type="Rewrite" url="/questions.aspx?lang={R:1}&tab={R:2}&pid={R:3}" />
</rule>

when I give the link en-us/questions/newest redirects to: /questions.aspx?lang=en-us&tab=&pid=
What is wrong with this? Its now about 5 hours I'm just reviewing the same things
Comment: Wouldn't `ask` do the same thing, and what is your expected result?
Comment: yes it would do the same thing but `/newest` was example give it a try with ask, I'm expecting : `/questions.aspx?lang=en-us&tab=newest&pid=15`
Comment: or `/questions.aspx?lang=en-us&tab=ask&pid=` or `/questions.aspx?lang=en-us&tab=&pid=15`
Comment: The problem is that neither `en-us/questions/newest` or `en-us/questions/ask` have a third capture group like `en-us/questions/newest/15`, so `pid` will always be blank.
Comment: well pid might be blank but not always. will it be blank in here: `en-us/questions/newest/15` or here: `en-us/questions/12/just-a-text-to-be-ignored` and my first problem is that as it rewrites to questions.aspx so it matches the pattern. lets say pid will be blank but why the tab is always blank?
Comment: Try using [the regex debugger with substitutions](https://regex101.com/r/gP4wL6/2) so you can see what the result should be.
Comment: but in all of the tab is not empty. do you have a suggestion for the pattern?
Answer: Since you have three different possible url endings that ultimately effect the outcome of the rewritten url you can either setup one all inclusive rule that will hopefully match everything you want, or you could setup three rules to handle each accordingly:
One Rule:
^(\w{2}-\w{2})/questions/(\w+)/?(\d+)?.*$

https://regex101.com/r/dN8bM9/1  - tries to handle all cases

<rule name="en-us questions" enabled="true" stopProcessing="true">
  <match url="^(\w{2}-\w{2})/questions/(\w+)/?(\d+)?.*$" />
  <action type="Rewrite" url="/questions.aspx?lang={R:1}&tab={R:2}&pid={R:3}" />
</rule>

 * note: one possible reason the original pattern was failing to capture the second group was the inclusion of (?:) - which means match but don't capture; leaving that out might solve most of the issues there.
Three Rules:
^(\w{2}-\w{2})/questions/(\w+)$

https://regex101.com/r/lI8bQ1/1  - en-us/questions/[single word]

^(\w{2}-\w{2})/questions/(\d+)/.*$

https://regex101.com/r/hV5fK3/1  - en-us/questions/[digits]/discard

^(\w{2}-\w{2})/questions/(\w+)/(\d+)$

https://regex101.com/r/kO0dJ0/1  - en-us/questions/[single
  word]/[digits]

Putting it all together into a ruleset:
<rule name="en-us questions case one" enabled="true" stopProcessing="true">
  <match url="^(\w{2}-\w{2})/questions/(\w+)$" />
  <action type="Rewrite" url="/questions.aspx?lang={R:1}&tab={R:2}" />
</rule>
<rule name="en-us questions case two" enabled="true" stopProcessing="true">
  <match url="^(\w{2}-\w{2})/questions/(\d+)/.*$" />
  <action type="Rewrite" url="/questions.aspx?lang={R:1}&tab={R:2}" />
</rule>
<rule name="en-us questions case three" enabled="true" stopProcessing="true">
  <match url="^(\w{2}-\w{2})/questions/(\w+)/(\d+)$" />
  <action type="Rewrite" url="/questions.aspx?lang={R:1}&tab={R:2}&pid={R:3}" />
</rule>

 * note: you might need to adjust this in some way, but it should give you an idea of how to accomodate three different variations (as you seem to have) for rewriting your urls.
Comment: Thanks, I devided into 2 rules and now working great. I actually was chalenging myself to do it in one rule. I know that it can be done but I couldn't. Thanks again for your nice answer
Answer: Note you have three lazy captures:

(?:/(\w+))?
(?:/(\d+))?
(?:/.*?)?

asp.net's regex implementation interprets ? as:

In addition to specifying that a given pattern may occur exactly 0 or 1 time, the ? character also forces a pattern or subpattern to match the minimal number of characters when it might match several in an input string.

So asp.net is assigning no characters to 1, no characters to 2, and collecting the rest of the characters 3.
To use greedy matching instead of the lazy matching ? forces use: {0,1}
So you're regex should look like:
^(\w{2}-\w{2})/questions(?:/(\w+)){0,1}(?:/(\d+)){0,1}(?:/.*?)?$

Live example
Comment: I'm really very sorry my internet connection have a problem and disconnects. I tried this and still the result is exactly the same
Comment: @AshkanMobayenKhiabani Hmm... I'm surprised by that. I think that you need to first confirm how your regex implementation is actually matching and then work on fixing it. For example, is my assumption correct that it's putting the rest of the link in capture **3**? You could determine that by doing: `^(\w{2}-\w{2})/questions(?:/(\w+))?(?:/(\d+))?(/.*?)?$` and printing capture **4**.
Comment: @AshkanMobayenKhiabani If that is in fact how your regex implementation works, maybe you could try implementing a regex that just matches the start instead of all the string: `^(\w{2}-\w{2})/questions(?:/(\w+))(?:/(\d*))`

Weird regex bug in iis url-rewrite


getting individual field values

Question: I'm trying to fetch each individual field of an items array. here is the image of the kint result

here is what i have tried.

{{items[0].content}} -- this one prints the whole thing in the items array.
{{items[0].content['#context'].value}} ---- does nothing
{{items[0].content['#markup'].getValue()}} ---  does nothing

Have looked for documentation, couldn't find one. Any explanation or url to documentation or example will be appreciated.
Answer: Try with items[0]['content'] or if its field of type text you will find the value of the text with items[0]['content']['#text'] or items[0]['content']['#value']
Comment: that does not work
Comment: i have tested with text field it works, which type of your field ?

getting individual field values


Scrollbar on Matplotlib showing page

Question: I want to know if there is a manner to put a scrollbar (horizontal or vertical) on a matplotlib showing page (plt.show) that contains several sublots (sublot2grid). At the moment, the only solution I find is to make the subplots very small, which isn't very elegant at all.
Answer: The window showing the matplotlib figure does not have the option to add scrollbars. It will automatically resize itself to the figure size. And inversely, if it is resized the figure will resize as well.
An option would be to build a custom window which does have this ability. For that purpose, one can use PyQt. An example is given below, where instead of calling plt.show() a custom class is called with the figure to draw as an argument. The figure size should be set to the figure fig beforehands and that custom class will not change it. Instead it puts the figure into a canvas with scrollbars, such that the figure retains it's original size and can be scrolled within the Qt window. You wouln't have to deal with the details inside the class but only the call at the end of the script.
This example is for PyQt4, see below for a PyQt5 example.
import matplotlib.pyplot as plt
from PyQt4 import QtGui
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt4agg import NavigationToolbar2QT as NavigationToolbar

class ScrollableWindow(QtGui.QMainWindow):
    def __init__(self, fig):
        self.qapp = QtGui.QApplication([])

        QtGui.QMainWindow.__init__(self)
        self.widget = QtGui.QWidget()
        self.setCentralWidget(self.widget)
        self.widget.setLayout(QtGui.QVBoxLayout())
        self.widget.layout().setContentsMargins(0,0,0,0)
        self.widget.layout().setSpacing(0)

        self.fig = fig
        self.canvas = FigureCanvas(self.fig)
        self.canvas.draw()
        self.scroll = QtGui.QScrollArea(self.widget)
        self.scroll.setWidget(self.canvas)

        self.nav = NavigationToolbar(self.canvas, self.widget)
        self.widget.layout().addWidget(self.nav)
        self.widget.layout().addWidget(self.scroll)

        self.show()
        exit(self.qapp.exec_())

# create a figure and some subplots
fig, axes = plt.subplots(ncols=4, nrows=5, figsize=(16,16))
for ax in axes.flatten():
    ax.plot([2,3,5,1])

# pass the figure to the custom window
a = ScrollableWindow(fig)

Here is a version for PyQt5.
import matplotlib
# Make sure that we are using QT5
matplotlib.use('Qt5Agg')
import matplotlib.pyplot as plt
from PyQt5 import QtWidgets
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt5agg import NavigationToolbar2QT as NavigationToolbar

class ScrollableWindow(QtWidgets.QMainWindow):
    def __init__(self, fig):
        self.qapp = QtWidgets.QApplication([])

        QtWidgets.QMainWindow.__init__(self)
        self.widget = QtWidgets.QWidget()
        self.setCentralWidget(self.widget)
        self.widget.setLayout(QtWidgets.QVBoxLayout())
        self.widget.layout().setContentsMargins(0,0,0,0)
        self.widget.layout().setSpacing(0)

        self.fig = fig
        self.canvas = FigureCanvas(self.fig)
        self.canvas.draw()
        self.scroll = QtWidgets.QScrollArea(self.widget)
        self.scroll.setWidget(self.canvas)

        self.nav = NavigationToolbar(self.canvas, self.widget)
        self.widget.layout().addWidget(self.nav)
        self.widget.layout().addWidget(self.scroll)

        self.show()
        exit(self.qapp.exec_())

# create a figure and some subplots
fig, axes = plt.subplots(ncols=4, nrows=5, figsize=(16,16))
for ax in axes.flatten():
    ax.plot([2,3,5,1])

# pass the figure to the custom window
a = ScrollableWindow(fig)

 While this answer shows a way to scroll a complete figure, if you are interested in scrolling the content of an axes check out this answer
Comment: Thank's a lot, this is what I am looking for ^^
Comment: I have a problem, when I close the Qt window to go back to Tkinter interface, the app crashes,
Comment: What does "go back to Tkinter interface" mean?
Comment: Sorry, my english is bad,
So, when I close the Pyqt window, my app crashes.
What is shown on Pyqt interface is a dashboard, and there is a Tkinter interface which call the dashboard which is supposed to remain pending;
Comment: Fore example if you put a print("hello world") after the script above, you'll get (Process finished with exit code -1073741819 (0xC0000005))
Comment: The point is that the code from the answer exits the program when the window is closed (`exit(self.qapp.exec_())`). If there is another part of the program still running it will be killed as well. So you need to somehow combine the two. But I guess this not part of this question.
Comment: Sorry but I never used QtGui, I used to use Tkinter,
Could you tell me please,  how to change the code above so that it leaves the interface Qt without leaving the program?
Comment: Replacing `exit(self.qapp.exec_())` by `self.qapp.exec_()` would not exit the script. I'm not whether this will work in combination with some other tkinter GUI though.
Comment: Thats exactly what I am looking for but I struggle to pass it on pyQt5 (and python 3.6), do you have any insights?
Comment: @Mayeul Since I don't know what you mean by "struggle" (i.e. I don't know the code you're using) and since I also don't have PyQt5 available, it's hard to say anything.
Comment: ok, by struggle I mean that I have the package PyQt5 but as i've never coded on UI, the translation is a bit complicated ahah
Comment: I edited to answer with a PyQt5 version. Can you test it and report back if it's working?
Comment: It is perfectly working, I am gonna compare it with my code, i couldn't get the figure to show inside the window
Comment: The PyQt5 version works well for me. But do you know how I can get the scrollable window's icon to match that of the figure that's passed in, please? `self.setWindowIcon(self.fig.???)`. I want the standard Matplotlib icon to show in the window's title bar.
Comment: The PyQt5 version works for me.  However, with ipython, I had to `import sys` and replace `exit` by `sys.exit`.
Answer: Because the code was a bit complicated and confused for me, I refactored it more readable, refactorable like below. I hope this will be helpful. Thank you!
import sys
import random
import matplotlib
import matplotlib.pyplot as plt

from PyQt5.QtWidgets import (
                        QWidget,
                        QApplication,
                        QMainWindow,
                        QVBoxLayout,
                        QScrollArea,
                    )

from matplotlib.backends.backend_qt5agg import (
                        FigureCanvasQTAgg as FigCanvas,
                        NavigationToolbar2QT as NabToolbar,
                    )

# Make sure that we are using QT5
matplotlib.use('Qt5Agg')

# create a figure and some subplots
FIG, AXES = plt.subplots(ncols=4, nrows=5, figsize=(16,16))

for AX in AXES.flatten():
    random_array = [random.randint(1, 30) for i in range(10)]
    AX.plot(random_array)

def main():
    app = QApplication(sys.argv)
    window = MyApp(FIG)
    sys.exit(app.exec_())

class MyApp(QWidget):
    def __init__(self, fig):
        super().__init__()
        self.title = 'VERTICAL, HORIZONTAL SCROLLABLE WINDOW : HERE!'
        self.posXY = (700, 40)
        self.windowSize = (1200, 800)
        self.fig = fig
        self.initUI()

    def initUI(self):
        QMainWindow().setCentralWidget(QWidget())

        self.setLayout(QVBoxLayout())
        self.layout().setContentsMargins(0, 0, 0, 0)
        self.layout().setSpacing(0)

        canvas = FigCanvas(self.fig)
        canvas.draw()

        scroll = QScrollArea(self)
        scroll.setWidget(canvas)

        nav = NabToolbar(canvas, self)
        self.layout().addWidget(nav)
        self.layout().addWidget(scroll)

        self.show_basic()

    def show_basic(self):
        self.setWindowTitle(self.title)
        self.setGeometry(*self.posXY, *self.windowSize)
        self.show()

if __name__ == '__main__':
    main()

Scrollbar on Matplotlib showing page


Drupal 7 custom module for menu

Question: I am developing a custom module that will have the functionality to show a particular menu link active on the desired path. For this I have implemented hook_form_FROM_ID_alter() hook for menu_edit_item(). In the edit form of each menu item there I have provided options similar to block modules page visibility options

Active on all pages except listed
Active only on listed pages
Active depending upon given php code

But I need to match the path listed with any existing path. Do I need to implement other hook or create more functions to implement this or this functionality should be implemented in hook_form_FROM_ID_alter(). I have also created a database table for this that stores

menu link ID
Visibility (i.e. flag to the option chosen by the user 0, 1, 2)
Pages (i.e the paths given by the user in the textarea eg: node/4,
etc)

Thanks for your help in advance
Answer: This is an existing module which does part of what you want to do. The module is Menu item visibility. I recommend looking at how it works and extending it (it says it's extendable) to add the extra pieces you are requesting: visible per page, or visible on PHP code. It probably has hooks you can use to complete your business logic.
In fact, this issue talks about adding "per page" or "php code" conditions essentially.
Comment: From the 7.x-1.x branch of the module. You would create a hook for `module_invoke_all('menu_item_visibility_load', $visibility, $mlid);` to supply a $visibility array for your FAPI elements to add your custom conditions (page, path, whatever). You would then have to patch function `menu_item_visibility_translated_menu_link_alter()` to use a hook so that your module can write the business logic to enforce your custom $visibility FAPI widget(s).    ... I'm just saying this module does most of what you want, you'd have to help finish that module and learn howto define/implement hooks.
Comment: Yeah I have referred that module but this module only provides per role visibility not per page visibility. Just tell me is there any special hook to implement that

Drupal 7 custom module for menu


What is the proper way to derive percentage from decimal

Question: I have been thinking about this for a while, and I want to know what the proper way to convert a decimal into a percentage is. Say you have a word problem that needs you to convert a decimal, say for instance 0.7, into a percentage. There would be 2 ways to do it:
0.7*100%=70%

or
0.7*100=70%

which is more correct? The first way has the flaw in which 100%=1. So the equation should rightfully be 0.7*100%=0.7. However in the second equation, the percentage is removed but the percentage sign at the very end of the equation pops up without warning. Which would be the proper way to calculate percentage in a word problem.
Comment: What is a "word problem"? As long as you can discern when you refer to a percentage and when you refer to a decimal I can't see the issue.
Comment: The first is "dimensionally correct". If you have e.g. $1=100\%$ it invites the question $100\%$ of what?
Comment: I would say your first method is more "proper". If you've taken any science classes like physics or chemistry, you'll have converted between different units. For example, to convert from meters to kilometers you would do $1\text{m} \left(\frac{1\text{km}}{1000\text{m}}\right) = 0.001 \text{km}$. If you consider "%" as a unit and "0.7" as unitless, then your situation mirrors this.
Comment: To clarify the calculation of @FullofDill, multiplying by $1$ is always legal and will never change the value of anything. Also, since $1\mathrm{km} = 1000\mathrm m$, we can take that equality, and divide it by $1000\mathrm m$ on both sides to get $\frac{1\mathrm{km}}{1000\mathrm m} = 1$. Since that fraction is the _same thing_ as $1$, we can freely mulltiply by it without changing the value of anything.
Answer:
It might be helpful to keep in mind:

The term percent is just a shorthand for hundredth, i.e. $\frac{1}{100}$. So whenever you say x percent this is the same as when saying: x-hundreth.

It follows $\color{blue}{0.7\cdot100\%}=0.7\cdot100\cdot\frac{1}{100}=\frac{70}{100}\color{blue}{=70\%}$ is correct, while the second statement is false.
Answer: You can just write
$$
.7 = 70/100 = 70\%.
$$
In other words, $70\%$ is literally just another notation for the number $70/100$, or $.7$.
Instead of saying, "$70\%$ of the cars are blue", it would be equally valid (if a bit strange-sounding) to say, "$.7$ of the cars are blue."
Answer: If it would make you more confident in your conversions, you could write
$$
(0.7 \times 100)\% = 70\%.
$$


Is a doubly augmented seventh a compound interval?

Question: I'm reading about the theory behind intervals, and Wikipedia says:

A compound interval is an interval spanning more than one octave.

Does "spanning" in this case refer to the number of staff positions or to the number of semitones?
Wikipedia lists the diminished ninth (12 semitones) as the smallest compound interval, so I'm guessing that the distinction between simple and compound intervals is based on staff positions instead of semitones: intervals encompassing 8 or less staff positions are simple, those with more are compound. Hence a double augmented ninth would still be a compound interval, whereas the doubly augmented seventh mentioned in the title would be simple.
Is my understanding correct?
Answer: Standing at what the article says:

Any compound interval can be always decomposed into one or more octaves plus one simple interval.

You cannot split a double augmented 7th in an octave + something else (basically because you haven't even reached the octave by staff positions), while you can express a 9-- as an octave plus a diminished second.
So yes, it is staff positions that matters.
Answer: Perhaps a simpler way to look at it is letter names. C4>C5 anything is simple, even if the space is more than an octave. C4>D5 anything is compound, even if the space is less than an octave.
Comment: C4>C5 is an interval. One octave. C4>Cb5 is a dim octave. C4>C#5 is an aug octave. All simple. C4>D5 is M9. C4>Db5 is m9. C4>Dbb5 is dim9. All compound.
Comment: Thanks for your input! I'm not sure I understand your "C4>C5" / "C4>D5" notation. Could you elaborate on that?
Comment: C4 is middle C. C5 is an octave above. D5 is the next note above that.
Comment: I'm aware of that, but I still don't understand what "C4>C5" is supposed to mean within the phrase "C4>C5 anything is simple".


Intermittent timeout when connecting to Sql Server, what do I look for?

Question: Sql Server 2008 Standard 64bit on Windows Server 2008 R2 virtual machine hosted on a Hyper-V server.
I'm getting intermittent timeouts when connecting to the server.  This happens for both windows and Sql Authentication.  May timeout every 2 out of 5 tries in different applications.
When the connection times out, I can see (in Profiler) that no connection was made.  Firewall is holey, server port is static (good ol' 1433).  If I ping /t the server I get a steady connection that wavers between 1 and 2 ms.
Any ideas what else to try would be appreciated, thanks.
Answer: Turned out the issue was with too many virtual machines sharing too few NICs.  Once new network cards were added the issue went away.
Answer: Is this on a server being hit by lots of users? @@MAX_CONNECTIONS will bring you back the maximum amount of connections that can be made to your SQL Server at a time. If you've got enough users that you're hitting this limit it could be causing the problem. This value can also be check/changed by going to the server properties of the instance through SQL Server Management Studio.
Is there any error on the timeouts? If the applications aren't providing good errors then you could try connecting through SSMS. You'd have to have it installed somewhere off of the server to test it this way but it's easy enough to install on a workstation.
Comment: Unfortunately it isn't a max connex issue.  Its not getting pounded at all.  The errors are all simple timeout errors.  Usually immediately attempting to connect again will succeed.


Having trouble with simple awk script

Question: This works in the command line:
awk -F, '$5>1900 {print $2}' presidents.csv

But I am not sure how to do it in an awk script.
This is what I have tried so far in the script.
BEGIN {FS=","}
if ($5>1900)
{print $2}

Answer: The stuff you have in single quotes:
awk -F, '$5>1900 {print $2}' presidents.csv

Is an AWK script.  Just stick that in a file:
BEGIN {FS=","}
$5>1900 {print $2}

Answer: AWK requires a pattern to search for and an action to be taken when it finds the pattern. Either pattern or action can be omitted but not both.
In your case first its a syntax mistake as you haven't enclosed your action within braces ( {} ) and so this will cause AWK to think both pattern and action got omitted. We can use relational expression as pattern in AWK and so instead of using an if statement you can use the relational expression as below.
$ cat test.awk
#!/usr/bin/awk -f

BEGIN {
    FS=",";
}

$5 >= 1900 {
    print $2;
}

Answer: AWk script would be like,
#!/usr/bin/awk
BEGIN {FS=",";}
{
if ($5>1900)
{print $2;}
}

That is, you need to enclose the if condition inside curly braces.
Comment: That is completely wrong. KannanMohan has the right answer (except for the useless semi-colons) and @larsks has it 100% correct.
Comment: How semicolons would make the answer wrong? Op uses the if condition.It would be possible to write without if condition. He's asking for how to rewrite the code to make it work.
Comment: semi-colons don't make it wrong they just add a bunch of useless characters, like if you put every symbol inside parens. Puting the condition inside an if statement inside the action part make it wrong. It's not how you write awk - awk scripts are written as ` {  }`, not ` { if () {  } }` and the OP deserves to know the right way to do it in awk.
Answer: If you'd like a shell script that runs your awk command you can put the following into a file:
#!/bin/bash
awk -F, '$1>1900 {print $2}' $1

To make it so you can actually execute the script, you can run:
chmod +x ./script_file_name_here

Then you can run
./script_file_name_here presidents.csv

and have the same thing happen.

Having trouble with simple awk script


Coefficient of variation of circular data

Question: The definition of coefficient of variation is as follows:
coefficient of variation= standard deviation/mean.
I am using circular (directional) statistics to find the mean and standard deviation. My question is: can I use the same definition of the coefficient of variation (given above) for circular data as well? Thanks in advance
Answer: This answer argues that there is no need for any version of coefficient of variation for circular data. Worse, attempts to define or calculate one lead to absurdity.
For a self-contained introduction to the topic setting out some notation and definitions this Wikipedia article will do fine.
Circular data take values on an outcome space that is a circle. Direction as a bearing relative to North or some other standard direction is a common example. Let's suppose for concreteness that the units of measurement are degrees, ranging from $0$ to $360^\circ$, noting that the key point is that $0$ and $360$ degrees are identical directions. Nothing depends on a choice of units: radians or time of day or time of year could make as much or more sense for particular applications.
Let's back up and consider that any kind of coefficient of variation is the ratio of a standard deviation, or at least some measure of scale or variability, and a mean, or at least some measure of level or location.
The standard definition of a mean for circular data is the vector mean. A standard mean is very often a poor choice, as for example if you have directions that are $1$ and $359^\circ$ feeding them to any ordinary mean routine will return $180^\circ$. The vector mean is the arctangent of the sum of sines of directions divided by the sum of cosines, or more plainly the resultant from adding directions as vectors geometrically, namely end to end.  The vector mean of $1$ and $359^\circ$ is $0$ or $360^\circ$, which as a diagram will suggest is a natural solution. More generally, the vector mean is well defined in all but a few pathological cases (such as data that are two opposite directions, which cancel).
Variability of directions on a circle is most simply measured as mean resultant length. (That name is the most popular among several in use; in my view the rarely used name consistency is much more evocative.) The mean resultant length is at its largest when all directions are identical (hence the alternative name consistency) and at its smallest when directions are equally common in  opposite directions (a circular uniform distribution is one but not the only possibility). Hence mean resultant length is an inverse measure of variability, and some have defined circular variance as its complement (in 1 or in 100%). This is likely to seem confusing at first sight, as either measure is reported  on a scale from 0 to 1 (or 0 to 100%). But any convention about using degrees, radians or other units for the original data does not bite, as those units wash out of the definition and calculation. Further, a circular standard deviation has been defined but, again confusingly at first sight, not as the square root of the variance. For details, see for example the article cited above.
To the point: Dividing any measure of circular variability by any mean is neither needed nor even justifiable. The vector mean could be zero, i.e. it could coincide with the reference direction, whether say North, or South, or midnight, or the start of a year. If the absurdity of dividing by zero is avoided by rotation, then one absurdity becomes another as now the result of dividing any measure of variability by any vector mean is utterly dependent on a convention about the reference direction.
Coefficients of variation have often been oversold in statistical applications, but their merit when they are well defined and useful is as measures of relative variability of counts or measurements of variables with positive mean and an unequivocal zero point. See for example this thread on CV. There is no analogue of that set-up in circular statistics, which concerns a quite different outcome space.
Comment: @gung But it is only a convention to have a zero at midnight.  If your  data are patient counts in time bins, then that's a different outcome space -- and in practice one analysis could be in terms of sine and cosine of hour of day as covariates, together with any others. Oddly, this territory is usually ignored in texts on circular statistics.  But however you parametrise your set-up, it is not what is generally considered as circular data. Although I disagree, partly or wholly, with the other answers to date, neither answer seems to understand the question in that way.
Comment: Thanks for your detailed answer Nick. I have a quick question to clarify sth for myself:
We are calculating our circular standard deviation using this formula: S=sqrt(-2*log(r)), where r is mean resultant vector length.
What is the unit of this standard deviation? or is it unit-less?
Comment: I did more search on this and I found that circular standard deviation has units (it can be in degrees or radians). You can see it in the following paper for instance: "Cell directional spread determines accuracy, precision, and length of the neuronal population vector".
Am I missing something?
Comment: I've simplified my answer to remove an assertion that was over-stated and indeed incorrect.
Comment: This is interesting, Nick, & it's a bit outside of my wheelhouse.  But consider a counterexample:  The number of new patients arriving at the ER by hour of the day.  Hour should be circular, but the number of patients is a count & it's not too outrageous to imagine that the SD scales w/ the mean.
Comment: The resultant vector length is a pure number (length relative to the length of the unit vector). Thus, the standard deviation as defined above must be a pure number, too. The time of day of arrival of each patient is a circular variable, and could be marked as unit vector on a circle of 2 pi \equiv 1440 min. The number of patients in a time slot is not circular in the same sense, see chapter 2 of Fisher (1993)
Answer: This type of coefficient of variation is generally only useful if the mean is not 'arbitrary', that is, the data means something very different if the mean is artificially raised. An example would be count data, but there are many practical situations where the coefficient of variation could be sensible with continuous data as well.
For circular data, however, we rarely have this situation; in most cases we expect to be able to rotate the data without changing its meaning too much. That is, the point where we put $0^\circ$ is arbitrary. So, in most cases, a coefficient of variation is not useful; on the flip side, the standard deviation is more directly interpretable, as it has a 'maximum' if the data is circular uniform.
An exception could be 'accuracy' style data, where $0^\circ$ is 'hitting the target'. In these cases, the $0^\circ$ point is not arbitrary, and the coefficient of variation can be useful. The exception to this exception is if the mean is very close to $0^\circ$ most of the time; this would make the coefficient of variation unstable due to the division with a small number.
Comment: Thanks for your explanation @keesmulder. I see what you mean by arbitrary mean. One thing that comes to my mind is that this arbitrary mean may happen in conventional (not circular) statistics as well. For example, in many problems the choice of origin is arbitrary. So maybe it is not only a problem in circular statistics. Is this correct?
Comment: Also, I think the standard deviation shows how close (or far) is our data to its mean (is this definition correct in the circular standard deviation as well?). Therefore, the coefficient of variation can show the variability in relation to the mean of the population. Is my interpretation correct?
Comment: This starts well but I think even your exception strains credulity. Please see my comment on a later answer. I can't see that there is ever a meaning to dividing by a vector mean using scalar arithmetic. The point is that measures of variability of circular data are already relative to the entire circle; there is no further scaling either needed or sensible.  Even when 0 deg is a natural reference point, the CV is absurdly unstable.
Comment: Thanks for your response @keesMulder. But I have a question, why you are saying that our mean is arbitrary when we use circular statistics (that takes periodicity of the data into account). In this case, can't we use the coefficient of variation (when mean is not zero) the way that I mentioned to show the variability of a specific data?
Comment: It's arbitrary in the sense that it is not 'natural' but a choice made by the analyst. A good example is the compass; there is no reason why would set north to be 0 degrees, or west, but we have to choose one. Therefore, the coefficient of variation would depend on this arbitrary choice of where is the zero. As I mentioned, this is not always the case, but is quite often with circular data.
Comment: Yes, you are right, this happens in non-circular statistics as well. And your interpretation of the coefficient of variation is correct as well.
Comment: I agree, @NickCox, thanks for your answer below extending this side of the story. I still think in bounded settings (ie. I ran across a vet at some point measuring the range of movements of horse joints) it might still be possible to carefully employ a cv-like measurement on angles, but I think we would both agree that those situations are barely circular statistics-settings at all. So I think the conclusion that cv-measurements should not be used in circular settings holds.
Answer: Given that you estimate the mean value and the standard deviation taking into account the circular statistics I suppose that you can estimate the CV with the same formula sd/mean. For example in the case of wind direction the well-known formula it works fine.
Comment: Not so. Suppose your mean direction is 0 deg N, except that it could be 359 or 1 deg. Dividing a standard deviation by 359, 0, 1 will give you variously a very small number, no quotable result at all, or a very large number, which is absurd.  The same problem applies with directions in radians, or any other units. This is even supposing that the SD of circular data has been calculated in a way that doesn't depend on the origin, as you do state.
Comment: Yes but in this case, you can convert the degrees to negative values for instance 359,0,1 could be written as -1,0,1 and hence you can estimate the CV without having a problem.
Comment: You make my point for me but more strongly. Not only does your version of the CV depend on your convention about origin it can  be positive, negative or not determined.
Comment: I understand your point and thanks for the answer, but I think with a simple code the problem could be resolved.


errors running HttpWebRequest.GetResponse - 401 unauthorized

Question: I am using Lymbix client library for sentiment analysis.
When I run the code I am getting an error in (WebResponse)httpRequest.GetResponse(): 401-Unauthorized
(available at https://github.com/lymbix/.NET-Wrapper)
The function containing 401 error is given below:
<code>    private static string Post(string url, string data, List<string> headers)
    {
        HttpWebRequest httpRequest = (HttpWebRequest)HttpWebRequest.Create(url);
        httpRequest.Method = "POST";
        httpRequest.Accept = "application/json";
        httpRequest.ContentType = "application/x-www-form-urlencoded";

        if (headers != null)
        {
            foreach (string header in headers)
            {
                httpRequest.Headers.Add(header);
            }
        }

        // write request?

        byte[] postData = Encoding.UTF8.GetBytes(data.ToString());
        httpRequest.ContentLength = postData.Length;
        httpRequest.GetRequestStream().Write(postData, 0, postData.Length);

        // read response
        WebResponse webResponse = (WebResponse)httpRequest.GetResponse();
        StreamReader webResponseStream = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8);
        return webResponseStream.ReadToEnd();
    }
</code>
Comment: No sure about Lymbix but do you need to send them credentials of some kind? 401 error would point to that pretty clearly.
Answer: It's saying you're not authorized, so you need to provide credentials.
<code>HttpWebRequest httpRequest = (HttpWebRequest)HttpWebRequest.Create(url);
httpRequest.Method = "POST";
httpRequest.Accept = "application/json";
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.Credentials = new NetworkCredential("username","password");
httpRequest.UseDefaultCredentials = false; //the default is false, but I included it here just to illustrate that it needs to be false in order to use the specified credentials
</code>


Complex Fourier-series

Question: Im supposed to show that $C_n = \frac{1(1-e^{-\pi}(-1)^n)}{\pi(1+n^2)}$
I'm computing the Fourier coefficients of the $2\pi$-periodic function $f(x) = \exp(-|x|)$ on $[-\pi,\pi]$. I need to show that:
$$\hat{f}(n) = \frac{1-e^{-\pi}(-1)^n}{\pi(1+n^2)}$$
So far, this is what I have done:
$$\hat{f}(n) = \frac{1}{2\pi} \int_{-\pi}^{\pi} f(x) e^{-inx} \ dx = \frac{1}{2\pi} \int_{-\pi}^{0} e^x e^{-inx} \ dx + \frac{1}{2\pi} \int_{0}^{\pi} e^{-x} e^{-inx} \ dx$$
Then, we have that:
$$\int_{-\pi}^{0} e^x e^{-inx} \ dx = \int_{-\pi}^{0} e^{x-inx} \ dx = \int_{-\pi}^{0} e^{x(1-in)} \ dx \frac{1}{1-in}(1-e^{-\pi(1-in)})$$
$$\int_{0}^{\pi} e^{-x} e^{-inx} \ dx = \int_{0}^{\pi} e^{-x(1+in)} \ dx = -\frac{1}{1+in} (e^{-\pi(1+in)}-1) = \frac{1}{1+in} (1-e^{-\pi(1+in)})$$
So, then:
$$\hat{f}(n) = \frac{1}{2\pi} \frac{1}{1-in} (1-e^{-\pi(1-in)}) + \frac{1}{2\pi} \frac{1}{1+in} (1-e^{-\pi(1+in)})$$
I have got so far, but I do not how to move forward to remove the $i$s.
Answer: First, let's recognize that:
$$\exp(i\pi n) = (\exp(i\pi))^n = (-1)^n$$
Right, so you have that:
$$2\pi c_n(1+n^2) = (1+in)(1-\exp(-\pi)(-1)^n) + (1-in)(1-\exp(-\pi)(-1)^n) = (1+in+1-in)(1-\exp(-\pi)(-1)^n)$$
I just multiplied throughout by $1+n^2$ and used the fact that $1+n^2 = (1-in)(1+in)$. Also, I've used the fact that:
$$\exp(-\pi(1-in)) = \exp(-\pi)\exp(i\pi n) = \exp(-\pi) (-1)^n$$
and I've done the same thing for the other term (this means that you should do the calculation yourself and verify that it's correct). So, it follows that:
$$2\pi c_n(1+n^2) = 2(1-\exp(-\pi)(-1)^n)$$
But this implies that:
$$c_n = \frac{1-\exp(-\pi)(-1)^n}{\pi(1+n^2)}$$
and you're done. For future reference, you should try to type out your work here in latex so that your questions don't get downvoted. It's just fine in this case (at least, for me) because your handwriting is gorgeous :)
Comment: okai thank you! this makes a lot of sense! Im sorry, im new here so im not that good in latex, but i will keep that in mind. Again, thank you!
Comment: No problem at all, please upvote and accept the answer if you found it helpful :D

Complex Fourier-series


Expectation $E[e^{\lambda B_{T}}]$ where $T$ is a stopping time w.r.t. Brownian Motion

Question: Consider a $1$-dimensional Brownian motion started from $0$. Compute $E[e^{\lambda  B_{T}}]$, where $\lambda>0$ and $T$ was the first time $t$ for which $B_t=1$.
If this were $E[e^{\lambda  {T}}]$, then I would know how to compute it. However, I am a bit confused with how one can compute this expectation with $B_T$ involved. I assume this should be quite straightforward; one just needs to find a suitable martingale and then apply the optional stopping theorem and dominated convergence. However, I am not seeing it.
Could I use that $e^{\lambda B_t-\lambda^2t/2}$ is a martingale?
Answer: $B_T$ is just the constant random variable $1$ so $Ee^{\lambda B_T}=e^{\lambda}$.
Comment: Wow, I definitely over complicated that! Thanks.


How to sync DAW VST automations to live performance?

Question: I'm looking for a tool to control VST instruments through my epiano midi.
My goal is to play a song, and having the DAW (or whatever) detect the tempo by analysing what I'm currently playing.
Question: is that possible at all?
I would want to create some kind of "song template" that syncs automatically with my play, and could automatically switch sounds in time. I could achieve this by using the DAW automation settings to control VST instruments.
Like: starting with a string sound in the intro and verse, and switching (or adding) additional piano sounds automatically in the chorus.
I could program the template beforehand (as number of bars are known for each verse/chorus). But I don't know how I could sync this to my live playback, so the DAW knows the exact time to switch?
Or if that's not possible to live-detect the timing just by what I play with my midi keyboard: is there any program that can switch sounds by midi signal? Like a C5 key could active the verse sounds, an D5 the chorus sounds?
Answer: That should be possible, especially with MIDI as notes are coded as orders. However, have you think about how risky that would be? An algorithm trying to detect the tempo and then counting the beats in order to manage a transition would have some chances to fail somewhere. If that would be the case you would have no way to recover which, for a live performance is a bit annoying…
Your second solution seems more practical. What you really want to send is a ControlChange or a ProgramChange order. On most MIDI keyboards you can assign a button (or several) to perform this. If you do not have the time to take your hands of the keyboard (the playing part) then I am sure there are programs (like QMidiRoute on Linux) which are able to detect a specific MIDI order (like a NoteOn on a predefined specific note) and modify it to output a ProgramChange for instance. In my opinion that would be way more reliable for live, but beware to press only this note when you want a change to happen!
A last way that you did not mentioned (maybe because it needs purchasing) would be to have a MIDI foot controller. By a foot press on one of the switches you can send MIDI signal to your DAW saying that it has to change the sound. This solution would not "remove" some keys from your keyboard to assign them to something else and would keep your hands free…
Answer: Using designated MIDI notes as control messages - 'key-switching' - is standard practice in sequencers such as Cubase.  It's also an internal feature of some virtual instruments.
Musical Theatre keyboardists routinely use MainStage on a Mac computer to play the complex and  ever-changing  patches that have become their stock-in-trade.  Program switching is typically by a footswitch, but could easily be by a 'hotkey' on the keyboard.
Tempo detection is sometimes used when a backing track needs to 'follow' rather than lead the tempo.  It can work, when the keyboard playing is very disciplined!   I think, however, that a footswitch or keyswitch will suit your purpose better and more reliably.   (It doesn't HAVE to be MainStage on a Mac.  It's just that they've cornered the MT market at the moment.)
Answer: I'll assume that your real question is, how to change sounds in the middle of a song. This is an age-old need and your idea of pre-programmed and timed song sequences is only one way to do it. There are many different strategies for it. I'll try to list a few.

Make a sequence ("template song") for each song's exact bar structure and the sound changes and sync your live playing to the sequence. You'll need to listen to a click track when playing. Computer says click click click and you obey. This is essentially the same as playing over a backing track, a commonly used strategy in many sequenced shows and mixed electronic/rock/vocal genres where some instrument parts (or lighting or special effects) are sequenced. The sequence could include played instrument tracks, but in your case it only changes sound settings.

The same as above, but adjust the sequencer's tempo with a tap-tempo sort of pedal or other syncing device that generates MIDI clock and you can make the sequencer follow your tempo. I'm not sure if this makes running the show easier or more difficult. There are even products that try to sync to the tempo of incoming audio, but I don't think they'll follow anyone's piano playing. And what if you make a mistake and the sync program decides that you just sped up by 50% and speed-runs five bars ahead of you.

Make a sequence for each song, but with a "scene" for each part like verse, chorus etc. and you can manually trigger the scenes with MIDI commands like key-presses or control changes, etc. from a keyboard or foot controller. Ableton Live supports this, and maybe other programs and utilities as well. Whenever the verse/chorus etc changes, you say to the sequencer "next scene please", with a foot controller, key-press, control-change etc.

If your keyboard or synth has lots of changeable user memories, you can program the sounds or program-changes for all parts of songs or entire sets of songs into the user bank programs. For example, preset 001: song 1 intro, preset 002: song 1 verse, preset 003: song 1 chorus, ... etc. You start the song (or whole set) from preset 001 and every time you need to change sounds you just press NEXT. Programming the songs or sets can be tedious, but some players use this method.

Limit your playing to only a few sounds and use the sounds as a single instrument and as your natural expression. Do you really need dozens of different sounds? Would three sounds be enough? Are you a cover band working as a human jukebox, expected to replicate all the same sounds and effects that are on the record everyone knows? How about using just a couple of sounds, and adding a piano when you want the extra power. Guitarists can dial in more power and distortion as a part of their musical expression, instead of trying to be a timed theater act. Use a MIDI volume pedal that brings in the piano when you want power. There's nothing wrong with playing the whole gig with just one Rhodes sound! If you want to change the sound, activate an effect or change the amount of drive or distortion. Learn to accept sound-changes as a part of your instrument, just like pressing the keys is. Drawbar organ players use the drawbars (and Leslie control) as part of their musical expression.

Use multiple keyboards. The lower one is your master keyboard where you have your main grand piano etc. and the upper ones are used for auxiliary sounds like pads, leads etc. This is like an organist using multiple manuals.
Comment: Another option that works in in some scenarios is to split the keyboard -- for instance, I might have a piano sound covering most of the keyboard and a synth lead in the upper octave or two. Also, great point on the volume pedal/expression. I often like to map the modwheel to a low-pass filter on my pads, as that way I can smoothly thicken or thin the sound during performance.


DHCP: DHCPREQUEST issues

Question: Maybe someone that knows more about DHCP and network that can help me out with this one.  I've been thrown for a look with this for awhile now, so I'm reaching out to the ServerFault community.
We migrated a server out of our production datacenter as we are repurposing the system.  In our local lab, I PXE booted the server into our environment and it PXE booted fine and loaded the image just fine.  However, after loading the image, it couldn't download the kickstart.  I noticed from the DHCP logs, it was trying to get a new IP address and couldn't.  What it seems to look like is, it PXE boots using a local PXE/DHCP server and then once it loads up and then tries to DHCP again, it reaches out to a random server outside our lab environment.  Here are the DHCP logs:
dhcpd: DHCPDISCOVER from 00:11:22:33:44:55 via 10.6.246.1
dhcpd: DHCPOFFER on 10.6.246.9 to 00:11:22:33:44:55 via 10.6.246.1
dhcpd: DHCPREQUEST for 10.6.246.9 (10.6.247.236) from 00:11:22:33:44:55 via 10.6.246.1
dhcpd: DHCPACK on 10.6.246.9 to 00:11:22:33:44:55 via 10.6.246.1
xinetd[3010]: START: tftp pid=26359 from=10.6.246.9
in.tftpd[26360]: RRQ from 10.6.246.9 filename /pxelinux.0
in.tftpd[26360]: tftp: client does not accept options
in.tftpd[26361]: RRQ from 10.6.246.9 filename /pxelinux.0
in.tftpd[26362]: RRQ from 10.6.246.9 filename /pxelinux.cfg/44454c4c-5800-104e-8057-b9c04f4b4e31
in.tftpd[26363]: RRQ from 10.6.246.9 filename /pxelinux.cfg/01-00-11-22-33-44-55
in.tftpd[26364]: RRQ from 10.6.246.9 filename //images/RHEL-6.4-x86_64/vmlinuz
in.tftpd[26365]: RRQ from 10.6.246.9 filename //images/RHEL-6.4-x86_64/initrd.img
dhcpd: DHCPDISCOVER from 00:11:22:33:44:55 via 10.6.246.1
dhcpd: DHCPOFFER on 10.6.246.9 to 00:11:22:33:44:55 via 10.6.246.1
dhcpd: DHCPREQUEST for 10.88.254.102 (10.88.254.22) from 00:11:22:33:44:55 via 10.6.246.1: ignored (not authoritative).

Things to note are:

00:11:22:33:44:55 is the MAC address I am referring to
In the DHCP server, the subnet 10.6.246.0 is set as authoratative
In the DHCP server, the option routers section is set to 10.6.246.1, which is the default gateway - which in our network, can get to anything on the network

Does this seem weird or is it just me?  I feel like it's reaching out to something that it shouldn't be.
Also - that 10.88.254.102 and .22 servers - those are not pingable from anywhere, however, when I run an nmap on them they show a squid-http servers (which would be proxy servers), however, my network team tells mem those don't have an ARP entry so they are "down".
I'm at a loss here.
Comment: Have you already checked the `ks` and `ksdevice` parameters in your `append` line?
Comment: Yes - found the issue though - I'll post the issue.
Answer: Was able to resolve the issue myself. There was a rogue DHCP server on the same switch (and thus, the same network/VLAN) as the server I was provisioning.
This was an weird mistake, due to the fact that both servers were plugged in at the same time, and apparently the second server actually booted up, and the server used to be an old DHCP server, therefore, it started issuing out DHCP offers without me even knowing that it used to be used for that purpose.  I had our network team trace down the offers and that's when we realized that it was the second server causing the issues.  So weird though.

DHCP: DHCPREQUEST issues


Fragment going over ActionBar in ViewPager

Question: I am using viewpager to show different fragments but in this one of fragment replaces the actionbar. Till now I have found that it is because of edittext which I am using for searching items. When I remove edittext from this it works fine.
The main activity layout is a CoordinatorLayout, containing the typical AppBarLayout and then a ViewPager and it is filling whole screen.
fragment_gst_rates.xml:
<LinearLayout
       xmlns:android="http://schemas.android.com/apk/res/android"
       xmlns:tools="http://schemas.android.com/tools"
       android:layout_width="match_parent"
       android:layout_height="match_parent"
       xmlns:app="http://schemas.android.com/apk/res-auto"
       android:orientation="vertical"
       android:padding="5dp"
       tools:context="com.kamalarora.mygst.FragmentGstRates">

       <LinearLayout
           android:layout_width="match_parent"
           android:layout_height="match_parent"
           android:orientation="vertical"
           android:id="@+id/linearLayout">

          <LinearLayout
              android:layout_width="match_parent"
              android:layout_height="wrap_content">

              <EditText
                  android:id="@+id/searchBox"
                  android:layout_width="match_parent"
                  android:layout_height="wrap_content"
                  android:hint="Search items . . ."
                  android:layout_marginTop="5dp"
                  android:layout_marginBottom="2dp"
                  android:drawableRight="@android:drawable/ic_menu_search"/>

          </LinearLayout>

          <LinearLayout
              android:layout_width="match_parent"
              android:layout_height="wrap_content"
              android:orientation="horizontal"
              android:background="@color/lightGrey">
              <TextView
                  android:layout_width="wrap_content"
                  android:layout_height="wrap_content"
                  android:text="Products"
                  android:textSize="18dp"
                  android:layout_marginLeft="1dp"
                  android:padding="5dp"/>
              <TextView
                  android:layout_width="match_parent"
                  android:layout_height="wrap_content"
                  android:text="GST Rate"
                  android:layout_marginRight="1dp"
                  android:textSize="18dp"
                  android:gravity="right"
                  android:padding="3dp"/>

          </LinearLayout>

          <ListView
              android:layout_width="match_parent"
              android:layout_height="match_parent"
              android:orientation="vertical"
              android:id="@+id/listView">
          </ListView>

      </LinearLayout>

   </LinearLayout>

activity_main.xml:
    <?xml version="1.0" encoding="utf-8"?>
<android.support.design.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/main_content"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:fitsSystemWindows="true"
    android:orientation="vertical"
    tools:context="com.kamalarora.mygst.MainActivity">

    <android.support.design.widget.AppBarLayout
        android:id="@+id/appbar"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:paddingTop="@dimen/appbar_padding_top"
        android:theme="@style/AppTheme.AppBarOverlay">

        <android.support.v7.widget.Toolbar
            android:id="@+id/toolbar"
            android:layout_width="match_parent"
            android:layout_height="?attr/actionBarSize"
            android:background="?attr/colorPrimary"
            app:layout_scrollFlags="scroll|enterAlways"
            app:popupTheme="@style/AppTheme.PopupOverlay">

        </android.support.v7.widget.Toolbar>

        <View
            android:id="@+id/appbar_bottom"
            android:layout_width="match_parent"
            android:layout_height="1dp"
            android:background="@android:color/transparent"
            android:visibility="invisible" />

    </android.support.design.widget.AppBarLayout>

    <android.support.v4.view.ViewPager
        android:id="@+id/container"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_marginBottom="104dp"
        app:layout_behavior="@string/appbar_scrolling_view_behavior" />

    <android.support.design.widget.BottomNavigationView
        android:id="@+id/bottomNav"
        android:layout_width="match_parent"
        android:layout_height="50dp"
        android:layout_alignParentBottom="true"
        android:layout_gravity="bottom"
        app:itemBackground="@color/white"
        app:itemIconTint="@color/mediumgrey"
        app:itemTextColor="@color/mediumgrey"
        app:menu="@menu/menu_bottom" />

</android.support.design.widget.CoordinatorLayout>

Comment: pls post your main activity xml code.
Answer: do you tried with the linear layout at the root there try change with android:layout_height="wrap_content" .


How to find grid neighbors from a string?

Question: If I collapsed a 2 dimensional grid into 1 dimension (e.g a 3x3 grid into a string):
1 2 3
4 5 6     =>    1 2 3 4 5 6 7 8 9
7 8 9

Is there a formula where I could verify if two points used to be neighbors?
Working it out on a markerboard, the best I could come up with is:
(a % r) - (b % r) = {0,1}

Where a & b are the points and r is the row length.
This works well enough for handling the wrapping problem (e.g. 3 and 4 are not neighbors). But it thinks everything in the same column are neighbors. This formula would say that 1 and 7 are neighbors because (1%3)-(7%3) => 1-1 => 0.
Is there a way to correct that? I have no idea what I'm doing, so I may be approaching this the wrong way. I would not be surprised if there was an existing algorithm, and that ultimately is what I am after. I don't need my formula fixed, I'm just providing it to hopefully explain whats going on.
Answer: You are on the right track. If you write an $r\times r$ square matrix this way,
\begin{bmatrix}
    x_{0,0} & x_{0,1} & x_{0,2} & \dots  & x_{0,r-1} \\
    x_{1,0} & x_{1,1} & x_{1,2} & \dots  & x_{1,r-1} \\
    \vdots & \vdots & \vdots & \ddots & \vdots \\
    x_{r-1,0} & x_{r-1,1} & x_{r-1,2} & \dots  & x_{r-1,r-1}
\end{bmatrix}
the elements $x_{i,j}$ and $x_{k,\ell}$ are neighbors if and only if $|i-k|$ and $|j-\ell|$ add up to $1$.
Your matrix is indexed with one number:
\begin{bmatrix}
    a_{1} & a_{2} & a_{3} & \dots  & a_{r} \\
    a_{r+1} & a_{r+2} & a_{r+3} & \dots  & a_{2r} \\
    \vdots & \vdots & \vdots & \ddots & \vdots \\
    a_{r(r-1)+1} & a_{r(r-1)+2} & a_{r(r-1)+3} & \dots  & a_{r^2}
\end{bmatrix}
Therefore, if you can find functions $i(n)$ and $j(n)$ so that $a_n=x_{i(n),j(n)}$, the adjacency condition will be that $a_n$ is adjacent to $a_m$ if and only if
$$|i(n)-i(m)|+|j(n)-j(m)|=1.$$
The functions $i$ and $j$ aren't hard to come up with. They are almost quotient and remainder modulo $r$, but since you are counting from $1$, not $0$, they are
$$ i(n)=(i-1)\backslash r\textrm {  and  } j(n)=(j-1)\% r,$$ where $\backslash$ is integer division (discarding remainder).
Note:If you want to consider an element its own neighbor, change $= 1$ in the formula to $\le 1$, and if you want to consider "diagonal neighbors," change $|i(n)-i(m)|+|j(n)-j(m)|$ to $\max\left(|i(n)-i(m)|,|j(n)-j(m)|\right)$
Answer: Lets make a map from the linear index of your numbers to the rows and columns. I will assume your linear index starts with $0$ instead of $1$ to make the formulae simpler. Given a $n \times m$ integers $0, 1, \ldots, n\times m -1$ we want to arrange these into a grid of $n$ columns and $m$ rows:
\begin{align*}
&0& &\cdots& &n-1& \\
&n& &\cdots& &2n& \\
&\vdots& &\vdots& &\vdots&\\
&(m-1)\times n& &\cdots& &m\times n -1
\end{align*}
Now we want a function which gives us the row $r(x)$ and the column $c(x)$, lets assume these have values at $0 \ldots (m-1)$ and $0 \ldots (n-1)$ respectively.
Then given some linear index $x$ we know $x = n \times r(x) + c(x)$ so $c(x) = x \mod n$ and $r(x) = \frac{x- c(x)}{n}$.
Then given $x,y$ in your grid you can define a neighbor function:
$neighbor(x,y) \,\,\textrm{if}\,\, (r(x) = r(y) \pm 1 \,\,\textrm{and}\,\, c(x) = c(y))\,\, \textrm{or}]\,\, (r(x) = r(y) \,\,\textrm{and}\,\, c(x) = c(y) \pm 1) $.


jquery/javascript - running two functions from an onComplete property

Question: I'm using a plugin which has a property "onComplete".
So I set something like this "onComplete: completeHandler"
And it runs the function defined in:
function completeHandler()

I just need an additional one line of code that I need to run, is there a way to say to run two functions? Or is my best way to write another function called "alternateCompleteHandler" and at the end of that function call completeHandler()?
Answer: This is what you're looking for:
onComplete: function(){
   function1();
   function2();
}

Answer: If you want two functions to be run simultaneously, Use this:
setTimeout(function1,1);
setTimeout(function2,1);

If you want them to run after each other, do this:
onComplete: function() {completeHandler();alternateCompleteHandler();}

its called anonymous function.
Comment: Please note that you cannot actually have two functions be processed at the same time in Javascript.  The Javascript engine in the browser will simply be context switching between the two functions, or running one after the other.
Comment: As you cant have two functions (or thread or even processes) run at the same time on a single CPU anywhere.
Answer: I would say the most flexible way is to create a new function. I think there are better options for naming it, though :)


dhsic.test returning different p-values?

Question: I am using dhsic.test in R, and I have this doubt:
For this example of clearly uncorrelated data I get different p-values:
ug=matrix(nrow=N,c(0.5,0,0,0.5))
r=mvrnorm(n=500,rep(0,N),ug)
dhsic.test(split(r,col(r)))$p.value

if I rerun just the last line (only the test) I get different values like .44, .37 ,.36
Shouldn't I get always the same p-value?
Answer: Your code does not run - you are missing the packages and the value of N - but the underlying reason is that the test (at least using the permutation or bootstrap methods and possibly eigenvalue, where permutation is the default) takes a Monte-Carlo approach.  So if you want identical results from repeated tests of the same data using the same method then you need to use an explicit seed each time.
Here is some R code to illustrate your point and mine. The arbitrary seed of 2020 should make your machine produce the same numbers as mine, while the arbitrary but repeated seeds of 1 produce the same answers for repeated use of the same method:
set.seed(2020)
library(dHSIC)
library(MASS)
N = 2
ug = matrix(nrow=N, c(0.5,0,0,0.5))
r = mvrnorm(n=500, rep(0,N), ug)
dhsic.test(split(r,col(r)))$p.value
# 0.1208791
dhsic.test(split(r,col(r)))$p.value
# 0.1348651
dhsic.test(split(r,col(r)), method="gamma")$p.value
# 0.1352237
dhsic.test(split(r,col(r)), method="gamma")$p.value
# 0.1352237
dhsic.test(split(r,col(r)), method="permutation")$p.value
# 0.1348651
dhsic.test(split(r,col(r)), method="permutation")$p.value
# 0.1298701
dhsic.test(split(r,col(r)), method="bootstrap")$p.value
# 0.1128871
dhsic.test(split(r,col(r)), method="bootstrap")$p.value
# 0.1398601
dhsic.test(split(r,col(r)), method="eigenvalue")$p.value
# 0.1214
dhsic.test(split(r,col(r)), method="eigenvalue")$p.value
# 0.1244
set.seed(1)
dhsic.test(split(r,col(r)))$p.value
# 0.1258741
set.seed(1)
dhsic.test(split(r,col(r)))$p.value
# 0.1258741
set.seed(1)
dhsic.test(split(r,col(r)), method="gamma")$p.value
# 0.1352237
set.seed(1)
dhsic.test(split(r,col(r)), method="gamma")$p.value
# 0.1352237
set.seed(1)
dhsic.test(split(r,col(r)), method="permutation")$p.value
# 0.1258741
set.seed(1)
dhsic.test(split(r,col(r)), method="permutation")$p.value
# 0.1258741
set.seed(1)
dhsic.test(split(r,col(r)), method="bootstrap")$p.value
# 0.1178821
set.seed(1)
dhsic.test(split(r,col(r)), method="bootstrap")$p.value
# 0.1178821
set.seed(1)
dhsic.test(split(r,col(r)), method="eigenvalue")$p.value
# 0.1268
set.seed(1)
dhsic.test(split(r,col(r)), method="eigenvalue")$p.value
# 0.1268


What is a proper, modern and cross-browser safe method of creating JavaScript classes?

Question: I am confused with hundreds of ways of creating JS classes. One says that I should use prototypes, while others say that noone uses prototype beacuse it is "bad". On the other hand CoffeeScript uses prototypes but wraps a construction with a function whick returns itself (or something). I've seen function that returns an object, function that returns a function that returns an object etc..
I think it should be easy and no frameworks should be needed to create classes in a language - maybe I'm missing something.
There are also two (al least) ways of creating methods : foo: function() {} and function foo() {}. I've even seen those two ways in single class. The problem is that the first way results in creating annonymous functions (wich happens to be assigned to a field of an object) and debuggers say that error happened in annonymous function called by annonymous function etc.
I understand that JS is intended to be functional rather than OOP, but sometimes a class is the best way to describe a concept (e.g. a UI widget wants to be a class).
I'd be grateful for an example of proper constructed class with few words of explanation.
Comment: JavaScript is a prototype-based language, so when you use constructor functions ("classes"), you're really supposed to use the prototype to add functionality. Only extending the prototype of other functions that are not your own is considered bad practice - that's like altering the code of a class.
Comment: You can declare method as `foo: function foo() {}` and not bump into issue with debuger.
Comment: This is really problem with javascript. there is not any documentation from JS authority or if it is then it is so complicated. :(
Answer: I think you can consider the code that CoffeeScript produces "good":
// Create a "class" named Foo = create a variable that contains whatever the
// self-invoking anonymous function returns (the function avoids the pollution
// the global namespace).
var Foo = (function() {

  // Create the "class" = the constructor function that is supposed to be used
  // with the "new" keyword to create instances (objects).
  function Foo() {}

  // Add a method "bar" (that's what the prototype is for!)
  Foo.prototype.bar = function(baz) {
    // Assign the value to a member variable of the current instance (this)
    this.foobar = baz;
  };

  // ...add more stuff.

  // Return only the function, every other local variable stays in this scope.
  return Foo;

})();

Comment: It looks reasonably, however it is rather complicated.
Comment: In my experience, that pays off when classes get bigger.
Answer: I think this article explains it well:
https://developer.mozilla.org/en-US/docs/JavaScript/Introduction_to_Object-Oriented_JavaScript
This is (I believe) the correct way to use classes in a prototype based language like javascript, and gives a good explanation of the concepts. I use this approach in my projects, and it seems to work in all modern browsers.
Answer: If you are confident in using CoffeeScript, it has solid approach to classes, and gives you much clear syntax, compared to any other OOP framework at the same time.
Comment: Actually I don't want (can't) use CoffeeScript. I was just looking how  others do this.


Java / Swing -> Creating a notification JFrame, and the error "The frame is displayable"

Question: Ok, I am not that versed in Java / Swing, and I am running into a problem. My application is throwing this error: "Exception in thread "AWT-EventQueue-0" java.awt.IllegalComponentStateException: The frame is displayable." Google isn't turning up much information on this error, and a quick search of StackOverflow is not showing anything yet.
What I am trying to do: just implement translucent windows using the code from the tutorials. That's it. I plan to use this as a general window for displaying notifications, such as "This program has not been run in 30 days" down near the system tray.
Are there any experienced Java / Swing gurus out there who can spot my beginner's mistake?
Thank You.
/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package energizingemerald;

/**
 *
 */
import java.awt.*;
import javax.swing.*;
import static java.awt.GraphicsDevice.WindowTranslucency.*;

public class NotificationFrame extends javax.swing.JFrame {

    /**
     * Creates new form NotificationFrame
     */
    public NotificationFrame() {
        initComponents();

        this.setLayout(new GridBagLayout());
        this.setUndecorated(true);
        this.setSize(300,200);
        this.setLocationRelativeTo(null);
        this.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        this.setOpacity(0.50f);

    }

    /**
     * This method is called from within the constructor to initialize the form.
     * WARNING: Do NOT modify this code. The content of this method is always
     * regenerated by the Form Editor.
     */
    @SuppressWarnings("unchecked")
    // <editor-fold defaultstate="collapsed" desc="Generated Code">
    private void initComponents() {

        setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE);

        javax.swing.GroupLayout layout = new javax.swing.GroupLayout(getContentPane());
        getContentPane().setLayout(layout);
        layout.setHorizontalGroup(
            layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
            .addGap(0, 400, Short.MAX_VALUE)
        );
        layout.setVerticalGroup(
            layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
            .addGap(0, 300, Short.MAX_VALUE)
        );

        pack();
    }// </editor-fold>

    /**
     * @param args the command line arguments
     */
    public static void main(String args[]) {
        /* Set the Nimbus look and feel */
        //<editor-fold defaultstate="collapsed" desc=" Look and feel setting code (optional) ">
        /* If Nimbus (introduced in Java SE 6) is not available, stay with the default look and feel.
         * For details see http://download.oracle.com/javase/tutorial/uiswing/lookandfeel/plaf.html
         */
        try {
            for (javax.swing.UIManager.LookAndFeelInfo info : javax.swing.UIManager.getInstalledLookAndFeels()) {
                if ("Nimbus".equals(info.getName())) {
                    javax.swing.UIManager.setLookAndFeel(info.getClassName());
                    break;
                }
            }
        } catch (ClassNotFoundException ex) {
            java.util.logging.Logger.getLogger(NotificationFrame.class.getName()).log(java.util.logging.Level.SEVERE, null, ex);
        } catch (InstantiationException ex) {
            java.util.logging.Logger.getLogger(NotificationFrame.class.getName()).log(java.util.logging.Level.SEVERE, null, ex);
        } catch (IllegalAccessException ex) {
            java.util.logging.Logger.getLogger(NotificationFrame.class.getName()).log(java.util.logging.Level.SEVERE, null, ex);
        } catch (javax.swing.UnsupportedLookAndFeelException ex) {
            java.util.logging.Logger.getLogger(NotificationFrame.class.getName()).log(java.util.logging.Level.SEVERE, null, ex);
        }
        //</editor-fold>

        /* Create and display the form */
        java.awt.EventQueue.invokeLater(new Runnable() {
            @Override
            public void run() {
                new NotificationFrame().setVisible(true);
            }
        });
    }

    // Variables declaration - do not modify
    // End of variables declaration
}

Comment: +1, pick the pack to the end after `setUndecorate()` method, solved other question too, thanks.
Answer: The error you are getting is calling setUndecorated(true) after the frame is made displayable by calling pack(). Call setUndecorated before pack and it should work.
Comment: Also, `this.setSize(300,200);` *after* the `pack();` pretty much makes the `pack()` useless there.
Answer:
+1 thanks for this thread, your code solved quite correctly undecorated conatiner, everything is about correct ordering of methods (wooolaaa we are back to Java_1.4.2 edges)
but bug still is there, isn't possible to create decorated JFrame with changed Look And Feel
based on proper ordering of methods

import java.awt.*;
import javax.swing.*;

public class TranslucentWindow extends JFrame {

    private static final long serialVersionUID = 1L;

    public TranslucentWindow() {
        super("Test translucent window");
        setLayout(new FlowLayout());
        add(new JButton("test"));
        add(new JCheckBox("test"));
        add(new JRadioButton("test"));
        add(new JProgressBar(0, 100));
        JPanel panel = new JPanel() {
            @Override
            public Dimension getPreferredSize() {
                return new Dimension(400, 300);
            }
            private static final long serialVersionUID = 1L;

            @Override
            protected void paintComponent(Graphics g) {
                super.paintComponent(g);
                g.setColor(Color.red);
                g.fillRect(0, 0, getWidth(), getHeight());
            }
        };
        panel.add(new JLabel("Very long textxxxxxxxxxxxxxxxxxxxxx "));
        add(panel);
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        setUndecorated(true);
        pack();
        setLocationRelativeTo(null);
        setOpacity(0.70f);
    }

    public static void main(String[] args) {
        try {
            for (UIManager.LookAndFeelInfo info : UIManager.getInstalledLookAndFeels()) {
                if ("Nimbus".equals(info.getName())) {
                    UIManager.setLookAndFeel(info.getClassName());
                    break;
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        //JFrame.setDefaultLookAndFeelDecorated(true);
        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                Window w = new TranslucentWindow();
                w.setVisible(true);
            }
        });
    }
}