Change default backend for matplotlib in Jupyter Ipython

Question: Right now the default backend for matplotlib is 'module://ipykernel.pylab.backend_inline'
I want to switch that to TkAGG. I edited the matplotlibrc file in
~/anaconda2/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc/
to add
backend      : TkAgg
and It did switch the backend for the python but not in Jupyter.
For now everytime I start a new notebook in Jupyter, I have to do %matplotlib tk, isnt there a nice way to make TkAGG the default backend in Jupyter?
Answer: The question is similar to Automatically run %matplotlib inline in IPython Notebook, except that you want to automatically use TK backend instead of inline backend.
So the idea is to locate you IPython configuration file. See configure IPython. It should be
/.ipython/profile_default/ipython_kernel_config.py

If it doesn't yet exist, create it via > ipython profile create.
Inside this file locate the setting c.InteractiveShellApp.matplotlib and set it to "tk". It should then look like
## Configure matplotlib for interactive use with the default matplotlib backend.
c.InteractiveShellApp.matplotlib = "tk"

Save the file and restart the kernel.
Comment: this did help. thanks. i went through similar questions but somehow missed this one.


How can the expectation of a martingale be a random variable

Question: The definition of a martingale is
$\mathbb{E}[X_{n+1} | X_n,...,X_1] = X_n$
Why does it work that the RHS is a random variable. How can we expect to have a variable which random and not a point like $0.25$.?
Answer: The left hand side is the conditional expectation of $X_{n+1}$ on the $\sigma$-algebra generated by the random variables $X_1, \dots, X_n$; it's not just the expectation of $X_{n+1}$.
The definition of $Y = \mathbb{E}[X_{n+1} | X_1, \dots, X_n]$ is that it's unique (up to a.e.) $\sigma(X_1, \dots, X_n)$- measurable random variable such that if $A \in \sigma(X_1, \dots, X_n)$ then $\mathbb{E}[Y 1_A] = \mathbb{E}[X_{n+1} 1_A]$. This means that you don't have that $\mathbb{E}[X_{n+1}|X_1, \dots, X_n]$ is a constant in general by the definition.
Answer: For example, let $\{X_n\}$ be unbiased random walk on $\mathbb Z$. Then defining $S_n=\sum_{k=1}^n $ we have
\begin{align}
\mathbb E[S_{n+1}\mid X_n] &= \mathbb E(S_n+X_{n+1}\mid X_n\}\\ &=  \mathbb E[S_n\mid X_n] + \mathbb E[X_{n+1}\mid X_n]\\ &= S_n + \mathbb E[X_{n+1}]\\&=S_n.\end{align} It is clear that $\mathbb E[S_{n+1}\mid S_n]$ is a random variable taking different values for different values of $\{S_n=k\}$.
Answer: Here is the context. Consider a p-space $(\Omega, \mathcal{F}, \mathbb{P})$ and a $\sigma$-sub algebra $\mathcal{G} \subseteq \mathcal{F}$. Let $X: \Omega \to \mathbb{R}$ be an integrable random variable on this p-space.
Then we call a random variable $Z: \Omega \to \mathbb{R}$ a version of the conditional expectation of $X$, given the sigma-algebra $\mathcal{G}$, if
(1) $Z$ is $\mathcal{G}$-measurable
(2) $\forall G \in \mathcal{G}: \mathbb{E}[XI_G]= \mathbb{E}[ZI_G]$
Showing that this exists might take some work (can be proven using Hilbert space techniques or Radon-Nikodym's theorem). Such a variable is also unique a.e.
We denote such a variable with $Z:=\mathbb{E}[X|\mathcal{G}]$
In your situation, $\mathcal{G}= \sigma(X_1, \dots, X_n)$, the smallest $\sigma$-algebra that makes the maps $X_1, \dots, X_n$ measurable.
One simplifies notation and defines
$$\mathbb{E}[X_{n+1}|X_1, \dots, X_n]:= \mathbb{E}[X_{n+1}|\sigma(X_1, \dots, X_n)]$$


Showing $\tan^{-1}(x)$ is Lipschitz

Question:
I am trying to show  $\tan^{-1}(x)$ is lipschitz for all $x\in\mathbb{R}$.

My attempt:
If $f$ is Lipshitz on $\mathbb{R}$, then $\exists L\in\mathbb{R}$ such that $\forall x,y\in\mathbb{R}$ $$|f(x)-f(y)|\leq L|x-y|.$$
Now consider the Mean Value Theorem ($f$ is continuous and differentiable on $\mathbb{R}$) $$\exists c\in\mathbb{R} \ \text{such that} \ \frac{|f(x)-f(y)|}{|x-y|}=|f'(c)|$$
Using the Mean Value Theorem we can see that, \begin{align}
L&\geq\frac{|f(x)-f(y)|}{|x-y|} \\
&=|f'(c)| \\
&=|f'(0)| \\
&=1
\end{align}
I have a two questions:

what determines the value of $c$?
what if $x=y$? Doesn't this not fulfill the inital condition that $f$ is Lipschitz $\forall x,y\in\mathbb{R}$?
Comment: Note that the MVT shows that if $f$ is differentiable and the derivative is bounded then it is Lipschitz.
Comment: I can't recall seeing it explicitly for $\mathbb{R} \to \mathbb{R}$ functions, but the MVT (or rather the mean value inequality, see same at the end of section https://en.wikipedia.org/wiki/Mean_value_theorem#Mean_value_theorem_for_vector-valued_functions) for $\mathbb{R}^m \to \mathbb{R}^n$ functions usually has it explicitly stated.
Comment: The $c$ comes from the MVT, but it is not necessarily unique. In this case you have $\arctan' x= {1 \over 1+x^2}$, so $|\arctan' x| <1$ for all $x$. The $x=y$ case doesn't matter, it is vacuously true.
Comment: That is a nice result. Do you know where I can read more on this?
Answer: The present question may be resolved with the aid of the principle that a differentiable function is locally Lipschitz, and in fact globally Lipschitz if the derivative is globally bounded.  To see this, let
$f \in \mathcal C^1(I, \Bbb R), \tag 1$
where $I \subset \Bbb R$ is an open interval.  The then for $x, y \in I$, we have
$f(y) - f(x) = \displaystyle \int_x^y f'(s) \; ds \tag 2$
by the fundamental theorem of calculus; thus if $y \ge x$,
$\vert f(y) - f(x) \vert = \left \vert \displaystyle \int_x^y f'(s) \; ds \right \vert \le \displaystyle  \int_x^y \vert f'(s) \vert \; ds \le M \vert y - x \vert, \tag 3$
provided
$\vert f(s) \vert \le M, \; s \in I; \tag 4$
if $x \ge y$ we obtain the same result by reversing the roles of $x$ and $y$ in (3); since we eventually take absolute values, in the end the result is independent of the relative order of $x$ and $y$; we therefore see that under such circumstances $f$ is Lipchitz continuous on $I$, with Lipschitz constant $M$; and if $I = \Bbb R$, $f$ is globally Lipschitz on all of $\Bbb R$.
We apply this observation to the question at hand:
An important and helpful aspect of solving this problem is keeping careful track of the domain and range of $\tan^{-1}x$; recall that, taking
$\tan: \left ( -\dfrac{\pi}{2}, \dfrac{\pi}{2} \right ) \to \Bbb R, \tag 5$
we have
$\tan^{-1}: \Bbb R \to  \left ( -\dfrac{\pi}{2}, \dfrac{\pi}{2} \right ); \tag 6$
since $\tan^{-1}$ is defined on all of $\Bbb R$, so is its derivative; we may estimate $(\tan^{-1} x)'$ as follows:
$y = \tan^{-1} x; \tag 7$
$x = \tan y; \tag 8$
$\dfrac{dx}{dy} = \sec^2 y = 1 + \tan^2 y; \tag 9$
$\left  \vert \dfrac{dy}{dx} \right \vert = \dfrac{1}{1 + \tan^2 y} =  \dfrac{1}{1 + x^2} \le 1, \; \forall x \in \Bbb R; \tag {10}$
in the light of (10), we see that in fact $\tan^{-1}x$ is globally Lipschitz on $\Bbb R$, with Lipschitz constant $1$, by virtue of the principle proved in the above.
As far as our OP Bell's two specific questions are concerned, it should be observed that $c$ will in fact depend on $x$ and $y$ if the mean value theorem is invoked, which we can't do directly in the case $x = y$; this is why I prefer the above argument based on the fundamental theorem.
Comment: There's no problem in using the Mean Value Theorem. I don't see what your complaint is in the last paragraph there.
Comment: If $x=y$, the mean value theorem gives $f(x)-f(x) = f'(x)(x-x).$ Not very useful, but it is true. Remember that one does not generally state the MVT as a quotient! Anyways it's not that important
Comment: @Shalop:  i.)  I didn't complain, I offerred my comment as per OP'ss request; ii.)  what happens with MVT if $x = y$? iii.) if you feel the MVT is the way to go, you should go with it.

Showing $\tan^{-1}(x)$ is Lipschitz


How to limit cursor movement in WPF based app?

Question: I'm working with a WPF app, more specifically a Canvas with draggable elements.
Once an item is being dragged, I'd like to limit the scope of cursor movement to inside the canvas where the items are being dragged about.
The event which can start a drag is shown below
<code>
private void WidgetCanvas_PreviewHeaderLeftMouseDown(object sender, MouseButtonEventArgs e)
{
     e.Handled = true;
     ... logic to figure out if this is a valid drag, blah blah blah ...

     this.IsDragging = true;
     // TODO: clip the available cursor movement to actual width of Canvas
}
</code>
On the Preview-MouseUp, I'd like to simply "unclip" the cursor movement back to it's normal state.
I'll be monitoring the movement of the mouse once I start dragging (PreviewMouseMove), so worst case, I could manually check the position of the mouse and keep it constrained to the canvas, but that seems a little ugly.
Anyone have a better way to limit the cursor boundaries?
Answer: There's no clean way to do this and the not-so-clean ways will make your mouse cursor "jitter" at the border of the clipping area.
Moreover, I'd question if this is really a good idea. The user should really own the mouse and he or she generally gets frustrated when you try to artificially limit things that he or she owns.
If you want to provide feedback when the mouse leaves your canvas, maybe you could leave the item being dragged stuck along the border while the mouse button is still down? This would tell the user that he or she has left the target area without trying to impose limitations on where the mouse can go.
Good luck!
Comment: After reading your comments, and few more on MSDN that echoed the same sentiment, I ended up doing just this. I monitor the items for an event which will start the dragging (as dragging only starts when the use mouse downs on the header of the items). Once dragging starts, I capture the mouse on the canvas, and allow the item to be dragged to the end of the canvas but no further. Positioning of the items can change even if the mouse is not on the canvas, as long as the there is no MouseUp.
Comment: This sounds like a good solution. Thanks for posting an update... it's nice to see what worked out for you.
Answer: You should be able to do it using the ClipCursor native API.


Building Abstract Syntax Tree

Question: I have my parse tree with me, now I have traversed in order on the parse tree and a Symbol table is generated too.
But how do I build the AST for this?
Here is what I have gathered (some unreliable resources):

In your parse tree you go inOrder I.e
Take the leftmost child, The parent then, and then the list of other children.
If you come across anything like SEMICOL, Parenthesis, do not add it to AST, if there is only one child to a node, remove that node, and use child instead.

Is this all I need to do?
Comment: Your system should not have any semantic information at this point.  You have just parsed it.
Comment: It is impressive your symbol table is generated since you can't know scope at this point.
Comment: @Hogan I have the parse tree, what will stop me from knowing the scope?
Comment: @Kraken - how do your parse tree and AST differ? They can be pretty much the same thing. Personally, I've always treated the parse tree as if it was the AST, though I'm no expert - I've only written a few unreleased domain-specific languages. That said, translating from one tree form to another is really just a combination of traversing the old and creating the new - if you can create a parse tree while parsing, you shouldn't have too much trouble generating a new tree while traversing the old one. You even get to choose the traversal order and limits to suit what you're generating.
Comment: @Hogan from my parse tree, after traversing it, i looked at all the declared IDs and added their info to the table.
Comment: @Steve314 - You are right for simple languages.  But then I believe based on the question Kraken is not working on a simple language or is taking a compiler course.
Comment: @Kraken - and how did you "look at" the declared IDs?  If for example you looked for elements in your parse tree that were of the form (var) -> (name), then you were performing lexical analysis and not parsing.
Comment: @Steve314 i have this as the project in my compiler course, here is what the problem statement asks:  The parse tree is fed to the module, all the unnecessary info like semicolon, open brackets etc are removed, the ast contains nodes only that are essential for the semantic analysis phase.
Comment: @Hogan from my grammar i know the productions that will help me declaring the IDs, now whenever i came across the non terminal that corresponds to the non terminal on the LHS of the production rule, i extracted the info related to ID from there itself.
Comment: @Steve314 why would i delay adding the scope info for later and not do it after building the parse tree itself. I know where in the parse tree my ID will lie.
Comment: @Hogan also, is my traversal pattern correct, that is firstly go to the leftmost child, then the parent, then the list of other child, considering my children of a node are maintained as a list.
Thanks.
Comment: I can't answer that without seeing what is contained in the parent and children.
Comment: @Kraken - that comment only existed for about three seconds before I deleted it! You might delay adding scope and other information to the symbol table because it's not obvious from the initial form of the parse tree - or because the parse tree isn't sufficiently built to provide that information yet. For example, LR parsing tends to build the tree bottom-up, so the node for the scope of a declaration may not exist when the declaration itself is recognised - you can't just step up the tree to look because those parts of the tree don't exist yet. Also "after" is delayed relative to "during".
Comment: And here I was thinking that an AST is also called a 'parse tree'.
Answer: Update based on comment
In your example the parse tree might contain a statement node like this
(int) -> (ageArray) -> ([) -> (30) -> (]) -> (;)
The professor expects something like
(intArray) -> (ageArray) -> (30)
I guess.

You are making the solution to simple.  It is not simply a matter of traversing your tree and throwing some stuff out. ASTs represent the constructs of the program language.  In order to build an AST you first have to design what the AST is going to look like.
For example, an "IF" node might be designed as having a boolean equation element (this in itself is probably a tree too), a true statement tree and a false statement tree.
You have to design what all the elements are going to look like based on the language you are compiling.  Once you have that design then often a lexer is used on the parse tree to generate the AST.
Comment: yeah my Production for declaration is like
Declarstmt - > DECLARE listid : type ;

So my parent node will be Declarestmt
Children List has DECLARE - listid - : -type : ;
Comment: Ah, you didn't say you were using a compiler generator.  In that case, in-order and drop un-needed elements like you said.
Comment: what made you say that i am using compiler generator, well,here is what i did.
Lexial Analysis, using DFA ,
Feed the token stream to parser, now the parser module, will look at the grammar rules, and then from there using FIRST FOLLOW SETS and the LL predictive parsing technique, i built the parser. ( using stack too).
Is that what you call compiler generator?
Comment: Yes, that is what compiler generators do.  When I've done that I don't need another step, the output of the parser is already an AST without any additional junk.
Comment: then the traversal will be in the same order as i asked earlier right? If you can answer it now. :)
Thanks.

Building Abstract Syntax Tree


prove a function is continuously differentiable

Question: $f(x,y) =\begin{cases}\arctan(y/x) & x\neq 0\\ \pi/2 & x=0,y>0\\-\pi/2 & x=0,y<0.\end{cases}$
$f$ is defined on $\Bbb R^2\smallsetminus\{(0,0)\}.$
Show that $f$ is continuously differentiable on all of its domain.
Also use implicit function to show the above proof again.
Thanks!
Comment: You're repeating *exactly* your question from 3 hours ago. You must be patient and wait until somebody deals with that, and not send over and over the same question.
Comment: I have no idea what your inequalities and bounds for $x$ and $y$ represent. Please fix those yourself.
Comment: So...are you dividing by $0$ in there? That's...bad.
Comment: that's a function with different values in different domains
Comment: Ah, I see. Please tell me if my interpretation is right.
Answer: Maybe rewriting your equation as
$$ x \tan f = y$$ does help?
Edit:
Given the fact that the first hint did not help. Here, is the second hint: you can rewrite your equation as
$$ F(x,y,f) = x \tan f - y =0.$$
Can you then use the implicit function theorem to learn something about $\partial_x f$ and $\partial_y f$?
Edit2:
I just did see that you have changed your question and thus the points with $f=\pi/2 + n\pi$ are not excluded any more. In this case you should rewrite your equation as (check the special points at $f=\pi/2 + n\pi$ separately)
$$F(x,y,f) = x \sin f - y \cos f =0.$$
and then apply the implicit function theorem.
Comment: doesn't help at all
Comment: could you give me more hint? how can I apply implicit function? Is there any conclusion of implicit function about continuously differentiable?
Comment: Did you take a look at the theorem on [wikipedia](http://en.wikipedia.org/wiki/Implicit_function_theorem#Statement_of_the_theorem). If $F$ is continuously differentiable (which should be obvious) and the Jacobian $\partial F/\partial f$ is invertible (which is just a number for us) then $f(x,y)$ is continuously differentiable.
Comment: never say something is obvious. when you say something is obvious, it could be not obvious
Comment: the part you cited as obvious is exactly the hard part
Comment: @Frank: I'm a bit confused:  I believe you know that $\sin f$ and $\cos f$ are in $C^\infty$? I truly don't understand to which subtlety you are hinting to...
Comment: but you have x and y here. you have to prove it's ctsly diff when x= 0
Comment: @FrankXu: $F(x=0,y,f)= - y \cos f$ I'm still not sure I understand your remark.
Comment: didn't you see x = 0 is a breaking points?
Comment: @FrankXu: no I do not see how the point $x=0$ is any special point for the function $F(x,y,f) = x \sin f - y \cos f $. I'm not sure if this discussion makes any sense so far you were not given any real indication what you are worrying about.
Comment: Because at x = 0, you have to prove F is continuously differentiable at that point.
Comment: @FrankXu: I thought discussed this already. I assume you know that the product/sum of two continuously differentiable functions is continuously differentiable. You agreed to know that $\sin f$ and $\cos f$ are continuously differentiable. I did not ask you if you know that $x$ and $y$ are continuously differentiable (because I thought you know it). Is you question why $x$ (or for that matter $y$) is a continuously differentiable function? Because as soon as this is clear you should agree that $F(x,y,f) = x \sin f - y \cos f$ is continuously differentiable in all its variables.


How can I calculate the length of intersection on a 2d line?

Question: The following diagram explains my question:
$\text{Line}A = [2,7]$
$\text{Line}B = [5,9]$
$X =$ The length of the intersection between LineA and LineB

Thanks
Update: I ended up using this algorithm
https://en.wikipedia.org/wiki/Marzullo%27s_algorithm
Comment: Note that the length is equal to $A\cap B$ if $A$ and $B$ are the sets of points in your line segments (the range of the empty set is $0$).
Comment: Can't you just see from the picture that the length is 2?
Answer: It is $$\ell ([A_1,A_2],[B_1,B_2])=\begin{cases}
\min \{A_2,B_2\}-\max \{B_1,A_1\} & \text{If }\min \{A_2,B_2\}\geq\max \{B_1,A_1\} \\
0 & \text{Otherwise}
\end{cases}$$
In your example $7-5=2.$
Comment: Why +1? From the pic the length is 2.
Comment: @ShubhamJohri Cause I have been in the discrete math section too long. I was thinking in cardinal. Thanks!


How can I apply list of functions consecutively to a variable?

Question: I have a list of functions I want to apply to a string consecutively, changing the string.  For example, a list of regular expressions I want to remove from a string, e.g.
to_remove = c('a','b')
original = 'abcabd'

In this case, I could use a simple regular expression, e.g. 'a|b'.
library(stringr)
str_remove( original, paste0( to_remove, collapse='|'))

The actual situation is more complex than this, and the regular expression gets a little hairy. Also, I am curious how to do this in a proper R way.
Another way to phrase the question is, "How can I implement the following for loop using a vector approach?"

for( rem in to_remove) {
      original = str_remove( original, rem )
}

'
Answer: stringi offers vectorized string replacement. The replacement argument can take on a vector of length 1, "", or a vector of the same length.
to_remove = c('a','b')
original = 'abcabd'

stringi::stri_replace_all_fixed(
    str = original,
    pattern = to_remove,
    replacement = "",
    vectorize_all = F
)
#> [1] "cd"

For repeated application of patterns, perhaps use an alternative pattern, per Stéphane's comment, replace_all_* with vectorize = FALSE is recursive, see for example the output of
stri_replace_all_regex(
    "abc",
    pattern = c("^a", "^b"),
    replacement = "",
    vectorise_all = FALSE
)
#> [1] "c"

Comment: With `vectorize_all=FALSE`, then `stri_replace_all_*` is recursive. Try e.g. `stri_replace_all_regex("abc", pattern = c("^a", "^b"), replacement = "", vectorise_all = FALSE)`. This removes `a` at the first position, then `b` at the first position.
Comment: @StéphaneLaurent You are right - thanks for the heads up, I have revised the answer
Answer: I don't think that str_remove_all can take a list of inputs, but you can just use str_replace_all with a named vector to circumvent the issue:
str_replace_all("abcdabcd", c("^a" = "",
                              "b" = "",
                              "c" = "",
                              "d$" = ""))

[1] "da"

This solution works with regular expressions as well. The str_remove functions are just an alias for str_replace(x, ""), so there is no loss in computational speed.
Alternatively, you can set the names of an empty vector as the regular expressions as such:
to_remove <- c('a','b', )
empty <- rep("", length(to_remove))
names(empty) <- to_remove

str_replace_all("abcd", empty)

Or you can sequentially pipe the expression into a sequence of str_remove calls as so:
"abcd" %>%
  str_remove_all("d$") %>%
  str_remove_all("a")


In what sense is one latent causal structure "preferred to" another? Definition 2.3.3 from Causality by Pearl

Question: In Causality - Models, Reasoning, And Inference by Pearl, definition 2.3.3 reads as follows -

One latent structure $L$ = $\langle D,O \rangle$ is preferred to
another $L^{'}$ = $\langle D^{'},O \rangle$ (written $L \preceq L^{'}$)
if and only if $D^{'}$ can mimic $D$ over $O$ - that is, if
and only if for every $\Theta_{D}$ there exists a $\Theta^{'}_{D^{'}}$
such that $P_{[O]}(\langle D^{'},\Theta^{'}_{D^{'}}\rangle)=P_{[O]}(\langle D,\Theta_{D} \rangle)$.
Two latent structures
are equivalent, written $L^{'} \equiv L$ if and only if $L \preceq L^{'}$
and $L \succeq L^{'}$.

In what sense is $L$ preferred over $L^{'}$? Is it because the probability distributions satisfied by $L$ are a subset of those satisfied by $L^{'}$?
In particular, please refer to the below example Pearl cites later -

For the independencies $(a \perp b)$, $(d \perp \{a,b\} \ | \ c)$, figure (a) captures all of them. Figure (c) does not necessitate $(a \perp b)$.
Does it mean that figure (c) can be made to fit not only all probabilities where $(a \perp b)$ holds, but also others where it does not hold? Is figure (a) preferred over (c) in this sense, given the independencies $(a \perp b)$, $(d \perp \{a,b\} \ | \ c)$?
Note: I am familiar with minimal I-maps, perfect maps etc., from Probabilistic Graphical Models by Koller. If necessary, one can answer in those terms. I have a hunch that figure (a) is a perfect map while (c) is a minimal I-map for the independencies cited; that might be the reason for preferring (a) over (c). Am I correct?
Answer: Definition 2.3.3 is in essence a statement about "excess edges" or "excess dependencies"
Let us for a second assume there are no hidden variables. Then, a structure $L'$ that can, with the right parametrization $\Theta'_{D'}$, mimic all probability distributions of an alternative structure $L$ (such that $P_{[O]}(\langle D', \Theta'_{D'}\rangle) = P_{[O]}(\langle D, \Theta_{D}\rangle)$, must contain all edges present in $L$ and possibly more (not necessarily the same direction). In this case, the structure with fewer or equal edges $L$ is preferred.
So if we denote the number of edges in $L$ as $|D|$, we could read Definition 2.3.3 as saying that we prefer $L$ over $L'$ if $|D| \leq |D'|$. We say $L \equiv L'$ if $|D| = |D'|$.
This isn't quite true when there are hidden variables (more on that below), but I think it gives good intuition as to why we would want to have this definition.
In terms of Occams razor we prefer the model with the fewest assumption (fewest edges) among models that fit the data equally well.
Figure 2.1
Figure 2.1 illustrates the concept by showing a minimal model a), another minimal model b) including a potential hidden variable, and a model with an excess edge c).
The model in a) appears to be a graph that fits all distributions $P_{[O]}$ without any unnecessary dependencies between observed variables.
The model in b) is basically the same model as a), except that a hidden variable is assumed to explain the dependence $a \to c$. Practically speaking, such a model is not very useful because a hidden variable may explain all distributions and can never be ruled out (see first paragraph in 2.3). So although the number-of-edges reasoning does not apply here directly because we have more variables to begin with, we could re-phrase the same idea using the number of dependence statements between observed variables.
In this sense, b) is equivalent to a).
The model in c) contains an "excess edge" in $a \to b$. The distribution $P_{[O]}$ can be explained without it as seen in a).


Android throws error everytime when "Camera camera = Camera.open();" is called

Question: I am having a problem using the camera in Android.
Developing on API 25 Nougat 7.1.1 SDK. Min target SDK is set to 15.
It throws error everytime upon calling:
Camera camera = Camera.open();

with the error "RuntimeException: Fail to connect to camera service", which can be seen from my emulator (Nexus 5X API 25 Android 7.1.1)
Testing on a REAL DEVICE (Android 5.1.1) the camera features does not work either - exactly the same problem.
Here is the code:
import android.hardware.Camera;

...

boolean hasCamera = false;

private boolean hasCamera(Context context) {
        return context.getPackageManager().hasSystemFeature(PackageManager.FEATURE_CAMERA);
    }

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    ...

    hasCamera = hasCamera(this);
    if(!hasCamera) {
        takePhotoButton.setText("No Camera Found");
        takePhotoButton.setEnabled(false);
    }
}

private class TakePhotoButtonListener implements View.OnClickListener {
    @Override
    public void onClick(View v) {
        if(hasCamera) {
            try {
                Camera camera = Camera.open();
                camera.startPreview();
                camera.takePicture(
                        null,
                        null,
                        new CameraPictureCallbackJPEG());
            } catch (Exception e) {
                Log.i("Error", e.getMessage());
            }
        }
    }
}

The camara is indeed detected (hasCamera == true).
I'm not using the Camera2 (from API 21) onwards because I'd like my app to be work on API 15 onwards.
I'e tried many answers here but nothing worked. Please advise!
Comment: http://stackoverflow.com/questions/23904459/android-java-lang-runtimeexception-fail-to-connect-to-camera-service
Comment: Permission problem?
Comment: Try rebooting the device. I usually see this when some app acquired the camera and did not release it.
Comment: Could you post a stack trace of the error that will help better in figuring out a solution to your issue.
Comment: Possible duplicate of [android java lang runtimeexception fail to connect to camera service](http://stackoverflow.com/questions/23904459/android-java-lang-runtimeexception-fail-to-connect-to-camera-service)
Answer: Please add following permissions to your Manifest file, if these are not added.
<uses-permission android:name="android.permission.CAMERA"/>
<uses-feature android:name="android.hardware.camera" />

Important note:
This is happening because you've opened Camera at first launch and after that you haven't release it's resources.
make sure you should do this
camera.stopPreview();
camera.release();
camera = null;

You have to do above process of cleaning and releasing resources when you are done with using camera.
Hope this helps.


Center image view on rotate

Question: I have an image view with an image and it looks find when the phone is portrait. When I turn it landscape, it rotates, however, the image isn't centered anymore. How do I go about doing that?
Answer: Create a layout-land directory in your res folder. And copy your layout files with landscape orientation. That should do it.
Answer: Rather than making a new folder and xml file and all that, I simply did
android:layout_width="match_parent"

on my image view and when I rotate the device, the image stays centered.

Center image view on rotate


Passing multiple parameter in app-engine Task Queue (JAVA)

Question: Is there a way to pass multiple parameters in a Queue in google-app-engine?
I use the code below
Queue queue = QueueFactory.getQueue("sms-queue");
queue.add(TaskOptions.Builder.url("/SQ").param("id",pId));

in my servlet this id is retrievd as a query string.
long pID = Long.parseLong(req.getParameter("id"));

I need to pass 6 parameters.
Answer: Have you tried doing this:
queue.add(TaskOptions.Builder
    .url("/SQ")
    .param("p1Name", p1Value)
    .param("p2Name", p2Value)
    .param("p3Name", p3Value)
    // etc
);


Persistence value is not correct in the code

Question: I am working on a program in C which is supposed to dynamically allocate memory to a 2d array of [x] rows and [y] columns with the help of malloc(). The things I was supposed to to do for this program are :

Create space to hold [r] x [c] integers.
Use a nested for loop to fill the array space.
Each cell needs to be assigned with the number i + j
You need to keep track of your own offset by i*c + j within a single subscript.
Walk the entire array and add up each element to form the sum.

This is how I have approached the problem
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char const *argv[])
{
  int r;
  int c;
  int sum = 0;
  int i;
  int j;
  int *A;

  printf("Enter number of rows : \n");
  scanf("%d", &r);
  printf("Enter number of columns : \n");
  scanf("%d", &c);

  A = (int *)malloc(sizeof(int) * r * c);

  printf("Enter the array elements : ");
  for (i = 0; i < r; i++)
  {
      for (j = 0; j < c; j++)
      {
          scanf("%d", &(*(A + i * c + j)));
          sum += (*(A + i * c + j));
      }
  }
  printf("The sum is : %d", sum);
  return 0;
}

As far as the code is, it works properly when compiled by allocating 2 bytes per integer for each slot in the integer array, but when I submit this same code to Bottlenose (Web based assignment submission for CS based courses), I am given these two prompts
not ok 1 - Test 1 FAILED: Did not print sum properly.
not ok 2 - Test 2 FAILED: Did not allocated and free
correct amount of memory with one call to malloc and one call to free.

What exactly am  I doing wrong in my program?
Comment: Don't the assignment instructions say that the value in each cell has to be calculated (sum of row number plus column number)?  Not the result of `scanf()`...  You should probably add a newline at the end of the final `printf()`.  And, were it me, I'd write `&A[i * c + j]` in preference to `&(*(A + i * c + j)))`.  It's a lot simpler to read and harder to get wrong.
Comment: "Did not allocated and free". That one is likely because you did not call `free`. For the first one, you have not given us the exact requirements so we don't know whether the requirement actually needs you to print `"The sum is"` part of the output or just the integer sum value.
Comment: There is no 2D array (heck, is this "let's call everything a 2D array" day?)
Comment: Persistence normally refers to values saved on disk (in a database, in a file, or something similar).  Your question is not self-evidently about persistence.
Comment: @kaylum Yes, the program also has to bring "The sum is" also with the sum of all array values.
Comment: @kaylum I edited the code and used free() function on my pointer *A, which makes my test 2 pass, but test one is still incorrect
Comment: Can you provide the test input that causes the incorrect output?
Comment: @M.M The input which causes the incorrect output is the array values which the user needs to input for addition
Comment: @JonathanLeffler Now that I see it, you are right about the formatting. I'll keep this in mind the next time I format. Thanks for your tip
Answer: As I read your problem statement, it appears you are confusing the directions. Specifically:

Each cell needs to be assigned with the number i + j

You are also told to "keep track of your own offset by i*c + j within a single subscript". Which would imply a simple assignment of the value of i + j to the offset array[i * c + j], which would imply:
array[i * c + j] = i + j;

Further, as indicated in the comments, you are NOT allocating a 2D array, you are allocating a block of memory of sufficient SIZE to hold r * c integer values. Whether you index values in that block sequentially (as a simple array) or index them as a simulated 2D array is entirely up to you, but the underlying block of memory is simply an allocated block of r * c * sizeof *array bytes.
Putting those pieces together, you could do something similar to the following:
#include <stdio.h>
#include <stdlib.h>

int main (void) {   /* you don't use any argument, use void */

    int *array, r, c, sum = 0;  /* declare, initialize variables */

    printf ("enter number of rows: ");
    if (scanf ("%d", &r) != 1) {    /* alway validate ALL input */
        fprintf (stderr, "error: invalid row value.\n");
        return 1;
    }

    printf ("enter number of cols: ");
    if (scanf ("%d", &c) != 1) {
        fprintf (stderr, "error: invalid column value.\n");
        return 1;
    }

    /* allocate & VALIDATE -- often useful to initialize to zero with calloc */
    if (!(array = calloc (r * c, sizeof *array))) {
        fprintf (stderr, "error: virtual memory exhausted.\n");
        return 1;
    }

    for (int i = 0; i < r; i++)          /* nested loops */
        for (int j = 0; j < c; j++) {
            array[i * c + j] = i + j;    /* assign i + j to correct address */
            sum += array[i * c + j];     /* add value to sum */
        }

    printf ("The sum is : %d\n", sum);

    free (array);   /* if you allocate it, track it and free it when done */

    return 0;
}

Example Use/Output
$ ./bin/arrayalloc
enter number of rows: 4
enter number of cols: 4
The sum is : 48

If you print the values in the simulated 2D array, you will get:
$ ./bin/arrayalloc
enter number of rows: 4
enter number of cols: 4
   0   1   2   3
   1   2   3   4
   2   3   4   5
   3   4   5   6
The sum is : 48

Look things over and let me know if you have any further questions. If you are interested in outputting the values as a simulated 2D array using you single subscript index, you can do something similar to:
    for (int i = 0; i < r; i++) {       /* nested loops */
        for (int j = 0; j < c; j++) {
            array[i * c + j] = i + j;   /* assign i + j to correct offset */
            sum += array[i * c + j];    /* add value to sum */
            printf (" %3d", array[i * c + j]);
        }
        putchar ('\n');
    }

Also note the requirement:

Walk the entire array and add up each element to form the sum.

could imply separate nested-initialization and summing loops requiring you to "walk the entire array" using a pointer to sum the values. In that case you could do something similar to:
#include <stdio.h>
#include <stdlib.h>

int main (void) {   /* you don't use any argument, use void */

    int *array, *p, r, c, sum = 0;  /* declare, initialize variables */

    printf ("enter number of rows: ");
    if (scanf ("%d", &r) != 1) {    /* alway validate ALL input */
        fprintf (stderr, "error: invalid row value.\n");
        return 1;
    }

    printf ("enter number of cols: ");
    if (scanf ("%d", &c) != 1) {
        fprintf (stderr, "error: invalid column value.\n");
        return 1;
    }

    /* allocate & VALIDATE */
    if (!(array = calloc (r * c, sizeof *array))) {
        fprintf (stderr, "error: virtual memory exhausted.\n");
        return 1;
    }
    p = array;  /* assign array to pointer */

    for (int i = 0; i < r; i++)         /* nested loops used to initialize */
        for (int j = 0; j < c; j++)
            array[i * c + j] = i + j;   /* assign i + j to correct offset */

    for (int i = 0; i < r * c; i++, p++)  /* separate loop to walk the array */
        sum += *p;                        /* sum values using pointer */

    printf ("The sum is : %d\n", sum);

    free (array);   /* if you allocate it, track it and free it when done */

    return 0;
}

That is more a matter of semantics, but using separate initialization and summing loops and using a pointer to walk the array complies more closely to your instructions.
Comment: May I know as of why would you use **fprintf()** instead of **printf()** to print statement? in your first solution?
Comment: Sure, whenever you are issuing warnings or errors, you want those printed on `stderr` instead of the normal `stdout`. Since `printf` outputs by default to `stdout`, you use `fprintf` to print to `stderr` (or any other file stream). It's really just a matter of doing it right. If you attempt to collect output by redirecting the output of your program, you often do not want warnings, etc.. in the output you capture. (e.g. `./myprog >> my_logfile.txt`) If you use `fprintf` to `stderr` for warnings and errors, only output goes to `my_logfile.txt` and warnings/errors go to the screen.


Preencher List com retorno de sql

Question: bom dia
Como faço para preencher uma Lista com um retorno de um sql?
O que eu tentei fazer com certeza vai me dar um outofrange exception
List<string> list = new List<string>();
string query = "select * from pedidos_distribuidos";
string connectionString = ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
SqlConnection con = new SqlConnection(connectionString);
{
    using (SqlCommand cmd = new SqlCommand(query))
    {
        con.Open();

        cmd.CommandTimeout = 300;
        cmd.CommandType = CommandType.Text;
        cmd.Connection = con;
        SqlDataReader reader = cmd.ExecuteReader();

        while (reader.Read())
        {
            list[0] = reader[0].ToString();
        }
        //using (SqlDataAdapter sda = new SqlDataAdapter())
        //{
        //    cmd.Connection = con;
        //    sda.SelectCommand = cmd;
        //    using (DataTable dt = new DataTable())
        //    {
        //        sda.Fill(dt);
        //        pedidos = sda.A ;
        //    }
        //}
    }
}

O que eu estou fazendo
var list = "";
            string connectionString = ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
            SqlConnection con = new SqlConnection(connectionString);
            {
                list = con.Query<string>("select * from pedidos_distribuidos" ).ToString() ;

            }

            int total = list.Count();
            if (!String.IsNullOrWhiteSpace(parametrosPaginacao.SearchPhrase))
            {
                //  areaClientes = areaClientes.Where("Area.Contains(@0) OR DescricaoGAreaCliente(@0)", parametrosPaginacao.SearchPhrase);
                list = list.Where("Protocolo.Contains(@0)", parametrosPaginacao.SearchPhrase).ToString();
            }

            var pedidosPaginados = list.OrderBy(parametrosPaginacao.CampoOrdenado).Skip((parametrosPaginacao.Current - 1) * parametrosPaginacao.RowCount).Take(parametrosPaginacao.RowCount);

            //int total = 0;
            DadosFiltrados dadosFiltrados = new DadosFiltrados(parametrosPaginacao)
            {
                rows = pedidosPaginados.ToList(),
                total = total
            };
            return dadosFiltrados;

}

Preciso que retorne o resultado do select
Comment: seria só fazer isso: `list.Add(reader[0].ToString());` mas, fica muito mais fácil se usar o dapper como o AR colocou
Comment: Acontece que esse drapper está me retornando dado 12 vezes mais do que o normal @RovannLinhalis
Comment: Como assim? Não entendi
Comment: @RovannLinhalis coloquei lá
Comment: Vamos [continuar esta discussão no chat](https://chat.stackexchange.com/rooms/80450/discussion-between-gabrielfalieri-and-rovann-linhalis).
Comment: Mesmo assim quando eu coloco   list = con.Query("select Protocolo from pedidos_distribuidos" ).ToString(); ele me retorna 48
Comment: Pelo amor de deus gabrielfalieri, você precisa estudar os conceitos da linguagem antes mano.

list = con.Query("select * from pedidos_distribuidos" ).ToString() ;

Você tem mais resultados porque dá um select * from (que deveria retornar um objeto e não string) e ainda me dá um list.ToString...
Comment: @GustavoSantos seja um pouco mais educado por favor?
Comment: Não faltei com o respeito, mas é claro que você não conhece a linguagem e está fazendo algo fora do que você conhece, é uma dica, estudar a linguagem antes.
Answer: Para que reinventar a roda? utilize o Dapper.:
string connectionString = ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
SqlConnection con = new SqlConnection(connectionString);
{
    var list = con.Query<string>("select coluna from pedidos_distribuidos");
}

Comment: Mesquista, havia me esquecido disso, até exclui minha resposta
Comment: Primeiro, quem é esse connection, o connectionString?

Segundo, está dando erro de "string does not contain a definition for query"
Comment: @gabrielfalieri vi que editou a resposta, conseguiu resolver o problema?
Comment: Não, continua dando erro
Comment: você declarou list como uma string.: `var list = "";`
vendo o seu código agora, você tem bem mais problemas do que o retorno da consulta.
Answer: Apenas complementando a resposta, e explicando melhor o problema relatado no chat:
Crie uma classe para seus resultados:
public class PedidosDistribuidos
{
    public string Protocolo {get;set;}
    public DateTime DataProtocolo {get;set;}
    public string Descricao {get;set;}
}

Depois execute a Query, retornando a lista tipada:
List<PedidosDistribuidos> list;

using (SqlConnection con = new SqlConnection(connectionString))
{
    list = con.Query<PedidosDistribuidos>("select protocolo, data_protocolo, descricao from pedidos_distribuidos").ToList();

}

Repare que não tem .ToString no final da execução da query; A lista é declarada antes do bloco using (pois você quer usa-la depois);
Comment: @gabrielfalieri ainda há problemas graves na sua abordagem, você deveria filtrar, paginar e ordenar direto na sua consulta. hoje você está trazendo toda a tabela para a memoria, lhe garanto que esta é uma das maneiras mais praticas para se cometer suicídio.

Preencher List com retorno de sql


IllegalStateException occured when deployed war in tomcat with spring boot 2.0.1

Question: I packaged a war with gradle + boot 2.0.1, gradle script as below
plugins {
    id 'org.springframework.boot' version '2.0.1.RELEASE'
    id 'java'
    id 'war'
}

apply plugin: 'io.spring.dependency-management'

dependencyManagement {
    imports {
        mavenBom org.springframework.boot.gradle.plugin.SpringBootPlugin.BOM_COORDINATES
    }
}

repositories {
    mavenCentral()
}
bootWar{
    mainClassName = 'api.abroad.Application'
}

configurations {
    providedRuntime
}
dependencies {
    compile 'org.springframework.boot:spring-boot-starter-web'
    providedRuntime 'org.springframework.boot:spring-boot-starter-tomcat'
}

The Application class with main function as below
@SpringBootApplication
public class Application {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(Application.class, args);
    }
}

and I also wirted a class extends SpringBootServletInitializer to override configure function, although it was not required in docs.
public class ServletInitializer extends SpringBootServletInitializer{

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
        return application.sources(Application.class);
    }

}

I installed tomcat 8.5.29 in my windows, and deployed the war. An exception occured as below
2018-04-09 14:40:19.978  INFO 11720 --- [ost-startStop-1] com.tbiedu.abroad.ServletInitializer     : Started ServletInitializer in 4.113 seconds (JVM running for 10.228)
2018-04-09 14:40:19.989  INFO 11720 --- [ost-startStop-1] c.t.abroad.controller.DemoController     : Root context already created (using as parent).
09-Apr-2018 14:40:19.992 严重 [localhost-startStop-1] org.apache.catalina.core.ContainerBase.addChildInternal ContainerBase.addChild: start:
 org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/abroad-service-api]]
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:986)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: No SpringApplication sources have been defined. Either override the configure method or add an @Configuration annotation
        at org.springframework.util.Assert.state(Assert.java:73)
        at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:127)
        at org.springframework.boot.web.servlet.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
        at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:172)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5204)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        ... 10 more

Did i missed something...
Comment: I am not sure what "docs" you're referring to but the section on `war` file has [an explicit section about it](https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-create-a-deployable-war-file). It looks like you're reusing what start.spring.io does when you request the generation of a war project. That war project works fine for me so perhaps you can share your project somewhere?
Comment: Thank you metioned "reusing", so I recheck my code and found another "extends SpringBootServletInitializer" on a controller class. It's working fine after i removed the repeated one.
Answer: you need to place @Configuration above this class
@Configuration//--> this line u need to place
public class ServletInitializer extends SpringBootServletInitializer{
@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
    return application.sources(Application.class);
}
}


Asking a "better" question

Question: If there's a question that we have substantial disagreement in terms of whether or not its premise is valid and whether or not it meets the criteria of being answerable, but we cannot edit it without changing it dramatically, what should we do?
Example:

What is the evidence that econometrics has empirical value?
Economies are extremely complex systems with many variables, not to mention the fact that they emerge from the interactions of complex beings. I agree that economies have certain underlying principles, but I remain skeptical of the overall value of econometrics as a science.
A truly useful econometrics would be a valuable tool in predicting the future behavior of the economy, particularly in predicting shocks like the recent financial crisis and Great Recession. But it did not happen.
If econometrics can't predict the future, how do we know it even effectively describes the past? What is the evidence that it has empirical value?

to me should be best rewritten as a series of questions along the lines of:

What are some pieces of empirical evidence that support econometric's predictive value?
Relating to macroeconomics:
Given that economies are extremely complex systems with many variables, not to mention the fact that they emerge from the interactions of complex beings, what predictions can be made by econometric systems, and how can they be tested?
Are there any meta-studies that catalog historical predictions and accuracies under differente econometric analyses?

With each subsequent question on a different discipline or subfield.
Answer: Ask a new question.
The intent of the original question should be preserved, and it can be downvoted or closed accordingly.
Answer: Vote to close it.
You should not edit a question to change the OP's intent.  Please do read the guidelines if you are at all unsure about that. Write your own question if you have a different intent to the OP.
The editing guidelines say explicitly:

clarify the meaning of the post (without changing that meaning)

Putting an unsuitable question on hold, which is what voting to close does, gives the OP chance to edit the question into shape. That way, askers can learn how to ask better questions.
At the same time, putting it on hold helps answerers by delaying the opportunity to answer, so they don't answer a bad question; this avoids situations where we get an answer to the bad question which then doesn't fit the edited and reopened question.
Use comments to get clarification from the OP, and to guide them to editing a better answer: or at least, making their intent more explicit in their comments, so that the question can then be edited by anyone to incorporate the OP's intent as expressed in their comments. If the comments go on too long, then switch over to chat to discuss the question and get clarification.
Answer: Edit the original question.
It's more important to contribute to the discussion and limit duplication than to preserve the intent of the OP.

Asking a "better" question


How to make vertically scrollable text area?

Question: I have a textarea html element, which may contain URLs. User should be able to click on such URLs to open them. Currently the code looks like below:
<textarea rows="26" class="form-control" disabled> {{ text|urlize }}</textarea>

Of course, links are not supported there currently. Looks like the textarea should be replaced with something else. I've tried to use div instead, but I am not sure how to restrict its height (for textarea I used rows="26") and make it scrollable.
Comment: Can't understand your question, what do you mean by clicking a link from a text area ?
Comment: @AbdelrahmanWahdan, looks at this fiddlet - http://jsfiddle.net/and7ey/o4zLap5r/ - you can not just click on the example.com to open it.
Answer: please check this code
html

CSS
.textarea{
  width:50%;
  background-color:#eee;
  overflow:hidden;
  height:250px;
  max-height:250px;
  border:1px solid #ccc;
  overflow-y: auto;
}

http://codepen.io/anon/pen/KpYOKq


Classification of $A^x=B$ vs $xA^x=B$

Question: Given these two types of a single-variable equation (with $x$ denoting the variable):

$A^x=B$, which can be solved via logarithm
$xA^x=B$, which can be solved using Lambert W function

Is there a standard terminology which classifies these types under two different categories?
I'm basically asking - what is the standard class name for each one of these equation types?
Answer: Both equations are transcendental equations.
Let $H$ a function of one variable, $x$ a variable and $c$ a constant. For naming the type of an equation $H(x)=c$, the name of the type of the function $H$ is used.
$A^x=B$ is an exponential equation.
An exponential equation is a kind of exponential polynomial equations.
$xA^x=B$ is an exponential polynomial equation.
Only some simple exponential polynomial equations can be solved in closed form.

Classification of $A^x=B$ vs $xA^x=B$


How many $6$-digit positive integers are there in which the sum of the digits is at most $51$?

Question: How many $6$-digit positive integers are there in which the sum of the digits is at most $51$?
Answer: Hint:  First find the number of all 6-digit integers.  Then count the ones that do not satisfy the requirements, that is, those with digit sum of at least 52.
Comment: Alternatively, you could also find all ways that six digits could sum up to at least 52 and work on permutations of those.
Comment: @Frank: Given that $6\times 9 = 54$, herpderp's answer is likely to be quicker, as well as reducing the complications of leading $0$s
Comment: @Henry Definitely possible.  I just thought I'd give an alternative solution in case it helped someone.
Comment: alright thx guys
Answer: $$6.9 > 51$$
$$5.9+8 > 51$$
$$4.9+8.9 =52 >51$$
$$3.9+3.8 < 51$$
Now just the permutation part. You can use maximum of three $9$'s in your 6-digit number(Only constrain).


some questions about open affine cover

Question: Suppose $X$ and $Y$ are separated noetherian schemes. Let $f: X\to Y$ be a morphism between them. If $\mathcal{U}=(U_{i})$ is an open affine cover of $X$ and $V$ is any open affine subset of $Y$, Hartshorne's III proposition 8.7 claims that $U_{i}\cap f^{-1}(V)$ are still open affine subsets of $X$.
Hartshorne gives a hint II Ex 4.3, but $f^{-1}(V)$ may not be an open affine subset of $X$. And I don't know how to use $V$ is an open affine subset of $Y$... And I don't know whether $f$ is separated and I can't use properties of fibre product... Maybe I misunderstand this problem...
Could you help me and give me an explanation?
Answer: $\require{AMScd}$First, if $S$ is a scheme, and $W, W' \subset S$ are open, then $W \cap W' = W \times_S W'$.
We may also write $f^{-1}(V) = X \times_Y V$, so that $$U \cap f^{-1}(V) = U \times_X (X \times_Y V) \cong (U \times_X X) \times_Y V = U \times_Y V.$$
To finish up, we write the corresponding cartesian diagram below.
\begin{CD}
U \times_Y V  @>>> V\\
@VVV @VVV\\
U @>>> Y
\end{CD}
Now, since $Y$ is separated, the morphism $V \to Y$ is an affine morphism by the mentioned exercise. Hence, as the base change of such a morphism, $U \times_Y V \to U$ is also affine, which implies that $U \times_Y V$ is an affine scheme.

some questions about open affine cover


Rewriting a binomial formula summation

Question: I am trying to obtain an explicit solution for $p$ from the following equation:
$p = 1 - \left[ \sum_{k=0}^{n} \binom{n}{k} p^k (1-p)^{n-k} \frac{1}{k+1} \right]^{\frac{1}{{n}}}$.
Obviously, I know that summing a binomial pdf we have
$\sum_{k=0}^{n} \binom{n}{k} p^k (1-p)^{n-k} = 1$
Perhaps there is a neat way to rewrite
$\sum_{k=0}^{n} \binom{n}{k} p^k (1-p)^{n-k} \frac{1}{k+1}$
so I get an explicit solution for $p$.
Answer: $$
\eqalign{
  & \sum\limits_{\left( {0\, \le \,} \right)\,k\,\left( { \le \,n} \right)} {{1 \over {k + 1}}\left( \matrix{
  n \cr
  k \cr}  \right)p^{\,k} q^{\,n - k} }  = \sum\limits_{0\, \le \,k\,\left( { \le \,n} \right)} {{1 \over {n + 1}}\left( \matrix{
  n + 1 \cr
  k + 1 \cr}  \right)p^{\,k} q^{\,n - k} }  =   \cr
  &  = {1 \over {\left( {n + 1} \right)p}}\sum\limits_{0\, \le \,k\,\left( { \le \,n} \right)} {\left( \matrix{
  n + 1 \cr
  k + 1 \cr}  \right)p^{\,k + 1} q^{\,\left( {n + 1} \right) - \left( {k + 1} \right)} }  =   \cr
  &  = {1 \over {\left( {n + 1} \right)p}}\sum\limits_{1\, \le \,j\,\left( { \le \,n + 1} \right)} {\left( \matrix{
  n + 1 \cr
  j \cr}  \right)p^{\,j} q^{\,\left( {n + 1} \right) - j} }  =   \cr
  &  = {1 \over {\left( {n + 1} \right)p}}\left( {\sum\limits_{0\, \le \,j\,\left( { \le \,n + 1} \right)} {\left( \matrix{
  n + 1 \cr
  j \cr}  \right)p^{\,j} q^{\,\left( {n + 1} \right) - j} }  - q^{\,\left( {n + 1} \right)} } \right) =   \cr
  &  = {{1 - q^{\,n + 1} } \over {\left( {n + 1} \right)p}} = {1 \over {\left( {n + 1} \right)}}{{1 - q^{\,n + 1} } \over {1 - q}} \cr}
$$

Rewriting a binomial formula summation


Show that in any metric space $(E; d)$, $cl(\{ y \in E: d(x, y) < \epsilon\}) \subseteq \{y \in E: d(x; y)\leq \epsilon\}$

Question: However, give an example where, $cl(\{ y \in E: d(x, y) < \epsilon\})  \neq \{y \in E: d(x; y)\leq \epsilon\}$. I'm completely stuck here and I don't know where to start from.
Comment: I think you may have mistyped the question, because the statement in the title of your post is false. Consider $B = \{y\in\mathbb{R}^2: d(0,y)<1\}$. Then ${\rm cl}(B) = B\cup S^1$, both the open disk *and* unit circle, which is obviously not contained in just the open disk.
Comment: Replace $<$ by $\leqslant$, once in the title and once in the body.
Comment: @Did Thanks, I've made the change
Answer: Here's one strategy for the proof: let $\overline{B}(x,r) = \{y\in E: d(x,y)\leq r\}$ be the closed ball of radius $r$, and $B(x, r) = \{y\in E: d(x,y)<r\}$ be the open ball of radius $r$. It will suffice to show that $\overline{B}(x,r)$ is closed, since it's a general face that if $S\subset T$ and $T$ is closed, then $\overline{S}\subset T$ (taking closure preserves inclusions, and the closure of a closed set is itself).
For fixed $x,r$, we show that $E\setminus \overline{B}(x,r)$ is open. Take $y\in E\setminus \overline{B}(x,r)$ and put $\epsilon = d(x,y)-r>0$. Then if $d(y,z)<\epsilon$, we have by reverse triangle inequality
$$d(x,z)\geq d(x,y)-d(y,z) > d(x,y)-\epsilon = r.$$
This proves that $B(y,\epsilon)\subset E\setminus \overline{B}(x,r)$. Since the complement is open, $\overline{B}(x,r)$ is closed. By my argument above, this completes the proof.
For an example where $\overline{B(x,r)} \neq \overline{B}(x,r)$, try playing around with the discrete metric:
$$d(x,y) = \begin{cases}0 & \text{if $x=y$}\\ 1 &\text{if $x\neq y$.}\end{cases}$$.


Android and GSON throwing No-Args Constructor not found exception

Question: I am having a problem with GSON in my Android app.
JSON Input
[
    {"id":"1202","parent_id":"0","agrpcount":"0","disp_order":"1","code":"PROD-NEW","name":"New Products","active":"1"},
    {"id":"1119","parent_id":"0","agrpcount":"0","disp_order":"2","code":"PROD-VID","name":"\\\"How To\\\" Videos","active":"1"},
    {"id":"1685","parent_id":"0","agrpcount":"0","disp_order":"3","code":"SUPP-RESOUR","name":"Support Resources","active":"1"},
    {"id":"49","parent_id":"0","agrpcount":"0","disp_order":"109","code":"EQ","name":"Equipment for Sign Making","active":"1"},
    {"id":"22","parent_id":"0","agrpcount":"0","disp_order":"110","code":"application-tape","name":"Application Tapes","active":"1"}
]

Container class for each array item
public class Category {
    public int id;
    public int parent_id;
    public int agrpcount;
    public int disp_order;
    public String code;
    public String name;
    public int active;
}

Class to load category array
public static List<Category> getCategories(){
    Category[] cats;

    Gson gson = new Gson();

    cats = gson.fromJson(strJSON, Category[].class);

    List<Category> CatsList = Arrays.asList(cats);
    return CatsList;
}

When I run this code I get the following in the Log
02-28 15:15:42.471: ERROR/AndroidRuntime(777): Caused by: java.lang.RuntimeException: No-args constructor for class com.signwarehouse.catalog.SWService$Category does not exist. Register an InstanceCreator with Gson for this type to fix this problem.
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.MappedObjectConstructor.constructWithNoArgConstructor(MappedObjectConstructor.java:64)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.MappedObjectConstructor.construct(MappedObjectConstructor.java:53)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonObjectDeserializationVisitor.constructTarget(JsonObjectDeserializationVisitor.java:40)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonDeserializationVisitor.getTarget(JsonDeserializationVisitor.java:56)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:109)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonDeserializationVisitor.visitChild(JsonDeserializationVisitor.java:107)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonDeserializationVisitor.visitChildAsObject(JsonDeserializationVisitor.java:95)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonArrayDeserializationVisitor.visitArray(JsonArrayDeserializationVisitor.java:72)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:117)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:64)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:49)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.Gson.fromJson(Gson.java:568)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.Gson.fromJson(Gson.java:515)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.Gson.fromJson(Gson.java:484)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.Gson.fromJson(Gson.java:434)
02-28 15:15:42.471: ERROR/AndroidRuntime(777):     at com.google.gson.Gson.fromJson(Gson.java:406)

I tried adding a no-args constructor to the Category class, but this didn't help. No surprise since Java creates it automatically.
Does anyone have any idea how to fix this?
Thanks.
Comment: Could you post the whole SWService.Category class? It seems it is an inner class. Maybe GSON can't access it. Did you try to make it a class of its own?
Comment: @gabe: He did and it's an inner public class.
Comment: Looks correct. Can you try getting just one of the objects from the array (editing the file to contain one object) and using Category.class?
Answer: Trying making the Category class a public static class...
public static class Category {/* fields */}

I bet what's happening is since you didn't make the class static it is trying to create your parent/outer class (com.signwarehouse.catalog.SWService) first which probably doesn't have a no-arg constructor.
Comment: +1 Nice catch that `Category` is an inner class from the exception message.
Comment: Perfect.  Worked like a charm.  Thanks.


Firebase login with phone & password

Question: I want the user to be able to login with phone and password.
If the user enters the app first time, he es registered by validating his phonenumber, thats the current state at firebase. But I dont want to always send a sms code to the user every time he wants to login.
Now in the next step of the authentification the user should enter an email and a password, so I can link the credentials with the phone. The user is now able to login with email and password and with phone and verification code.
But I want to have the function that the user can login with phoneNumber and password. So I can write a cloud function for "doesPhoneNumber" exist which goes through the user-db and tells me if the user can login or have to sign up.
The next function is now my actual problem, i want to send the phone number and the password to a extra cloud function to receive the authToken but how can I verify if the credentials are correct within that function?
Answer: It can be achieved using firebase custom token
The cloud function need to save the registered phone and password first.
When client request to login, the cloud function then doing query to validate the phone and password. If it is valid, then cloud function will create custom token and send it back in response.
The client then using that custom token to login to firebase auth.
The documentation for sign in with custom token can be seen in :

for android client
for web client
for ios client

Other solution is make the phone as email by putting @somesite.com at the end of number.
For example email@example.com then make client sign in with email and password.
Comment: And how would you validate the password without storing it in the database?
Comment: Well, there is another "hack" solution, make the phone as email. For example email@example.com So you can use sign in with email and password
Comment: By this approach you disable the abillity for Users to get their passwords reset.
Comment: Nope, we can still easily update the user password using firebase admin sdk directly without sending reset password to email. The documentation can be read in https://firebase.google.com/docs/auth/admin/manage-users#update_a_user
Comment: You can use firebase cloud functions or your own server to create webservice for updating / resetting password. There is security risk, but you can still make it more secure with instead sending userId as parameter you can send `idToken` for the server to verified.
Comment: @faruk firebase admin sdk allows to create users with phone number and password, is there any way to login with phone number and password ?
Comment: You can find the email of the user that has that phone number, and then use the Firebase Javascript SDK to sign in with email and password. See this question: https://stackoverflow.com/questions/51752457/firebase-verify-email-password-in-cloud-function/51755426#51755426
Answer: You shouldn't handle your users login.
There are endpoints available for each platform for loggin in. Those endpoints tell you whether the login was successfull or not and return your desired user informations
Read more: https://firebase.google.com/docs/auth/
Comment: That doesnt answer my question in any kind of way.

Firebase login with phone & password


The possible values of $x$, if $\tan^{-1} x>\cot^{-1}x$

Question: What are the possible values of $x$, if $\tan^{-1}x >\cot^{-1}x$?
We have $\tan^{-1}x >\cot^{-1}x\implies \tan^{-1}x -\cot^{-1}x>0 \implies \tan^{-1} x-\tan^{-1}1/x>0\implies \tan^{-1}\dfrac{x-1/x}{1-x.1/x}>0$.
What can I do now?
Comment: There is problem $x^2>1\implies (x+1)(x-1)>0\implies  x<-1  \&\,  x>1$. Why not $x<-1$? @Nosrati
Comment: As you found $\tan^{-1}\dfrac{x-1/x}{1+x.1/x}>0$ then $x>1$.
Answer: HINT:
$$\cot^{-1}x=\frac\pi2-\tan^{-1}x\implies \tan^{-1}x-\cot^{-1}x=2\tan^{-1}x-\frac\pi2>0$$
Answer: Given $\displaystyle \tan^{-1}(x)>\cot^{-1}(x)\;\forall\; x\in\mathbb{R}.$
So  $\displaystyle \tan^{-1}(x)>\frac{\pi}{2}-\tan^{-1}(x)$
So $\displaystyle \tan^{-1}(x)>\frac{\pi}{4}\Rightarrow x>1$
So we get $x\in(1\;,\infty).$
Comment: It seems you have lost a part of the solution for $x<0$.
Answer: A plot might help to guide the derivation.
Recall that for $x\ge0$
$$\cot^{-1}(x)=\frac{\pi}{2}-\tan^{-1}(x)$$
and hence
$$\tan^{-1}(x)>\frac{\pi}{2}-\tan^{-1}(x)$$
$$ \tan^{-1}(x)>\frac{\pi}{4}\implies x>1$$
and for $x<0$
$$\cot^{-1}(x)=-\frac{\pi}{2}-\tan^{-1}(x)$$
hence
$$\tan^{-1}(x)>-\frac{\pi}{2}-\tan^{-1}(x)$$
$$ \tan^{-1}(x)>-\frac{\pi}{4}\implies x>-1$$
therefore the solution is
$$x\in (-1,0)\cup(-1,\infty)$$


Collection of subsets of Naturals with containment

Question: Question: Let $S$ be a collection of subsets of $\Bbb N$ such that for every $A, B \in S$ we have $A \subset B$ or $B \subset A$. Can $S$ be uncountable?
Answer: Yes, although this is often found to be surprising. To see it, replace $\mathbb{N}$ with $\mathbb{Q}$, which has the same size, and consider the collection of Dedekind cuts in the rationals.
Comment: The following is undoubtedly excessively vague: Is there a natural collection of sets of natural numbers with the desired property?
Comment: @AndréNicolas, here is another way to do it, which you might find to be more natural: Consider the tree $2^{\lt\omega}$ consisting of all finite binary sequences. We may identify nodes in that tree as natural numbers in a natural way, for example, by interpreting them as binary representations (appending a $1$ to address the leading zeros problem). Now, for each infinite binary sequence $s$, let $A_s$ consist of the numbers that lie on $s$ or to the left in that tree under the lexical order. It follows that $s<t\to A_s\subset A_t$.
Comment: Is there any way of doing it without replacing N with Q?
Comment: Well, $\mathbb{Q}$ is countable, so enumerate it as $\{q_n\mid n\in\mathbb{Q}\}$, and then for each real $r$ consider $A_r=\{n\mid q_n<r\}$. That is what I meant by "replace $\mathbb{N}$ by $\mathbb{Q}$."
Comment: Thank you for clearing that up.
Comment: Definitely tied more naturally to the integers. Thanks.


How to increase the topic consumer throughput by using `Function, Flux>`?

Question: I have a service based on webFlux and will consume then produce message from a kafka topic.
My code is just like this
@Bean
public Function<Flux<String>, Flux<String>> reactiveUpperCase() {
    return flux -> flux.map(val -> val.toUpperCase());
}

What I found is when I have 2 instance, I could consume 750 message per 30 minutes, but my CPU is never higher than 10%.
As time goes by, the lag keeps increasing, so I'm wondering how could I increase the consumer throughput.
From the documents, the concurrency doesn't take effect for reactive, link
Does any one know how could I increase the throughput without adding more instance?
Answer: As I'm using kotlin, what I found is using Flow.flatMapMerge(parallelCount)


Notice: Undefined index: field_brand_image in include() in views field template

Question: I've got another question about undefined index that I can use help with.
The error message I get is: Notice: Undefined index: field_brand_image in include() on line 15
I'm pretty sure that this is the same lnaguage and "und" portion of the field array issue I had in my other question here: Notice: Undefined index: und in include()
And only displays when the field is empty
Except this time the error is coming from a views template, the line throwing the error looks like:
php print $fields['field_brand_image']->content;

I "think" the answer involves an isset conditional, but I don't know what the syntax should be.
I'm using D7.23 and views 7.x-2.3
I'd appreciate some help.
Thanks.
Answer: One way of doing that is using a empty to check the field as following:
<?php print (!empty($fields['field_brand_image'])) ?
  $fields['field_brand_image']->content : ''; ?>

Comment: Just empty on its own is enough. It doesn't give an error when the expression does not exist.
Comment: @AlfredArmstrong Yeah right, Thanks for correcting me
Comment: Thank you both again for the help, I really do appreciate it.


Is there a way to reduce the size of the Git folder?

Question: It seems like my project is getting bigger and bigger with every Git commit/push. Is there a way to clean up my Git folder?
Comment: Related: *[Make Git consume less disk space](https://stackoverflow.com/questions/1398919/)* and *[Reduce Git repository size](https://stackoverflow.com/questions/2116778/reduce-git-repository-size/2116892#2116892)*
Answer: I'm not sure what you want. First of all, of course each time you commit/push the directory is going to get a little larger, since it has to store each of those additional commits.
However, probably you want git gc which will "cleanup unnecessary files and optimize the local repository" (manual page).
Another possibly relevant command is git clean which will delete untracked files from your tree (manual page).
Comment: **`WARNING`** The command as written above by @Kalle will remove **EVERY _>UNTRACKED< FILE AND DIRECTORY_ WITHIN YOUR GIT ROOT**, _not_ just "files listed in .gitignore". Anything that is not being tracked by Git, regardless of whether or not it is listed in `.gitignore` will be wiped. `git clean -dfX` (note the case on the `X`) will only remove items which have an applicable rule in `.gitignore`. **Please heed this warning:** _Never run `git clean` without either running it in interactive mode, with `-i` instead of `-f`, or at least doing a dry run first — `-n` and then again with `-f`._
Comment: git clean -d -f -x deletes files listed in .gitignore and such. E.g. workspaces that don't belong in git, Pods folder, etc.
Comment: Or making a backup :-)
Comment: And making a backup
Answer: Run:
git remote prune origin

It deletes all stale tracking branches which have already been removed at origin, but which are still locally available in remotes/origin.
git gc --auto

'G arbage C ollection' - runs housekeeping tasks (compresses revisions, and removes loose/inaccessible objects). The --auto flag first determines whether any work is required, and exits without doing anything if not.
Comment: Some explanation of what those do? I know that we can Google them and search for their documentation, but it is a common practice to provide short description of your answer when it involves just code or commands.
Answer: One scenario where your git repo will get seriously bigger with each commit is one where you are committing binary files that you generate regularly. Their storage won't be as efficient than text file.
Another is one where you have a huge number of files within one repo (which is a limit of git) instead of several subrepos (managed as submodules).
In this article on git space, AlBlue mentions:

Note that Git (and Hg, and other DVCSs) do suffer from a problem where (large) binaries are checked in, then deleted, as they'll still show up in the repository and take up space, even if they're not current.

If you have large binaries stored in your git repo, you may consider:

managing those binaries in an external repository.
manage your .git repo size
try and remove those binaries from your history with git filter-branch (warning: this will rewrite the history, which is bad if you have already pushed your repo and if other have pulled from it)

As I mentioned in "What are the file limits in Git (number and size)?", the more recent (2015, 5 years after this answer) Git LFS from GitHub is a way to manage those large files (by storing them outside the Git repository).
Comment: git large file support is helpful if you have large binary files (like images) routinely being added / updated. See https://git-lfs.github.com/. Super easy to implement, supported by github. All team members need to install it to make use of it collaboratively.
Comment: @EricWoods True. I mentioned Git-LFS before (64 times: https://stackoverflow.com/search?tab=newest&q=user%3a6309%20git-lfs). I have edited this old answer accordingly.
Comment: Ha, indeed! Funny how a 9+ year old answer is still relevant (and now even more so with the LFS info).
Answer: Yes, yes, git gc is the solution, naturally.
And locally, you can just delete the local repository and clone it again
But there is something more important here...
The seconds you wait for that huge Git and externals to process are collected to long minutes in which are collected to hours of inefficient time spent.
Create a new (entirely, not just a branch) repository from scratch, including the only recent version of files, naturally you'll lose all the history.
But when in the code world, it is not time to get sentimental. There isn't any point in dragging along the entire 5 years of code for every commit or diff.
You can still store the old Git and externals somewhere, if you get nostalgic :]
But, at some point you really have to move along :]
Your team will thank you!
Comment: Completely agree, we recently took this approach with an old repository and haven't looked back; well, mainly because we can't, but you know what I mean :)
Answer: Running this command is extremely dangerous, but it will shrink your repository by erasing all your Git recovery/backup files:
git reflog expire --expire=now --all && git gc --prune=now --aggressive

It will erase all files Git uses to recover your repository from some bad command. For example, if you did git reset --hard, you can usually recover the files lost. But if you do git reset --hard before the git reflog expire... command, then you lost everything. Now, your only hope is to use some tool which analyses your file system and try to recover the erased files, if they were not overridden.
Comment: I really wouldn't call this *extremely dangerous*. I'd just label it something you have to be *careful* with. In my experience, very few indeed ever touch the reflog or unreachable objects—most don't even know they're there or how to interact with them, and so get stuck in situations where they'd be useful, or do things a terribly inefficient way. I'd go so far as to say that if you don't know and can't figure out what these commands will do, then you can safely run them!
Answer: git clean -d -f -i is the best way to  do it.
This will help to clean in a more  controlled manner.
-i stands for interactive.
Comment: While the OP's question is vague, and this is a good answer in that respect, I do want to point out that `git clean` is not for cleaning up the repo so much as cleaning up the directory. For users who blindly copy/paste, beware; this removes untracked files/dirs you might actually want locally.
Comment: git clean -d -x -f works nicely if you want to deep clean
Answer: I don't know if it will shrink it, but after I run git clean, I often do git repack -ad as well, which reduces the number of pack files.
Comment: repack is a part of `git gc` process, so no need to run it separately


Cost-effective method of serving around 300-500 GB of static data for a new site?

Question: I'm building a website that needs to serve about 300-500 GB of static data from the get-go, with about 5-10 GB added every month.  The files are mostly photos.
All of my potential users will be located in Canada, with about 95% of them in a single city (Toronto).
I have many options, but I don't know which is best for my situation.
First of all, I don't think I need to use CDN since the majority of my users will be based in Toronto.  I just need to find a database provider based in/near Toronto.
I want to host my website on a VPS in the beginning, since it's easy to set up and should have no problems accommodating my initial users.
The problem is where I should host the 500 GB of static files.  Should I host them alongside my website on the VPS?  Or should I host them on a cloud platform like BackBlaze B2 or Amazon S3?  HDD is sufficient for my static files, although I would like to host website itself on a SSD.
I estimate that each user will be served about 10 MB of data per visit. Assuming I get about 5000 visitors a day, I should be able to serve 50GB of data daily with no visible decrease in performance.  The user will spend most of his time on a map (I'm using Google Maps JavaScript API), so it will be highly dynamic with continuous ajax calls.
I'm not too worried about getting perfect availability in the beginning, as long as the site is up most of the time.
Comment: A CDN has plenty of benefits even if your users are all in one place.
Comment: To clarify: 300 - 500GB of files, but what volume of downloads - how many GB per day? Do you need very high bandwidth? What's the average size are the files?
Comment: @Tim I estimate that each user will be served about 10 MB of data per visit.  My goal is to get 5,000 daily visitors.  So I should be able to serve 50GB of data daily with no visible decrease in performance.
Comment: Do you need high availability? Is price a significant factor - what's your budget? How critical is performance locally, and globally? Is your website largely static or does every user log in, making it largely dynamic?
Comment: @Tim High availability would be nice, something like 99.9% is good enough.  Cost is certainly a factor (I'm a poor plebeian student), but having good performance is more important to me than slightly higher costs.  The user will spend most of his time on a map (I'm using Google Maps JavaScript API), so it will be highly dynamic with continuous ajax calls.
Comment: High availability generally requires duplicating infrastructure - two VPS servers in different data centers, behind a load balancer that spans both. Data stores need to be synchronized. This will probably double to triple your server costs. Think again if it's worth the additional cost and complexity. Have you got a preferred VPS?  Suggest you take all the information you've entered in comments and clarify your question, it's easier for people to answer if you give them all your requirements in one place.
Comment: Are the static files something like map files, dynamically populated as users scroll? Or as they pictures that are one-off downloads in their own window. Map tiles require low latency, but are likely easy to cache, so long as the caching headers and URL structure are set up properly.
Comment: @Tim The static files are mostly pictures, either 5 KB or 50 KB in size depending on whether the thumbnail or full image is served.
Comment: Just in passing, "*99.9%*" and "*poor student*" don't usually go together.  I tell my clients dreaming about five-nines on a shoestring that you get 99% uptime for free (ie, for no more cost than the basic hosting).  Each extra nine costs between four and ten times as much.
Answer: I'm not going to go through costs, as you can work them out yourself easily using public pricing. Generally VPS hosting of large amounts of files is most expensive, followed by AWS, followed by B2.
If you want high availability you'll want two servers and a load balancer. This is fairly easy in AWS, Azure, and Google, and some smaller providers can do it as well.
AWS S3
AWS S3 is great, and has redundancy built in - from memory your data is stored in and potentially served from three data centers. Infrequently accessed class storage can reduce the bill, and is fine for low volume file hosting. At your 1.5TB per month will cost something like $30 per month. Good system, reliable, easy to add a CDN. AWS has a region in Canada.
AWS LightSail / VPS
AWS Lightsail VPS has a lot of bandwidth quite cheap, for AWS at least. Digital Ocean and the other decent VPS providers are similar. You should consider just having plenty of storage on your VPS. The downside here is SSD storage will probably be quite expensive, and you have no redundancy - but if you're running from a single VPS you have a single point of failure anyway.
BackBlaze B2
B2 is cheap storage, but more expensive bandwidth. I think they only have one data center, so there's less redundancy there. They'll be somewhere in the USA, so latency and bandwidth should be pretty good, but not as good as AWS. Another single point of failure.
CDN
If you use a VPS then you should consider using a CDN. A CDN can significantly reduce your bandwidth costs if set up correctly.
CloudFlare free tier is fantastic, and you can pay for a better class of service and more features. As well as serving users faster than your server can, it will serve remote users more quickly. Google likes fast websites too, and their crawlers could connect from anywhere.
Recommendation
Your priority seems to be performance, price and reasonable availability. AWS S3 IA class is probably a good trade-off, but if you can put up with it being slightly slower you can try B2.
You can put CloudFlare's free (or paid) services in front of Amazon S3. Instructions here.
Comment: AWS bandwidth is incredibly expensive. What spaces does for $5 is $96 on AWS, $90 of that is bandwidth fees. If you use a LOT of bandwidth it might be worth using. If you're low bandwidth and are ok with infrequently accessed storage then AWS might not be too bad.
Comment: Great answer.  Thank you very much.  I think I'll probably go with S3 since it's the industry-standard
Comment: AWS S3 is a safe bet, but I wouldn't say "industry standard" as such. Try the IA storage class first, I think it'll be fine, but be prepared to move to standard class. With a CDN IA should be fine. You could look at Google Cloud as well. I'm not a huge Azure fan, it just doesn't feel as well designed to me. If you have cheap disk for a VPS hosting yourself via a CDN to reduce bandwidth and latency is a decent option, but VPS disk is often quite expensive.
Comment: I just discovered Digital Ocean came out with their own object storage service called Spaces, competing against AWS S3.  They are aimed at smaller developers and the primary selling point is their significantly lower bandwidth cost.  S3 charges $0.09 for each GB of bandwidth, while Spaces charges a mere $0.01 for each GB.  Spaces also offer free requests (S3 charges $0.005 per 1000 requests)
Answer: I only add this because it seems to be missing from the excellent lists of suggestions so far put forth, but if you have a lot of data don't overlook the possibility of co-locating your own kit.  That might seem rather 1990s, but it's still very much an option.
If you get/build a decent 1U server you can put at least two large, fast discs in it, and RAID-1 them.  Your 1.5TB of throughput a month is well within the spec of most colocation offers.  And although it's possible that you'll have a couple of days outage when something goes pop, using quality hardware will minimise the likelihood of such an outage, and having a few judiciously-chosen shelf spares (PSU!) will help reduce the duration of such an outage.


How to show latest and high price value first using Views?

Question: I am using views to show products in my site. I need to filter my view with high price and latest product . Using expose filter not a sort criteria.
Comment: is your requirement is to implement sort by price or relevance option for anonymous users..??
Comment: yes for all user..
Answer: To achieve your requirement, you have to expose the sort criteria. Select checkbox Expose this sort to visitors, to allow them to change it under sort criterion configuration.
Note: When you add any field into SORT CRITERIA, then under this field configuration you can see this option.
Comment: ok let me check. But I need same functionality via filter not sort
Comment: When you add any field into SORT CRITERIA, then under this field configuration you can see this option.


How do Unix Domain Sockets differentiate between multiple clients?

Question: TCP has the tuple pairs (IP Addr/port/type) to tell one client from another.  UDP passes the client IP and port.  How does the unix domain keep track of different clients?
In other words the server creates a socket bound to some path say /tmp/socket.  2 or more clients connect to /tmp/socket.  What is going on underneath that keeps track of data from client1 and client2?  I imagine the network stack plays no part in domain sockets so is the kernel doing all the work here?
Is there a unix domain protocol format like there is an IP protocol format and TCP/UDP formats?  Is the format of domain socket datagram protocols published somewhere?  Is every unix different or does something like POSIX standardize it?
Thanks for any illumination.  I could not find any information that explained this.  Every source just glossed over how to use the domain sockets.
Comment: @MarcB What you are describing seems dubious. In page 449 5th paragraph of `Linux Programming 2nd Edition Unleashed` by `Kurt Wall, et al`, it is stated thus: `...with named pipes you cannot tell one process data from another. Using UNIX Domain sockets, you will get a separate session for each process.`
Comment: Can that be true?  If a server writes data the first client that reads gets the data regardless if it was intended for that client or not? That make them almost useless.
Comment: @MarcB that should be an answer
Comment: Talking over a unix domain protocol is basically just file i/o. unless the data you're passing through the socket contains source identification, there's no way to tell which process sent a particular string through.
Answer: If you create a PF_UNIX socket of type SOCK_STREAM, and accept connections on it, then each time you accept a connection, you get a new file descriptor (as the return value of the accept system call).  This file descriptor reads data from and writes data to a file descriptor in the client process.  Thus it works just like a TCP/IP connection.
There's no "unix domain protocol format".  There doesn't need to be, because a Unix-domain socket can't be connected to a peer over a network connection.  In the kernel, the file descriptor representing your end of a SOCK_STREAM Unix-domain socket points to a data structure that tells the kernel which file descriptor is at the other end of the connection.  When you write data to your file descriptor, the kernel looks up the file descriptor at the other end of the connection and appends the data to that other file descriptor's read buffer.  The kernel doesn't need to put your data inside a packet with a header describing its destination.
For a SOCK_DGRAM socket, you have to tell the kernel the path of the socket that should receive your data, and it uses that to look up the file descriptor for that receiving socket.
If you bind a path to your client socket before you connect to the server socket (or before you send data if you're using SOCK_DGRAM), then the server process can get that path using getpeername (for SOCK_STREAM).  For a SOCK_DGRAM, the receiving side can use recvfrom to get the path of the sending socket.
If you don't bind a path, then the receiving process can't get an id that uniquely identifies the peer.  At least, not on the Linux kernel I'm running (2.6.18-238.19.1.el5).
Comment: There is also SOCK_SEQPACKET for AF_UNIX in Linux which allows connections like in SOCK_STREAM, but also preserves message boundaries like in SOCK_DGRAM.
Comment: @rob-mayoff from your explanation I understand that there's no "send queue" on a UNIX socket, since the data is directly pushed to the peer's receive queue. But it seems that `sock.sk_wmem_alloc` is incremented on the sending socket when sending data, whereas I would expect `sock.sk_rmem_alloc` to be incremented on the peer's socket instead.
Comment: @rob mayoff nicely explained.


Highlight words in document using office.js for Word 2013

Question: There is a list of words that needs to be highlighted in the word document, how can the words be highlighted from the add-in when using office.js (common APIs) for Word 2013?
Answer: There isn't a straightforward way to do this with Office.js in Word 2013.
With Word 2016 you can use WordApi to search for content and format the results. These APIs are not applicable to Word 2013 however.
For Word 2013, would need to extract the raw OOXML, manipulate it externally yourself and then replace the OOXML in the document. Since the user would need to first ""select all", this isn't a very practical solution.


How to access SSH from web without connecting from client

Question: I have a server which is behind a firewall and the SSH port is disabled.
I want to access via SSH to monitor services on a server. My web server is Apache Tomcat. How can I achieve this? is there any java application for this?
Comment: ok, in that case what are you hoping to get out of SO answers that you can't just from googling "applications for ssh over http"?
Comment: are you talking about ssh over http?
Comment: @Dmitry  yes i am talking ssh over http
Comment: @talnicolas hi i am seroiously working on my english , and accepting answers on stack over flow.
Answer: There are 2 Solutions(which you may combine together too!):

Web-Terminals:There are many "web based terminals" written in different languages, i'm not sure which one simply runs on Apache Tomcat, but i'm sure there are some written in Java like WebTerm or JCTerm.
Tunnels: You could also try "TCP over HTTP Tunnels".

I believe a good way to start is to take a look at this comparison of these Tools by AnyTerm.


Linking surface integral of a gradient field to a contour integral

Question: I have a vector field $F$ deriving from a scalar potential $f$, i.e. $F=\text{grad}(f)$. I want to compute the integral of $F$ over a surface (To evaluate the flux of $F$). I think there exists a theorem linking this integral over a surface to the integral of $f$ over the edge of the surface but I can't find it.
What I find is :
-Green theorem linking integration of $\text{div}(F)$ over a surface to $F$ over a line
-Stokes theorem linking integration of $\text{curl}(F)$ to integration of $F$.
But no theorem linking integration of $F$ over a surface to $f$ over a contour.
Can someone help me ?
Thanks
Answer: It sounds like you are trying to find the flux of a vector field through a surface with a boundary (i.e. not a closed surface around a voluminous region). Unfortunately, there isn't an integral theorem that allows you to related the flux through the surface to an integral around the boundary. See the second figure here.
I played around with this idea last time I was teaching Calc 3 and found a few counter examples. One problem is that there isn't an obvious way to orient the vector field around the boundary of the surface, and it depends on how you want to integrate the vector field over the surface.
How do you want to integrate the vector field over the surface? There are several ways to do it. Do you want to take its 'spatial' curl, it's 'spatial' divergence  , or something else. If you want to take the divergence of the component of the vector field which is tangential to the surface, this can be done: see this post. I like to think of it a projecting the vector field onto the surface and then applying the divergence theorem on that two-dimensional surface. It works and it is another special case of the Stoke's Theorem.


weak classifier with weak features

Question: I have a set of weak features, and I am looking to create classifier based on them (it will only be weak given the noisy data). I am only trying to be right in ~60% of the case (that's enough for my problem).
Is there a literature on this kind of problem? I can only find articles where the output accuracy is very high.
Thank you
Comment: I think that if your features are truly weak then any optimized model you build will be, at most, weak. We can't really pinpoint which one it will be, thanks to **No free lunch**, which still applies when you actually want a "not so good" model.
Answer: Your question makes me ask the following:

Why are you 'looking to create a weak classifier'? As in why are you deliberately looking for a 'weak classifier', when potentially you could train a 'strong classifier'?
Why would you deliberately try to be right only about 60% of the time, when potentially a model could be trained to be right about say 66% of the time?

Your self-imposed '60%' accuracy limit suggests that you are aware of working in some noisy domain, (perhaps financial time series).
If that is the case, it seems that you are wary of over-fitting a classifier and seeing too-good-to-be-true accuracy on in-sample data. In which case, a better way is to simply to penalize model complexity when training your model and keep aside a pristine subset of data for out-of-sample testing.
In case, 60% accuracy is some lucky magic number, then stop training your classifier once it reaches 60% accuracy on the subset of data used for validation; which you can later check against the as yet pristine out-of-sample test data.
Having said all that, I think better to use regularization. In say boosting, not using interaction terms in the base learners and a low learning rate and a smaller number of iterations will inevitably to lower in-sample accuracy than when using greater interaction depth, fast learning rate and high number of iterations, holding other things constant.
Comment: I am not looking to create a weak classifier, but given my data set there is little hope I will get anything better than 60%. This number is a bit arbitrary, just to illustrate the kind of ratio I am hoping for.
Comment: In which case it is better to rephrase the original question. Instead of saying we have weak features and we desire to train a weak classifier and only trying to attain 60% accuracy; we might do better saying that we are operating in a noisy domain with weakly predictive features - is there a way to obtain useful predictions in such a situation?

Literature relevant to data mining in noisy domains might be of some use.

weak classifier with weak features


getting images from iPhone simulator

Question: I have just created an image and saved it in iPhone simulator phone library. I want to view this image(not from Photo Library) to view its properties from document resources. But I am not able to access the contents of photo library.
Thanks
Pankaj
Answer: With Xcode 6 the simulator image files are now located in:
~/Library/Developer/CoreSimulator/Devices/<device UDID>/data/Media/DCIM/100APPLE

Where the value of <device UDID> can be determined using the command: xcrun simctl list devices.
EDIT - Path is still valid for Xcode 10.1 version.
Comment: You can also get the `` from **Devices** window (Shift+CMD+2)
Comment: `xcrun simctl list devices | grep Booted | grep -E  '\w+-\w+-\w+-\w+-\w+' -o` to reduce noise
Answer: Go to this folder (Finder, Shift+Command+G):
~/Library/Application Support/iPhone Simulator/(version)/Media/DCIM/100APPLE
replacing "(version)" with your Simulator version. You should find all the photos there.
Comment: This folder no longer existed in version 7.0+
Answer: In newer versions of the Simulator, there is a button which will let you take a screenshot (the camera icon in the picture below):

The new screenshot will show up next to Simulator window and you can drag it to your desktop to save the file:

getting images from iPhone simulator


Potluck (Food) Menu Sign up form

Question: I am looking a way to create a food menu sign up sheet.
Login user can select food menu from the list, If somebody picks the food, it is disabled, so nobody can sign up for that item again. Is there anything like this out there?
Answer: You could try a wishlist or gift registry module. Here's one. I haven't tried this but it should disable or remove items from the list once people select them.
https://www.drupal.org/project/wishlist
Please note that this question has multiple or too many possible answers, so it might get disabled.

Potluck (Food) Menu Sign up form