Riemann sums and optimization

Question: I have a question that is related to a question asked on another stackexchange page:
integral partition, real analysis
The question on the other page is written out here:
If we have $f(x) = x^2 $ and $P_n $ which partitions $[1,3]$ into $n$ sub-intervals, each equal in length,how can I write the formulas for $L(f,P_n)$ and $U(f,P_n)$ in terms of $n$?
Do they converge to the same limit as $n$ goes to infinity?
How large does $n$ need to be so that $U(f,P_n)$ is within $0.01$ of $\int_1^3 \! f(x) \, \mathrm{d}x$ ?
My question is about the last part. I have set up the following
$$\text{min $n$ such that} \frac{26}{3} - (2 + \frac{8(n+1)}{2n} + \frac{4}{3} \cdot \frac{2n^3 + 3n^2 +n}{n^3 } ) \leq 0.01$$
Is there some intuition from the Riemann Integrability of this function that will help me solve for n in a quick way? What is the standard way to do this?
Thanks.
Answer: You are asking about error estimates for Riemann sums in particular.  For some reason the error estimate for the endpoint sums is typically missing from the standard, calculus-level discussion of numerical integration which usually includes the trapezoidal, midpoint and Simpson approximations.  I think this is a shame because the endpoint error esimate is simpler -- so simple that when I taught second semester calculus this past semester, I decided that it was worth including a derivation of this error estimate (and it certainly was not for the other error estimates).  Here is the result:

Endpoint Approximation Theorem: Let $f: [a,b] \rightarrow \mathbb{R}$ be differentiable, with bounded derivative: there is $M \geq 0$ such that $|f'(x)| \leq M$ for all $x \in [a,b]$.  Let $L_n(f)$ be the left endpoint sum -- i..e, the Riemann sum obtained by dividing $[a,b]$ into $n$ equally spaced subintervals and taking the left endpoint of each subinterval as the sample point.  Then

$\left| \int_a^b f - L_n(f) \right| \leq \left(\frac{(b-a)^2M}{2} \right) \frac{1}{n}$.

Exactly the same result holds for the right endpoint sums.

For a proof, see Section 9.3 of these notes.  (See Section 7.1 for the Racetrack Principle.) Let me say though that they are for a more advanced course.  In my freshman calculus course I eventually gave a derivation like that, but I spent much more time motivating the basic idea: imagine that you are an FBI agent trying to capture an escaped criminal.  The key questions to ask are: (i) Where did he escape?  (ii) When did he escape?  (iii) What kind of transportation might he have access to?  Or in mathematical terms, (iii$'$): What is an upper bound on his maximum speed?  If you know where someone starts, how fast they could possibly be travelling, and how much time has elapsed, you can draw a circular disk on your map in which your criminal must lie.  If you want to think of the disk as expanding as a function of time then you get a cone.  (This is similar to what is studied in physics, the light cone.)  This is for two-dimensional motion.  Here we have a real-valued function which corresponds to motion on a line, so the picture is simpler: using the initial position and the maximum speed, you can draw a triangle in which the graph of the function must lie.  This is the basic idea which gives you an estimate for a single subinterval.  Adding this up over all the subintervals you get the result.
The FBI business is less silly than it sounds: the idea is to get you to see intuitively that the maximum value of $|f'(x)|$ must be taken into account in order to get any bound.  As long as $f'$ is continuous, it is bounded on the closed interval $[a,b]$, and that happens e.g. if $f''$ exists.  So the hypothesis is mild.  And getting bounds on $|f'|$ are exactly what the methods of differential calculus (using $f''$ and perhaps $f'''$) will do for you.  The shape of the bound though is that you should expect the error to be about a constant times $\frac{1}{n}$.  Note also that this formula is sharp in the sense that it is an equality when $f$ is linear, so it really is "the truth" in some reasonable sense.
You can absolutely use this formula to answer your question about how large $n$ needs to be to get the error less than $.01.$  Try it!  I just did a quick mental calcuation and got $n \geq 1200$, but you should check me on that.  Here, since $f$ is increasing, the upper sum is just the right endpoint sum.  Actually, when $f$ is increasing you know that the error is at most the difference between the upper sum and the lower sum, which is just $(f(b)-f(a))(b-a)) \cdot \frac{1}{n}$, which is an even easier way to see that the error is on the order of $\frac{1}{n}$.  But for a non-monotone function the upper and lower sums are rather theoretical: you will need to solve $n$ different optimization problems to compute them exactly...good luck with that.  Thus we estimate the error with endpoint sums instead because we can actually compute them.
Notice though that all this says that in order to get $k$ decimal places of accuracy for $\int_a^b f$ we will need to add together some constant times $10^k$ terms.  Do we really need to compute so many terms to get $k$ decimal places of accuracy?  No, at least as long as $f$ is reasonably smooth.  This is where the other numerical integration (or quadrature) methods come in.  They are cleverly weighted averages of the endpoint sums which converge much faster.  For instance, Simpson's Rule says that if $f$ has a bounded fourth derivative then some slightly strange-looking variant $S_n(f)$ of the endpoint sum approximates $f$ to within $C \frac{1}{n^4}$, where the constant $C$ depends upon the size of $|f''''(x)|$.  There are infinitely many more rules: in other words, if $f$ has a bounded $k$th derivative, then there is some weighted average of Riemann sums that when you add up $n$ terms of it, you will approximate $\int_a^b f$ to within $C \frac{1}{n^k}$, where the constant $C$ depends upon the maximum value of $|f^{(k)}|$ on $[a,b]$.
Note finally that there is a version of all of these rules which gives an equality for the error rather than the inequality.  This expression for the error is in terms of a certain derivative of $f$ at an unknown point $c \in [a,b]$.  These types of results come from a similar exact expression of the error in Taylor's Theorem and ultimately are just souped up versions of the Mean Value Theorem: $f(b) = f(a) + (b-a)f'(c)$.  However, in general you don't know what $c$ is, so the only obvious way to use these expressions is to get an upper bound for the error by bounding $|f^{(n)}(x)|$ on $[a,b]$.  Thus although in some sense the exact expressions look more elegant, in practice I don't know a single situation in which it is helpful to the have them rather than the corresponding inequality.  I would be very interested to learn of such an instance.
Comment: An error bound for a **generic Riemann sum** $A$ is missing too usually$$\left|\int_a^bf-A\right|\le(b-a)^2M\frac 1n$$
Comment: @Tony: Yes, that's true.  I should probably incorporate that into my answer.
Answer: There are error calculations for most of the standard numerical methods of computing integrals, such as the trapezoid rule, Simpson's rule, and yes, even the rectangle rule.  These depend both on n and f.
For example for the trapezoid rule the error term on the interval [a,b] can be given as :
  E = $\frac {(b-a)^3}{12n^2} f''(c)$  where $a \le c \le b$.
This formula assumes that f'' exists.  So for your case the error above would work.  Of course you have to assume something about f, or there is no way to bound the error.  Assumptions based on the smoothness of f are natural; also easy to prove.
There is a nice write up of this material here: http://pages.cs.wisc.edu/~amos/412/lecture-notes/lecture19.pdf.
Comment: Thank you, is there a particular c I should use in this case?
Comment: The essence of the situation is that we don't know what c is, only that it exists. In the case of f(x) = $x^2$ you know f'' = 2, so the c is irrelevant.  In other cases, you can try to bound the entire expression, which means looking carefully at your f.

Riemann sums and optimization


GLM instead of Mann-Whitney

Question: stats newbie here so please make replies easy to understand!
With the type of data I generate, my colleagues use Mann-Whitneys to generate p values.
However,
I did a stats course recently and we were told there was no longer any excuse to do:
• ANOVA, routinely transforming data to make residuals normal
• non-parametric stats (M-W, K-W, runs, etc) to cope with complex parametric structures
and instead we had to use GLMs.
I have data where my independent variable is a categorical/discrete variable and my dependent variable is continuous variable (non-integer)
I have tried to do a linear model but my data is non normal, and my residuals vs fitted in R looks like this:

I have tried transforming the data using lny 1/y sqrt(y) and it doesn't help at all.
From my stats course the next step would be to try a generalised linear model,
The only GLMS I know are poisson and negative-binomial but my data aren't integers. Any suggestions on what I can do?
Comment: 1. Your plot is not residuals vs fitted. That's a scale-location plot. 2. Please describe your response variable in more detail.  Its not even clear whether your response is ordered.  3. The use of the phrase 'no longer' in your course seems odd in relation to the use/availability of glms. Software for fitting GLMs has been around for more than 45 years, whereas that phrase seems to suggest they're fairly  recent.
Answer: First of all, while the advice you received in your course reflects the general sentiment of the stats community, there is nothing wrong with a transformation, and due to simplicity, I would prefer this solution to a GLM in your case. So, before you move to complicated GLM / GLS solutions, try e.g. if https://www.rdocumentation.org/packages/forecast/versions/8.13/topics/BoxCox produces a model with acceptable residuals.
If this is not the case, you can move to either GLM (changing distributions) or GLS (modelling variance), or both.
From eye-balling your data, it seems as if you have a strictly positive response. In this case, you could try a Gamma GLM, and possibly add a variance term (e.g. in glmmTMB) if you still have variance problems (for checking the latter, you can use e.g. https://cran.r-project.org/web/packages/DHARMa/index.html).
Alternatively, to avoid the entire distributional problem, you could just run a quantile regression, e.g. https://cran.r-project.org/web/packages/qgam/index.html
Comment: Hi thank you so much this is so helpful. I did a Box Cox transformation, using boxcox() in r and plotting it to see my optimal value for lambda. I then did the lm based on the boxcox transformation but my residuals vs fitted looks pretty much exactly the same as the one in the original post. So should I move onto lm now?
Comment: Yes, if you cannot find a suitable transformation, I would move to the options I mentioned.

GLM instead of Mann-Whitney


How do we calculate the gradient from numerical data

Question: I have a 3D irregular mesh. Each coordinate point holds a specific data, say , for example, temperature.
For a specific point, how do I calculate the temperature gradient ?
Comment: You have a reasonable answer below (not from me btw) - you might consider accepting.
Answer: Ideally in this scenario, you would not have the data just at the mesh points, but would rather be doing some kind of a finite element type construction, which would give you the temperature as a function on each mesh element. Then the gradient would be given as part of the profile (i.e. you could analytically differentiate at every point except perhaps points on the boundary of a mesh element).
The next best thing would be data points on a regular mesh. Then you could approximate the partial derivatives directly, and assemble them into the gradient vector.
Since you don't even have that, you can do the following. At a mesh point $x$, find three nearby mesh points $x_1,x_2,x_3$. Then $\frac{f(x)-f(x_i)}{\| x-x_i \|}$ should be approximately the directional derivative of $f$ at $x$ in the direction $x-x_i$. That directional derivative is exactly $\nabla f(x) \cdot \frac{x-x_i}{\| x-x_i \|}$.
Now assume for numerical purposes that
$$f(x)-f(x_i)=\nabla f(x) \cdot (x-x_i).$$
This is a system of three linear scalar equations in three scalar unknowns (namely the components of $\nabla f(x)$). Provided the $x-x_i$ are linearly independent, it has a unique solution, so you can solve it in principle. Depending on how irregular your mesh is, you could have issues with numerical instability.
Comment: I cant use a finite element type method using the shape functions as my values are derived quantities, i have 1 value per element (esp at the integration point).

I have found one more method using the weighted difference formula involving surrounding points
Comment: @prithivirajanVeerappan 1. I know that you can't always use a finite element type method. It is just preferable in these kinds of situations. 2. That weighted difference formula can be derived using the procedure I just suggested.


Presentability of natural numbers (number theory)

Question: Let $a,b,c,n \in \mathbb{N}$ with $gcd(a,b) = 1$ and $ab = c^n$.
Task: Show that there exist $d,e \in \mathbb{N}$, so that $a = d^n$ and $b = e^n$.
My thoughts about that are, that because of $gcd(a,b) = 1$ the following equality holds: $$ax + by = 1 \text{ with some } x,y \in \mathbb{Z}$$
Now I tried different approaches, e.g. writing $a = \frac{c^n}{b}$ and substituting this in the equality above. This lead to $$\frac{c^n}{b}x + by = 1 \iff b^2y - b + c^nx = 0$$
But putting this into the p-q-formula, I can't show the b is of the desired form and in analog way, that a if of the desired form.
I also tried multiplying the whole equality with b and got
$$abx + b^2y = b,$$ this lead me to $c^nx + b^2y = b$ and so to the same equality like above.
What could be a constructive approach to get this task solved?
Comment: Frequently, $\gcd(a,b) = 1$ is applied with expressing $a$ and $b$ by their prime factorizations, since the gcd tells you no prime appears (with positive power) in both factorizations.
Answer: Let $d=(a,c)$. In particular $d|a$, and so $(d,b)=1$. We also have $d^n|c^n=ab$, and since $d^n$ is coprime to $b$ it follows that $d^n|a$.
Now, we clearly have $(\frac{a}{d}, \frac{c}{d})=1$. Hence also $((\frac{a}{d})^n, (\frac{c}{d})^n)=(\frac{a^n}{d^n}, \frac{c^n}{d^n})=1$. But note that both $\frac{a^n}{d^n}$ and $\frac{c^n}{d^n}$ are divisible by $\frac{a}{d^n}$. Thus $\frac{a}{d^n}=1$, and so $a=d^n$. And then:
$b=\frac{c^n}{a}=(\frac{c}{d})^n$
Comment: @X3nius If $g$ is a common divisor of $d$ and $b$ then since $d|a$ it is also a common divisor of $a$ and $b$. So we must have $g=1$, since $(a,b)=1$. As for the second question: since $d|c$ we can write $dm=c$ for some $m\in\mathbb{N}$. Then $d^nm^n=c^n$, and so $d^n|c^n$.
Comment: I've got two questions:
1st) Can we say that $(d,b) = 1$ because if $(d,b) > 1$, then we would have a common divisior $g > 1$ that divides $d$ (and so $a$ because of $d|a$) and that divides b because of $d|b$, and this would be a contradiction to $(a,b) = 1$?

2nd) Why do we have $d^n \mid c^n = ab$?


ImageView Scanning Simulation Animation

Question: Briefly,
Here's my Background.

I want to simulate scanning view, first, I'll see a RECT of

[ 0 , 0 , DisplayWidth , DisplayHight ]

After a static period of time I'll translate smoothly to another RECT of

[ 0 + Ratio , 0 + Ratio , DisplayWidth + Ratio ,
  DisplayHight + Ratio ]

I know I'm still have lots of logic to be added to my following code, but I'm asking about another solution with HIGHER performance!
<code>final Bitmap Picture = BitmapFactory.decodeResource(getResources(),
        R.drawable.world);
final ImageView Background = (ImageView) findViewById(R.id.Background);
Display Display = getWindowManager().getDefaultDisplay();
Point Size = new Point();
Display.getSize(Size);
final int Width = Size.x;
final int Height = Size.y;

final int[] Motion = new int[1];
final Handler Handler = new Handler();
final Runnable Runnable = new Runnable() {
    @Override
    public void run() {
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                Console.Record("Running");
                Background.setImageBitmap(Bitmap.createBitmap(Picture, 0 + Motion[0], 0 + Motion[0], Width + Motion[0], Height + Motion[0]));
                Motion[0]++;
            }
        });
        Handler.postDelayed(this, 1);
    }
};
Runnable.run();
</code>
Answer: use custom View to display moving world map? you would then avoid creating a new Bitmap for each frame of animation

ImageView Scanning Simulation Animation


Printing taxonomy terms in node.tpl

Question: I am trying to print taxonomy terms using this code:
<?php foreach ((array)$taxonomy as $item) { ?>
<a href="<?php print base_path() . "taxonomy/term/" . $item->tid; ?> "class='tags'><?php print $item->name ?>,
<?php } ?>

It works fine if I put in contemplate, however it doesn't in the full node (node.tpl.php), the tags are empty in the full node. Can any one tell me what is the reason for this and how to fix this?
Comment: Are you sure the $taxonomy variable is populated in node.tpl.php of your theme?
Comment: No, how can I do this?
Comment: Place this:
 value of tax var is:
In your node.tpl.php page. That is prob. the easiest way. Unless you have Devel module installed, if so just use  in the template instead.
Answer: First verify that the $taxonomy variable is available and populated in your node.tpl.php file. ...
EDIT:
Your accessing the values in your $item variable like it is an object but your casting to an array. Try accessing as an array:
 <a href="<?php print base_path() . "taxonomy/term/" . $item['tid']; ?> "class='tags'><?php print $item['name'] ?>,

Comment: OK, the variable is available, however still it is not printed..
Comment: Nothing..It seems liek some declaration is missing as the code works correctly when (if teaser) but doesnt work wjen (if full node)...
Comment: @Vafello Wait. Can you post the results of the dsm print of the $taxonomy variable?  It may be that you have to specify which Vocabulary you want printed.

Printing taxonomy terms in node.tpl


Characteristic polynomial of square of matrix?

Question: Let $A \in M_n(\mathbb{C})$ with $p_A(x)=(-1)^n\cdot (x-λ_1)\dots(x-λ_n)$ it's characteristic polynomial.
Prove that $p_{A^2}(x)=(-1)^n\cdot (x-λ_1^2)\dots(x-λ_n^2)$
Any tips on where to start from?
Comment: Hint: $A^2-x^2I=(A-xI)(A+xI)$, hence $\det{(A^2-x^2I)}=\det{(A-xI)}\det{(A+xI)}$.
Answer: The characteristic polynomial of a matrix has roots that are equal to its eigenvalues. Let $x$ be an eigenvector of $A$ with corresponding eigenvalue $\lambda$.  Then
$$
AAx = A \lambda x = \lambda^2 x.
$$
So $x$ is an eigenvector of $A^2$ too, but its eigenvalue is $\lambda^2$.  So the roots of the characteristic polynomial of $AA$ are the squares of the roots of $A$.  Its characteristic polynomial is completely determined by this fact.


I broke remote access when configuring an AD-DS

Question: I bought a VPS and today is the first day. I admit that Im new on server administration and yay, I broke it on my first day.
The objetive of this question is to learn from my errors, dont wan't to break it twice.
I have a domain (domain.com) and I wanted to access my websites using the domain. the company of the VPS doesn't provide me any DNS so I was told that I needed an Active Directory Domain Service.
Ok, I picked up a guide, I configured a new forest, used my domain.com domain, said that is W2008R2 and clicked next until the wizard ended. Next, it configured a DNS service and rebooted the machine. Since that, I lost the remote access to my machine. It says that is powered off, without network or without remote access. The machine is on, I can see my IIS hello world.
So, what happened and how can I prevent that?
If is not much to ask: the AD-DS is needed just to redirect my domain to my IIS? Or Im wrong?
Comment: You've confused AD domains with internet domains.  They aren't interchangeable concepts here.  AD domain s are a authentication and configuration tool, while internet domaina and for name resolution.  No need to add AD to serve up web pages.  You need IIS for that.
Answer: You can run a DNS server on Windows (either the built-in or third party) without having an Active Directory infrastructure. In fact, unless you have a compelling need to have Active Directory I would recommend against it. (Personally, I'd use a third-party DNS provider with nice geographically distributed servers, etc.)
Promoting the machine to being a domain controller shouldn't have disabled RDP access, though. I'm at a loss as to describe what happened there, unless the VPS provider has somehow "customized" the load of Windows to act differently than stock Windows Server 2008 R2.
If I were in your position with your experience-level I'd probably start with asking the VPS provider to return the machine back to the default configuration. That ought to be something very easy for them to do.
If you can access RPC on the machine you could probably use the Remote Server Administration Tools to get RDP running again (though, realistically, you should block RPC from the Internet, along with all other unnecessary traffic in a production configuration).
Comment: Thanks for the response Evan (and uSlackr for the comment). I was told that I needed AD. the vps provider have reinstaled the Windows.I will try to no disturb them again :P About the DNS thing, I will make a new question :) For now, vote up + accept answer.


How to decide what voicing/register to use when writing an accompaniment?

Question: Today I wrote this piano accompaniment to a melody. I first filled in the bass notes and tweaked it a bit to find the best possible bassline. Thereafter, I just chose a random register that sounded good on the piano and filled in the right hand chord shapes trying to keep the voice leading smooth going from one chord to the next. I like it but I am not sure if I should have voiced the chords differently? Perhaps higher or lower so that the vocal line would stand out more. What are some guidelines and tips regarding this? Is what I have done ok or would a better register have been more suitable and why?
Comment: Too broad? I have already filled in chords. how can it be too broad to tell me why another voicing would work better than the one I have chosen?
Comment: lets say sing it or have a solo player play it on another instrument.. the melody is separate by a different instrument
Comment: This question is too broad IMO. The guidelines & tips are what you're taught in a composition course. And that's literally just the basics. There's just too many topics to cover.
Comment: Are we meant to play the melody and the accompaniment on the same instrument at the same time, or sing the melody and play the accompaniment at the same time?
Answer: Maybe it´s not extrictly speaking the same, but SATB writing and accompaniments follow the same principal of spreading voices in search of independence, specially when talking of the main voice.
Personally I would take the melody an octave up. I was starting to work it but suddenly I end up making barely the same as you but an octave lower, so yeah here´s wonderwall:
Comment: Thank you but the whole point was to use that register and that melody as specified in the OP. SATB writing is not the same as an accompaniment because the melody is the soprano in SATB. This is not the same as what I did
Comment: Ok but the main focus should be the melody and that can be made just by raising the volume up. At the end of the day what really matters is what sounds good for you. In my case I´m more used to hear melodies in a higer register. That doesn´t mean there couldn´t be concertos for tubas and basses. It´s up to you.
Comment: @armani I agree that changing the octave is probably not strictly within the constraints of the exercise, but it's not clear from this question that this _is_ an exercise.  Not everyone has read your earlier question.  Anyway, it's certainly possible to write an SATB accompaniment for a melody without having the soprano part carry the melody.  For example, a choir can accompany a solo singer or even instrumentalist, or SATB voicing can be used for a band accompanying a soloist.  Whether or not this is within the bounds of the exercise, it is certainly an interesting exercise in itself.
Comment: so when the SATB choir is acting as the accompaniment the role of the soprano no longer is to provide the melody right? The soloist or singer will have that role and the whole choir will be further away more like a background instrument right?
Comment: That's what happens in most of concertos, like in Tchaikovsky's Violin concerto in D, where the main violin is different to the soprano violín on the SATB. Here and in many other concertos the main violin is higher than the rest of the SATB accompaniment, but again bass concertos are a thing, maybe less popular but still existent.
Answer: You don't need to worry about the piano accompaniment being higher than the solo melody.  The tune will come through.  When a song is published for male or female voice, an octave apart, the piano part remains the same.   And when accompanying a sonata for string bass, the piano isn't restricted to the bottom octave of the keyboard!
Write each instrument where it sounds good.
Comment: @phoog Probably as much about levels and choice of stops as about registers.   Was the music intended to be played on organ?
Comment: "You don't need to worry about the piano accompaniment being higher than the solo melody": I recently attended a recital by a truly excellent flutist playing various historical flutes.  The accompanist played a small pipe organ.  The slow movement of a sonata played on a baroque transverse flute was ruined because the solo part was in its lower reaches, and the organ was playing well above the flute, obscuring it almost entirely in places.  It may frequently be acceptable for the accompaniment to be higher than the solo part, but it can also be a problem at times.
Comment: A solo flute playing in its lowest register could easily be drowned out by a carelessly composed and/or arranged and/or performed accompaniment.


Server attack, how to fix it

Question: Looks like the server is attacked. Contents of /var/log/auth.log are as follows. Its trying to ssh with all these username, how can I shut it off. The server is Ubuntu.
    Feb 22 16:29:15 server sshd[23413]: Failed password for invalid user mirror from 22.214.171.124 port 43881 ssh2
Feb 22 16:29:15 server sshd[23414]: Failed password for invalid user justice from 22.214.171.124 port 43882 ssh2
Feb 22 16:29:15 server sshd[23416]: Failed password for invalid user london from 220.127.116.11 port 43885 ssh2
Feb 22 16:29:15 server sshd[23415]: Failed password for invalid user justice from 126.96.36.199 port 43884 ssh2
Feb 22 16:29:17 server sshd[23421]: Invalid user oxford from 220.127.116.11
Feb 22 16:29:17 server sshd[23421]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:17 server sshd[23421]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.8.131.52
Feb 22 16:29:17 server sshd[23422]: Invalid user london from 188.8.131.52
Feb 22 16:29:17 server sshd[23422]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:17 server sshd[23422]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.8.131.52
Feb 22 16:29:17 server sshd[23424]: Invalid user london from 18.104.22.168
Feb 22 16:29:17 server sshd[23424]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:17 server sshd[23424]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=18.104.22.168
Feb 22 16:29:17 server sshd[23423]: Invalid user mirror from 126.96.36.199
Feb 22 16:29:17 server sshd[23423]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:17 server sshd[23423]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.8.131.52
Feb 22 16:29:19 server sshd[23421]: Failed password for invalid user oxford from 22.214.171.124 port 43959 ssh2
Feb 22 16:29:19 server sshd[23422]: Failed password for invalid user london from 18.104.22.168 port 43962 ssh2
Feb 22 16:29:19 server sshd[23424]: Failed password for invalid user london from 126.96.36.199 port 43967 ssh2
Feb 22 16:29:19 server sshd[23423]: Failed password for invalid user mirror from 22.214.171.124 port 43964 ssh2
Feb 22 16:29:20 server sshd[23429]: Invalid user pacific from 18.104.22.168
Feb 22 16:29:20 server sshd[23429]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:21 server sshd[23429]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.108.40.206
Feb 22 16:29:21 server sshd[23430]: Invalid user mirror from 188.8.131.52
Feb 22 16:29:21 server sshd[23430]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:21 server sshd[23430]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=18.104.22.168
Feb 22 16:29:21 server sshd[23432]: Invalid user oxford from 126.96.36.199
Feb 22 16:29:21 server sshd[23431]: Invalid user mirror from 126.96.36.199
Feb 22 16:29:21 server sshd[23432]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:21 server sshd[23432]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.108.40.206
Feb 22 16:29:21 server sshd[23431]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:21 server sshd[23431]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.8.131.52
Feb 22 16:29:22 server sshd[23429]: Failed password for invalid user pacific from 184.108.40.206 port 44073 ssh2
Feb 22 16:29:22 server sshd[23430]: Failed password for invalid user mirror from 126.96.36.199 port 44078 ssh2
Feb 22 16:29:23 server sshd[23432]: Failed password for invalid user oxford from 18.104.22.168 port 44082 ssh2
Feb 22 16:29:23 server sshd[23431]: Failed password for invalid user mirror from 18.104.22.168 port 44079 ssh2
Feb 22 16:29:24 server sshd[23437]: Invalid user pizza from 220.127.116.11
Feb 22 16:29:24 server sshd[23437]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:24 server sshd[23437]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=220.127.116.11
Feb 22 16:29:24 server sshd[23438]: Invalid user oxford from 184.108.40.206
Feb 22 16:29:24 server sshd[23438]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:24 server sshd[23438]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.108.40.206
Feb 22 16:29:24 server sshd[23441]: Invalid user oxford from 126.96.36.199
Feb 22 16:29:24 server sshd[23441]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:24 server sshd[23441]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=22.214.171.124
Feb 22 16:29:24 server sshd[23440]: Invalid user pacific from 22.214.171.124
Feb 22 16:29:24 server sshd[23440]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:24 server sshd[23440]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=220.127.116.11
Feb 22 16:29:26 server sshd[23437]: Failed password for invalid user pizza from 126.96.36.199 port 44173 ssh2
Feb 22 16:29:27 server sshd[23438]: Failed password for invalid user oxford from 220.127.116.11 port 44184 ssh2
Feb 22 16:29:27 server sshd[23441]: Failed password for invalid user oxford from 18.104.22.168 port 44186 ssh2
Feb 22 16:29:27 server sshd[23440]: Failed password for invalid user pacific from 220.127.116.11 port 44185 ssh2
Feb 22 16:29:28 server sshd[23445]: Invalid user quality from 22.214.171.124
Feb 22 16:29:28 server sshd[23445]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:28 server sshd[23445]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=126.96.36.199
Feb 22 16:29:29 server sshd[23446]: Invalid user pacific from 188.8.131.52
Feb 22 16:29:29 server sshd[23446]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:29 server sshd[23446]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=184.108.40.206
Feb 22 16:29:29 server sshd[23448]: Invalid user pacific from 220.127.116.11
Feb 22 16:29:29 server sshd[23448]: pam_unix(sshd:auth): check pass; user unknown
Feb 22 16:29:29 server sshd[23448]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=22.214.171.124
Feb 22 16:29:29 server sshd[23450]: Invalid user pizza from 220.127.116.11
Feb 22 16:29:29 server sshd[23450]: pam_unix(sshd:auth): check pass; user unknown

Answer: Been brought up a few times I believe:
Securing SSH on Linux Ubuntu
Hundreds of failed ssh logins
BTW these attempts are VERY common, usually automated scripts.
Answer: Install denyhosts.
apt-get install denyhosts
Deny hosts is a daemon that watches your servers logs, generally /var/log/secure, for suspicious access patterns and if found, adds the IP addresses of curious visitors to /etc/hosts.deny causing sshd to block them outright.
It also has a mode that allows it to exchange the local block list with lists from other machines, in a way, crowdsourcing known bad IP addresses. Similar to the way RBL lists work for SMTP.
I would also recommend you disable keyboard-interactive authentication on your ssh daemon to prevent someone accidentally creating a test user account, with an easy to guess password.
Comment: I used to use fail2ban, but I liked deny hosts because it wasn't reliant on using iptables.
Comment: Yep, denyhosts is excellent.
Comment: fail2ban is another option
Answer:
Change SSH to a different port
Block the IP addresses for a short time and they might give up

Server attack, how to fix it


converting multiple text value in one column into different column

Question: I have a large df file from a survey platform.  the columns contains one question(as a title) with all potential answer. I'm trying to separate each of this answer column into a new set of columns (with the name of answer as the the name of the new column).
after that i would like each new column to indicate if the value appears in the original column(with "1") to process the data more easily
     df<-data.frame("name"= c("John","mark","bell","elsa"),"what do you like to
     eat"=c("apple","fries apple","peach","bread"))

original df

name
What.do.you.like.to.eat

John
apple

Mark
fries apple

bell
peach

elsa
bread

i'm using this code which works, but i'm sure there must be  more efficient/easier way of doing it, as i have a over 50 column like this.
    df<-df %>%
  separate(what.do.you.like.to.eat, c("apple","fries","peach","bread",NA ), remove = F)
df[,3:6]<-""
{
  df[,3] =  with(df, ifelse(grepl("apple", df$what.do.you.like.to.eat,ignore.case = T),
                                             paste('1', df[,3]),
                                             paste("", df[,3])))
  df[,4] =  with(df, ifelse(grepl("fries", df$what.do.you.like.to.eat,ignore.case = T),
                            paste('1', df[,4]),
                            paste("", df[,4])))
  df[,5] =  with(df, ifelse(grepl("peach", df$what.do.you.like.to.eat,ignore.case = T),
                            paste('1', df[,5]),
                            paste("", df[,5])))
  df[,6] =  with(df, ifelse(grepl("bread", df$what.do.you.like.to.eat,ignore.case = T),
                            paste('1', df[,6]),
                            paste("", df[,6])))
}

desire output

name
What.do.you.like.to.eat
apple
fries
peach
bread

John
apple
1

Mark
fries apple
1
1

bell
peach

1

elsa
bread

1
Comment: Honestly I'm struggling to understand it's kind of vague so could you provide a sample of your data and a sample of the expected result ?
Comment: just did, submitted by mistake with no example
Answer: Ok i've done this tell me if this works for you :
my_df <- data.frame("name" = c("John","mark","bell","elsa"),
                "what do you like to eat" = c("apple","fries apple","peach","bread"),
                stringsAsFactors = FALSE)
my_var <- unique(sort(str_split(string = my_df$what.do.you.like.to.eat, pattern = " ", simplify = TRUE)))
my_pos <- which(my_var == "")
if (length(my_pos)) {
  my_var <- my_var[-my_pos]
}
my_col <- c(colnames(my_df), my_var)
my_miss <- setdiff(my_col, colnames(my_df))
my_df[my_miss] <- NA
my_f <- function(x, y) {
  my_var <- grep(pattern = colnames(my_df)[x], x = my_df[, y])
  if (length(my_var)) {
    my_df[my_var, x] <<- 1
  }
}
lapply(3:ncol(my_df), function(x) my_f(x, 2))

you can change this part to this :
my_df <- data.frame("name" = c("John","mark","bell","elsa"),
                "what do you like to eat" = c("i like apple","i love fries apple","i'm kind of peach","bread all the way"),
                stringsAsFactors = FALSE)
my_var <- unique(sort(str_split(string =
my_df$what.do.you.like.to.eat, pattern = " ", simplify = TRUE)))
my_food <- c("apple", "fries", "bread", "peach")
my_var <- my_var[which(my_var %in% my_food)]
my_pos <- which(my_var == "")
if (length(my_pos)) {
  my_var <- my_var[-my_pos]
}

Comment: it's work great with this example, so thanks. the problem is that in the original df the answer are mainly  distinct string like "i like to eat apples";" i like bread very much"; etc. is there a way to define my_var so it could detect such string, or does it have to be added manually to this variable?
Answer: You could use purrr::map to apply over your vector of answers and for each, check their presence in the string.
library(tidyverse)

df <- data.frame(
  name = c("John", "mark", "bell", "elsa"),
  "what do you like to eat" = c("apple", "fries apple", "peach", "bread")
)

ans <- c("apple", "fries", "peach", "bread")

map_dfc(ans,~ transmute(df, !!sym(.x) := str_detect(what.do.you.like.to.eat, .x))) %>%
  bind_cols(df, .)
#>   name what.do.you.like.to.eat apple fries peach bread
#> 1 John                   apple  TRUE FALSE FALSE FALSE
#> 2 mark             fries apple  TRUE  TRUE FALSE FALSE
#> 3 bell                   peach FALSE FALSE  TRUE FALSE
#> 4 elsa                   bread FALSE FALSE FALSE  TRUE

Comment: thanks this works great, also with strings. is there an option to bind the outcome column with the bind_cols  function,  at specific location in the data frame.? (similar to : dplyr::relocate(ans, .after = name)


Parse doesn't work in app release version, but works on simulator or when installed on device via xcode

Question: I have a parse deployed in Heroku for my app. When I run app in simulator or on my test device after installing via xcode, everything works fine. I can fetch data and see images downloaded on initial view controller. But, whenever app is on app store review or being installed via Test Flight, Parse just stops working. I can't fetch any data and app becomes almost useless.
The code looks like this:
class ListOfCategoriesViewController: UIViewController {

@IBOutlet weak var tableView: UITableView!

var categories: [DisplayCategory] = []

override func viewDidLoad() {

    super.viewDidLoad()

    // loader
    let spinner: UIActivityIndicatorView = UIActivityIndicatorView(activityIndicatorStyle: .Gray)
    spinner.center = self.view.center
    spinner.startAnimating()
    view.addSubview(spinner)

    //setting text
    self.title = "Explore"

    let query = PFQuery(className: "Category")

    query.findObjectsInBackgroundWithBlock { (result: [PFObject]?, error: NSError?) -> Void in

        self.categories = result as? [DisplayCategory] ?? []
        spinner.stopAnimating() // loader stops here and code below never executes

        for category in self.categories {

            category.imageCategoryFile?.getDataInBackgroundWithBlock { (imageData: NSData?, error: NSError?) -> Void in

                let image = UIImage(data: imageData!, scale: 1.0)
                category.imageCategory = image

                self.tableView.reloadData()
            }
        }
    }
}

DisplayCategory class looks like this:
class DisplayCategory: PFObject, PFSubclassing {

@NSManaged var imageCategoryFile: PFFile?
@NSManaged var titleCategory: String?
var imageCategory: UIImage?

static func parseClassName() -> String {
    return "Category"
}

Any ideas what might cause this issue here? Appreciate your help.
Answer: Try adding the following in your Info.plist:
<key>NSAppTransportSecurity</key>
<dict>
  <key>NSAllowsArbitraryLoads</key>
  <true/>
</dict>

Comment: Just checked. I have this in my info.plist. Any other ideas what might cause the problem?
Answer: The solution was trivial. I needed to register subclass in AppDelegate and call it in didFinishLaunchingWithOptions launchOptions:
So, it now looks like this in AppDelegate:
 func application(application: UIApplication, didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool {

    configureParse()

    return true

}

func configureParse() {
    DisplayCategory.registerSubclass()

}


Piano - Triplets turn into sixteenths

Question: I am having trouble with playing triplets at higher speeds (such as at the end of the Pathetique Sonata, 3rd movement). I find that when I speed things up to tempo that the triplets always turn into 16th notes in sets of 4 rather than remaining as groups of 3. Does anyone have any suggestions?
Comment: Do your 16th notes still start with that really high F and end with the B above Middle C? If yes, are you inserting chromatic notes in between? If you make sure to strictly follow the sheet music, I don't think you'd encounter this 16th-note problem unless you constantly make mistakes (like I do).
Comment: The trouble happens on the second C of the run when the thumb comes down and the cross to the 4 happens. So the beginning of the third set of triplets. The notes are never wrong, only the timing is wrong.
Answer: If I were having this problem, I would approach it with the following four-step process:
1. Use the beats as anchors
By replacing pitches that aren't on the beat with rests, we can focus to make sure we know where these beats are to be placed. It's also important that you use accurate fingering here, so I've included what my fingerings would be:

(Note that I've kept the entirety of the beat-4 pickup, but you can adjust that as necessary.)
2. Add to that downbeat profile
Now that you have the on-the-beat pitches placed correctly, add to it by adding in the third pitch of each triplet:

Again, make sure you're using your eventual fingering when you do this, or else you'll be practicing something different from what you'll perform.
3. Add again to that downbeat profile, but differently
We have the first and third pitch of each triplet grouping, but we haven't yet focused on the second. So now, instead of playing the first and third, we play the first and second:

4. Now put it all together
And play it as written!

A Final Thought
This excerpt is really just a C harmonic minor scale in triplets. As such, take some time warming up each day by playing scales in triplets. Most people typically practice scales in sixteenth notes, and this may be the ultimate source of your issue: since you're so used to playing scales in sixteenth notes, that muscle memory is taking over. So incorporate some triplet scales into your routine to make this feel more normative, and this triplet scale in the Beethoven should be no sweat.
Comment: I will try these. The trouble happens on the second C of the run when the thumb comes down and the cross to the 4 happens. So the beginning of the third set of triplets.
Answer: First, check your fingering. It may be that you are playing the passage so that your strongest fingers are striking each fourth pitch and resulting in an accent.
Then, practice slowly, placing a light pulse on the first note of each triplet. Listen as you go to make sure the accented and unaccented notes (i.e., first note of each triplet versus the other two) sound "in context" (no unwanted accents).
Once you have the feel of that at a slow tempo, gradually speed up. Be very careful to listen along the way, because as we speed up, the tendency is for our muscle memory to kick in, and presumably you're current muscle memory is producing the inappropriate accents.
Answer: Richard's answer is great. I have no criticisms of it, only another trick to add to your practicing vocabulary: break the motif into groups. I'll share some images to show what I mean more clearly, but before I do I'll explain in words. Beethoven's motif goes Mi Re Do - Fa Mi Re - Do Ti La - So Fa Mi - Re Do Ti - Do; I'm using the hyphens to indicate the triplet phrasing. My practice technique is to change the grouping—play the first two notes quickly together, then the next two notes, and so on; then go in groups of three quickly, and so on; then groups of four, groups of five, and you can go as high as you want. You can also change which note each groups starts on; for example Mi - Re Do Fa - Mi Re Do, etc. You can practice each grouping at slow tempos, medium tempos, and fast tempos, in that order, until you're comfortable.

Piano - Triplets turn into sixteenths


Por que o NetBeans avisa que deve não acessar variáveis globais diretamente?

Question: O NetBeans sugere que não acessemos as variáveis globais do PHP tipo $_SERVER[''] diretamente, qual a sugestão neste caso?
Normalmente é feito assim:
<?php

   $ip = $_SERVER["REMOTE_ADDR"];

?>

Qual a forma aconselhada de se pegar uma variável global?
Comment: O netbeans não argumenta mais nada, só isso? acho que essa recomendação pode mudar conforme o contexto.
Answer: Não faço a menor ideia mas não vejo problemas em fazer isto. Se fosse acesso a um $_POST eu até entenderia que a sugestão seria para filtrar o conteúdo com um filter_input ou técnica semelhante. Mas neste caso não pode acontecer falhas de segurança.
Eu já vi sugerirem filtrar até mesmo este tipo de variável, mas me parece insano. Se você não puder confiar no que o servidor HTTP te passa você está lascado.
Posso estar enganado mas eu chutaria que é um falso positivo, que é comum em analisadores estáticos de código.
Tem 4 caminhos:

Conviva com isto;

desligue o analisador estático;

colocar um comentário de dica para ele parar de avisar isto (tem que procurar na documentação);

contrariadamente faça o filtro e satisfaça o analisador.
  filter_input(INPUT_SERVER, 'REMOTE_ADDR')

Coloquei no GitHub para referência futura.
Comment: Pois é também acho que o Netbeans está neurótico com relação a segurança, vou seguir a dica para filtrar, não custa nada.
Answer: No exemplo da pergunta não existe nenhum problema, informações da requisição, sessão e do servidor no PHP(puro) vem através de variáveis globais, em outras
linguagens essas infomações vem através de objetos como request do java, frameworks PHP também fornecem objetos para manipular essas informações.
O que é não muito correto fazer é acessar/manipular uma variável global dentro de uma função, o correto é passar essa global como argumento da função, para evitar efeitos colaterais como a quebra de funcionamento, uma função não deve sofrer interferencia de nada externo deve apenas depender dela mesma.
Answer: Existe a recomendação de não utilizar a variável global $_SERVER devido o risco de cross-site scripting que é alguém conseguir executar scripts no seu servidor ou, em outras palavras, injetar código.
Conforme descrito neste artigo (em inglês).
Quando exposto ao cliente (em formulários, por exemplo) pode dar aberturas à ataques e invasões, por isso é recomendado o uso do filtro.
Answer: /* Esta é a maneira correta de se declarar uma superglobal */
$post = filter_input_array(INPUT_POST, FILTER_DEFAULT);
$get = filter_input_array(INPUT_GET, FILTER_DEFAULT);

/* Esta é a maneira correta de se atribuir uma variável a uma informação oriunda  de uma superglobal */
$nomeDoname = $post["nomeDoname"];
$nomeDoget = $get["nomeDoget"];

Answer: Eu já vi o W3School recomendando o uso desta forma:
htmlspecialchars($_SERVER["PHP_SELF"]);

Você pode ver a explicação aqui, procure por esse título:

How To Avoid $_SERVER["PHP_SELF"] Exploits?


nav menu styling issues

Question: I have been given a psd of a nav menu and im trying to recreate it using css and html.
I have set up a jsfiddle instead of posting my code on here so people can mess about with it.
http://jsfiddle.net/XdefV/
The code in the js fiddle is my attempt, but as you will be able to see its a little bit off.
What im trying to recreate is this:

Can anyone help me better recreate the image?
Comment: I see it perfect. What change do you expect?
Answer: This Fiddle should get you on the right track.
Answer: here you go.  Much closer:
http://jsfiddle.net/XdefV/4/
Answer: You can try this.
(i'm basing this on your 968px image width)
http://jsfiddle.net/XdefV/6/

nav menu styling issues


Using NDMP as an alternative to CIFS mount

Question: I have a weird but interesting use-case. I use CIFS to mount shares from a File Server (NetApp, EMC etc) to an application server (win/linux server where my application runs). My application needs to process each of the file from the shares that I mount via CIFS. My application also needs access to the meta-data of these files such as Name, Size, ACLs etc.
I would like to see if I can achieve the same via NDMP. I have some very basic questions regarding this use-case. It would be great if you guys can help me out here.

Is this even something which is achievable?
Can I just transfer share that interest me instead of entire volume?
Answer:
1.Is this even something which is achievable?

Theoretically yes, practically no - not unless you want to entirely rewrite your code using a wildly inappropriate API - but technically yes you could, but it's an inappropriate method.

Can I just transfer share that interest me instead of entire volume?

Certainly NDMP does let you backup/restore whole shares instead of whole volumes yes, so if you wanted to do this you would have that level of granularity, but it would still be like completing every third page of your tax return in Latin - achievable but kind of pointless.


Multiple imputation and normality assumption

Question: I am reading the an online book by Stef Van Buuren (link at bottom) regarding multiple imputation. In Section 3.2.1 he lists 4 different approaches to multiple imputation:

Later on in Section 3.3 he states:
"The imputation methods discussed in Section 3.2 produce imputations drawn from a normal distribution."
I'm struggling to see how this is the case. Obviously I can see the error term added is drawn from a normal distribution. But surely the distribution of the imputations depend on the distribution of each of the predictor variables in X? Seeing as it is a linear regression, I would have thought the distribution of Y would be a combination of all of the distributions of the X's, as well as the error term.
SciKit learn's IterativeImptuter documentation also states:
"The version implemented assumes Gaussian (output) variables. If your features are obviously non-normal, consider transforming them to look more normal to potentially improve performance"
I feel like this is because the default option for IterativeImputer draws the regression coefficients from a gaussian posterior distribution.
Based on all of the above I have the following questions:

Where in the  above equations is normality assumed, and why do the equations "produce imputations drawn from a normal distribution".
Is the distribution of any imputed value determined by the model/equations used, which surely themselves depend on the distribution of X's, which are not necessarily normal?

Sci kit learnt docs:
https://scikit-learn.org/stable/auto_examples/impute/plot_missing_values.html#sphx-glr-auto-examples-impute-plot-missing-values-py
Stef Van Buuren book: https://stefvanbuuren.name/fimd/sec-linearnormal.html
Answer: Under the usual assumptions for a linear regression model, the coefficient estimates have a multivariate normal distribution, as described in the coefficient covariance matrix. The predicted response, as a sum of normally distributed random variables, thus also has a normal distribution (Method 1). Method 2 just adds yet another (independent) normally-distributed term. Similar arguments apply to the other Methods. That is, multiple predictions for any particular missing data case will have a normal distribution.
That said, I think that the first highlighted used of "normal" in the quote might be in the sense of an "ordinary" linear model rather than in the technical sense of a "normal distribution." Be careful before you put too much weight on particular uses of words like "normal" that have both technical and colloquial meanings. Even the most careful author aided by a rigorous editor can still produce a written product with ambiguous usage of such words.


embed wordpress or other php application to a django/python site?

Question: Is that possible to  embed wordpress or other php application to a django/python site ? I am running apache web server on linux...
Thanks
Answer: i figured it out myself.
Just redirect the subdomain of using php application to a different location at apache configure
if you like blog.yourdomain.com be a php applicaiton just at a subdomain
at apache configure
<code><VirtualHost *:80>
        ServerName   blog.yourdomain.com
        UseCanonicalName Off
        ServerAdmin  "email@example.com"
        DocumentRoot /var/www/vhosts/yourdomain.com/blogapplicaiton
</VirtualHost>
</code>
and put your php code in /var/www/vhosts/yourdomain.com/blogapplicaiton
you get all other web application through django but blog.yourdomain.com using php applicaiton.


How to decide between a docker volume container and a docker volume?

Question: After reading the docs I found myself somewhat confused as to how best to manage productive application/service data.
There seem to be 3 options:

Simply map volume to host directory (i.e. -v argument for docker run)
Create a docker container image for data (i.e. separate container and --volumes-from)
Creating a docker volume (i.e. docker volume create)

Now, it seems that the accepted practice is option #2, but then I wonder what is the purpose of #3.
Especially how do you correctly handle these scenarios with docker volume and is it better to use a data volume container or this for each situation?

You need application data in a separate volume and/or storage tier in your server
Backing up
Restoring data
Comment: @MichaelHampton Why?, data may not be dockerized but the host OS is still managed by an infrastructure team who monitors and backups
Comment: http://serverfault.com/q/576490/126632
Comment: @MichaelHampton I realized I should rephrase my question
Comment: #1 is not a serious option for production; it should basically never be done if an alternative exists.
Comment: @dukeofgaming Not to mention that you can run `btrfs scrub` on it to find and correct damaged files. I am not sure how dockerized stuff works, but I guess it does not protect against data rot, so I always need a full restore if something bad happens instead of just restoring individual files. Another thought that it adds another layer of abstraction, so it slows down file reading and writing even more. I somehow don't see the advantages of #2 and #3, but I am not experienced with docker, so this might change.
Answer: As of Docker 1.9, creating Named Volumes with the Volumes API (docker volume create --name mydata) are preferred over a Data Volume Container. As of February 2016, the Docker volumes documentation is woefully out-of-date. Folks at Docker themselves suggest that Data Volume Containers "are no longer considered a recommended pattern," "named volumes should be able to replace data-only volumes in most (if not all) cases," and "no reason I can see to use data-only containers."
Answer: I think #2 and #3 are pretty much the same thing, the main difference is that there is no stopped container with #3 (it is literally, just a named volume). For example, you can create a named volume and do similarly what you would do with #2 with -v instead.
Create a named volume:
$ docker volume create --name test

Mount and write some data to that volume from a container:
$ docker run -v test:/opt/test alpine touch /opt/test/hello

You can then mount that same test volume in another container and read the data:
$ docker run -v test:/opt/test alpine ls -al /opt/test
total 8
drwxr-xr-x    2 root     root          4096 Jan 23 22:28 .
drwxr-xr-x    3 root     root          4096 Jan 23 22:29 ..
-rw-r--r--    1 root     root             0 Jan 23 22:28 hello

The advantage here is that the volume won't accidentally disappear if you remove the data-only container. You now manage it with the docker volume sub-command.
$ d volume ls
DRIVER              VOLUME NAME
local               test

It also opens the possibilities for volume drivers down the road so you might be able to do shared volumes between hosts (ie. named volumes over NFS). Examples of this might be Flocker and Convoy. To your point specifically about moving or backing up data, Convoy has specific sub-commands for backing up data and allows for storage on NFS or EBS external to your host.
For this reason, I think the more new-school way (Docker 1.9+) is to use a named volume rather than a data-only container.
Comment: Thanks, you answered most of my questions, but the point on managing container data in a different physical volume tier is still unanswered and that is the critical one... suppose this is a git repo management solution and I need that part of the container data (which is a volume defined in a dockerfile) in Tier 0 storage located in a different physical host volume (i.e. another partition, physical disc, or whatever)
Comment: I _sort of_ did with the mention of volume drivers. Right now, to store data outside the physical local storage driver, you would need to use one that did specifically what you are looking to do. Off the top of my head, there is https://github.com/rancher/convoy and https://github.com/ClusterHQ/flocker. Convoy has support for NFS and GlusterFS at the moment which sounds closer to what you are after. I'll modify the answer to clarify this.
Comment: Using the devicemapper driver seems to answer my question, thanks! https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/
Comment: `the volume won't accidentally disappear if you remove the data-only container`. Could you elaborate ? Thanks.


Correct way of formatting dates for output in node theming

Question: I am theming a node where I need to show the created date. Since Drupal stores it as timestamp, what is the best method to retrieve it? I have found this post: http://drupal.org/node/101993. But the problem is that whe using this...
<?php
print date("D, j M, Y \a\\t G:i", $node->changed);
?>

... it delivers me the content in English. My portal default language is Portuguese. Is there a Drupal function that will deliver the timestamp as a regular date that takes into account the user's selected language? And  that maybe takes into account the preformatted dates set in the portal configuration? Thank you.
Answer: Instead of date(), try using format_date()
It has an otional last parameter $langcode that tells it what language code to use for the date.
Comment: Yes, thank you, I found the documentation here: http://api.drupal.org/api/drupal/includes--common.inc/function/format_date/7


Pager for basic pages in Drupal

Question: I am working on site, on site for certain set of basic pages i want to provide a numeric navigation with previous/next buttons. Is there a module to achieve it easily or else we need to hard code it.
Please suggest me some solutions to get this functionality.
Answer: Smart Paging have the option to navigation with previous/next links with numeric navigation see the below image and add a css style to prev/next class to achieve the button effect. See the demo

also worth checking this module Pagerer for providing a collection of pager themes to enhance Drupal and Views standard pagers. See Demo

Pager for basic pages in Drupal


The Normalization of Image gives black image

Question:
I want to Enhance Latent fingure print image for that i have do the
  normalization of image first I have Write the code for the
  normalization of fingureprint image but it gives output as totally
  black image I want know what is wrong in the code below is the java
  class.

package com.quality;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.awt.image.WritableRaster;
import java.io.File;
import java.io.IOException;

import javax.imageio.ImageIO;

class ImageArray
{
    double M;
    double VAR;
    BufferedImage img;
    int w;
    int h;
  double[][] imagedata;
  double [][] outputdata;

public ImageArray()
        {
            try {
                img = ImageIO.read(new File("Sample7.jpg"));
                w = img.getWidth();
                h = img.getHeight();
                imagedata = new double[w][h];
                outputdata = new double[w][h];
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }

        public void Mean()
        {

    for(int x=0;x<imagedata.length;x++)
    {
        for(int y=0;y<imagedata[x].length;y++)
        {
            Color color = new Color(img.getRGB(x, y));

            imagedata[x][y] = color.getRed();
        }
    }

    for(int x=0;x<imagedata.length;x++)
    {
        for(int y=0;y<imagedata[x].length;y++)
        {
            M+=(imagedata[x][y]);
        }
    }

    M =   M/(w*h);
    System.out.println("mean"+M);

    }

public void Variance()
        {

            for(int x=0;x<imagedata.length;x++)
            {
                for(int y=0;y<imagedata[x].length;y++)
                {
                    VAR+=Math.pow(imagedata[x][y]-M,2);
                }
            }

        VAR = VAR/(w*h);
            System.out.println("varience"+VAR);
        }

public void normalization(double mean,double varience)
        {
            int M0 = 100;
            int VAR0 = 100;

            for(int x=0;x<imagedata.length;x++)
            {
                for(int y=0;y<imagedata[x].length;y++)
                {
                    if(imagedata[x][y]>mean)
                    {
                        outputdata[x][y]=M0+(Math.sqrt(VAR0*Math.pow(imagedata[x][y]-M, 2)))/VAR;
                    }else
                    {
                        outputdata[x][y]=M0-(Math.sqrt(VAR0*Math.pow(imagedata[x][y]-M, 2)))/VAR;
                    }
                }
            }

        }

public void DrawImage() throws IOException
        {
            BufferedImage outputImage = new BufferedImage(w, h, BufferedImage.TYPE_BYTE_GRAY);

            for(int x=0;x<outputdata.length;x++)
            {
                for(int y=0;y<outputdata[x].length;y++)
                {
                    outputImage.setRGB(x, y, (int) outputdata[x][y]);
                }
            }

            Graphics2D g2 = outputImage.createGraphics();
            g2.drawImage(outputImage, null, null);

            File imageFile = new File("output.jpg");
            ImageIO.write(outputImage, "jpg", imageFile);

        }

 public static void main(String args[])
            {

                try {
                    ImageArray abc = new ImageArray();
                    abc.Mean();
                    abc.Variance();
                            abc.normalization();
                    abc.DrawImage();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
}

Answer: You may want to parenthesize the expression:
M =   M/w*h;

as
M =   M/(w*h);

Comment: Yes, same goes for VAR = VAR/w*h;
Comment: I was leaving that as an exercise for the student.  ;-)


How to make Feeds import only one post at a time when cron run?

Question: I have a website which will import many images through Feeds. But Feeds import too much posts which affect the bandwidth.
I want Feeds import only one post at a time when cron run.
The setup:

feeds-6.x-1.0-beta10
Drupal 6.20
Answer: There's no way (without hacking the Feeds module) to set this. You can affect how often the Feed updates, so if it's a feed with lots of posts set it to update more often and run cron more often, and that way fewer items will be imported per run because there won't be large amounts of new data to import each time.


VBScript. Error on data set return

Question: I don't have any experience programming with VBScript and I'd like to use some OOP approach. If you'll look at my code below, you will see that I'm trying to create DB access class with a set of methods, that returns data sets. Have no idea why, but receiving error:
Microsoft VBScript runtime error '800a01c2'

Wrong number of arguments or invalid property assignment

/TrashCan/library/BLL/CreditBLLClass.asp, line 18

I have an .asp page:

    <%
        Dim creditBLL
        Dim reader
        creditBLL = new CreditBLLClass

            reader = creditBLL.GetData()
            If reader.EOF = False Then
                Do While reader.EOF = False
                    Response.Write(reader.Fields("a").Value)
                    reader.MoveNext
                Loop
            End If
    %>

CreditBLLClass.asp:

<%
Class CreditBLLClass
    'Private variables
    Dim connection

    Public Default Function Init()
        Set Init = Me
      End Function

    Public Function GetData ()
        connection = new DatabaseConnectionClass
        GetData = connection.DBGetRecords("select a from b")
    End Function
End Class

%>

And database connection class
<%

Class DatabaseConnectionClass

'Private variables
dim pConnection

Public Default Function Init()

    Set Init = Me
  End Function

Public Function DBGetRecords ( sSQL )

    Set pConnection = Server.CreateObject( "ADODB.Connection" )
    With pConnection
            .ConnectionString = "string"
            .Open
         End With
        DBGetRecords = pConnection.Execute ( sSQL )
End Function

End Class

%>

Please tell me what am I doing wrong? Maybe there is a common approach of how to build data access layer?
Comment: Remember that VBScript was not made for OOP style programming. Though organizing your code in classes is possible and not a bad practice, don't force OOP on VBScript too much. You will run into problems / performance issues later on.
Comment: What are those `Init` methods doing?
Comment: can see that you're right... but before migrate web site to APS.NET have to refactor code somehow or it will take ages...
Answer: The error is in the invalid property assignment part of the error.
From your code:
connection = new DatabaseConnectionClass
GetData = connection.DBGetRecords("select a from b")

You have to use Set when using objects.
Change it to this:
Set connection = new DatabaseConnectionClass
Set GetData = connection.DBGetRecords("select a from b")

VBScript. Error on data set return


Como dar um refresh no context? ASP.NET

Question: Eu estou começando a desenvolver em ASP.NET e tenho sempre um pequeno problema.
Quando eu preciso gerar um contexto (Aquela classe que vai fazer a referência com o banco), exemplo abaixo:

Eu faço o seguinte procedimento: Botão direito em cima do projeto -> Add -> new item -> data -> ADO.NET Entity Data Model depois disso eu dou o nome da classe conecto com o banco e gero todas as entidades baseadas no meu banco...
Quando eu mudo ou acrescendo qualquer campo no meu banco, toda vez eu tenho que deletar essa classe e cria-la novamente, eu gostaria de saber se há uma maneira de dar um refresh, e ela atualizar com os novos campos que estão no banco?
Se alguém saber essa solução, por favor compartilhe comigo, porque será de grande ajuda e de grande produtividade, obrigado!!!
Answer: Em seu caso, estás utilizando o Database First, que consiste em gerar o modelo de acordo com as tabelas em seu banco de dados.
Você pode atualizar seu .edmx normalmente, basta seguir esses passos:

Clique com o botão direito em seu arquivo .edmx e vá em Update Model from Database.;
Basta seguir os passos normais e pronto, mas vale lembrar que algumas coisas podem não atualizar, isso devido a N fatores.

A Microsoft explica isso neste artigo e neste outro.

Como dar um refresh no context? ASP.NET


VS Xamarin Agent losing build host connection

Question: I'm having an issue for the 3rd time now! VS loses connection to the mac build host. Trying to remember how it happened the other times I think it might be related to updates. Yesterday I updated to Parallels 12 and the new OSX version, 10.11.6. Please take a look at my pictures and let me know if you've been in this situation and how you solved.
The other times I fixed by try-and-error, so I have no idea about what was the actual fix…
By the way, I can ping, remote login is enabled and mono is set as allowed in the firewall.

Ping mac host:

Remote login:

Firewall:

PuTTY SSH working:

Everything said here is done:
https://developer.xamarin.com/guides/ios/getting_started/installation/windows/#Configuration
Any ideas?
Thanks!
EDIT:
Well, from the moment I posted this question to now it started to work again. But instead of closing this answer I'm gonna leave it open until we can find a solution for this. I'm not the only one facing this issue and it would be cool to document it for the next time it happens.
Here's the connection working now... :
Comment: You are connected to the mac by cable or wifi?
Comment: @MarcosJoséPérezPérez Parallels
Comment: What if you connect by IP and not by DNS name?
Comment: Check the log/  help -> xamarin -> open logs.
-----------------------------------------------------------
Check you have the same version of Xamarin in the Mac OS and Windows.
Comment: @MarcosJoséPérezPérez cool, just one sec
Comment: @valdetero I'll check, just one sec
Comment: Well, it is working now (check my updated answer) but if you have more ideas I'm willing to check that. I'd rather get to the bottom of it this time.
Comment: @valdetero not the issue, I was able to ping the name :/
Comment: @MarcosJoséPérezPérez yes I do.
Comment: I think the problem is related with intermittent network.
Comment: @MarcosJoséPérezPérez but why only when I update the OS?
Comment: may be you are working with dynamic ip. I don't know, I have the last version of the captain and I don't have any problem.
Answer: We're including some reliability improvements on the connectivity between Visual Studio and the Mac build host as part of Xamarin for Visual Studio 4.2.0.
If you hit this issue again, please let me know what Xamarin version are you using, and also share the logs after the failing connection. (Help - Xamarin - Zip Logs).
If the issue is DNS related, you can try connecting by IP (adding the Mac manually) as suggested by @valtedero.
Another interesting test to diagnostic the issue is trying to build an iOS project from a developer command line, passing the ServerAddress, ServerUser and ServerPassword. That will try to connect to the Mac using the same mechanism used by Visual Studio without interference of editors, extensions or any other Visual Studio artifacts.
Finally, if you're not using the latest Xamarin version (currently 4.2.0) please give it a try, it does not only includes connectivity improvements, but it also improves the tracing around connectivity, which can give us valuable information to diagnostic other potential issues.
Thanks!
Comment: Thanks for the answer. Is this update available in the stable channel or only beta? The last version I have installed (even after force-checking) is 22.214.171.124
Comment: Hi @eestein, this update is currently only in Beta and Alpha channels (4.2.0.x). Please keep in mind that you will need to switch channel on the Mac machine as well.
Thanks!
Comment: Thanks for keeping this open.  I have had issues with Xamarin connectivity to the mac for almost two years; keep giving up and trying again...  using the latest everything, including Visual Studio 2017.
Comment: +1 for using msbuild without VS; that was able to connect properly for me and I learned that there were more issues on the Mac side.


How browser calculate size for elements with percents width or height

Question: How browsers calculate size for elements with percents width or height? Is it calculated based on size of the parent node? What if parent node does not have explicit size attributes?
Comment: Even if the parent element has no explicit dimensions set it will fit to the size of it's contents, therefore any other element that is set to 100% width will be 100% of the parents calculated width.
Answer: Yes it is calculated based on the parent node. Coming to your point if no explicit size has been given to the parent...Actually I have never done this so cannot guide you on that point.
But remember in case of font-size..if  you do not specify any size then browsers go for some pre-defined settings. Every browser has a base font size..When you set a custom value of font-size then the browser actually adds or subtracts the values from that base size...
In short:

Yes its calculated on the basis of parent tag.
Its not safe to not give a custom size and rely on browsers pre-defined settings.
If you dont specify something mostly browsers fall back to their pre defined settings.


what is difference to use or not to use parenths before function in javascript

Question: What is difference between following code?
Code 1:
var f = function() {

  // function body code
  // ...
  // ...

}();

Code 2:
var f = (function(){

  // function body code
  // ...
  // ...

})();

Which one is better to use?
Comment: Check this article: [An Important Pair of Parens](http://michaux.ca/articles/an-important-pair-of-parens)
Answer: They are equivalent. The second one might be more clear, though.
Answer: They're exactly equivalent given the syntax of the language.
Comment: True. It might be worth it to note that both snippits are setting 'f' to the result of a self-executing anonymous function.
Answer: Since it would be invalid syntax without the assignment, as in
function() {
    // ...
}();

it may be clearer to always wrap anonymous functions in parentheses.


Line-Triangle Intersection Check Returns Wrong Intersection Point

Question: I'm writing a very basic raycaster for a 3D scene with triangulated objects and everything worked fine until I decided to try casting rays from points other than the origin of the scene (0/0/0).
However, when I changed to origin of the ray to (0/1/0) the intersection test suddenly returned a wrong intersection point for one of the triangles.
I'm deliberately "shooting" the rays into the direction of the center of the triangle, so obviously this should be the intersection point. I just simply don't know what's exactly leading to the wrong results in my code.
(I'm not using Möller-Trumbore at the moment because I'd like to start out with a simpler, more basic approach, but I will switch to Möller-Trumbore when optimizing the code.)
These are the coordinates of my the three vertices of the above mentioned triangle:
-2.0/2.0/0.0 | 0.0/3.0/2.0 | 2.0/2.0/0.0
This is the center of the triangle:
0.0/2.3333333333333335/0.6666666666666666
This is my ray (origin + t * Direction):
Origin: 0.0/1.0/0.0
Direction (normalized): 0.0/0.894427190999916/0.4472135954999579
This is the obviously wrong intersection point my program calculated (before checking and finding out that the point is not even on the triangle:
0.0/5.0/1.9999999999999996
So yeah, it's not hard to see (even without a calculator) that the ray should hit the triangle at its center at roughly t = 1.5. My code, however, returns the value 4.472135954999579 for t.
Here's my code for the intersection check:
    public Vector intersectsWithTriangle(Ray ray, Triangle triangle) {
    boolean intersects = false;

    Vector triangleNormal = triangle.getNormalVector();
    double normalDotRayDirection = triangleNormal.dotProduct(ray.getDirection());

    if(Math.abs(normalDotRayDirection) == 0) {
        // parallel
        return null;
    }

    double d = triangleNormal.dotProduct(triangle.getV1AsVector());
    double t = (triangleNormal.dotProduct(ray.getOrigin()) + d) / normalDotRayDirection;

    // Check if triangle is behind ray
    if (t < 0) return null;

    // Get point of intersection between ray and triangle
    Vector intersectionPoint = ray.getPosAt(t);

    // Check if point is inside the triangle
    if(isPointInTriangle(intersectionPoint, triangle, triangleNormal)) {
        intersects = true;
        return intersectionPoint;
    }

    return null;
}

Any ideas what's wrong with the line that calculates t?
Comment: If your code works correctly for (0, 0, 0), maybe you can just translate the triangle vertices via v - p where p is the point, and then apply the code for (0, 0, 0).
Comment: hmm, I think you should combine both steps into one method - see the Möller-Trumbore algorithm: https://en.m.wikipedia.org/wiki/Möller-Trumbore_intersection_algorithm
Answer: If the ray is given by o + t*v and the triangle plane is defined by normal vector n and point p, then we are looking for t s.t. n*(o + t*v) = n*p which gives t = (n*p - n*o)/(n*v). So you seem to have a sign error and the correct computation for t should be:
double t = (d - triangleNormal.dotProduct(ray.getOrigin())) / normalDotRayDirection;

As long as the ray origin was (0,0,0) the wrong sign did not matter.
Comment: Sorry for my late response! Yup, this was indeed the mistake! Thanks!


jspdf is generating a corrupted pdf

Question: https://jsfiddle.net/yt92n1pt/49/
The pdf generated in the above fiddle is corrupted according to Acrobat reader, but renders fine in Google Chrome. Why does this happen and how can I avoid it?
Edit: The question was originally asking why the contents of a zipped pdf were different when compared to the original pdf. But that's not the case.

window.create_zip = function() {
  var pdf = new jsPDF('p', 'pt', 'a4');
  addHtml(document.getElementById('tables').outerHTML, pdf).then(function(){
    var zip = new JSZip();
    zip.file("notok.pdf", pdf.output());
    zip.generateAsync({
      type: "blob"
    })
      .then(function(content) {
      saveAs(content, "example.zip");
    });
  });
}

window.create_pdf = function() {
  var pdf = new jsPDF('p', 'pt', 'a4');
  addHtml(document.getElementById('tables').outerHTML, pdf).then(function(){
    pdf.save('ok.pdf');
  });
}

function addHtml(html, doc) {
  var canvas = doc.canvas;
  canvas.pdf = doc;

  html = html.replace(/

jspdf is generating a corrupted pdf


How to prove $B\subset A$ for $A=\{3n; n\in Z^+\}$ and $B=\{9m; m\in Z^+\}$

Question: I have been trying to figure how to do the following problem,
Let $A = \{3n : n \in Z^+\}$ and $B = \{9m : m \in Z^+\}$. Prove $B \subset A$.
I see that we need to show that $A$ is a proper subset of $B$ then all elements of $A$ are in $B$ but $B$ has one element that is not in $A$. I can kind of visualize it but not put it in writing.
Answer: See $x\in B\iff \frac{x}{9}\in \mathbb{Z_+}$.

Now the proof:
$x\in B\iff \exists m\in \mathbb{Z_+}$ such that $x=9m\implies x=3\times 3m=3n$ where $n=3m\in \mathbb{Z_+}$$\implies x\in A$.
So we get $\forall x\in B\implies x\in A$.
Hence by definition $B\subseteq A$.
Now $y=3\in A$. But $\frac{y}{9}=\frac{3}{9}=\frac{1}{3}\notin \mathbb{Z_+}\implies y \notin B$
So $\exists y\in A$ such that $y \notin B\implies A\neq B$.
Now $B\subseteq A$ and $A\neq B\implies B\subset A$.
Answer: All set theory equalities and containments boil down to what you're saying. Show that for any $a\in B$, you also have $a\in A$. For example, $9=9\cdot 1 \in B$ and also $9=3\cdot 2\in A$. The point is, you know that every element in $A$ has the form $3m$ for $m$ positive integer and $9m$ for $B$. Use this knowledge.
To show proper containment, find something in $B$ that doesn't belong to $A$.
Comment: So for A we can have 3*1=3 but for be we cant have 3 the lowest we can go is 9, is that what you mean?
Comment: @russiandobby: for strict containment, yes.


Hibernate connection with HSQLDB

Question: I have a problem while setting up my first Hibernate project. I follow the tutorial and get stuck when I launch my code. What happens is that the program get stuck, I believe, when starting the connection with HSQLDB. It does not provide any error message.
The verbose output in the console is the following:
172 [main] INFO org.hibernate.annotations.common.Version - Hibernate Commons Annotations 3.2.0.Final
172 [main] INFO org.hibernate.cfg.Environment - Hibernate 3.6.10.Final
172 [main] INFO org.hibernate.cfg.Environment - hibernate.properties not found
187 [main] INFO org.hibernate.cfg.Environment - Bytecode provider name : javassist
187 [main] INFO org.hibernate.cfg.Environment - using JDK 1.4 java.sql.Timestamp handling
265 [main] INFO org.hibernate.cfg.Configuration - configuring from resource: /hibernate.cfg.xml
265 [main] INFO org.hibernate.cfg.Configuration - Configuration resource: /hibernate.cfg.xml
375 [main] INFO org.hibernate.cfg.Configuration - Reading mappings from resource : org/itg/ACLwithHibernate/User.hbm.xml
484 [main] INFO org.hibernate.cfg.Configuration - Configured SessionFactory: null

now it is configured, I call buildSessionFactory():
562 [main] INFO org.hibernate.cfg.HbmBinder - Mapping class: org.itg.ACLwithHibernate.User -> USER
577 [main] INFO org.hibernate.cfg.Configuration - Hibernate Validator not found: ignoring
593 [main] INFO org.hibernate.cfg.search.HibernateSearchEventListenerRegister - Unable to find org.hibernate.search.event.FullTextIndexEventListener on the classpath. Hibernate Search is not enabled.
593 [main] INFO org.hibernate.connection.DriverManagerConnectionProvider - Using Hibernate built-in connection pool (not for production use!)
593 [main] INFO org.hibernate.connection.DriverManagerConnectionProvider - Hibernate connection pool size: 10
593 [main] INFO org.hibernate.connection.DriverManagerConnectionProvider - autocommit mode: false
609 [main] INFO org.hibernate.connection.DriverManagerConnectionProvider - using driver: org.hsqldb.jdbcDriver at URL: jdbc:hsqldb:hsql://localhost:9001
609 [main] INFO org.hibernate.connection.DriverManagerConnectionProvider - connection properties: {user=sa, password=****}

and... nothing more. It never goes on to the next instructions in my main() class. It is stuck somewhere.
When I debug, I get this:
overview of the debugger
Here is my hibernate.cfg.xml:
<hibernate-configuration>

    <session-factory>

        <property name="connection.driver_class">org.hsqldb.jdbcDriver</property>
        <property name="connection.url">jdbc:hsqldb:hsql://localhost:9001</property>
        <property name="connection.username">sa</property>
        <property name="connection.password"></property>

        <property name="connection.pool_size">10</property>

        <property name="dialect">org.hibernate.dialect.HSQLDialect</property>

        <property name="current_session_context_class">thread</property>

        <property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>

        <property name="show_sql">true</property>

        <property name="hbm2ddl.auto">update</property>

        <mapping resource="org/itg/ACLwithHibernate/User.hbm.xml"/>

    </session-factory>

</hibernate-configuration>

My HSQLDB server is running and outputed the following:
[Server@3ed5dee4]: [Thread[org.hsqldb.Server.main(),5,org.hsqldb.Server]]: checkRunning(false) entered
[Server@3ed5dee4]: [Thread[org.hsqldb.Server.main(),5,org.hsqldb.Server]]: checkRunning(false) exited
[Server@3ed5dee4]: Startup sequence initiated from main() method
[Server@3ed5dee4]: Loaded properties from [C:\Users\dbr\workspace\test_hibernate\server.properties]
[Server@3ed5dee4]: Initiating startup sequence...
[Server@3ed5dee4]: Server socket opened successfully in 0 ms.
[Server@3ed5dee4]: Database [index=0, id=0, db=file:target/data/tutorial, alias=] opened sucessfully in 630 ms.
[Server@3ed5dee4]: Startup sequence completed in 640 ms.
[Server@3ed5dee4]: 2012-07-18 12:36:53.717 HSQLDB server 2.0.0 is online on port 9001
[Server@3ed5dee4]: To close normally, connect and execute SHUTDOWN SQL
[Server@3ed5dee4]: From command line, use [Ctrl]+[C] to abort abruptly

When server.silent = false, I get in addition the following when my client connects:
[Thread[HSQLDB Server @1af5458a,5,org.hsqldb.Server]]: handleConnection(Socket[addr=/127.0.0.1,port=49438,localport=9001]) entered
[Thread[HSQLDB Server @1af5458a,5,org.hsqldb.Server]]: handleConnection() exited
[Thread[HSQLDB Connection @9437a04,5,HSQLDB Connections @1af5458a]]: ODBC client connected.  ODBC Protocol Compatibility Version 0.0

Have you encountered a similar issue before? Do you have an idea about what is going wrong?
Thank you very much in advance.
Dany
Comment: Is your database running on localhost? Can you telnet to localhost:9001?
Comment: @Funtik You are right, I didn't think about trying that. Yes, I can telnet and I get `HSQLDB JDBC Network Listener. User JDBC driver with Network Compatibility Version220.127.116.11 and a JDBC URL like jdbc:hsqldb:hsql://hostname...` then the connection is dropped.
Comment: I'm not familiar with the HSQLDB, but I would try to create a database alias, and use it as a DB name in hibernate connection
Comment: To debug, add server.silent=false to server.properties. Try with HSQLDB 2.2.8 (from Maven) as this is more compatible with the Hibernate dialect.
Comment: Thank you @fredt, I tried with version 2.2.8 & server.silent=false and I edited my post to include the inherent logs.
Comment: The server message should not say ODBC client. Therefore you are propbably using different HSQLDB jars for your program and the server. Make sure the same 2.2.8 jar is used and verify the server is reporting "HSQLDB server 2.2.8 server is online .." and not saying ODBC when a connection is made.
Comment: @fredt: it WORKED! So it was a version problem... Thank you. Please submit it as an answer so I can mark it as accepted.
Answer: When an HSQLDB Server is used with Hibernate or other frameworks, it is essential to check the HSQLDB jar version used by the client and server and make sure they are both the same version of HSQLDB. It is also necessary to use an up-to-date version of HSQLDB.
It is easy to troubleshoot an HSQLDB Server if you start it with the property server.silent=false (or silent=false as a command line argument). This will show the attempted connections.
In this example, the server is showing:
[Thread[HSQLDB Connection @9437a04,5,HSQLDB Connections @1af5458a]]:
      ODBC client connected.  ODBC Protocol Compatibility Version 0.0

which indicates a version incompatibility problem, as the ODBC is not actually used for Java to HSQLDB connections.
Answer: It stuck in connecting to your DB.
for debugging remove the connection pool size element from the configuration and set hbmtoddl to false.
Comment: Unfortunately, it gives me the very same behavior and output. Somehow, the connection seems to be established well, because when I terminate the 'frozen' code I get a `java.net.SocketException: Connection reset` on the ouput of my server.
Comment: I have though `log4j.logger.org.hibernate=debug`, `log4j.rootLogger=debug, stdout` and `log4j.logger.org.hibernate.SQL=debug` in my log4j.properties.
Comment: but you run in info mode and not debug mode. you can see that log level is info

Hibernate connection with HSQLDB


Remove/Disable selected options from a dropdown(Which is dynamically created) in angular 8

Question: I have a dynamic dropdown with some fields.

press + button will generate a new row.We can have any number of rows here.
I need to Remove/Disable the previously selected dropdown values from here

I have stored selected items and my entire list like this.
<code>SelectedFieldList =["ProductCode","Name"];
FieldList=["ProductCode","Name","Code","Image","Category", etc..]
</code>
I have tried to disable the selected Options using this code
<code> <select formControlName="FieldName">
                        <ng-container *ngFor="let FieldItem of FieldList">
                          <option *ngIf="!SelectedFieldList.includes(FieldItem)"  [value]="FieldItem">{{FieldItem}}</option>
                          <option [disabled]="true" *ngIf="SelectedFieldList.includes(FieldItem)"  [value]="FieldItem">{{FieldItem}}</option>
                        </ng-container>
                      </select>
</code>
This is working fine inside the dropdown. But the issue is after disabling the previously selected item in the next button click. This is changing the already selected dropdown values. Means it's also disabling from the already selected dropdowns and showing the next related value.
In the above screenshot, I have selected the "Product number" .But after clicking + button it's disabled from the next dropdown successfully. but it's changed the value to "Product code" in the first dropdown(Product number is also there but it's a disabled format).

any solution will be appreciated.
Thanks in advance
Answer: Why dont you bind the condition to disabled input?
<code> <ng-container *ngFor="let FieldItem of FieldList">
    <option [disabled]="SelectedFieldList.includes(FieldItem)"  [value]="FieldItem">{{FieldItem}}</option>
 </ng-container>
</code>
Comment: Hi @Pinka , I have maintained an array for all selected items in the dropdown change event and wrote a condition on the select-options.

`{{FieldItem}}
{{FieldItem}}`

please compare with my aforementioned example.
Comment: @Ajas Aju did this solution work for you? I am having same requirement but this is not working.
Comment: I am disabling and changing the colour if the `fieldItem` in the `SelectedSkunestFieldList` array.Otherwise it will list all fields normally.


Python sql returning list

Question: got some functions with sqlstatements. My first func is fine because i get only 1 result.
My second function returns a large list of errorcodes and i dont know how to get them back for response.
TypeError: <sqlalchemy.engine.result.ResultProxy object at 0x7f98b85ef910> is not JSON serializable

Tried everything need help.
My Code:
def topalarms():
    customer_name = request.args.get('customer_name')
    machine_serial = request.args.get('machine_serial')

    #ts = request.args.get('ts')
    #ts_start = request.args.get('ts')

    if (customer_name is None) or (machine_serial is None):
        return missing_param()
    # def form_response(response, session):
        # response['customer'] = customer_name
        # response['serial'] = machine_serial

    # return do_response(customer_name, form_response)
    def form_response(response, session):
        result_machine_id = machine_id(session, machine_serial)
        if not result_machine_id:
            response['Error'] = 'Seriennummer nicht vorhanden/gefunden'
            return
        #response[''] = result_machine_id[0]["id"]
        machineid = result_machine_id[0]["id"]
        result_errorcodes = error_codes(session, machineid)
        response['ErrorCodes'] = result_errorcodes

    return do_response(customer_name, form_response)

def machine_id(session, machine_serial):

    stmt_raw = '''
        SELECT
            id
        FROM
            machine
        WHERE
            machine.serial = :machine_serial_arg
        '''
    utc_now = datetime.datetime.utcnow()
    utc_now_iso = pytz.utc.localize(utc_now).isoformat()

    utc_start = datetime.datetime.utcnow() - datetime.timedelta(days = 30)
    utc_start_iso = pytz.utc.localize(utc_start).isoformat()

    stmt_args = {
        'machine_serial_arg': machine_serial,
        }

    stmt = text(stmt_raw).columns(
        #ts_insert = ISODateTime
        )

    result = session.execute(stmt, stmt_args)

    ts = utc_now_iso
    ts_start = utc_start_iso

    ID = []

    for row in result:
        ID.append({
            'id': row[0],
            'ts': ts,
            'ts_start': ts_start,
            })
    return ID

def error_codes(session, machineid):

    stmt_raw = '''
        SELECT
            name
        FROM
            identifier
        WHERE
            identifier.machine_id = :machineid_arg
        '''

    stmt_args = {
        'machineid_arg': machineid,
        }

    stmt = text(stmt_raw).columns(
        #ts_insert = ISODateTime
        )

    result = session.execute(stmt, stmt_args)

    errors = []

    for row in result:
        errors.append(result)
    #({'result': [dict(row) for row in result]})
    #errors = {i: result[i] for i in range(0, len(result))}
    #errors = dict(result)
    return errors

My problem is func error_codes somethiing is wrong with my result.
my Output should be like this:
ABCNormal
ABCSafety
Alarm_G01N01
Alarm_G01N02
Alarm_G01N03
Alarm_G01N04
Alarm_G01N05

Answer: I think you need to take a closer look at what you are doing correctly with your working function and compare that to your non-working function.
Firstly, what do you think this code does?
   for row in result:
        errors.append(result)

This adds to errors one copy of the result object for each row in result.  So if you have six rows in result, errors contains six copies of result.  I suspect this isn't what you are looking for.  You want to be doing something with the row variable.
Taking a closer look at your working function, you are taking the first value out of the row, using row[0].  So, you probably want to do the same in your non-working function:
   for row in result:
        errors.append(row[0])

I don't have SQLAlchemy set up so I haven't tested this: I have provided this answer based solely on the differences between your working function and your non-working function.
Answer: You need a json serializer. I suggest using Marshmallow: https://marshmallow.readthedocs.io/en/stable/
There are some great tutorials online on how to do this.
Comment: there must be another possibility like in the func machine_id
Comment: What is you aversion to using Marshmallow? It was literally design for this exact purpose, and works beautifully with SQLAlchemy.

Python sql returning list


Unbalanced control and experiment group sample size

Question: I am working on testing the significance of a training program for businesses. We offer the training to the experimental group and want to compare some metrics to the control group. Because it takes time to implement such training, currently we only have $n=10$ for the experiment group and $n=150$ for the control group.
Should I use the whole control group to compare the metrics or should I do random sampling and test out different experiment:control ratios? I'm also thinking about picking businesses from the control group that that are more similar to the experiment group (same types of product, etc) - does it make sense to do so?
I'm super new to experimental design/testing so any advice is appreciated. Note that I have to use that experimental group for now and it's not possible to increase the size of experimental group at this point.
Answer: There is no requirement for group comparisons that data should be balanced (but, if you can plan to get equal group sizes, that is usually better).  You should analyze the data you have. Of course experimental group size of 10 might be few, but you do not better the situation by subsampling the larger group.
For more advice, we need more details of the situation.
Comment: Thanks a lot! What of additional details should i provide?
Comment: In what does the experimental treatment consist? What are the units? (businesses? What kind of ...)   In short, describe the experiment and your goal in everyday language.
Comment: Sure! So as mentioned, we provide training programs to companies and want to see if there is a significant difference in some metrics for companies that went through the training program. The units are companies. The experiment group contains companies that already did the training program, and the control group contains those that did not. We want to compare metrics like: time to complete a certain task of the group of companies who did the training vs who did not and see if the difference is significant
Comment: I do realize that such metric could be influenced by other factors (company size, industry, type of products, etc) of the company, I wonder if I should subsample the control group to find those that are most comparable to those in the experimental group
Comment: Please add this new details to the Q as an edit!
Comment: To make the parenthetical point about equal sample size a bit more clear, this means that if you have a fixed total N, you'll get the best power by splitting equally between groups. But more data is always better, there'd be no reason to limit your sampling just to keep the groups equally sized.


Roll $1$ die and square or roll $2$ dice and multiply; which has higher mean?

Question: I am wondering if there is a fast way to tell which expected value is higher: "Roll 1 die and take the square of the number that comes up or roll 2 dice and multiply them". Thank you!
Answer: Let $\xi$ and $\eta$ be values of the two die rolls. The expected value of squaring the the first roll is $\operatorname{E}(\xi^2)$, and the expected value of multiplying the two values is $\operatorname{E}(\xi\eta)$. Since we can assume the two die rolls are iid, we have $$\operatorname{E}(\xi\eta)=\operatorname{E}(\xi)\operatorname{E}(\eta)=\operatorname{E}(\xi)^2$$
Since the difference between these two expected values is equal to the variance,
$$\operatorname{Var}(\xi)=\operatorname{E}(\xi^2)-\operatorname{E}(\xi)^2$$
it must be nonnegative. Therefore, we have
$$\operatorname{E}(\xi^2)\ge\operatorname{E}(\xi)^2$$
so the expected value of squaring the result of one die roll is higher.
Comment: perfect! I was trying to do something like this. Thanks a lot
Comment: Nice and general. +1
Answer: Not sure whether you consider this faster.  The first probability is
$$\frac{1}{n}\sum_{k=1}^n k^2 = \frac{1}{n}\cdot\frac{n(n+1)(2n+1)}{6}=\frac{(n+1)(2n+1)}{6}.$$
The second probability is
$$\left(\frac{1}{n}\sum_{k=1}^n k\right)^2 = \left(\frac{1}{n}\cdot\frac{n(n+1)}{2}\right)^2 = \frac{(n+1)^2}{4}.$$
The first one is larger when $(2n+1)/6 > (n+1)/4$, that is when $n > 1$.
Comment: yeah this is definitely faster. Thanks a lot!