Grace notes: when are they played on the beat, and when are they played off?

Question: Since my performance experience is mostly in music after 1850, and since my instrument (tuba) very rarely has any grace notes, I've been mixed up for some time on how to play grace notes in different musical eras.
Specifically, how are grace notes played differently when they're written in the Classical style versus being written in the Romantic? (Perhaps the dichotomy isn't fair and is much more nuanced than that.) What about a composer like Beethoven, who straddles the two eras? And is there a similar rule for modern music, or does it vary by composer?
Different types of grace notes and Grace notes in Minuet by Boccherini both come close to the answer, but neither discuss the specific distinction I'm looking for.
I'm asking about grace notes for all instruments and for all historical eras. Ultimately the question is this:

Given a sheet of music with an acciaccatura and/or appoggiatura, how do I know when to play them on the beat compared to off the beat? If it's an issue of when in history the piece was written, what are these guidelines?
Comment: A thought-provoking question. +1. Including all sorts of grace notes will produce a hefty tome rather than a concise answer!
Answer: There is no universal agreement on this topic. While there may be a general consensus, you can always find a scholar arguing the opposite view.
The standard ornaments in the practice of Mozart and Haydn can be derived from the earlier, larger Baroque repertoire. Leopold Mozart and C. P. E. Bach are standard sources here. The general consensus today is to start all ornaments on the beat, except perhaps the acciaccatura which is so short that distinguishing is difficult. More often than not this is carried forward into Beethoven as well. For example, Arrau's edition recommends this practice. Broadly speaking, the generation before him is most likely to be heard beginning ornaments before the beat.  Czerny Op. 500 is a useful source here.
The interpretation of the acciaccatura is fairly consistent through time, the appoggiatura less so.  The length it absorbs from the following note in the Romantic occurrences that spring to mind is very much at the performer's discretion. On the other hand, its notation in Mozart and Haydn is more grammatical. Finding a small quaver appoggiatura preceding a crotchet, typically one would simply play two quavers, likely with a small extra "leaning" accent.
The 19th century saw the appearance of a large number of primers with something to say about this topic. The general view was that appoggiaturas begin on the beat. It is of course hard to tell how closely these texts reflect the contemporary performance practice.
In response to a comment that an acciaccatura is distinguished by a stroke through the note: this is the general view but the picture is more complex than that. The forceful keyboard acciaccatura and the short appoggiatura in music for various instruments have both been notated with the stroke, in the latter case to distinguish it from the long appoggiatura.
Comment: Just for completeness: acciaccatura is notated by a slash through the  stem of the grace note.
Comment: Interestingly, or perhaps annoyingly, I have seen sheet music in which a theme is written once as two eighth notes but later on as a grace-note + quarter note.  I don't know whether to blame the composer or the typesetter.
Comment: If it was an Urtext edition, it might simply be a fastidious rendering of inconsistency on the part of the composer. Also, if I were pressed on what the difference might be between two eights and a quarter with an appoggiatura, I'd say that the appoggiatura has overt stylistic meaning, and would emphasize the grace heavily and lighten on its resolution.
Comment: In addition to the composer and period, you can also take into account the *style* of a particular piece--does it evoke  something outside the piece itself, and does this helps you decide on ornament execution? For example, Haydn, Mozart, and Beethoven all wrote works in the "Turkish" style characterized, among other things, by many acciaccaturas.
Answer: In addition to user48353's excellent answer, I'd like to add: We want to target our historical knowledge as narrowly as possible to the composer (or even the piece!). Don't settle for acquiring one rule for music 1600-1750, another for 1750-1825, etc. If the composer wrote about ornamentation, like CPE Bach, read that. If not, check contemporary treatises. For well-known pieces, the execution of the graces might even have a written reception history. Unfortunately, ornamentation has always been a field plagued by gnostic systems of arcane symbols, but the conclusion doesn't always have to be to shrug and do whatever we want. The first question is always "what did the composer mean?"


Copy Aws launch template to another region

Question: I want to know if it is possible to copy the launch template create in one region to another region.
The idea behind this is, if for some reason the region goes down, I would like to start my service in another region with this launch template.
Answer: You cannot copy the launch template to another region, however, you can use aws cli to get launch template data and then create a template in the target region with the launch template details
Get: https://docs.aws.amazon.com/cli/latest/reference/ec2/get-launch-template-data.html
Create: https://docs.aws.amazon.com/cli/latest/reference/ec2/create-launch-template.html
Comment: do I need to modify the queried json content with the new region?
Comment: If your template specifies an AMI, you will need to change that and you may need to copy the existing AMI if it is a custom one.


Installing Private and Public Key Certificates On IIS7


Python Nested Loop unique pairs

Question: I'm trying to write a nested loop that prints out all possible "unique pairs" of numbers from a certain range. For example, if the range was from 1 to 3 the unique pairs would be:
(1,2)
(1,3)
(2,3)
If the range was from 1 to 4 the unique pairs would be:
(1,2)
(1,3)
(1,4)
(2,3)
(2,4)
(3,4)
Here's how I did it for 1 to 3:
for i in range(1,4):
    for j in range(2,4):
        if (i != j & j != (i-1)):
            print (i,j)

which prints out (1, 2), (1, 3),(2, 3). But this is a hack because it doesn't work when I change the range to 1,5. It prints out duplicate pairs such as (1,5) and (5,1).
Answer: for i in range(1,4):
    for j in range(i+1,4):  # <-- the key is to make j start at i+1
        print (i,j)

Answer: Use itertools.combinations():
>>> import itertools
>>> print list(itertools.combinations(range(1, 5), r=2))
[(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]

As long as your inputs are unique, there will be no repeated combinations:

itertools.combinations(iterable, r)
Return r length subsequences of elements from the input iterable.
Combinations are emitted in lexicographic sort order. So, if the input iterable is sorted, the combination tuples will be produced in sorted order.
Elements are treated as unique based on their position, not on their
  value. So if the input elements are unique, there will be no repeat
  values in each combination.
Answer: You are probably looking for something like this:
n = 4
x = [(i,j) for i in range(1,n+1) for j in range(i+1, n+1)]
print x

Cheers,
Alex
Comment: Because i wrote it for a general purpose. You just change the "n" to whatever and the code works :)
Comment: Why `n+1`? Isn't `n+1` just 5? Maybe I'm misunderstanding something here?
Comment: You need n+1 because range (i, j) goes from i to j - 1. So if we used (1, n) we would have 1 to n - 1. So if n was 4, then we would be missing 4, we would only have 1, 2 and 3.
Comment: @AlexandruGodri I think he is asking why not replace all `n+1`'s with 5's.
Answer: You can use itertools.combinations:
>>> from itertools import combinations
>>> print list(combinations(range(1,4), 2))
[(1, 2), (1, 3), (2, 3)]

Python 2 Documentation
Python 3 Documentation
Answer: See itertools module.
Maybe what you want is
list(itertools.combinations(range(1,4),2)) == [(1, 2), (1, 3), (2, 3)]

Python Nested Loop unique pairs


What is a better practice for extracting icons?

Question: I have created an explorer window that scans the directories upon loading for filetypes, etc. when it first loads, i use
internal static extern uint ExtractIconEx(string szFileName, uint nIconIndex, IntPtr[] phiconLarge, IntPtr[] phiconSmall, uint nIcons);

and
internal static unsafe extern int DestroyIcon(IntPtr hIcon);

to scan for currently registered icons and then i associate them with teh files im using. it works fine on my machine but when i run it on others, i get a low resource error.
so my question is what is a better practice? to go the way that i am, and load them all up, or everytime i need to enumerate a directory to look for the filetype and associate it.
Comment: What makes you think that the icons are the source of your resource problems? How many icons is your program loading? How many is it freeing? Can you show the relevant code?
Answer: As an alternative you could do this with system.drawing
var icon = System.Drawing.Icon.ExtractAssociatedIcon(@"c:\xxx\some.file");

(You would need to resize this for the small shell size)


How does flexibility of an algebra imply this identity?

Question: In a paper from Schafer, he mentions that the flexible identity:
$$(xy)x = x(yx)$$
can be "linearized" to become:
$$(xz)y + (yz)x = y(zx) + x(zy)$$
In other words, an flexible algebra (one that satisfies the flexible identity) also always satisfies this equation. However, I'm not sure how to prove this.
The way that this is presented in the paper makes it seem like this is the case for any algebra, but the paper is mostly concerned with algebras over fields, so perhaps that is required. However, even with those conditions, I'm not sure how to prove it.
I'm also not really sure what he means when he says to "linearize" the identity.
So how should this proof go?
Comment: Just linearize the equation. You'll obtain $(x,y,z)+(z,y,x)=0$ for the associators. For a detailed proof see [here](https://books.google.at/books?id=Z1D2BwAAQBAJ&pg=PA8&lpg=PA8&dq=linearize+identity+for+flexible+algebra&source=bl&ots=VroHK_9fcN&sig=ACfU3U0an-ZLzu6jG41s9_Qi9P6pZGnWBA&hl=de&sa=X&ved=2ahUKEwj07enmp5XoAhVsMewKHfdnAQcQ6AEwAnoECAkQAQ#v=onepage&q=linearize%20identity%20for%20flexible%20algebra&f=false).
Comment: @DietrichBurde Like I said, I don't really know what it means to "linearize" the identity. All that link says is to linearize the identity, without really saying what that means, which is the same as the paper I'm reading.
Comment: @DietrichBurde Ah, I scrolled through the book a bit more, and found where they said what it meant to linearize the equation (though vaguely). But yeah, your answer is very clear, thanks.
Answer: Substituting $x+z$ for $x$ in $(xy)x=x(yx)$ we obtain
$$
(xy)x+(zy)x+(zy)z+(zy)z=x(yx)+x(yz)+z(yx)+z(yz)
$$
where the terms $(xy)x$ and $x(yx)$ cancel, as well as $(zy)z$ and $z(yz)$. This gives
$$
(zy)x-z(yx)=x(yz)-(xy)z,
$$
or
$$
(x,y,z)+(z,y,x)=0.
$$
Renaming $x,y,z$ appropriately gives your identity.


Multi RDP connections as an Admin to the domain controller

Question: We have a 2008 r2 server that's everything in one.  It's a domain controller plus it runs some applications.  I know it's a not a good design but it is what it is.
Since it's a domain controller, it has restrictions on who can access it via RDP.  I added bunch of folks to the domain administrator group.  When the users login they bump each other off (these are different users with different login and password).  It appears that there is only one connection as an admin allowed on the domain controller(?).
The questions is, how can I allow multiple-admin-RDP-connection to the domain controller?  Please note that the server is fully licensed up to five connections (which is the number of users that I have).
Edit
Folks, the Remote Desktop Session Host is on and the License Server is also running.  The license that I mentioned WAS for RDS Not windows.  We have five licenses.
We are having issues running more than one RemoteApp at the same time to same server.  RemoteApp relies on RDS.  Currently, only an administrator can launch the RemoteApp. I suspect this is due to the fact that the server is also a DC.  We need to be able to run five connections at the same time all running the RemoteApp.  I hope this clarifies things a bit.
If were you guys reading this I would be throwing insults too.  But just take it from me...again...it is what it is.
Update  Thank you everyone.  We demoted the server from DC and now everything works fine.  We now can have multi connections at the same time.  For the next foolio -> if it's a DC, only administrators can RDP to DC.
Comment: Having the DC run a few extra services is one thing; it's no one's first choice, but we've all done it. Putting standard users into the Domain Admin group is whole other league of stupid.
Comment: Please see my edits and any input is appreciated.
Answer: Have a look at https://technet.microsoft.com/en-us/library/cc753380.aspx it should help you.
By default the number should be two connections.
Answer: The maximum you can have is two simultaneously, unless you enable the RD Session Host feature. Note: RDS CALs are different from windows CALs. There's no need to have the users in the domain admins group.
Comment: We are talking about RDS CALs..not windows.  RD SEssion host is turned on.  The license server is running and has five RDS licences.  Thanks
Comment: @Zuzlx Ah, I see; that would make a difference. IMO it sounds like a good reason to take it to management and get the resources to take the app/users off the DC.
Answer: Questionable administrative group practices aside, you can get into a third session by launching mstsc with /admin.
<code>mstsc /admin
</code>
Answer: The maximum simultaneous allowed RDP connectons is two regardless of being a domain admin or not. I would never add non admin users to the domain admins group.
To allow more than two connections you can make your server a terminal services server - add that service, and requires licesing as well.
The licensing you are talkig about has nothing to do with RDP, it is the number of people who can logon to your domain at the same time.(someone correct me if I'm wrong).


In Android, how to shift a bitmap?

Question: I am trying, unsuccessfully, to "shift" a bitmap.  By "shift", I mean:  if I have a 500x300 and I shift it 50 pixels to the right, the result should be a 500x300 bitmap where every pixel has been shifted 50 pixels to the left, and the 50 right most pixels are blank.  So, the new image is the same as the old one except the left side is cropped, and a blank area is appended to the right side.  The below code successfully crops the left side, but it does not produce a blank area on the right side, it just produces a narrower bitmap:
Bitmap newBitmap = Bitmap.createBitmap(oldBitmap.getWidth(), m_CurrentBitmap.getHeight(), Config.ARGB_8888);
newBitmap = Bitmap.createBitmap(oldBitmap.getWidth(), deltaX, 0, oldBitmap.getWidth() - deltaX, oldBitmap.getHeight());

So then I tried creating a new Bitmap with the desired dimensions, and manually setting the pixels (with an offset).  However, this still produces a narrower Bitmap, lacking the blank area on the right side:
Bitmap newBitmap = Bitmap.createBitmap(oldBitmap.getWidth(), oldBitmap.getHeight(), Config.ARGB_8888);

int[] oldPixels = new int[(int)(oldBitmap.getWidth() - deltaX) * oldBitmap.getHeight()];
            m_CurrentBitmap.getPixels(oldPixels, 0, (int) (oldBitmap.getWidth() - deltaX), (int)deltaX, 0, (int) (oldBitmap.getWidth() - deltaX), oldBitmap.getHeight());

newBitmap.setPixels(oldPixels, 0,  (int) (oldBitmap.getWidth() - deltaX), 0, 0, (int) (oldBitmap.getWidth() - deltaX), oldBitmap.getHeight());

Answer: EDIT: Okay, so you basically want to draw the following of the old bitmap onto the new bitmap.
You need to draw from the OldBitmap pixels from deltaX to OldBitmap.Width onto the NewBitmap pixels from 0 to NewBitmap.width - deltaX.  This should leave a right margin of the NewBitmap (NewBitmap.Width - deltaX to NewBitmap.Width) blank (assuming that the canvas behind it is white or whatever "blank" color is).
Comment: if i use deltaX as an offset, then any width > oldBitmap.getWidth() - deltaX will give an outOfBounds exception.
Comment: @ab11 I updated my response with at least the math portion, let me know if it doesn't make sense ;)  I'm not sure how to exactly translate drawing only a certain segment of a bitmap onto another in code because I haven't done it before.

In Android, how to shift a bitmap?


$\frac{\sqrt{12x}}{2+2\sqrt{3}}$ simplifies to $\frac{3\sqrt{x}-\sqrt{3x}}{2}$ but I get $-12x + 6x\sqrt{3}$

Question: I am asked to simplify $\frac{\sqrt{12x}}{2+2\sqrt{3}}$ and the solution is provided as $\frac{3\sqrt{x}-\sqrt{3x}}{2}$. I arrived at $-12x + 6x\sqrt{3}$ and I'm not sure how to arrive at the text book solution.
My working:
$$\frac{\sqrt{12x}}{2+2\sqrt{3}} = \frac{\sqrt{12x}}{2+2\sqrt{3}}\frac{2 - \sqrt{3}}{2 - \sqrt{3}} = \frac{12x(2-\sqrt{3})}{(2+2\sqrt{3})(2-\sqrt{3})} = \frac{24x-12x\sqrt{3}}{4+(2\cdot(-3))}=\frac{24x-12x\sqrt{3}}{-2}$$
Then, multiplying out the denominator I get:
$-12x+6x\sqrt{3}$
Is my thought process sound up to a point? Where did I go wrong and how can I arrive at $\frac{3\sqrt{x}-\sqrt{3x}}{2}$?
Comment: Why do you get $\sqrt {12x}(2-\sqrt{3}) = 12x(2-\sqrt 3)$?  You arbitrarily just removed the radical sign over the $\sqrt{12x}$.  That was your error.  Do it again without removing the radical sign.
Comment: Your second error is multiplying numerator and denominator by $2-\sqrt 3$ instead of $2-2\sqrt 3$.
Comment: And your third and fourth errors occur when you try to calculate $(2+2\sqrt 3)(2-\sqrt 3)$.
Comment: @TonyK good catch.
Answer: First error:
$$
\sqrt{12x}(2-\sqrt{3})\ne 12x(2-\sqrt{3})
$$
Second error:
$$
(2+2\sqrt{3})(2-\sqrt{3})=4+4\sqrt{3}-2\sqrt{3}-2\cdot3=2\sqrt{3}-2\ne-2
$$
Third error: in order to rationalize the denominator, you have to multiply by $2-2\sqrt{3}$.
On the other hand, you can proceed more simply:
$$
\frac{\sqrt{12x}}{2+2\sqrt{3}}=\frac{2\sqrt{3x}}{2(\sqrt{3}+1)}
=\frac{\sqrt{3x}}{\sqrt{3}+1}\frac{\sqrt{3}-1}{\sqrt{3}-1}=
\frac{\sqrt{9x}-\sqrt{3x}}{3-1}=\frac{3\sqrt{x}-\sqrt{3x}}{2}
$$
Answer: $\frac{\sqrt{12x}}{2+2\sqrt{3}}=\frac{\color{green}{\sqrt{12x}}}{2+2\sqrt{3}}\frac{2 - \sqrt{3}}{2 - \sqrt{3}}=\frac{\color{red}{12x}(2-\sqrt{3})}{\color{purple}{(2+2\sqrt{3})(2-\sqrt{3})}} = \frac{24x-12x\sqrt{3}}{\color{orange}{4+(2\cdot(-3))}}=\frac{24x-12x\sqrt{3}}{-2}$
You magically turned $\color{green}{\sqrt{12x}}$ into $\color{red}{12x}$ for no reason whatsoever.
And you incorrectly calculated $\color{purple}{(2+2\sqrt{3})(2-\sqrt{3})} = \color{orange}{4+(2\cdot(-3))}$.
You should have done:
$\frac{\sqrt{12x}}{2+2\sqrt{3}}=\frac{\sqrt{12x}}{2+2\sqrt{3}}\color{green}{\frac{2 - 2\sqrt{3}}{2 - 2\sqrt{3}}}=\frac{\sqrt{12x}(2-2\sqrt{3})}{\color{purple}{(2+2\sqrt{3})(2-2\sqrt{3})}} = \frac{...}{\color{purple}{4-4*3}}=...$
Comment: Thanks for the feedback and explanation. My first error re 12x was an outright error. I was not sure how to multiply out the denominator so thanks for clarifying how I do that here.


Customizing user module output

Question: I am using Drupal 7 to develop a REST API to create an iPhone app. The basic feature for this app user login and logout process. I am using the User module. The authentication process involved is session authentication. JSON is the output format used for the API.
How do I customize the JSON output from the login service? I am getting a bunch of other information which I don't require. I am not sure how to add more tags to the JSON format. Can anyone point me to the files where the information is?
For example, for a wrong username and password I get the following output:

Status 401 Unauthorized - Wrong username or password
["Wrong username or password."]

I would like to change the output to have something like <code>{status: 0, message: 'Wrong username and passowrd'}</code>.
Can anyone help me?
Comment: As an aside, when developing a RESTful API I would always advocate bypassing those provided by Services and writing one's own. That way it is a lot easier to add abstraction to the API. The ones provided by Services are very low level and can be hard to work with. Field values, for example, are in their raw form.
Answer: Use this for errors:
<code>/**
 * Implements hook_rest_server_execute_errors_alter().
 *
 * Reformat error output.
 */
function up_coaching_rest_rest_server_execute_errors_alter(&$error_alter_array, $controller, $arguments) {
  $error_alter_array['body_data'] = array(
    'success' => FALSE,
    'error' => $error_alter_array['body_data'],
  );
}
</code>
For success output you need clone needed resources from user resource definition.

Customizing user module output


Como aumentar altura conforme scroll desce a página mas não diminuir com scroll subindo?

Question: Estou em um projeto de página única, e preciso de uma div que aumente conforme eu desço a página, porém ao subir ela não pode diminuir, tem de ficar parada, do tamanho que estava até descer novamente.
Estou usando JQuery com o seguinte código:
o HTML:

o CSS:
header, footer{
    display:block;
    height:200px;
    background:red;
    width:100%;
}
#content{
    display:block;
    width:100%;
    height:1500px;
    background:blue;
}
#broca{
    width:50px;
    height:0;
    background:#000;
}

o JQUERY:
$(function(){
$(window).scroll(function() {
    var $broca = $('#broca');
    var st = $(this).scrollTop();

    $broca.height( st );
    if( st == 0 ) {
        $broca.hide();
    } else {
        $broca.show();
    }
}).scroll();

jsFiddle de exemplo.
Comment: Olá Mauro, podes juntar o HTML (idealmente um jsFiddle) para a pergunta ficar mais completa?
Comment: Olá Sergio. Como você comentou, acrescentei um jsFiddle na postagem inicial.
Answer: Mauro, precisa apenas verificar se o scroll é maior que  height da div:
$(function(){
    $(window).scroll(function() {
        var $broca = $('#broca');
        var st = $(this).scrollTop();
        if (st > $broca.height()){
            $broca.height( st );
        }
        if( st == 0 ) {
            $broca.hide();
        } else {
            $broca.show();
        }
    }).scroll();
})

JsFiddle
Comment: Diego, esse exemplo que você alterou faz com que o #broca mantenha a altura enquanto dá scroll pra cima, mas ao mandar scroll pra baixo, logo de cara ele diminui a altura logo na primeira rolada. O ideal seria "esperar" até chegar na altura máxima anterior e continuar aumentando só a partir dai.
Comment: Alterei Mauro, vê se resolveu!
Comment: Resolveu perfeitamente, Diego. Muito obrigado.


Formal proof that if $a-b = \frac{a-b}{ab}$, $a-b$ has to be zero.

Question: Just out of interest, how would I provide a proof that if $a-b = \frac{a-b}{ab}$, then $a$ has to be equal to $b$? It appears really logical, I just want to know how to formally prove it. Thanks for any suggestions.
Comment: It's not true.  If $a=2, b=\frac 12$ then $ab=1$ and $a-b\neq 0$.  If you had intended to restrict the possible values of $a,b$ you should do so explicitly.
Answer: Per lulu's comment, it's not true, but this is true:
$a-b=\dfrac{a-b}{ab}\implies (a-b)ab=a-b \implies (a-b)(ab-1)=0\implies a=b$ or $ab=1$
Comment: If $a,b\in\Bbb N$ and $ab=1,$ then $a=b=1,$ so $a-b=0$
Comment: Thanks. And if I restricted to $\forall a,b \in \mathbb{N}$, I could exclude $ab = 1$, right?


How do I change an element in a nested list?

Question: I am a Python beginner and in need of some advice.
I defined ten variables that all have an X and Y value.
Then I put these variables (which are actually a list) in another list called "pointlist".
Now, I want to use a for-loop to change the X-value of each variable.
Usually, I can access the X-value of the first variable through "pointlist[0][0]".
When I applied this to the for-loop I got the following error:
"TypeError: list indices must be integers or slices, not tuple"
Does anyone have an idea what I missed? Thank you so much in advance!
import random
X=1350
Y=250

p1=[X, Y]
p2=[X, Y]
p3=[X, Y]
p4=[X, Y]
p5=[X, Y]
p6=[X, Y]
p7=[X, Y]
p8=[X, Y]
p9=[X, Y]
p10=[X, Y]
p11=[X, Y]
pointlist = [p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11]

limit1=1150
limit2=1250

for i in enumerate (pointlist, start=1):
    pointlist[i][0] = random.randrange(limit1, limit2)
    limit1-=100
    limit2-=100

Answer: Pythonize your loop using:
for i in pointlist:
    i[0] = random.randrange(limit1, limit2)
    limit1-=100
    limit2-=100

This way i becomes an entry of pointlist for easy access
Answer: you might want to use range instead of enumerate.
import random
X=1350
Y=250

p1=[X, Y]
p2=[X, Y]
p3=[X, Y]
p4=[X, Y]
p5=[X, Y]
p6=[X, Y]
p7=[X, Y]
p8=[X, Y]
p9=[X, Y]
p10=[X, Y]
p11=[X, Y]
pointlist = [p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11]

limit1=1150
limit2=1250

for i in range(len(pointlist)):
    pointlist[i][0] = random.randrange(limit1, limit2)
    limit1-=100
    limit2-=100

if you still want to use enumerate.
import random
X=1350
Y=250

p1=[X, Y]
p2=[X, Y]
p3=[X, Y]
p4=[X, Y]
p5=[X, Y]
p6=[X, Y]
p7=[X, Y]
p8=[X, Y]
p9=[X, Y]
p10=[X, Y]
p11=[X, Y]
pointlist = [p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11]

limit1=1150
limit2=1250

for i in enumerate(pointlist, start=0):
    index = i[0] # get the index from here.
    value = i[1]
    pointlist[index][0] = random.randrange(limit1, limit2)
    limit1-=100
    limit2-=100

Answer: The enumerate call returns a tuple on each loop, not a single value. So, your i is being set to a tuple, which consists of the index (starting at 1) and the element in the pointlist. You can fix this many ways. One is to change:
for i in enumerate(pointlist, start=1):

to:
for i in range(len(pointlist), start=1):

Or you can keep the enumerate call and drop the returned element of pointlist, which doesn't really make much sense, but it's an option:
for i, _ in enumerate(pointlist, start=1):

Here, the _ is a placeholder that takes the pointlist value to prevent i from being a tuple. The first option is much better.
Note that list indeces start at 0, not 1, so you're start=1 is probably going to break your logic. You might be much better off with the default start value of 0.
Answer: Off-topic.
Adding to Mat's amazing answer:
I believe you explicitly created p1...p11 so when you change the x-coordinate of a point, other points will not be affected. If you are not going to use the references p1...p11, I suggest using a list comprehension:
point_list = [[X, Y] for _ in range(11)]

The inner list [X, Y] is created 11 times, and they are 11 separate objects, so changing any one of it will not affect the others. You can visualise it using Python Tutor.
I personally prefer using custom objects than dealing with nested lists in your case, as it better displays the intent of the code. Here is how I would  have coded it:
import random

class Coordinate:
    """
    A container that represents a coordinate point.
    Functions to manipulate the coordinates (called methods)
    can be added here.
    You can optionally convert it to a dataclass using
    functools.dataclass decorator to clean up the code further.
    """

    def __init__(self, x: float, y: float) -> None:
        """ At creation of the Coordinate object,
        we set its x, y-coordinate values. """
        self.x = x
        self.y = y

    def __repr__(self) -> str:
        """ Defines how a Coordinate should be represented as a string."""
        return f'Coordinate({self.x}, {self.y})'

X = 1350
Y = 250

coords = [Coordinate(X, Y) for _ in range(11)]

lower_bound = 1150
upper_bound = 1250

for coord in coords:
    # Here we are saying we want to change the x-coordinate,
    # instead of the ambiguous coord[0] = ...
    coord.x = random.randrange(1150, 1250)
    lower_bound -= 100
    upper_bound -= 100

print(coords)

Comment: Thank you so much! Although it's off-topic you actually helped me with a totally different problem I was having! I appreciate it! Best reagrds.


How do I remove the rollover effect from Rubik tooltips

Question: I am using Rubik and the tooltips are rollovers. Generally this is fine, but for one site I need them to display with the field instead. Is there a setting to disable the rollover effect and just make it look like help text?
Answer: I have not used this theme before, but there is a checkbox on the theme settings page to toggle "Display form field descriptions inline". I believe this is what you are looking for.


Moving the "active" class in a dropdown menu to the users current page

Question: So i thought i would give the Foundation framework a go and the first problem i run into is class="active" in a dropdown menu.
What i want is the class to apply to the page link (in the dropdown list) the user is currently viewing, if this is easily done any help would be appreciated.
I have scoured the Foundation Docs and the world wide web and gotten nowhere unfortunately.
Answer: What you are tying to achieve is usually done by the backend or my your front-end MVC (like angular or backbone) by testing where you are and printing an 'active' class on the right link.
Foundation isn't an MVC framework. It simply helps quickly developing some HTML, CSS, and JavaScript by leveraging their modular, pre made code.
That being said, if you wanted to achieve this purely on the front-end, you would need to create a function that parses your URL, finds the link with that matching URL, and add a class of 'active' on the matched on.
Comment: Damn.. I was hoping there would be a built-in function or script to do this. Not to worry, thanks for your time. I'll be adding some basic PHP to the site soon anyway so i'll sort something out there.


How to exclude value that is equal to zero while using Django aggregate queries for average or minimum

Question: I have a database table where some of the values in a column for sales are zero.
So far my query returns the min of all the values.
Sale.objects.all().aggregate(minimum=Min('sale'))
I want to exclude the values that are equal to zero while calculating the minimum or the average.
So my question is how is that query done?
Answer: Try:
Sale.objects.filter(sales > 0).aggregate(minimum=Min('sale'))

or:
Sale.objects.exclude(sales = 0).aggregate(minimum=Min('sale'))


Should I break a larger mysql table into multiple?

Question: I have a pretty large social network type site I have working on for about 2 years (high traffic and 100's of files)  I have been experimenting for the last couple years with tweaking things for max performance for the traffic and I have learned a lot.  Now I have a huge task, I am planning to completely re-code my social network so I am re-designing mysql DB's and everything.
Below is a photo I made up of a couple mysql tables that I have a question about.  I currently have the login table which is used in the login process, once a user is logged into the site they very rarely need to hit the table again unless editing a email or password.  I then have a user table which is basicly the users settings and profile data for the site.  This is where I have questions, should it be better performance to split the user table into smaller tables?  For example if you view the user table you will see several fields that I have marked as "setting_" should I just create a seperate setting table?  I also have fields marked with "count" which could be total count of comments, photo's, friends, mail messages, etc. So should I create another table to store just the total count of things?
The reason I have them all on 1 table now is because I was thinking maybe it would be better if I could cut down on mysql queries, instead of hitting 3 tables to get information on every page load I could hit 1.
Sorry if this is confusing, and thanks for any tips.
alt text http://img2.pict.com/b0/57/63/2281110/0/800/dbtable.jpg
Comment: You have a pretty large _what_?
Comment: I think you meant to tag this question as "schema", not "scheme".
Comment: I see some parenthesis', don't you?
Answer: As long as you don't SELECT * FROM your tables, having 2 or 100 fields won't affect performance.
Just SELECT only the fields you're going to use and you'll be fine with your current structure.
Comment: I understand this, sorry I wasn't more clear in this case by large I mean by how manycolumns are in the user table
Comment: It's ok, what I meant is that number of columns in the table won't affect performance. On most DB Engines anyway... are you using InnoDB?
Comment: actually I believe the current user table is not InnoDB but I will probably make this new table be InnoDB for the row locking vs table locking
Answer: You'll need to compare performance testing results between the following:

Leaving it alone
Breaking it up into two tables
Using different queries to retrieve the login data and profile data (if you're not doing this already) with all the data in the same table

Also, you could implement some kind of caching strategy on the profile data if the usage data suggests this would be advantageous.
Comment: All good points and I have done probably hundreds of hours of testing over the past 2 years on this same site, I got it pretty fast but now I am re-coding everything and this is the perfect chance to re-arrange any DB tables.  The hard-er part is testing is and breaking it up and testing it because honestly the difference can not be that great, however when you have a lot of traffic and millions of mysql records it could change.  thanks for the tips though
Comment: Absolutely. It might be helpful if you posted your testing results, as those are what really matter here.
Answer:
should I just create a seperate setting table?
So should I create another table to store just the total count of things?

There is not a single correct answer for this, it depends on how your application is doing.
What you can do is to measure and extrapolate the results in a dev environment.
In one hand, using a separate table will save you some space and the code will be easier to modify.
In the other hand you may lose some performance ( and you already think ) by having to join information from different tables.
About the count I think it's fine to have it there, although it is always said that is better to calculate this kind of stuff, I don't think for this situation it hurt you at all.
But again, the only way to know what's better your you and your specific app, is to measuring, profiling and find out what's the benefit of doing so.  Probably you would only gain 2% of improvement.
Answer: You should consider putting the counter-columns and frequently updated timestamps in its own table --- every time you bump them the entire row is written.
Answer: I wouldn't consider your user table terrible large in number of columns, just my opinion.  I also wouldn't break that table into multiple tables unless you can find a case for removal of redundancy.  Perhaps you have a lot of users who have the same settings, that would be a case for breaking the table out.
Answer: Should take into account the average size of a single row, in order to find out if the retrieval is expensive. Also, should try to use indexes as while looking for data...
The most important thing is to design properly, not just to split because "it looks large". Maybe the IP or IPs could go somewhere else... depends on the data saved there.
Also, as the socialnetworksite using this data also handles auth and autorization processes (guess so), the separation between login and user tables should offer a good performance, 'cause the data on login is "short enough", while the access to the profile could be done only once, inmediately after the successful login. Just do the right tricks to improve DB performance and it's done.
(Remember to visualize tables as entities, name them as an entity, not as a collection of them)
Comment: thanks for the tips, could you ellaborate on what you mean by (Remember to visualize tables as entities, name them as an entity, not as a collection of them)?  Thanks
Comment: Right. What I meant was (as an old habit, somewhat useful) to name the tables in singular, heh. No big deal, it just means you have a collection of rows, each one related to a... login,... user,... location,... and so on. Nothing special, though.
Answer: Two things you will want to consider when deciding whether or not you want to break up a single table into multiple tables is:

MySQL likes small, consistent datasets. If you can structure your tables so that they have fixed row lengths that will help performance at the potential cost of disk space. One thing that from what I can tell is common is taking fixed length data and putting it in its own table while the variable length data will go somewhere else.
Joins are in most cases less performant than not joining. If the data currently in your table will normally be accessed all at the same time then it may not be worth splitting it up as you will be slowing down both inserts and quite potentially reads. However, if there is some data in that table that does not get accessed as often then that would be a good candidate for moving out of the table for performance reasons.

I can't find a resource online to substantiate this next statement but I do recall in a MySQL Performance talk given by Jay Pipes that he said the MySQL optimizer has issues once you get more than 8 joins in a single query (MySQL 5.0.*). I am not sure how accurate that magic number is but regardless joins will usually take longer than queries out of a single table.


polynomial extrapolation using numpy

Question: I have a signal that I wish to fit using polynomial fitting first and then use that polynomial expression for extrapolation. Meaning, originally, my x axis range is 192.1-196THz. And I have corresponding y axis values for this range. But I want to plot the signal from 191.325-196.125 THz, meaning I need to predict y axis values outside of my original x axis range. Here is my code
import numpy as np
import matplotlib.pyplot as plt
import numpy.polynomial.polynomial as poly

y=[9.996651009,10.13327408,10.27003353,10.43754263,10.60523071,10.71136134,10.81753489,10.92299416,11.02853271,11.10093835,11.17349619,11.16565259,11.15787539,11.16191795,11.16626826,11.16087842,11.15561384,11.16940196,11.18346164,11.22434472,11.26537218,11.35981126,11.45427174,11.6102862,11.76633923,11.89058563,12.01486743,12.21133227,12.40782023,12.55682327,12.7059072,12.88910065,13.07234341,13.20139753,13.33048759,13.46960371,13.60875466,13.71956075,13.83042354,13.91157425,13.99277241,14.08548465,14.17824161,14.26091536,14.34364144,14.43504833,14.52651725,14.59548578,14.66449439,14.73751377,14.81058218,14.88687742,14.96322511,15.038278,15.11342222,15.1644832,15.21556914,15.31798775,15.42044285,15.51000031,15.59959896,15.68089263,15.76229354,15.86213633,15.96214484,16.0269976,16.0923806,16.16483146,16.23762336,16.24233076,16.24719576,16.26116812,16.27543692,16.28311024,16.29128992,16.23458771,16.17830522,16.12636211,16.07478778]

y.reverse()

# x axis for polyfit
i = 192.1e12
x=[]
while i <= 196e12:
  x.append(i)
  i += 50e9

plt.plot(x, y)

# x axis for extrapolation
i1= 191.325e12
x2=[]
while i1 <= 196.125e12:

  x2.append(i1)
  i1 += 50e9

z = poly.polyfit(x, y, 20)
f = poly.polyval(x, z)

plt.plot(x2, f(x2), linestyle='--')

Problems:
1.
The error I am getting : 'numpy.ndarray' object is not callable. (line 37)
I cannot find a work around.
2.
If I put more than 21 as the number of coefficients then I run into the following problem: SVD did not converge in Linear Least Squares. (line 34)
I basically have no idea about this problem.
Can anyone give me some ideas?
Thank you.
Comment: `If I put more than 21 as the number of coefficients then I run into the following problem: SVD did not converge` Is there a good reason to expect a really high degree polynomial to provide a good fit? I would bet that the polynomial goes crazy as soon as you move slightly outside of the area where you have known values.
Comment: @NickODell it is true that the polynomial will go crazy. however i do want to know about this error message i am getting in general and also i would like to do a comparison of accuracy for different orders of polynomial fitting.
Answer: Extrapolation is inaccurate, but you can do it using scipy.interpolate.interp1d by telling it not to give bounds errors (bounds_error=False) and to fill the values via extrapolation (fill_value="extrapolation"). In my code, I used a cubic spline fit. Also, make use of numpy functions to generate the x arrays.
import numpy as np
import matplotlib.pyplot as plt
from scipy.interpolate import interp1d

plt.close("all")

y = [9.996651009, 10.13327408, 10.27003353, 10.43754263, 10.60523071,
     10.71136134, 10.81753489, 10.92299416, 11.02853271, 11.10093835,
     11.17349619, 11.16565259, 11.15787539, 11.16191795, 11.16626826,
     11.16087842, 11.15561384, 11.16940196, 11.18346164, 11.22434472,
     11.26537218, 11.35981126, 11.45427174, 11.6102862, 11.76633923,
     11.89058563, 12.01486743, 12.21133227, 12.40782023, 12.55682327,
     12.7059072, 12.88910065, 13.07234341, 13.20139753, 13.33048759,
     13.46960371, 13.60875466, 13.71956075, 13.83042354, 13.91157425,
     13.99277241, 14.08548465, 14.17824161, 14.26091536, 14.34364144,
     14.43504833, 14.52651725, 14.59548578, 14.66449439, 14.73751377,
     14.81058218, 14.88687742, 14.96322511, 15.038278, 15.11342222,
     15.1644832, 15.21556914, 15.31798775, 15.42044285, 15.51000031,
     15.59959896, 15.68089263, 15.76229354, 15.86213633, 15.96214484,
     16.0269976, 16.0923806, 16.16483146, 16.23762336, 16.24233076,
     16.24719576, 16.26116812, 16.27543692, 16.28311024, 16.29128992,
     16.23458771, 16.17830522, 16.12636211, 16.07478778]
y = np.array(y)[::-1]

step = 50e9
x = np.arange(192.1e12, 196e12+step, step)
x2 = np.arange(191.325e12, 196.125e12+step, step)

f = interp1d(x, y, kind="cubic", bounds_error=False, fill_value="extrapolate")

fig, ax = plt.subplots()
ax.plot(x, y, ".", label="data")
ax.plot(x2, f(x2), "--", label="fit")
ax.legend()
ax.set_xlabel("x")
ax.set_ylabel("y")
fig.show()

Edit: If you do need a polynomial fit, then make use of np.poly1d. This returns a function that can be evaluated, unlike polyval which only returns the values (hence the error you were getting).
import numpy as np
import matplotlib.pyplot as plt

plt.close("all")

y = [9.996651009, 10.13327408, 10.27003353, 10.43754263, 10.60523071,
     10.71136134, 10.81753489, 10.92299416, 11.02853271, 11.10093835,
     11.17349619, 11.16565259, 11.15787539, 11.16191795, 11.16626826,
     11.16087842, 11.15561384, 11.16940196, 11.18346164, 11.22434472,
     11.26537218, 11.35981126, 11.45427174, 11.6102862, 11.76633923,
     11.89058563, 12.01486743, 12.21133227, 12.40782023, 12.55682327,
     12.7059072, 12.88910065, 13.07234341, 13.20139753, 13.33048759,
     13.46960371, 13.60875466, 13.71956075, 13.83042354, 13.91157425,
     13.99277241, 14.08548465, 14.17824161, 14.26091536, 14.34364144,
     14.43504833, 14.52651725, 14.59548578, 14.66449439, 14.73751377,
     14.81058218, 14.88687742, 14.96322511, 15.038278, 15.11342222,
     15.1644832, 15.21556914, 15.31798775, 15.42044285, 15.51000031,
     15.59959896, 15.68089263, 15.76229354, 15.86213633, 15.96214484,
     16.0269976, 16.0923806, 16.16483146, 16.23762336, 16.24233076,
     16.24719576, 16.26116812, 16.27543692, 16.28311024, 16.29128992,
     16.23458771, 16.17830522, 16.12636211, 16.07478778]
y = np.array(y)[::-1]

step = 50e9
x = np.arange(192.1e12, 196e12+step, step)
x2 = np.arange(191.325e12, 196.125e12+step, step)

z = np.polyfit(x, y, 20)
p = np.poly1d(z)

fig, ax = plt.subplots()
ax.plot(x, y, ".", label="data")
ax.plot(x2, p(x2), "--", label="fit")
ax.legend()
ax.set_xlabel("x")
ax.set_ylabel("y")
fig.show()

Edit: Using the more updated np.polynomial.Polynomial methods, we get a similar result using a 5th-order polynomial fit.
import numpy as np
import matplotlib.pyplot as plt
from numpy.polynomial import Polynomial

plt.close("all")

y = [9.996651009, 10.13327408, 10.27003353, 10.43754263, 10.60523071,
     10.71136134, 10.81753489, 10.92299416, 11.02853271, 11.10093835,
     11.17349619, 11.16565259, 11.15787539, 11.16191795, 11.16626826,
     11.16087842, 11.15561384, 11.16940196, 11.18346164, 11.22434472,
     11.26537218, 11.35981126, 11.45427174, 11.6102862, 11.76633923,
     11.89058563, 12.01486743, 12.21133227, 12.40782023, 12.55682327,
     12.7059072, 12.88910065, 13.07234341, 13.20139753, 13.33048759,
     13.46960371, 13.60875466, 13.71956075, 13.83042354, 13.91157425,
     13.99277241, 14.08548465, 14.17824161, 14.26091536, 14.34364144,
     14.43504833, 14.52651725, 14.59548578, 14.66449439, 14.73751377,
     14.81058218, 14.88687742, 14.96322511, 15.038278, 15.11342222,
     15.1644832, 15.21556914, 15.31798775, 15.42044285, 15.51000031,
     15.59959896, 15.68089263, 15.76229354, 15.86213633, 15.96214484,
     16.0269976, 16.0923806, 16.16483146, 16.23762336, 16.24233076,
     16.24719576, 16.26116812, 16.27543692, 16.28311024, 16.29128992,
     16.23458771, 16.17830522, 16.12636211, 16.07478778]
y = np.array(y)[::-1]

step = 50e9
x = np.arange(192.1e12, 196e12+step, step)
x2 = np.arange(191.325e12, 196.125e12+step, step)

p = Polynomial.fit(x, y, 5)

fig, ax = plt.subplots()
ax.plot(x, y, ".", label="data")
ax.plot(x2, p(x2), "--", label="fit")
ax.legend()
ax.set_xlabel("x")
ax.set_ylabel("y")
fig.show()

Comment: thank you the detailed explanation. However, the documentation (https://numpy.org/doc/stable/reference/routines.polynomials.html)  states clearly to avoid np.polyfit, np.polyval, and np.poly1d, and instead to use only the new(er) package. That is why I tried to avoid np.poly1d.
Comment: I've updated my answer to include code using `np.polynomial.Polynomial`.
Comment: @MRR does this now answer your question?
Comment: thank you for your revised answer. it answers my first problem. however i am waiting for the 2nd answer.
Comment: As per the [documentation](https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html), "Note that fitting polynomial coefficients is inherently badly conditioned when the degree of the polynomial is large." It recommends using `x - x.mean()` to improve the fit, and from my tests the fit is, in fact, improved, but the extrapolation is worse.
Answer:
it is true that the polynomial will go crazy. however i do want to know about this error message i am getting in general and also i would like to do a comparison of accuracy for different orders of polynomial fitting

It's useful to take a moment and think about what intermediate values the polynomial fitting code will have while it is fitting your polynomial. Your X values are on the order of 1.9 * 10^14. If you then take the 22nd power of that, you have the number 2.8 * 10^314. This is bigger than the maximum representable double-precision float.
Interestingly, even when your X^20 term fits into the range of a float, it seems that it still is suffering from floating-point precision problems. Here are the coefficients it fits for a degree 20 polynomial:
>>> print(z)
[ 7.01389718e+008 -7.23272471e-006 -6.18262251e-021  1.28113394e-034
  6.59034262e-049 -5.97742101e-066 -1.75197282e-077 -9.00617222e-092
  1.16972981e-106  3.58691274e-120 -9.24694178e-135  0.00000000e+000
  0.00000000e+000  0.00000000e+000  0.00000000e+000  0.00000000e+000
  0.00000000e+000  0.00000000e+000  0.00000000e+000  0.00000000e+000
  0.00000000e+000]

In other words, the coefficients it fits for everything beyond X^10 are zero. This is happening due to floating point precision problems.
How can this be fixed? You could take inspiration from machine-learning practitioners, and standardize your inputs. "Standardization" refers to subtracting the mean and dividing by the standard deviation of the input. Since this is a linear transformation, it is mathematically equivalent to fitting without standardization, while avoiding issues of overflow. When I do this, the coefficient it chooses for X^22 is -0.11, which is the equivalent of -5.3 * 10^-316 without standardization. Without standardization, this number cannot be expressed as a float.
Here is the code I used:
import numpy as np
import matplotlib.pyplot as plt
import numpy.polynomial.polynomial as poly
from sklearn.preprocessing import StandardScaler

y=[9.996651009,10.13327408,10.27003353,10.43754263,10.60523071,10.71136134,10.81753489,10.92299416,11.02853271,11.10093835,11.17349619,11.16565259,11.15787539,11.16191795,11.16626826,11.16087842,11.15561384,11.16940196,11.18346164,11.22434472,11.26537218,11.35981126,11.45427174,11.6102862,11.76633923,11.89058563,12.01486743,12.21133227,12.40782023,12.55682327,12.7059072,12.88910065,13.07234341,13.20139753,13.33048759,13.46960371,13.60875466,13.71956075,13.83042354,13.91157425,13.99277241,14.08548465,14.17824161,14.26091536,14.34364144,14.43504833,14.52651725,14.59548578,14.66449439,14.73751377,14.81058218,14.88687742,14.96322511,15.038278,15.11342222,15.1644832,15.21556914,15.31798775,15.42044285,15.51000031,15.59959896,15.68089263,15.76229354,15.86213633,15.96214484,16.0269976,16.0923806,16.16483146,16.23762336,16.24233076,16.24719576,16.26116812,16.27543692,16.28311024,16.29128992,16.23458771,16.17830522,16.12636211,16.07478778]

y.reverse()

# x axis for polyfit
i = 192.1e12
x=[]
while i <= 196e12:
    x.append(i)
    i += 50e9

plt.plot(x, y)

# x axis for extrapolation
i1= 191.325e12
x2=[]
while i1 <= 196.125e12:

    x2.append(i1)
    i1 += 50e9

x = np.array(x)
x2 = np.array(x2)

scaler = StandardScaler()
scaler.fit(x.reshape(-1, 1))
x_rescaled = scaler.transform(x.reshape(-1, 1))
x2_rescaled = scaler.transform(x2.reshape(-1, 1))

z = poly.polyfit(x_rescaled.reshape(-1), y, 20)
f = poly.polyval(x2_rescaled.reshape(-1), z)

plt.plot(x2, f, linestyle='--')
plt.ylim([7, 20])

I should mention that doing an extrapolation on a 20th degree polynomial is likely to be useless outside the region you are fitting. A polynomial of that degree is extremely sensitive to small changes.
For example, I added random noise of magnitude 0.01, then re-fit the polynomial. I repeated this process nine times. Visually, you can barely see the noise, but the noise has an enormous effect on the polynomial fit. Half of the resulting polynomials point up, and half of them point down.

See also Why is the use of high order polynomials for regression discouraged?
Comment: thank you for this informative insight. but i am a bit curious as to why the extrapolated part (outside of my fitting region) goes up sometimes and down the other times- why am i not getting a consistent pattern?
Comment: @MRR The random noise I'm adding ends up having a non-zero correlation with the high-order elements of the polynomial. It's a small correlation, sometimes negative, sometimes positive, but it's not zero. Therefore, the polynomial fit can be made very slightly better by changing the high-order polynomial coefficients. Changing the high-order polynomial coefficient has a small effect in the middle of the data, but a massive effect outside the fitting region.

polynomial extrapolation using numpy


How to recreate a sqlite database in ionic v3?

Question: I am using sqlite with ionic v3. Problem is that when i delete the sqlite db and recreate it will throw an error No such table table1.
this.sqlite.deleteDatabase({
        name: 'db.db',
        location: 'default'
      })
       .then(() => {
           return this.sqlite.create({
             name: 'db.db',
             location: 'default'
           })
       })
       .then((db: SQLiteObject) => {
        // create table queries
       });

Comment: Why you need to delete db i look they are same
Comment: I want to wipe out all the data.
Comment: actually best practices is if you delete table
Comment: If I delete and recreate all the tables then the problem is that previous data exist until I close the app from the background and reopen. In my app, I want to delete all the data on logout and recreate on login.
Comment: First of all you need to create service for table for all type of database operation then when you want to logout then drop table by services.
Comment: I am using service for db operations.
Answer: becasuse of you only create the database. you need to create table after  creating database
your code looks like this way
this.sqlite.deleteDatabase({
        name: 'db.db',
        location: 'default'
      })
       .then(() => {
           return this.sqlite.create({
             name: 'db.db',
             location: 'default'
           })
       })
       .then((db: SQLiteObject) => {
        // create table queries
             db.executeSql('create table table1(name VARCHAR(32))', {})
            .then(() => console.log('Executed SQL'))
            .catch(e => console.log(e));
         });
       });


solr schema design, multiple key/vals

Question: I have a schema I'm trying to design in solr the data is simple with just a productKey, then I take that product key, hit a few different websites and store their product descriptions of that product for searching.
for example:
productKey: 1234, I hit siteA and got back "cool razor", siteB I back "neat razor"
so ideally in a document I'd like to store something that's similar in concept to:
(PSUDOCODE)
<code><doc>
<productKey>1234</productKey>
<site>siteA</site>
<desc>coolrazor</desc>
<site>siteB</site>
<desc>near razor</desc>
</doc>
</code>
but I'd like to be able to do searches like, show me all the docs where siteA thought something had "cool razor" in it only. right now it would pull any site that had razor in the desc.
any fix for that?
Comment: why dont you use redis for your task instead of solr...this problem is well suited for redis.
Comment: also please improve your sentence a bit starting from "site A though something had......" how is site A thinking?
Comment: this problem is not suited for redis, redis is a key lookup, not a search index.
Answer: What type of Analyser are you using for desc field? I think, you will need to use solr.KeywordTokenizerFactory on the description field.
Refer to http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
Comment: I'm not using anything at the moment, trying to figure out how to get started on a correct schema
Answer: I would create one document per site, having something else than productKey as the unique id field.
Comment: assume 100 million prouducts, and data from 50 different sites, so a doc per site would yield 500 million docs in that case.

solr schema design, multiple key/vals


Using derivative information to improve Gaussian Process regression

Question: I do not exactly know whether the question fits SE stats criteria, if it doesn't let me know.
I am looking for a approach which can be used to improve a GP regression esimate if the derivatives are also known. This paper by Solak et al. discussions similar to this, but I could not understand it properly. I tried to search another paper on same topic but I couldn't get. Can someone suggest me another paper or an example on this.
Answer: I actually just stumbled on this paper and got very interested in this. Since this was posted 2 years ago, I am not sure OP is still looking for an explanation of how this works.
The idea is that it is essentially a multi-output Gaussian process, except that the full covariance function is specified once you specify the covariance function for the function observations. This is because the derivative is a linear operation and the resulting process for derivative is also Gaussian. Furthermore, the relationship between function and derivative imposes constraints on what the covariance function looks like. So if you have a function with input $\mathbf{x}=(x_1, x_2,...,x_p)$ (each one of your input is $p$-dimensional) , you are then fitting a Gaussian process with $(p+1)$ outputs ($1$ for function values and $p$ partial derivatives) and the covariance function is as specified in the paper.
The problem with adding derivative information is drastic increase in computational time if your function has high dimensional input. When you only have function observation, you only need to solve for a $n\times n$ matrix assuming you have $n$ data points. But with $p$-dimensional input, you now need to solve for a $n(p+1)\times n(p+1)$ matrix which will be extremely costly for large $p$. So it depends on whether you're willing to have this tradeoff between computational cost and accuracy.
Hope this helps.


Three unknowns, the last row of the matrix contains all zeros.

Question: I'm in the process of solving a system of three linear equations in three unknowns, the last row of the matrix contains all zeros. How does this affect the solution of the system of equations?
Comment: Do you mean  *how* to solve? It doesn't affect things at all; just continue solving. We may be able to get qualitative information about solutions; information about the rows tells us something about the row space.... Or just finish solving and see the behavior for yourself.
Answer: It means that there is a "free variable", and infinitely many solutions to the system, because that "free variable" can be assigned any value. The other variables can often be expressed in terms of the free unknown variable; that is, their values will often depend on the the unknown value, depending on how "reduced" your matrix is.
E.g. If we have the augmented coefficient matrix reduced to the following, representing a system of three equations in the three unknowns $x, y, z$:
$$
\begin{pmatrix}
1 & 1 & 1 & 1 \\
0 & 1 & 0 & 2 \\
0 & 0 & 0 & 0 \\
\end{pmatrix}
$$
Then $z$ is the free variable. $y = 2$, and $x = 1 - y - z = 1 - 2 - z = -(1 + z)$.
So if we let $z = \alpha$, we'd have the an infinite "family" of solutions which can be represented as
$$ \begin{pmatrix}\\ \\
-(1 + \alpha) \\ \\
2 \\ \\
\alpha \\ \\
\end{pmatrix}$$
which depends on the value of $\alpha$, which can be any of infinitely many values.
Hence, there are an infinite number of solutions.
But this still allows us to impose some constraints on the solutions: specifically, $y$ must be $2$, and whatever the value of $\alpha$, $x = -(1+ \alpha)$.
Answer: that means the system has infinitely many solutions, and if your last matrix is in ref or rref then you will have somthing like this$$ x_1+2x_2+x_3=3\\
x_2-x_3=4 $$which implies $$x_1=3-2x_2-x_3\\ \text{and}\hspace 2mm x_2=4+x_3$$ let$x_3=t$ then you will get the value of $x_1,x_2$ and hence your solution.
Comment: If $x_3=t$, then $x_2=4+t$. There is only one parameter needed here.
Comment: @julien Yes you are correct. Just edited it, Thank you.


SQL Server "NOT IN" taking a lot of time

Question: I'm using the SQL query shown below in SQL Server to load the data every week. To avoid duplicates, I have been using NOT IN to check the table which I'm inserting.
This query takes long time and it hangs occasionally. Only a few thousands rows are being inserted every week
INSERT INTO [ST_FACT]
    SELECT
        H.WK_DATE,
        H.[M_Code]
    FROM
        T_FACT H
    WHERE
        H.WK_DATE NOT IN (SELECT DISTINCT(WK_DATE)
                          FROM ST_FACT
                          WHERE RETAILER LIKE '%MSI%')

Comment: Are you sure it's the `not in` that's slow? Could it be that the `WHERE RETAILER LIKE '%MSI%'` is the slow part? See how long that subquery takes to run.
Comment: Thanks,it runs very quickly
Answer: I would suggest using NOT EXISTS:
INSERT INTO [ST_FACT]
    SELECT H.WK_DATE, H.[M_Code]
    FROM T_FACT H
    WHERE NOT EXISTS (SELECT 1
                      FROM ST_FACT f
                      WHERE H.WK_DATE = f.WK_DATE AND RETAILER LIKE '%MSI%'
                     );

For this to work, create an index on ST_FACT(WK_DATE, RETAILER).


Expected value question solution

Question: We flip 2 coins:Our random value is the number of heads
Possible values: $$\begin{array}{|c|c|c|c|}\hline &TT& HT ~~~ TH & HH\\\hline
X&       0&       1&       2\\ \hline
p(X=x)&  0.25&    0.5&     0.25\\\hline\end{array}$$
To get the expected value we find this sum : $$\begin{align}\left.\sum_{x=0}^2 \middle(x\cdot p(X=x)\right)~&=~ 0\cdot 0.25+1\cdot 0.5+2\cdot 0.25\\[1ex]&=0.5+0.5\\[1ex]&=1\end{align}$$
Is that correct? I just started studying probabilities so go easy on me.
Comment: Yes, it is correct (assuming the coins are fair).  Another way to see it is to note that the expected number of heads must equal the expected number of tails and that the two must sum to $2$.
Comment: You would notice in how you originally wrote your post, the stars were not appearing.  That is because if you want to type something *with italics* you use the stars around it to write it like `*with italics*`.  Similarly **bold words** are written as `**bold words**` with two stars surrounding them.  In order to have math appear correctly, have a look at how I edited this and take a look at this [MathJax primer](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) to get an idea how to properly typeset equations with MathJax and $\LaTeX$.
Comment: Your table seems accidentally shifted.  I believe that row labels $X$ and $P[X=x]$ should shift to the left, to leave room for four numerical values in each of the two rows.
Comment: @Michael I hadn't put much thought into how the OP had originally formatted their table, just went and put how it appeared on the edit page before the parser got a hold of it and typeset an equivalent table.  I've gone ahead and taken your observation into account and edited further.
Comment: @JMoravitz : I couldn't help making a new column and making hte $X$ labels correct.  =)
Comment: Oh you just undid what I did, I think to make the X labels correct by changing the labels not the numbers.
Comment: @Michael as for having HT and TH in separate columns, I disagree with that, as the purpose to try to treat the different values of $X$ as the index is common.  Else, you'll have the third row not make much sense as $p(X=1)$ is $0.5$ but in your edit you'd have two columns each labeled as $X=1$ each of which with the implication that $p(X=1)$ was $0.25$.  If the third row were instead written as $p(\{HT\})$ in the one column and $p(\{TH\})$ in the other, then I'd agree with separating them.  Regardless, this discussion of presentation is a bit off topic.
Comment: Either way is fine, now the table makes sense.  I was thinking of the sample space as $$S = \{(T,T), (T,H), (H,T), (H,H)\}$$ and  random variable $X:S\rightarrow\mathbb{R}$ defined by $X(T,T)=0$, $X(T,H)=1$, $X(H,T)=1$, $X(H,H)=2$.
Answer: Another way to look at the problem is to observe that flipping 2 fair once coins or flipping twice a single fair coin is the same. So the number of H (or T) is distributed with a Binomial law
$$X\sim Bin(n;p)=Bin(2;\frac{1}{2})$$
with mean $\mathbb{E}[X]=n\cdot p=1$

Expected value question solution


angularui bootstrap typeahead- prepopulate dropdown list onfocus

Question: I am using angularui bootstrap typeahead module for my project.
I am unable to pre-populate the drop-down with predefined value.
Whenever user click on input text of typeahead, it should automatically show the typeahead suggestion dropdown(from a static JSON).
Whenever user starts typing, then the behaviour should be normal.
I tried this solution but unfortunately it stopped working when I upgraded my angularjs to version  1.3
Answer: So I essentially accomplished this by making a custom directive and making a minor modification to the ui bootstrap typeahead code. You have to trick the typeahead into thinking someone has typed something in it.  I placed a little pulldown arrow to the right of the field so essentially it looked like a pulldown and clicking on that arrow would show ALL choices. You should be able to do it using onfocus.
Basically find the code in the bootstrap typeahead that binds to the key events and I changed it to check for a keydown event of 40
<code>           if(scope.matches.length === 0 && evt.which === 40) {  // Added
         //   COMMENT OUT  modelCtrl.$setViewValue(modelCtrl.$viewValue);
                          modelCtrl.$setViewValue('');  // Added
            }
</code>
You'll also need to make sure the typeaheadMinLength is set to 0.  Note in the code above it checks for the case of the user NOT having typed anything AND the special keydown event that I trigger. You should be able to change this to trigger on onfocus. I copied the bootstrap typeahead to mytypeahead.js and then modified it as above, along with a few other minor mods that you may or may not need depending on if the field is "required" or not.


attach file to a node by code

Question: I wanted to associate a file to a node. so far so good. create a cck type file, and the problem was solved. but I can not do this, I do not want the user to choose the file. the file in question is already in the system.
I have tried to place the file as # default_value field and hide it with the hook_form_FORM_ID_alter, but failed.
function my_module_form_node_form_alter(&$form, $form_state, $form_id) {
    if(isset($form['type']) && isset($form['#node'])) {
        $type = $form['#node']->type;

        if(stripos($type, 'node-type') === FALSE)
            return;

        switch($type) :
            case 'node-type_xyz':
                $fid = arg(3);
                $file = file_load($fid);

                // make a cck field_invoice a hidden field
                $form['field_invoice']['#prefix'] = '
';
                $form['field_invoice']['#suffix'] = '
';

                $form['field_company']['und'][0]['value']['#default_value'] = 'ABC';
                $form['field_account_number']['und'][0]['value']['#default_value'] = '09879';
                break;
        endswitch;
    }
}

anyone have any suggestions?
Answer: This can be a bit tricky, there are a few steps to get right. First create and save the node, then add the file to the node as a second pass. Here's an example implementation.
/**
 * Add a file to a field on a node
 * @param int $nid  node ID to add the content to
 * @param string $field_name the machine name of the field to add to
 * @param string $filedata the raw binary data as a string
 * @param optional boolean is the attachment an image (should image validation
 *        be performed)
 */
function mymodule_addfile($nid, $field_name, $filedata, $is_image = FALSE) {
  $node = node_load($nid);

  $field = content_fields($field_name, $node->type);

  // Load up the appropriate validators
  $validators =
  filefield_widget_upload_validators($field),
  if ($is_image) {
    $validators = array_merge(
      $validators,
      imagefield_widget_upload_validators($field)
    );
  }

  // Where do we store the files?
  $tmp_files_path = file_directory_temp()."/$file_name";
  $files_path     = filefield_widget_file_path($field);

  // Put the file in the temp folder
  $ret = file_put_contents($tmp_files_path,$filedata);
  unset($filedata);
  if ($ret === FALSE) {
    watchdog('mymod_error',"Could not write $nid ($file_name) to temp: $tmp_files_path");
    drupal_set_message("Could not write $nid ($file_name) to temp: $tmp_files_path", 'error');
    return FALSE;
  }

  $path = $files_path.'/'.$file_name;

  // Create the file object
  $file = field_file_save_file($tmp_files_path, $validators, $path);

  if ($file == 0) {
    watchdog('mymod_error',"Failed to save file $file_name");
    drupal_set_message("Failed to save file $file_name",'error');
    return FALSE;
  }

  // Apply the file to the field
  $node->{$field_name}[0] = $file;

  node_save($node);
  return TRUE;
}

A variation on the bottom works too, just assign the result to the file field.

function _fhi_add_existing_file($file_drupal_path, $uid=1, $status=FILE_STATUS_PERMANENT) {
  $fs       = filesize($file_drupal_path);

  $res = db_query("SELECT fid FROM files WHERE filepath = '%s' AND filesize = '%s'", $filepath, $fs);

  $fid = db_fetch_array($res);

  if ($fid) {
    //drupal_set_message("$filepath already found. fid: $fid");
    $fid = $fid['fid'];
    return field_file_load($fid);
  } else {
    $file=(object)array(
      'filename'  => basename($file_drupal_path),
      'filepath'  => $filepath,
      'filemime'  => file_get_mimetype($file_drupal_path),
      'filesize'  => $fs,
      'uid'       => $uid,
      'status'    => $status,
      'timestamp' => time()
    );

    drupal_write_record('files',$file);
    return field_file_load($file_drupal_path);
  }

}

This code drawn from memory and without interpreter (I've got too much time on my hands this morning) so be wary of typos I may have introduced. As with anything, not responsible for death of poodles as a result of the use of this code.
Comment: thx for responding. I have a question, how does the first step of the step to the second step?
Comment: depends on your implementation, most probably you're adding nodes in a module in which case,
    // Do your node create stuff
    node_save($node);
    mymodule_addfile($node->nid, "field_name", $filedata);

attach file to a node by code


Run a local script on a remote server using ssh with out having to worry about quotes

Question: So I have been running local scripts fine on a remote server:
ssh user@server "`cat local-script.sh`"

However, today I have a script that has both single and double quotes in it. Which causes the script to fail because the output of cat local-script.sh is wrapped in quotes. With out modifying the script itself, is there a better way to handle this?
I thought this may work:
ssh user@server $(<local-script.sh)

But is does not seem to do anything...
Answer: cat local-script.sh | ssh user@server bash

or whatever shell you want to use instead of bash
Answer: With the following test script:
#!/bin/bash
uname -n
a=(b c 'd e' f)
echo "${#a[@]}"
b=(1 "2 3" 4)
echo "${#b[@]}"

The output should be the remote host name followed by 4 and 3 when executed using:
ssh user@server "$(


Dynamic content view using fancybox

Question: I need to view the dynamic content in fancybox
my code looks like
View Profile
View Profile
View Profile

and fancy box content

Dynamic Profile

fancybox call
$("#featureExpertPopup").fancybox({
        'titlePosition'     : 'inside',
        'transitionIn'      : 'none',
        'transitionOut'     : 'none'

    });

Now the problem is popup opening for first anchor link only, i want to open popup for each link with different content (from database)based on profile id.
Answer: You can't use same ID for different elements. Use class instead of ID
View Profile
View Profile
View Profile

$(".featureExpertPopup").fancybox({
        'titlePosition'     : 'inside',
        'transitionIn'      : 'none',
        'transitionOut'     : 'none'

    });

Comment: Thanks Miqdad Ali, I fixed it.
Answer: you should not use one ID for multiple elements, use class instead:
View Profile
View Profile
View Profile \

$(".featureExpertPopup").fancybox({ ... })

$('a').click(function(e){
    e.preventDefault();
    var href = $(this).attr('href') + $(this).attr('id');
})

Comment: Hi, thanks for immediate response. Its working fine. Now i want send the id to generate dynamic content-`View Profile `
Comment: can we get the id using php get method or some other way? i need to pass the id with `href` like `href="profile.php?id=1"`
Comment: @LearneR unfortunately no, try the updated answer, you can also modify the `href` attribute manually.
Comment: Finally i got it, I made a separate file for popup content and passed the id as query string. Working fine. Thanks for your time.

Dynamic content view using fancybox


Fluxo de trabalho no R: estratégias para organizar um projeto de análise de dados

Question: Tomando como base esta pergunta do SOEN, pergunto:
Quais estratégias vocês recomendam para organizar um projeto de análise de dados no R? O projeto geralmente envolve as seguintes etapas (não necessariamente nessa ordem):

Carregar e "limpar" uma base dados "bruta";
Manipular a base de dados para deixá-la nos formatos necessários para visualização e análise;
Realizar as análises, construir os gráficos e tabelas;
Produzir os relatórios finais;
Também, em geral, será necessário criar suas próprias funções para executar os passos anteriores.

Especificando melhor alguns pontos que podem ser abordados (não precisa abordar todos os pontos, são sugestões para orientar as respostas):
Qual a estrutura de pastas e arquivos .R utilizar?
É recomendável colocar todas as funções no mesmo arquivo ou em arquivos separados?
Valeria a pena criar um pacote para essas funções, ao invés de ficar dando source?
Que funções, pacotes etc são recomendados para gerenciar esse processo? Controle de versão é recomendável em um projeto deste tipo?
Com relação aos scripts de execução da análise, seria melhor colocar toda análise em um único script ou separar um script por atividade (criar base dados, limpar base dados etc)?
Answer: Eu tenho trabalho em projetos que envolvem a integração de diversas fontes de dados, transformação dos dados e geração de relatórios. Os scripts são principalmente em R, mas às vezes recorro a outras linguagens.
Tenho criado os seguintes diretórios: report/, script/, lib/, data/, raw-data/ e doc/. O código-fonte fica em script/ e lib/, os dados em data/ e raw-data/ e os relatórios em report/. A ideia geral é criar pequenos scripts R que transformam os dados sucessivamente até chegar a dados formatados para serem usados em relatórios.
Em raw-data/ guardo dados que foram criados ou obtidos manualmente, geralmente em arquivos .csv ou similares. Os scripts leem dados de raw-data (ou de data/), possivelmente realizam algumas transformações (filtro, agrupamento etc.) e criam arquivos em data/, geralmente usando a função saveRDS, de forma que possam ser lidos rapidamente usando a função readRDS. Cada script é pequeno e geralmente grava apenas um arquivo rds contendo um data frame. Se existirem funções usadas em mais de um script, elas ficam em arquivos na pasta lib/ e são carregados usando a função source (com a opção chdir=TRUE). Os scripts usam extensivamente o pacote dplyr.
Na pasta doc/ eu procuro manter dois diagramas atualizados: um com os data frames e suas colunas e outro que descreve o pipeline de transformação de dados (um diagrama com scripts e arquivos de dados, indicando para cada script, quais arquivos ele lê e quais ele escreve). A vantagem de documentar o pipeline é que, quando algum arquivo muda (por exemplo, por causa da chegada de dados mais atuais), é fácil determinar quais scripts precisam ser executados e em que ordem para atualizar os dados usados nos relatórios finais. Uso o yEd para criar os diagramas.

Alguns dos scripts em script/ geram relatórios. Eles são escritos para serem compilados com o knitr::spin e criam arquivos HTML em report/, frequentemente contendo gráficos gerados com o rCharts.
O projeto é mantido sob controle de versão usando o Git. Eu evito manter no controle de versão os arquivos em data/, já que eles podem ser grandes e muitos deles podem ser gerados a partir dos scripts e dos dados em raw-data/. A exceção é para arquivos de dados que são derivados de bancos de dados externos. Nesse caso eu coloco o arquivo em controle de versão para garantir que pessoas sem acesso ao banco de dados possam executar o projeto.
Um exemplo de projeto que usa esse fluxo de trabalho pode ser encontrado em https://github.com/rodrigorgs/arch-violations-bugs
A vantagem de usar vários scripts especializados é que, se alguma fonte de dados é atualizada ou se uma nova coluna precisa ser calculada em um data frame, só é preciso executar novamente os scripts que lidam com aquele data frame. Isso é especialmente importante quando o projeto envolve transformações de dados lentas ou acesso a fontes de dados externas como a web e sistemas de gerenciamento de bancos de dados.
Comment: A documentação do pipeline e até a execução dos scripts na ordem certa é um processo que pode ser automatizado. Não encontrei nenhuma solução pronta para isso, mas se alguém tiver interesse em desenvolver uma, vamos conversar.
Comment: Muito boa sua resposta, bem mais interessante inclusive que aquelas do SOEN!
Comment: Nunca parei pra estudar empacotamento em R, mas um dia tenho que parar pra olhar. Você usa https://github.com/hadley/devtools ?
Comment: Você não costuma compilar as funções em um pacote? Isso é algo que estou cada vez mais achando que vale a pena, mesmo quando são poucas funções!
Comment: Uso, mas não para tudo - uso muito o Roxygen2 também, é uma mão na roda para documentar as funções!


Epsilon-delta proof of If $\lim (x_n y_n) = z$, then $\lim(y_n) = \frac{z}{x}$

Question: I would like to prove the statement

Let $(x_n)$ and $(y_n)$ be convergent sequence with limit $x$ and $y$ respectively. If $x_n >1$ for all $n$ and $\lim(x_n y_n) = z$, then $$\lim(y_n) = \frac{z}{x}$$

My attempt:
I observed that $$\vert y_n - \frac{z}{x} \vert = \vert \frac{xy_n-z}{x} \vert =  \frac{1}{|x|}\vert xy_n-z \vert$$
From the assumption, I believe that I should get $|x_n -1|$ somewhere. But I can't made it from the equation above...
Comment: The assumption is really just to guarantee $x \not = 0$
Comment: But how can I manipulate the equation so that I can use the assumption of $\lim(x_n y_n)=z$
Answer: A very short sketch of the proof may be given as follows,

Observe that (assuming $Y$ to a non-zero the bound of $(y_n)$), \begin{align}\left|x_ny_n-xy\right|&=\left|x_ny_n-xy_n+xy_n-xy\right|\\&=\left|y_n(x_n-x)+x(y_n-y)\right|\\&\le\left|y_n(x_n-x)\right|+\left|x(y_n-y)\right|\\&\le Y\left|x_n-x\right|+\left|x\right|\left|y_n-y\right|\\&< \color{crimson}{Y\cdot\dfrac{\varepsilon}{2Y}+|x|\cdot\dfrac{\varepsilon}{2|x|}}\\&<\varepsilon\end{align}for all sufficiently large $n$. From this it follows that $xy=z$ and consequently $y=\dfrac{z}{x}$.

Can you justify the coloured step?
Answer: $\lim\limits_{n\rightarrow\infty}|y_n-\frac{z}{x_n}|=\lim\limits_{n\rightarrow\infty}|\frac{x_ny_n-z}{x}|=\lim\limits_{n\rightarrow\infty}\frac{1}{|x_n|}|x_ny_n-z|=0$ since $x_ny_n\rightarrow z$ by assumption and $x_n>1\implies x_n\neq 0$.
Comment: I am supposed to prove it in epsion-delta, But thanks!


divide 4 digit of integer into 2 in one variable as a tuple in python

Question: split(aTime), which accepts a parameter representing a time in 24-hour mode. The function will return the time as a tuple consists of two integers – the hour and minute of the parametric time.
from :
split(1240)
so aTime have 2 integer aTime(12,40)
like how we divide the 3/4 digit into two, so if the input 725 it will be 7,25 or 1234 it will be 12,34
Comment: What is the question, exactly?
Comment: The python divmod will give the result you need: divmod(240,100) -> (2,40).
I think you should omit the leading 0 in you number, because that can cause the interpreter to use octal encoding.
Comment: `0240` is invalid syntax. Do you have an integer like `240` or a string like `'0240'`?
Comment: like how we divide the 3/4 digit into two, so if the input 725 it will be 7,25 or 1234 it will be 12,34
Answer: If your time is given as an integer, then you can use divmod.
def time_split(time):
    return divmod(time, 100)

my_time = time_split(1240)

print(my_time) # (12, 40)

If your time is given as a string, then you can use slicing.
def time_split(time):
    return int(time[:-2]), int(time[-2:])

Comment: how we keep the (12,40) in one variable?
Comment: @IvandyNezha It is. in the above example it has been stored in `my_time`
Answer: Since it's unclear if the input is a string or an integer I took a generic approach.
def time_split(value):
    normalized_value = f'{value:0>4}'
    return int(normalized_value[:2]), int(normalized_value[2:])

Alternative version:
def time_split(value):
    return divmod(int(value), 100)

Comment: sorry, the input is integer
Comment: This works with both. You might want to update the answer from Olivier Melançon, because he came up with the `divmod` idea first.
Answer: In basic shool you probably learned division like this: 13 divided by 5 is 2 with 3 remaining.
We have this concept in computer programming as well, but it is usually split in two operations:

Part 1: Integer division: 13 // 5 = 2 (gives you the result)
Part 2: modulo: 13 % 5 = 3 (gives you the remainder)

For your problem, you can use 100 for that division operation. 725 divided by 100 is 7 with 25 remaining (7 hours 25 minutes). 1234 divided by 100 is 12 with 34 remaining (12 hours and 34 minutes).
def time_split(number:int) -> tuple:
    hours = number // 100
    minutes = number % 100
    return (hours, minutes)   # A tuple holding both numbers

Comment: Nice explanation.


.htaccess Rewrite rule doesn't work

Question: I know there are already a ton of these questions but I don't get it...
I'm trying to rewrite my /register.php to /register but I can't.
.htaccess code
RewriteEngine on
RewriteBase /
RewriteRule ^register$ register.php [L]

Does any of you know the answer?
Sorry for the stupid question but I'm kinda new to the whole .htaccess stuff...
Comment: What URL are you loading that makes you think this isn't working?
Comment: @user3065852: What is the location of above .htaccess file?
Answer: Do you have enabled the Apache module mod_rewrite?
I use this pattern to create user-friendly URLs. It's pretty variable. It means, that every address in format letters-numbers with a certain file extension (PHP or HTML) will be translate into letters-numbers.php (for example).
RewriteRule ^([a-z]+)-([0-9]+)\.(php|html)$ /$1-$2.$3 [L]

.htaccess Rewrite rule doesn't work


Touch Scroll on View Flipper in Android?

Question: I have to achieve that the Touch Scroll on the ViewFlipper. For Example. I have two Images. At First, ViewFlipper shows an First Image. Now I Flung the view from right to left. The First Image view Slide out left and the Second Slide in from Left. I can achieve it By this Post. But I want to Scroll the image. That is, on the Action_Move Event I want to do Touch Scroll. For Example, when I move the touch from right to left it will flung how much the touch moves. on that time the output should show both images partly.
How to do that? What I have to measure the Screen levels(height & width). Example codes are more helpful.
Answer: If you need to detect scroll on only viewflipper which is not occupying entire screen, then try the below
gestureDetector = new GestureDetector(new MyGestureDetector());

viewFlipper.setOnTouchListener(new View.OnTouchListener() {

        @Override
        public boolean onTouch(View v, MotionEvent event) {
            if (gestureDetector.onTouchEvent(event)) {
                return false;
            }
            return true;
        }
  });

and MyGestureDetector will be same as in http://www.codeshogun.com/blog/2009/04/16/how-to-implement-swipe-action-in-android/
Answer: package com.appaapps.flipper;

import android.app.Activity;
import android.content.Context;
import android.graphics.*;
import android.os.Bundle;
import android.view.MotionEvent;
import android.view.View;
import android.widget.ViewFlipper;
//------------------------------------------------------------------------------
// Flipper - Philip R Brenan at gmail.com
//------------------------------------------------------------------------------
public class FlipperActivity extends Activity {
    ViewFlipper f;
    DrawView a, b, c;
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        f = new ViewFlipper(this);
        a = new DrawView(this, "aaaaa");
        b = new DrawView(this, "BBBBB");
        c = new DrawView(this, "ccccc");
        f.addView(a);
        f.addView(b);
        f.addView(c);
        setContentView(f);
    }
    //------------------------------------------------------------------------------
    // Draw
    //------------------------------------------------------------------------------
    class DrawView extends View implements View.OnTouchListener {
        final String text;
        DrawView(Context Context, String Text) {
            super(Context);
            text = Text;
            setOnTouchListener(this);
        }
        public void onDraw(Canvas Canvas) {
            super.onDraw(Canvas);
            Paint p = new Paint();
            p.setColor(0xffffffff);
            p.setTextSize(20);
            Canvas.drawText(text, 0, 20, p);
        }
        public boolean onTouch(View v, MotionEvent event) {
            final int a = event.getAction();
            if (a == MotionEvent.ACTION_DOWN) {
                final int i = f.getDisplayedChild(), n = f.getChildCount();
                f.setDisplayedChild((i + 1) % n);
            }
            return true;
        }
    }
}


Piece of Optional Stopping Theorem

Question: Given some probability space $(\Omega, \mathcal{F},\mathbb{P})$, $F_{n}$ some filtration, and $X_{n}$ some martingale with $\tau$ stopping time. We know some major things such as $\tau$ is finite almost surely, $\mathbb{E}|X_{\tau}|$ is finite, and $\lim_{n\to\infty}\int_{\tau>n}|X_{n}|d\mathbb{P}=0$. The goal being to show the big result of the Stopping Theorem, $\mathbb{E}X_{\tau}=\mathbb{E}X_{0}$
I have already proven that $\lim_{n\to\infty}\mathbb{E}(X_{\tau\wedge n})=\mathbb{E}(X_{\tau})$
I am stuck trying to show that $\mathbb{E}(X_{\tau \wedge n})=\mathbb{E}(X_{0})$.
A suggestion I found was the show that $\lim_{n\to\infty}\mathbb{E}|X_{\tau \wedge n}-X_{\tau}|\to 0$, but not really exactly sure what the punchline is?
Any suggestions?
Comment: @SextusEmpiricus  From the context presumably $\tau \land n$ (*logical and*, aka *[meet](https://en.wikipedia.org/wiki/Join_and_meet)*)  is supposed to mean $\min(\tau, n)$
Comment: Not a full answer, but my intuition about it. The process underlying the martingale will always have the same expectation while it progresses because for any value that is not stopped the expectation remains the same. The matter/problem is whether the expectation of the stopping time is finite. For instance, a random walk starting at $0$ with an absorbing boundary at $a$ will have $E[X_\tau]=a\neq 0$. That is because there's a tiny amount of probability that $\tau$ takes infinitely long and why $E[X_\tau]$ can deviate from 0. At any time it's still $E[X_t]=0$ and the non-stopped paths balance.
Comment: What do you mean by "$\tau \land n$"?
Answer: To prove $E(X_{\tau \wedge n}) = E(X_0)$, you may show that $\{(X_n^*, \mathscr{F}_n): n = 1, 2, \ldots\}$ is a martingale. Here $X_n^* = X_{\tau \wedge n}$.
First, $X_n^*$ is measurable $\mathscr{F}_n$:  since $[\tau > n] = \Omega - [\tau \leq n] \in \mathscr{F}_n$, for all $H \in \mathscr{R}^1$, it follows that
\begin{align*}
[X_n^* \in H] = \bigcup_{k = 0}^n[\tau = k, X_k \in H] \cup [\tau > n, X_n \in H] \in \mathscr{F}_n.
\end{align*}
Second, $E[|X_n^*|] < \infty$ for every $n$.  This is because
\begin{align*}
E[|X_n^*|] =
\sum_{k = 0}^{n - 1}\int_{[\tau = k]}|X_k|dP +
\int_{[\tau \geq n]}|X_n|dP \leq \sum_{k = 0}^nE[|X_k|] < \infty.
\end{align*}
Third, for every $A \in \mathscr{F}_n$,
\begin{align*}
& \int_A X_n^* dP = \int_{A \cap [\tau > n]}X_n dP + \int_{A \cap [\tau \leq n]}X_\tau dP, \tag{1} \\
& \int_A X_{n + 1}^* dP = \int_{A \cap [\tau > n]}X_{n + 1}dP + \int_{A \cap [\tau \leq n]}X_\tau dP. \tag{2}
\end{align*}
Because $\{(X_n, \mathscr{F_n})\}$ is a martingale and $A \cap [\tau > n] \in \mathscr{F}_n$, the right-hand sides of $(1)$ and $(2)$ coincide, which shows that $E[X_{n + 1}^*|\mathscr{F}_n] = X_n^*$.
The above three items showed that $\{(X_n^*, \mathscr{F}_n): n = 1, 2, \ldots\}$ is a martingale, it then follows that $E[X_n^*] = E[X_0^*] = E[X_0]$. This completes the proof.

Piece of Optional Stopping Theorem


Postfix DDoS mitigation: how to whitelist IPs for IMAP access to a single mailbox

Question: Does (Zimbra using) Postfix have an option to whitelist IPs or domains for IMAP logins for a specific mailbox only?
This is what we would like to achieve:

all IPs can open an IMAP connection on port 993 with valid
email@example.com credentials
all IPs can open an IMAP connection on port 993 with valid firstname.lastname@example.com credentials
only 184.108.40.206, 184.108.40.206/24 and foo.example.com can open an IMAP connection on port 993 with valid email@example.com credentials

Preferably when someone from 188.8.131.52 tries to open an IMAP session with email@example.com credentials the same error should be returned as for an unsuccessful login attempt due to wrong password or non existant recipient address, with the same timeout (e.g. 10 seconds).
Background: our mailserver (Zimbra using Postfix) is under intermittent attacks on specific mailboxes resulting in those mailboxes getting locked out due to the number of failed IMAP login attempts per hour for that mailbox. The login attempts are coming as a slow trickle from unique IPs (2-3 login attempts per minute on the same mailbox) so banning IPs is not a solution. Increasing the number of failed attempts per hour limit only allows the attack to proceed and opens the door for larger concurrent attacks.
Firewalling the IMAP port is not an option since we would like to keep access open from outside the VPN/corporate network except for those mailboxes that are under a current/persistent attack. Inspecting the traffic there is also not possible since it's encrypted.
So we are looking for something like postscreen_access_list but for specific recipients only as we would like to limit access only to those mailboxes that are under attack.
Comment: Found not quite an answer to the specific question but a partial solution to the underlying problem: Zimbra has an option to disable (external) IMAP for the specific mailbox, this prevents login attempts at least, so brute force guessing no longer works at least via IMAP. There are two separate options, one for "external IMAP", not sure yet what the difference is, disabled both for now.
Comment: What are you using for DDoS protection?
Comment: We don't have a specific service like Cloudflare if you mean that and currently no DNS blacklists configured.
Comment: You speak about "Postfix DDoS mitigation", what is it?
Comment: We are experiencing what I think can be termed a DDoS since mailboxes get locked out and I'm looking for a mitigation. This in combination with Postfix yields the subject. I didn't mean to imply I'm looking for a mitigation for Postfix, but for a mitigation for the DDoS, if there is one using a mechanism in Postfix, or a Zimbra construct of some sort, or any other ideas are welcome. But I'm mainly curious if there is such a possibility in Postfix itself.
Comment: So you are looking for a firewall with the possibility of whitelisting?
Comment: If that firewall is somehow capabe of distinguishing between what mailbox the IMAP session is opened to, then sure. It would need to deal with the encryption too.
Answer: I'd look at fail2ban first. It is quite simple and does not require any more hardware or network setups. fail2ban works by watching logfiles and then blocking IPs with iptables/nftables etc.
The patterns to look for in logfiles can be defined - thus you can exclude the logins for certain mailboxes by excluding them in the regular expression.
fail2ban can't mitigate all kind of DDoS but it is the first tool I'd look at. Also, you did not provide any more details about your needs and what kind of attacks you are faced with.
For every "jail" in fail2ban there failregex's and ignoreregex's. Just insert the according regular expressions there.
Comment: fail2ban will not be useful in this instance since the attack is coming from unique IPs each time (I did specify this above).

It also doesn't answer the question whether there is a way to white/blacklist for certain mailboxes only, iptables will filter for all.


How do I find the remainder of $4^0+4^1+4^2+4^3+ \cdots + 4^{40}$ divided by 17?

Question: Recently I came across a question,

Find the remainder of $4^0+4^1+4^2+4^3+ \cdots + 4^{40}$ divided by 17?

At first I applied sum of G.P. formula but ended up with the expression $1\cdot \dfrac{4^{41}-1}{4-1}$. I couldn't figure out how to proceed further. Secondly I thought of using the fact $(a+b+\cdots) \pmod {17} = (r_a+r_b\dots) \pmod {17}$ but it is getting more messier.
Please explain in detail. And also mention the formula being used.
Comment: Is the $4^2$ deliberately missing? And are all other exponents  less than $40$ supposed to be present?
Comment: In the future, even if you don't remember all the other manipulations, it may be useful to keep in mind that (for example) $4^{40} \bmod 17 = 4\cdot4^{39} \bmod 17 = 4(4^{39} \bmod 17)$.  You can extend that chain all the way down to the bottom.
Comment: @AndréNicolas It was a typo. It is a geometric progression with common ratio 4.
Comment: Two typos, it is still in the title. But then things are simple. sums by groups of $4$ are congruent to $0$, so the sum is congruent to $1$.
Answer: $$\frac{4^{41}-1}{4-1}\equiv \frac{2^{82}-1}{4-1}\equiv \frac{2^2-1}{4-1}\equiv 1\pmod{17}.$$
Fermat's little theorem was used: $2^{16}\equiv 1\pmod{17}$ implies:
$$ 2^{82} = 2^{5\cdot 16+2} = 4\cdot \left(2^{16}\right)^5 \equiv 4\pmod{17}.$$

Another approach. Let $a_n = 4^0+4^1+\ldots+4^n$. Then obviously $a_{n+1}=4a_n+1$, so given $a_n\pmod{17}$, to compute $a_{n+1}\pmod{17}$ is straightforward. $a_0=1$, so our sequence $\pmod{17}$ goes this way:
$$ 1 \to 5 \to 4 \to 0 \to 1 \to 5 \to 4 \to \ldots $$
hence it is $4$-periodic. That implies $a_{40}\equiv a_{0}\equiv 1\pmod{17}$.
Comment: Maybe it should be mentioned that Euler's theorem was used.
Comment: @Amad27, there are no series involved, this is just a partial sum of geometric sequence: $1 + x + x^2 + \cdots + x^n = \frac{x^{n+1}-1}{x-1}$ which works for any $x\neq 1$.
Comment: @user103816: Assuming that $c-d$ is an invertible element, that is trivial.
Comment: @orion: suggestion accepted.
Comment: @JackD'Aurizio, for a geometric series isnt $|r| < 1$ required?
Comment: Why is $ \frac{2^{82}-1}{4-1}\equiv \frac{2^2-1}{4-1}$?
Comment: @user103816: because $2^{82}\equiv 2^{2}\pmod{17}$. It is just proved below.
Comment: I'm asking how is $\left(\dfrac{a-b}{c-d}\right) \pmod N = \left(\dfrac{a\pmod N-b}{c-d}\right) \pmod N$?
Answer: HINT:
Observe that for any non-negative integer $a,$ $$1+4+4^2+4^3=(1+4)(1+4^2)\equiv0\pmod{17}$$
$$\implies\sum_{a=m}^n4^{4a}(1+4+4^2+4^3)\equiv0\pmod{17}$$
Here $m=0,n=9$
Comment: So we have to find only $4^{40}/17$'s remainder?
Comment: Yes, that's correct.
Comment: How can we find $(17-1)^{20} \pmod {17}$? My idea is to use $(17-1)^{20}=17f(n)+(-1)^{20}$, where $f(n)$ is some natural number and then use the fact $xN \pm b \pmod N = \pm b \pmod N$.
Comment: @user103816, True. Do you know http://mathworld.wolfram.com/Congruence.html ?
Comment: Yes I know that if $a-b$ is divisible by $N$ then $a$ and $b$ leave same remainders when divided by $N$.
Answer: If you calculate $4^{k}\ (\operatorname{mod}\ 17)$ for some $k$'s, you might quickly notice that the values are periodically $1,4,-1,-4$. This is not coincidence, since $4^4 \equiv 1 \ (\operatorname{mod}\ 17)$, and thus $$4^{4k+l}\equiv 4^{4k}\cdot 4^l \equiv (4^4)^k\cdot 4^l \equiv 4^l \ (\operatorname{mod}\ 17)$$ So, we conclude $$4^{4k+1}+4^{4k+2}+4^{4k+3}+4^{4k+4}\equiv 4 + (-1) + (-4) + 1 \equiv 0 \ (\operatorname{mod}\ 17)$$ and can calculate $$4^0+4^1+4^3+\cdots 4^{40} \equiv (4^0+4^1+4^3+4^4)+\sum_{k=1}^9(4^{4k+1}+4^{4k+2}+4^{4k+3}+4^{4k+4}) \equiv 4^0+4^1+4^3+4^4 \equiv 1 + 4 + (-4) + 1 \equiv 2\ (\operatorname{mod}\ 17)$$ Of course, if $4^2$ is accidentally missing in the question, sum just as easily evaluates to $1$.
Comment: Thank you, your method is quite understandable..
Answer: Hint: $$4^2 = 16 \equiv -1 \mod 17$$
$$4^4 \equiv (-1)^2 \equiv 1 \mod 17$$
Comment: And what about odd powers of $4$?
Comment: $4^3 \equiv -1\cdot 4 = -4 \mod 17$


What's the difference between bias in model error in regression?

Question: Is model error the same as bias in regression? For example, if I construct data by $y_i=N^{\text{th}}$ degree polynomial plus uncorrelated noise, and do a regression with the $M^{\text{th}}$ degree polynomial, is the 'sweet spot', where bias + variance is minimized, at $N=M$?
Comment: @PeterEllis has provided a good answer below. Let me add one small detail regarding your concluding question (which I suspect might have been an oversight): It depends on how much data you have. Eg, if you use an 8th degree polynomial, your model would be saturated if you had only 10 data.
Answer: There are two questions here.
The answer to your general question is "no", model error is not the same as bias.  Bias is the difference between the expected value of your estimator and the true parameter.
In many instances of model error, the parameter will not even feature in the model.  With a bit of effort you could reparameterise your model so that it has the parameter in it implicitly, perhaps forced to have a value of zero (if it is a linear model) so you can say that the expected value of your estimator is zero, which unless that is also the true value, is a bias of sort.
Alternatively, the model might look fine with regard to your particular parameter of interest but be misspecified elsewhere, which will lead to biases in the estimate of your parameter of interest.  So yes you can generally express the problems of model error as leading to bias in an estimate of one or more of your parameters.
It doesn't work as a general equation "...the same as..." however.  There is lots wrong with model error other than just bias.  And bias can come about from a particular estimation technique even if the model is exactly right.
Comment: Thanks! But I'm having a hard time following some aspects of your response. First, the definition of bias makes it appear like we need to know the true parameter. Also, in the simple case of regression -- fitting a polynomial -- doesn't the use of a model with a polynomial of degree too small have model error as well as bias?
Comment: @JohnFInn - Yes in both cases.  Bias is normally thought of in terms of "if XX were the true model, and we had YY as our model (which may or may not be the same as XX) and used estimation method ZZ, what would the difference be on average between the estimate and the true parameter?"  So either model misspecification or estimation method may lead to bias.