On conjugacy class size of finite groups.

Question: Suppose $G$ is a finite group such that the set of all the conjugacy class size is $\{1,2,\dots,n\}$, where $n$ is a natural number. Is it true that $n\leq 3$? Thanks in advance.
Comment: So are you saying that there is one conjugacy class of size 1, one conjugacy class of size 2, and so forth? If so, then because the size of conjugacy class must divide the order of the group, we have $n-1 | n(n+1)/2$. But $(n,n-1) = 1$, and $(n+1,n-1)$ is 1 or 2, this is impossible unless $n-1 = 1$ or 2.
Comment: I don't mean that. There may be so many conjugacy classes of size $m$ for any $m\leq n$ and atleast one conjugacy class of size $m$ exist.
Comment: Is there anyone out there who could produce an example or better counterexample with GAP? I have tried to get my finger around this very intriguing question. Noluck sofar. Perhaps its warrants a bounty?
Comment: I have check in GAP and i have found that no group of order less than or equal to 100 is a counter example to the question.
Answer: Yes, $n$ should be $\leq 3$.  It is proved by Bianchi, Gillio and Casolo (see here) that in a finite group $G$ such that the two largest (non-central) conjugacy class sizes $n < m$ are coprimes, then any conjugacy class in $G$ has size $1$, $n$ or $m$.
Comment: Yassine, thanks for the reference, a very nice paper indeed!!!


What is needed in order to start learning percussion?

Question: I'm thinking about learning percussion in the conservatory and I want to know what I'll need to have in my house to practice. Percussion covers a lot of different instruments and objects.
Does it need the student to move somewhere to practice the instrument? Does it need the student to buy some basic ones? How does it work?
EDIT:
The question is simply about the needs in order to start learning percussion (from the beginning) in a official way.
Comment: @SysDragon - that's great then, you have lots of prior knowledge. This will help tremendously.Sounds like you may need to build a 'conservatory' onto your house to put all the percussion instruments in !! I'd make a list of the common percussion instruments, and start collecting the smaller and cheaper ones now. Any percussion instrument will be useful. Guitarists collect guitars, harp players have a selection of harmonicas, etc.The tuned ones in particular will be good - xylophone, marimba, and as a piano player how could you not have a set of vibes ?
Comment: If you're planning to study percussion at a conservatory, I presume you have a percussion teacher, right? If so, ask them which instruments you need to have access to every day. If not, I really think you need a teacher, if you are planning to study at that level.
Comment: @BobBroadley If I'm asking this is because (obviously) I don't have a teacher yet. The course starts on september and I'm just planning my options
Comment: Right, sounds good that you are planning now! If you *have* already taken percussion lessons, I would suggest that you get some advice from your previous teacher. Your new teacher will also have advice. I would imagine you will be doing most of your practice at the conservatory. But most percussionists I know have at least one piece of tuned percussion at home (eg. marimba) and would also practise rudiments on a practice-pad.
Comment: @BobBroadley I never had any lesson, that's why I'm saying "start studying percussion"
Comment: Okay, well, usually if you want to study an instrument at a conservatory you will need to have already studied it to quite a high level. It might be a good idea for you to contact the conservatory to see what the entry requirements are. Or do you already have a place to study another instrument?
Comment: @BobBroadley In Spain you can start in the conservatory from the very beginning. 4 years the elemmental grade, then 6 years the professional one, and 4 to 6 years the superior one (depending on the expertise)
Comment: Many apologies, I think I have misunderstood your use of the word "conservatory" to mean "conservatoire" as in music college or university level. Or your system in Spain is completely different to what I'm used to! Apologies again...
Comment: @BobBroadley So the question then is simply about the needs in order to start learning percussion (from the beginning) in a official way
Comment: Sorry to ask you so many questions @SysDragon, but it will help me answer. Am I right that you are going to study percussion at a conservatory for 14-16 years? Is this full-time or part-time? Will you also being studying other subjects at the conservatory, or is percussion your main study there?
Comment: @BobBroadley I'm already a pianist and I'm learning Oboe in a band (not in the conservatory). I played Clarinet for three years (not anymore), and I'm planning to start percussion from the beggining but idk how many years.
Comment: @SysDragon - you need to explain more about the whole course. I guess you will have to also study a tuned instrument, not least tuned percussion. Going in 'cold' doesn't seem to me to be the best way - most people of your sort of age will by that stage already have some knowledge and experience of playing music.assuming you are in your teens now.
Comment: @Tim I'm 26 and a pianist. I learned clarinet for 3 years some years ago, and I'm currently learning oboe. I always wanted to learn percussion and oboe (apart from piano)
Comment: @SysDragon, sorry to ask again, is this a part-time or full-time course? How many days (or hours) a week will you be there? This will give some idea of how quickly you will be expected to progress and, so, what equipment you might need.
Comment: @BobBroadley Conservatory has, usually, a lot of subjects, but since I can get a lot of them "convalidated" (idk the word in english) I will only need instrument. On elemmental it will be one hour per week, I think.
Comment: *Right*, now I understand! I am so sorry! This sounds like a part-time course; I'll write an appropriate answer.
Answer: TL;DR: If you are a beginner, your teacher will probably ask you to get a practice-pad and some drum sticks.
If you are just starting as a percussionist, you will first need to learn basic rudiments, which will allow you to play a range of percussion instruments, and then specialise at some instruments if you wish to (unless you are starting to learn a very specific type of music, see below). To do this, you can just get started with a practice-pad and some drum-sticks. Although this site is specifically aimed at beginner drummers, the advice would be good for any beginner percussionist.
However, if you already know that you may be specialising in one particular area of percussion playing, you could get the necessary instrument(s). For instance, if you know that you want to become a drummer (i.e. using drum-kit), you can start with just the equipment above, but you might want to get a simple drum kit quite soon. Or, if you know that you want to specialise as a tuned percussionist, you could get a tuned percussion instrument, but I would imagine that you would still need to learn basic percussion skills first.
On the other hand, if you know that you are going to focus upon learning percussion instruments related to a specific type of music, you might instead get the associated instrument(s) quite soon. For instance, if you are studying Indian Classical music, or a specific kind of Latin, African or other world music, the basic skills that need to be developed as a beginner may require a specific instrument (for example, tabla, cajon etc…)
If you are just starting to study percussion part-time, you probably won't need much equipment to get you started; as you become more advanced and maybe more specialised you will need the necessary equipment to practice at home. Hopefully, you will be able to use your conservatory as a place to practice between lessons; this will mean you can use their equipment.
Finally, as you become more advanced, you may find that although you specialise in some areas, you will gain experience of a wide variety of percussion instruments, that you will want to have regular access to, either for practice or for performances. But even then, it is common for percussionists to borrow or hire less common instruments when needed, there are very few percussionists that will own every instrument that they are likely to play. (Although I do know percussionists with garages full of marimbas, vibraphones, drums of all sizes…) Realistically, you are most likely to gradually gather a collection of smaller percussion instruments: shakers, triangle, cowbell, tambourine, cabasa etc., and a variety of sticks, alongside your "main" instrument(s).
Note for other readers: the OP is starting a conservatory course as a beginner, having not had percussion lessons before. This is not the same as my usual understanding of a music-college, university or conservatoire course, where a certain level of previous study is expected before starting the course. (Hopefully this should save you reading through our extended comments discussion...!)
Comment: @SysDragon, good luck with starting your course in September! As a pianist and oboist already, you should make a flying start to percussion studies. I've often thought about starting a new instrument, that I have not played before (would love to play the harmonica…), but have never had the courage to start, so I have a lot of respect for you...
Comment: Is necessary to specialize? in the end, will I need all the instruments that I'm going to play later (as a part of a orquestra, for example)?
Comment: Most percussionists I know have experience of a wide variety of percussion instruments, but are more specialised in some (especially with kit drummers or tuned players, for instance). If you are playing in an orchestra you would probably use *their* instruments for rehearsals and performances. You wouldn't be expected to have *every* instrument at home to practise with. I updated my answer...
Comment: Start, then! Life is for doing what you like! (at least, in the free time)
Answer: Just to add something to the above answers.
Since percussive instruments are rhythmic, you should really really focus on rhythmic exercises.
Something that really helped me when I began studying drums were the rhythm solfege books. There many of them and they start from the very basics of the rhythms.
I suggest getting the first one and start practicing -- A teacher is most likely to tell you to get one.
Answer: The best person to enquire to is the Director of the particular Conservatory you wish to go to. There must be a syllabus to peruse, and a list of appropriate equipment needed, as well as a synopsis of the course you wish to follow and hopefully complete.This sounds similar to the Greek tutor courses which take, I believe, about 15 yrs to complete.


Jittery Behavior on First Scroll in React Header Component

Question: Description
I'm encountering an issue with the header component in my React based next.js web application. Upon the initial load of the page, when I first scroll, there's a noticeable jittery behavior before the header hides as intended. Once this initial jittery behavior occurs, the header behaves perfectly fine for subsequent scrolls.
<code>function Header() {
    const [mounted, setMounted] = useState(false);
    const { theme } = useTheme();
    const [prevScrollPos, setPrevScrollPos] = useState(0);
    const [visible, setVisible] = useState(true);

    useEffect(() => {
        const handleScroll = () => {
            const currentScrollPos = window.scrollY;
            setVisible(prevScrollPos > currentScrollPos || currentScrollPos < 200);
            setPrevScrollPos(currentScrollPos);
        };

        window.addEventListener("scroll", handleScroll);

        return () => window.removeEventListener("scroll", handleScroll);
    }, [prevScrollPos]);

    useEffect(() => {
        setMounted(true);
    }, []);

    if (!mounted) {
        return null;
    }

    return (
        <>
            {(theme === 'light' || theme === 'undefined' || theme === null) && visible && <LightHeader />}
            {theme === 'dark' && visible && <DarkHeader />}
        </>
    )
}
</code>
(I am using next-themes for theme switching. The mounting logic is to mitigate hydration error)
I suspect I might be missing a crucial React concept or best practice here. Could someone please provide insights into what might be causing this jittery behavior on the first scroll and how I can resolve it? Would de-bouncing help in this scenario? Additionally, could this issue be related to React Snapshot?
Any help or guidance would be greatly appreciated! Thank you.

warning suggests that I read scroll anchoring
Comment: Browser scrolling (*and mouse events in general*) are quite noisy. Have you tried using a passive event listener? Think you could create a ***running*** [codesandbox](https://codesandbox.io/) demo that reproduces this scrolling issue that we could inspect live?
Comment: Scroll anchoring was disabled in a scroll container because of too many consecutive adjustments (10) with too little total distance (0 px average, 0 px total).

this is consoled out as well
Answer: The issue was with scroll not able to anchor due to dynamically adding and removing the header component. I was rendering either the header component or null which was affecting the height of the document.
I mitigated this issue by using visibility: visible and hidden such that it is still in the document flow but hidden.
NOTE: I still don't know why the issue resolves itself after initial jitter tho. Is it because the height of the header is saved by the document?


After upgrading to IntelliJ IDEA 2022.3.1 all my projects stopped loading in it - directory structure is completely gone

Question: Yesterday, I upgraded my IntelliJ IDEA (Community Edition) to the 2022.3.1 (Build #IC-223.8214.52, built on December 20, 2022) version. All the folders, files and an entire directory structure is gone, leaving only pom.xml.

I've tried closing and re-opening the project 10 times, but it didn't help.
I've also tried invalidating all the cache - didn't help as well.
Any help?
Btw, IntelliJ IDEA gets worse and worse in the course of last few years, and I know plenty of people around saying the same.
Comment: Have you tried to run "Reload" from Maven tool window? Also you may try to remove IDE related data (.iml file and .idea folder) and reimport project.
Comment: @y.bedrov I've tried reloading Maven project - w/o any success; and then I just did what you wrote - removed IntelliJ IDEA related data and re-created IntelliJ IDEA project.. but the latter is super inconvenient to do with all the projects you may have.
Comment: Do you face it with every project?
Comment: @y.bedrov initially - yes, then I removed IntelliJ related files/folders and recreated projects.
Answer: Try File > Invalidate caches... > Check the first checkbox > Invalidate and Restart.
Comment: I did this. Didn't help.


VSCode: How to export a python file that was imported from a Jupyter Notebook back to Jupyter format?

Question: Probably a silly question, but I couldn't find it. Visual Studio Code editor has a really nice way to work with Jupyter Notebooks. I can edit the cells directly in vscode and run them. Now it would be easy to work with version control.
But I couldn't find a way to convert it back to a Notebook! How do I generate a notebook back from the generated python file?
I understand that the notebook wouldn't have the output cells in it.
Answer: There is an option in the interactive Python window that has the notebook output:

This is really cool, now you can work in a Python file and have a really nice interface with your version configuration system (Git).
Yes, it was a silly question :-)
Comment: Note that this functionality is [implemented in TypeScript](https://github.com/microsoft/vscode-python/blob/master/src/client/datascience/jupyter/jupyterExporter.ts).
Answer: If you need more control over how conversion to ipynb is done (or you need to have cross-references) then you can give a try to Pandoctools. It can export VSCode *.py documents to any Pandoc output format or to Jupyter notebook.
For example you can create and register Jupyter kernel. For example is can be named "nn". That should be the same kernel that you selected in VSCode (there you select it by path but VSCode still uses installed kernels specs under the hood). Then add hat to the Python file, split document to cells, provide settings and set Markdown cells (commented metadata line would export to pdf instead of ipynb; I recommend to open ipynb in nteract native app):
"""
---
kernels-map:
  py: nn
jupyter:
  kernelspec:
    display_name: nn
    language: python
    name: nn
pandoctools:
  # out: "*.pdf"
  out: "*.ipynb"
...

# Markdown section title 1

Some **static** Markdown text.
"""

# %% {echo=False}
import IPython.display as ds
import math
import sugartex as stex

# %% {markdown}
"""
# Markdown section title 2

The quick brown Fox jumps over the lazy dog.
"""

# %%
ds.Markdown(stex.pre(f'''

Some **dynamic** Markdown text with SugarTeX formula: ˎα^˱{math.pi:1.3f}˲ˎ.
It works because of the `Markdown` display option and `sugartex` Pandoc filter.
Acually `stex.pre` is redundant here but it is needed when the text is imported
or read from somewhere instead of being written in the same document.

'''))

Then convert the file via pandoctools: drag and drop file to pandoctools shortcut/executable or "open with" pandoctools executable.
Also see:

Two introduction articles are at the beginning of this README,
examples of input to output conversion that have cross-references!
how to use Pandoctools and it's CLI,
how to use Knitty that collects Jupyter outputs and change it's settings.
Answer: The Jupytext library supports the percent-based cell format used by VSCode-Python, as well as other text-based notebook interchange formats:
https://github.com/mwouts/jupytext
Comment: @PeterZagubisalo from the jupytext readme: "It can also convert these documents into Jupyter Notebooks, allowing you to synchronize content in both directions."
Comment: Jupytext is useless if you use VS Code Scientific Python mode to run code cells (or Atom-Hydrogen). But Jupytext can be a nice alternative to it: write code in VS Code but run code cells in Jupytext. As of late both VS Code and Atom had severe bugs on Windows so having a third alternative is nice.
Answer: Use jupytext library:
Install:
pip install jupytext

Now open CMD or internal terminal to the folder with your .py file
Run this:
jupytext --set-formats py:percent,ipynb filename.ipynb


List Embdedding and Defined Property

Question: Suppose I have a matrix.
The question is, how do I show that whether a given list embedded within a list, the following rule is satisfied:
Given any three (or more) members of the list, X([i][j])[k] = X[i]([j][k])
?  If you are confused you can think of this in terms of a identity matrix, for instance, the identity matrix is A= [[1,0],[0,1]]. The identity does not follow the property because =(0.0).1 != 0.(0.1). Where (0.0) means A[0][0] and (0.1) means A[0][1]. So how do I apply the row and column of 1 to the row and column that contains (0.0)?  Note that (0.0) =1 in this case and (0.1) = 0
Well what I have to do is to show that
A[i][j]*k != i*A[j][k] but that doesn't really make sense in python because of *.
A = [[1,0],[0,1]]

for i in A:
    for j in A:
        for k in A:
          if A[i][j]*k != i*A[j][k]:
              return print("Property Not Satisfied")

Comment: Is the matrix `A` always square?
Comment: I and others can help if we can understand clearly what you want the output to be. So please help me understand - say for i = 0, j = 0, and k = 1, what should A([i][j])[k] be? Let's go step by step. A([0][0]) is 1. So next how should the [k] part be applied? It becomes something like 1[1] at this step, which is meaningless. I am sure you are not formulating your question right.
Comment: What is `(0.0).1` exactly? `(0.0)` means `A[0][0]`, and then how do you apply the remaining `.1` on it?
Comment: That's the question i'm asking. (0.0) = A[0][0]. I want to know if theres a way to apply the remaining 1, which is an entry in the matrix.
Comment: @FatihAkici I want to know a way to do this: X([i][j])[k] = X[i]([j][k])
Comment: Do you mean you are trying to show X[X[i][j]][k] = X[i][X[j][k]]?
Comment: Yes,  that's all that I wanted to show. I just didn't know how to write it that way. HAHA. Thanks so much.
Answer: If I have no misunderstanding, you can implement exactly like you said, just compare integer-part, fraction-part separately:
def check_matrix(A):
    row, column = len(A), len(A[0])

    for i in range(row):
        for j in range(column):
            # if j out of index in row, skip
            if j >= row:
                break
            for k in range(column):
                # compare integer-part, fraction-part separately
                if A[i][j] != i or k != A[j][k]:
                    print("Property Not Satisfied")
                    print(A[i][j], i, k, A[j][k])
                    return False
    return True

def test():
    A = [[1, 0], [0, 1]]
    check_matrix(A)

output:
Property Not Satisfied
1 0 0 1

Hope that will help you, and comment if you have further questions. : )
Comment: Okay but that's only for the identity matrix, what about for other lists of lists? In General I mean. For instance [[1,3,4,5],[4,5,7,6],[4,3,5,6],[3,2,4,3]] Is there a more general way to do it?
Comment: I want to know a way to see if this holds in a general matrix: X([i][j])[k] = X[i]([j][k])
Comment: sorry, I think I'm totally confused about your requriement.

List Embdedding and Defined Property


What is the mixing time of a random walk of a rook

Question: Let $G(V,E)$ be the following graph: The vertex set $V$
is a $n\times n$ grid, and two vertices are connected $(E)$ if they lie on either the same row
or the same column. This is the rook's graph: It can be thought of as a $n\times n$ chess board with two vertices connected $i$ they represent a legal move by a rook.
What is the mixing time of a random walk of a rook on the chess board (graph $G$)?
So far, I noticed that the graph is ergodic (strongly connected and aperiodic) - so there is a stationary distribution for this random walk. Moreover, I observe that this problem is closely related to the random walk on  $K_n$ with a rapid mixing time of $e^{\Omega(-t)}$. But I could not procced
Answer: First lets reference an abbreviation of a theorem from Levin, Peres, and Williams:
Theorem 12.4 (LPW): Let $P$ be the transition matrix of a reversible, irreducible Markov chain with state space $\mathcal{X}$ and let $\displaystyle\pi_{\min} := \min_{x\in\mathcal{X}} \pi(x)$. Then $t_{\text{mix}} (\varepsilon) \leq t_{\text{rel}} \log (\frac{1}{\varepsilon\,\pi_{\text{min}}})$.
You noted that the Rook's walk's graph is ergodic. This is good, and we can say further it is symmetric, so we know that the stationary distribution, $\pi$, is uniform. Kim calculated the relaxation time, $t_\text{rel} = \frac{d(n-1)}{n}$, which therefore gives a loose bound on the mixing times. Note, $(d,n)$ is the dimension and length of the board. Chessboards in real life have $(d,n) = (2,8)$.
Proposition 2.3 (SSK): For the rook's walk on $\mathbb{Z}_n^d$ where $n>2$,
$$ \frac{d(n-1)}{n} \log \left(\frac{1}{2\varepsilon}\right) \leq t_{\text{mix}} (\varepsilon) \leq \frac{d(n-1)}{n} \log \left(\frac{n^d}{\varepsilon}\right).$$
Later, McLeman et al. calculated bounds for the Rook's walks by employing a path coupling technique. These results for the 2-dimensional case are asymptotically tight.
Theorem 6.1 (MORS): For the $n^d$ rook's walk, we have $$ t_{\text{mix}}(\varepsilon) \leq \left\lceil \frac{\log \frac{d}{\varepsilon}}{\log \frac{d(n-1)}{(d-1)(n-1)+1}} \right\rceil.$$
Sources:
(LPW): Markov Chains and Mixing Times, second edition. David A. Levin, and Yuval Peres, with contributions by Elizabeth L. Wilmer, 2009.
(SSK): Mixing Time of a Rook's Walk, Steven S. Kim, 2012
(MORS): Mixing times for the rook's walk via path coupling. Cam McLeman, Peter T. Otto, John Rahmani, Matthew Sutter, 2017


How does browser back button work in web application?

Question: I dont have any body regarding this.
I have tried reading the concepts in internet but still confusing about this one please suggest how it is works.
Answer: When you are going through the pages, the URL changes. Back button basicly returns you to the previous URL (previous page).
Nowadays some web applications use the same URL all the time and reloads content dynamically - in this case the back button is nearly useless.
More informations: How does the Back button in a web browser work?


How to run svcutil.exe from behind authenticating proxy

Question: I want to run the svcutil.exe tool to access a web service on the internet. Unfortunately, whenever I try, I get a bunch of errors that include the following message:

The request failed with HTTP status 407: Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.

As I have learned from this related post (with more details here), the problem is that I am sitting behind an authenticating proxy. That post explains that I need to edit the app.config file, but I can't figure out how to do that. I think I will use the  /svcutilConfig:alternate_app.config switch, but I don't know how to construct a valid .config file to pass to that switch. What is the default app.config file that svcutil.exe uses?
Answer: Well, I think I have figured out the answer to my question:
It turns out that the default .config file used by svcutil.exe is called svcutil.exe.config, and (at least for me) it lives in this folder:
C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\bin

I decided to just edit that file directly (rather than fumble around with the /svcutilConfig switch). I needed Admin privileges to do so.
The final contents of that file looked like this:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <runtime>
    <generatePublisherEvidence enabled="false" />
  </runtime>
  <system.net>
    <defaultProxy useDefaultCredentials="true" />
  </system.net>
</configuration>

(with the secret sauce buried in the <system.net> element.)
In order for this all to work, I had to start the Command Prompt as Administrator, navigate to the folder listed above, and run the svcutil.exe command from there.
Hope this helps some other poor soul who gets stuck in this mess! Thanks to @AndrewWebb for the clues that got me going!
Comment: Yes, I've had to do this for SvcUtil too.  For web apps, web.config remains as "web.config", but for desktop apps, app.config in the source code becomes ".exe.config", as you have found.  Hand-editing svcutil.exe.config (and similar files) has become very familiar to me!  Also: if you come across a desktop app *without* a config file in the first place, you can always create one with the correct name, whack in the defaultProxy config, and save it to the folder the .exe lives in.
Comment: If the svcutil.exe.config does not exist, you can create it like kmote has above and it will work just fine.


Finding Branch Values

Question: I have a function $f(z) = (\frac{8}{7}z^3-\frac{64}{7})^\frac{1}{3}$ which I have found to have branch points at $z_1 = 2$, $z_2 = 2\exp(\frac{2i\pi}{3})$, and $z_3 = 2\exp(\frac{-2i\pi}{3})$. The part I am stuck on I am suppose to consider branch cuts from $z_1$ to $z_2$. Then, let $F(z)$ denote the branch when Im[$f(10)$] = 0. What is the value of $F(1)$. I'm confused on how exactly I find $F(z)$.
Comment: Before you tackle this problem, do you understand the simpler case $g(z) = z^{1/3}$?  The branch points are 0 and $\infty$.  If Im[$g(1)$] = 0, what is $g(-1)$ if you consider a branch cut from 0 to $\infty$ going upwards along the imaginary axis?  What is $g(-1)$ if you consider a branch cut from 0 to $\infty$ going downwards along the imaginary axis?
Comment: I don't really know how to even this simpler case. For the case with a branch cut from 0 to $\infty$ upwards along the imaginary axis would I first evaluate $g(z)$ at 1, resulting in $g(1) = 1$. Then the Im[$g(1)$] = 0? I really am not sure how to approach it.
Answer: Before we tackle this problem, consider the simpler case $g(z)=z^{1/3}$, with branch points at 0 and $\infty$.  Suppose Im[$g(1)$] = 0.  For each of the possible branch cuts joining 0 and $\infty$, what is $g(-1)$?  Clearly $g(-1)$ has to be one of the 3 cube roots of -1.  The three cube roots of -1 are -1, $e^{-\pi i/3}$, and $e^{\pi i/3}$. So which one is it?
The main reason that branch cuts are necessary for $g(z) = z^{1/3}$ is because the arg function $\theta$ cannot be defined continuously for all nonzero $z$.  If we could define $\theta$ continuously, then we could take cube roots continuously by using $\theta/3$.  The reason $\theta$ can't be defined continuously is because if, say, the positive real axis is assigned $\theta = 0$, then by the time you wander all the way around a circle centered at 0, you find that the same point wants to be assigned either $\theta = 2\pi$ or $-2\pi$, depending on which direction you wander.  This is explained in more detail here.
By taking a branch cut, we remove that curve from consideration and try to define $\theta$ continuously on the rest of the complex plane, and then divide it by 3 to take a cube root (i.e., define a branch of $g(z)$).  This can be done.  Suppose we take a branch cut going from 0 to $\infty$, upwards along the imaginary axis.  We are given that Im[$g(1)$] = 0 (i.e. $g(1)=1$), and asked to find $g(-1)$.  Since Im[$g(1)$] = 0, we should choose $\theta = 0$ along the positive real axis, so that $\theta/3 = 0$ agrees with the arg of the given value $g(1)=1$.  Now we need to find a path from 1 to -1 which avoids the branch cut.  This can be any curve joining 1 to -1 that swings downwards into the lower half plane.  Because we swing downwards, by the time we get to -1, $\theta$ must be $-\pi$ (not $\pi$).  Then $\theta/3$ = $-\pi/3$.  So the cube root of -1 we want must have its argument equal to $-\pi/3$, so $g(-1) = e^{-\pi i/3}$.
Similarly, if we were to choose a branch cut going downwards from 0 to $\infty$, then $g(-1) = e^{\pi i/3}$ instead.
Now back to our original problem with $f(z)$.  You choose a branch cut between $z_1$ and $z_2$, and find a path between 10 (where you are given a value) to 1 (where you need to find the value).  This path must avoid the branch cut.  Near each of the branch points, you have a similar situation as in the $g(z)$ example above.  I'll let you work out the details.
Comment: Thank you! So looking at the original problem I know Im[$f(10)$] = 0, so I know my initial location is 10 along the realy axis. This will give me an initial value of $\theta = 0$. To get from $f(10)$ to $f(1)$ I would need to travel around either the branch cut $z_1$ or $z_2$, thus traveling $\pi$ or $-\pi$. So, taking my original function, $f(z) = (\frac{8}{7}z^3-\frac{64}{7})^\frac{1}{3}$ I can rewrite this as $f(z) = (\frac87r^3\exp(3\theta i)-\frac{64}{7})^\frac{1}{3}$. Then plugging in $r=1$ and $\theta = \pi$ I get $f(1) = (\frac{8}{7}exp(3\pi i)-\frac{64}{7})^\frac{1}{3}$, right?
Comment: This isn't right: (1) Your final answer cannot have a 1/3 exponent in it, or else you really haven't specified which branch you're taking.  (It would be like writing $g(-1)=(-1)^{1/3}$ for the $g(z)$ example... *which* cube root of -1 do you mean?  That's the whole point of the question.)  (2) The branch point in the $g(z)$ example was at 0, but for $f(z)$, the branch points are elsewhere.  You need to shift the coordinates to put a branch point (either $z_1$ or $z_2$) to 0 before you can apply the $g(z)$ example to this problem.
Comment: So I'm going to try to do what you said in part (2). So to do this I am going to shift coordinates to make $z1$ at 0. So I would say $w=z-2$, rearranging $z=w+2$. Substituting this into f(z) I get $f(w) = (\frac87 (w+2)^3-\frac{64}{7})^\frac13$. This reduces down to $f(w) = (\frac87(w^3+6w^2+12w))^\frac13$ This has roots at w = 0, $2(1-exp(2\pi i/3))$, and $2(1-exp(-2\pi i/3))$. These are those of $f(z)$ but shifted to the left by 2. So for Im[f(10)]= 0 I know that $\theta = 0$. I can not see exactly I would go about getting rid of the 1/3 exponent. I've tried several things but can't get it.
Comment: When I said that you can't have a 1/3 exponent, perhaps I should have said instead: you can use a 1/3 exponent, but you have to specify which cube root you mean. Anyway, in the new coordinates, you take a path joining w=8 to w=-1. Since your branch point is now at 0, you can factor out a $w^{1/3}$.  That factor is exactly the $g(z)$ example. The remaining factor $w^2 + 6w + 12$ has no real roots between 8 and -1 (assuming you did the algebra correctly, I haven't checked) so you can take the straight path from 8 to -1, which (given the initial value at 8) is just taking the real cube root.
Comment: I followed all of that up until you said we can take the straight path from 8 to -1. Wouldn't that make the path pass through the branch cut. I thought the whole purpose of the branch cut was that we couldn't go through them. Wouldn't you have to go around the branch cut? Thus making $\theta = \pi\  or -\pi$?
Comment: For $(w^2+6w+12)^{1/3}$ there is no branch point at 0.  For that factor, it doesn't matter whether you go through 0 or around it; it will lead to the same value.  Only the $w^{1/3}$ factor is affected by the branch point at 0.  For $(w^2+6w+12)^{1/3}$ it's a little easier to work out if you take a direct path, because then all the values of $w^2+6w+12$ are real and positive, so you can take the real cube root function as the branch.

Finding Branch Values


Conditional expressions with Terraform deploying AWS resources

Question: My current Terraform setup consists of 3 template files. Each template file is linked to a launch configuration resource which is then used to launch instances via auto scaling events. In each template file, there is an AWS CLI command used to attach an existing EBS volume to the new instance being launched via autoscaling. I am having some trouble writing a conditional expression to pass in a variable to this AWS CLI command being used to attach a specific drive. Since I have 3 template files and 3 EBS volumes I'm looking to attach to each instance in its own autoscaling group, I don't believe I can have more than 2 expressions within my conditional expression. Any advice on how I can achieve this would be helpful.
Template_file
data "template_file" "ML_10_user_data" {
count    = "${(var.enable ? 1 : 0) * var.number_of_zones}" // 3 templates
template = "${file("userdata.sh")}

  vars {
    ebs_volume = "${count.index == 0 ? vol-xxxxxxxxxxxxxxxxx : vol-xxxxxxxxxxxxxxxxx}" // how to include 3rd EBS volume?
  }
}

Userdata.sh
#!/bin/bash
aws ec2 attach-volume --volume-id ${EBS_VOLUME} --instance_id `curl http://169.254.169.254/latest/meta-data/instance-id` --device /dev/sdf

EBS_VOLUME=${ebs_volume}

Any advice on how I can fulfill what I am trying to accomplish would be appreciated.
Comment: Do you have a 3 separate ASGs and 3 separate launch templates here? Is there a reason they're as ASGs instead of normal instances if you are just doing single instance ASGs?
Comment: The application we are hosting in this environment recommends autoscaling groups.
Comment: Is there some reasoning to that if there's no actual autoscaling happening?
Answer: The best way would be to put it in a list:
variable "volumes" {
  default = ["vol-1111","vol-2222","vol-333"]
}

data "template_file" "user_data" {
  count    = "${(var.enable ? 1 : 0) * var.number_of_zones}"
  template = "${file("userdata.sh")}"

  vars {
    ebs_volume = "${var.volumes[count.index]}"
  }
}

However, if these instances should be in ASG, this is not a very good design. Instances in ASG should be identical, interchangeable and disposable. They can be terminated at any time by AWS or scaling activities, and you should treat those instances as a group, not as individual entities.


Como buscar por arquivos antigos no Linux?

Question: Preciso utilizar o comando tail -f, para fazer uma busca em logs passados.
Sei que existem o tail -f | grep "".
Porem não sei como buscar os antigos com parâmetro que desejo.
Alguém pode me ajudar?
Answer: Para buscar arquivos que tenham determinada idade, pode-se usar o grep em junção com o find através da opção -mtime para realizar a busca.
O exemplo abaixo vai procurar pela palavra foo em todos os arquivos modificados até 20 dias atrás no diretório atual.
find . -type f -mtime -20 -print0 | xargs -0 grep -li 'foo'

Para buscar por arquivos modificados há n minutos atrás use -mmin ao invés de -mtime.
Answer: Não sei se entendi bem a pergunta. Você precisa buscar algo em logs antigos?
Se for esse o caso, você não deve precisar do tail -f, ele só é útil para acompanhar logs que estão rodando nesse momento (por exemplo, voce precisa ver alguma exception durante a navegação no seu site).
Para arquivos antigos (que não são mais alterados), só o grep deve funcionar.
Indo no diretorio dos logs voce pode rodar
grep -i 'oquevoceprocura' *
(o -i é para não diferenciar maiusculas de minusculas)
Isso ira buscar essa string em todos os arquivos do diretorio atual.
Mais sobre o grep:
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_04_02.html
http://www.cyberciti.biz/faq/howto-use-grep-command-in-linux-unix/
Comment: Eu vi a questão do -i. e agradeço a ajuda. Eu estava entendendo errado! Muito obrigado!


Prove: $4^{2k+1}>1 \cdot 3 \cdot \dots \cdot (2k+1)$ where $k$ is a positive integer

Question: Prove: $$4^{2k+1}>1 \cdot 3 \cdot \dots \cdot (2k+1)$$ where $k$ is a positive integer.
The difference is very large but I cannot find any way to prove it.
Comment: Do you know factorials ? If you do, multiply the rhs by $2.4.6...(2)$, put the same in a denominator which needs to be simplified.
Comment: Are you asking for the largest possible $k$? If so, please re-word your Q. If $f(k)=4^{2k+1}/(1.3...2k+1)$ then $f(k+1)/f(k)=16/(2k+3)$ so $\lim_{k\to \infty}f(k)=0$.
Comment: As explained in my answer, the inequality is true only if reversed and for $k \ge 19$.
Comment: Looking at your activity on MSE, I note that so far you have asked 11 questions but accepted no answer. Please note that this attitude is discouraged here, and if you do not show appreciation of the work and time that people put into answering your questions for free, you will discover that they will stop helping you. Please review the received answers and whenever you find a good one, do not hesitate to accept it.
Answer: This isn't true. Indeed, the ratio $\frac{4^{2k+1}}{1\cdot 3\cdot...\cdot(2k+1)}$ tends to zero (while if your inequality was true, it would be bounded below by $1$).
To see this, let $a_k=\frac{4^{2k+1}}{1\cdot 3\cdot...\cdot(2k+1)}$. It's then easy to see that $\frac{a_{k+1}}{a_k}=\frac{16}{2k+3}$, which is less than $\frac{1}{2}$ for $k\geq 15$. So we have for $k\geq 15$:
$$a_k=a_{15}\cdot\frac{a_{16}}{a_{15}}\cdot\frac{a_{17}}{a_{16}}\cdot...\cdot\frac{a_k}{a_{k-1}}<a_{15}\cdot\frac{1}{2^{k-15}}$$
which clearly tends to zero.
Answer: The thing you wrote in the right hand side is the semifactorial $(2k+1)!!=\prod_{i\le 2k+1, i\equiv 2k+1p\mod{2}}i$. It turns our that
$$
\forall i=1,\ldots,k, 2i+1>2i \implies (2k+1)!=(2k+1)!!(2k)!!<(2k+1)!!^2
$$
Therefore for all $k>64$
$$
(2k+1)!!>\sqrt{(2k+1)!}>\sqrt{63!\cdot 64^{2k+1-63}}>c\cdot 8^k
$$
for a positive constant $c$. Therefore $(2k+1)!!>4^{2k+1}$ whenever $k$ is sufficiently large.
Answer: The statement is mistaken: the inequality should have been reversed. Let us prove then that $4^{2k+1} < 1 \cdot 3 \cdot \dots \cdot (2k+1)$ for $k \ge 19$ by induction.
Let us verify it for $k=19$: $4^{39} = 302231454903657293676544 < 319830986772877770815625 = 1 \cdot 3 \cdot \dots \cdot 37 \cdot 39$.
Assume the statement true for $k$ and let us prove it for $k+1$:
$$4^{2(k+1)+1} = 4^{2k+1} \cdot 16 < 1 \cdot 3 \cdot \dots \cdot (2k+1) \cdot 16 < 1 \cdot 3 \cdot \dots \cdot (2k+1) (2k+3)$$
because clearly $16 < 2k+3$ whenever $k \ge 19$.
(Note that the given inequality is not true for $0 \le k \le 18$.)


Erro no build do Vue.js no Laravel

Question: Estou recebendo o seguinte erro ao salvar o arquivo app.js:
https://i.stack.imgur.com/4R27w.jpg
Os arquvios são esses:
app.js:
require('./bootstrap');

window.Vue = require('vue');

Vue.component('example-component', require('./components/ExampleComponent.vue'));
Vue.component('topo',require('./components/Topo.vue'));

const app = new Vue({
    el: '#app'
});

Topo.vue:
<template>

                {{ titulo }}

            <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="{{ __('Toggle navigation') }}">

            </button>

</template>

O erro só acontece quando coloco a linha "Vue.component('topo',require('./components/Topo.vue'));" no app.js
Alguém sabe qual é o problema?
Answer: O erro estava no <button> em
aria-label="{{ __('Toggle navigation') }}"

o correto seria :
aria-label="('Toggle navigation')"

Erro no build do Vue.js no Laravel


Creating special RSS channels

Question: How can I create RSS channels for certain content type or it's subtype (certain taxonomy terms from that content type, so that users can effectively filter the information they need).
thank you
Answer: You can build RSS feeds easily with views, after creating your first display (generally a page), you can click "+Add" and add a feed display, then you configure your fields and you're almost done.

In the "Filter Criteria" section you select your Content Types and or the taxonomy you want to target.
Comment: In views 2 there was automatically a RSS icon available at the bottom of the page, I'm not sure how it works for blocks, but you can surely add an HTML footer to your block and paste an `` there with a link to your feed (if of course views doesn't provide an easier way.
Comment: I created the RSS channel. But can I display e.g. RSS icon in that block? Or how can I visitors get know that the block has RSS channel and they can subscribe to it? I'd like it to be visible.

Creating special RSS channels


Functional Equation and totally multiplicative functions

Question: Find all $f:\mathbb{C}\rightarrow\mathbb{C}$ s.t. $\forall a, b\in \mathbb{C}, f(ab) = f(a)f(b)$.
I could deduce a lot of things about what happens at the roots of unity, and 0, but I can't find out anything more.
Comment: What happens at $a=0$ (holds iff $f(0)\neq0$)?
$$f(0\times b)=f(0)=f(0)f(b)\\
\implies f(b)=1$$ $b$ is arbitrary, and hence, $\forall b\in\mathbb{C},f(b)=1$.
Comment: @SanathDevalapurkar Or, $f(0) = 0$.
Comment: @DanielFischer Right. I'll edit the comment.
Comment: $f(a)=f(1)f(a)$, so $f(a)=0$ for every $a \in \mathbb{C}$ or $f(1)=1$
$f(a^2)=f(a)f(a) = f(a)^2$ and by induction $f(a^n)=f(a)^n$. Also
$1=f(1)=f(a a^{-1}) = f(a) f(a^{-1})$ and so $f(a^{-1})=f(a)^{-1}$. So, as far as we know: $f(a^k) = f(a)^k, \forall k \in \mathbb{Z}$
If you dont assume that $f$ is at least continuous I dont think much more can be said about $f$.
Comment: What can be said if it is continuous?
Answer: In addition to what I said in the comments,
let $z =e^{i\theta}$ and let $\{p_n\}$ be a sequence of rationals such that $p_n \to \theta/2\pi$. Then for every $p_n = \frac{a_n}{b_n}$, $$|f(e^{ip_n 2\pi}) |^{b_n} = |f(e^{ip_n 2\pi})^{b_n} | = |f(e^{ia_n 2\pi})| = |f(1)|=1$$ And thus $|f(e^{i p_n 2\pi})|=1$. If $f$ is continuous, $lim_{n\to\infty} |f(e^{i p_n 2\pi})| = |f(z)|=1$. That is: if $|z|=1$ then $|f(z)|=1$
If $f$ is bounded, then lets suppose $|f(z)| >1$ for some $z$. Thats absurd, because $lim\ |f(z^n)|=lim \ |f(z)|^n = \infty$. If $|f(z)|<1$ then $|f(z^{-1})|>1$ and thats also absurd.
So, if $f$ is bounded, $|f(z)| = 1$ for every $z$. And $f(z) = e^{i \theta(z)}$ with $\theta(x)+\theta(y) = \theta(xy)$


What happens to kernel regression (Nadaraya–Watson estimator) at a kink point?

Question: Suppose $Y_i=g(X_i)+e_i$, where $g(\cdot)$ is a function unknown to the researcher, and $E(e_i|X_i)=0$. Suppose $X_i$ is a random variable in $[-1,1]$ with a density that is everywhere positive, and the true regression function satisfies $g(x)=-x$ if $x\leq 0$, $g(x)=x$ if $x>0$. Note that here 0 is a kink point at which $g(x)$ is not differentiable.
We have data $\{Y_i,X_i\}_{i=1}^{n}$ and estimate $g(x)$ using
$\widehat{g}(x)=\frac{\sum_{i=1}^{n} Y_i K_{h}(X_i-x)}{\sum_{i=1}^{n}  K_{h}(X_i-x)}$, where $x\in(-1,1)$ and $K_{h}(\cdot)=\frac{1}{h}K(\cdot)$ with $K(\cdot)$ being the kernel function.
Question:
Is $\widehat{g}(0)$ consistent for $g(0)$?
If it is consistent, how does the rate of convergence of $\widehat{g}(0)$ differ from that of $\widehat{g}(x)$ where $x\neq 0$ and is in the interior of $[-1,1]$?
Thanks!
Answer: Of course this depends on the bandwidth and kernel you choose. To fix things, we take the rectangular Kernel (uniform on $[-1,1]$).
Despite the kink, your function is quite nice; it's just the absolute value function which is $1$-Lipschitz. The Nadaraya-Watson kernel nicely adapts to precisely this type of smoothness (while it does not do as well with higher order smoothness; in that case you should use higher order local linear regression).
Let us conduct a quick study of the uniform Kernel Nadaraya Watson estimator at $0$ with bandwidth $h$: $\hat{g}(0)$ is just the average of the $Y_i$'s corresponding to $X_i$'s that lie in the interval $[-h,h]$. Notice that for $X_i \in [-h,h]$ we have that: $\lvert\mathbb E[Y_i \mid X_i] - g(0)\rvert \leq h$ and so the bias of $\hat{g}(0)$ is of order $O(h)$. On the other hand, we will be averaging approximately $O(n \cdot h)$ points and so if $\text{Var}[Y_i \mid X_i]$ is bounded, we have that $\text{Var}(\hat{g}(0)) = O(1/(n \cdot h))$.
So the mean squared error (MSE) is $O(h^2) +  O(1/(n \cdot h))$. Thus as long as we choose $h$ such that $h \to 0$ and $n \cdot h \to \infty$, the estimator will be consistent. This answers your first question.
Optimizing over bandwidth will yield $h = n^{-1/3}$ and so the MSE will be of order $O(n^{-2/3})$. What happens away from $0$? If you use the same bandwidth as you did at $0$, then the rate will be the same. However you can repeat the above argument, with a more careful decomposition for the Bias, to get that the bias is of order $h^2$ (if you make some additional smoothness assumption on $p$). So away from $0$, you can pick a larger bandwidth, leading to a MSE decaying as $O(n^{-4/5})$.
Comment: Kinks are more generally not an issue, as long as e.g., you are continuous at the kink. Continuity means that for any $\delta$ by picking $h$ small enough, your bias will be smaller than $\delta$ (and you can drive the variance down slowly too). Lipschitz continuity in an interval around the kink gets you the specific bias decay rate $O(h)$ as in your example.
Comment: I see. Thanks! So to summarize, consistency (asymptotic unbiasedness plus variance going to zero) always holds at a continuous kink. But without Lipschitz continuity around the kink, there will be slower rate of (MSE) convergence due to bias shrinking at rate slower than $O(h)$, right?
Comment: Yes, precisely!
Comment: Thanks! This is quite enlightening. I got a few more questions:
Comment: In the analysis of bias, for $x\in [-h,h]$, the property of a uniform kernel is used to get $|E[\frac{\sum_{i=1}^{n} Y_i K_{h}(X_i-x)}{\sum_{i=1}^{n}  K_{h}(X_i-x)}|x]-g(0)|=|E(Y|x)-g(0)|$, and the property of the absolute value function is used to get $|E(Y|x)-g(0)|<h$, giving rise to the $O(h)$ order of bias.  Is the bias order still $h$ if for example, $g(x)=x^2 $ if $x<0$ and $g(x)=10x$ when $x\geq 0$?  Or, more generally, what are some intuitive general requirements on the kernel and $g(\cdot)$ around the kink point for bias to go to zero asymptotically?
Comment: Or, can you give an example in which the Nadaraya-Watson estimator does not adapt to a kink (a continuous but nondifferentiable point)?


OOM using Spark window function with 30 days interval

Question: I have this data frame:
df = (
spark
.createDataFrame([
    [20210101, 'A', 103, "abc"],
    [20210101, 'A', 102, "def"],
    [20210101, 'A', 101, "def"],
    [20210102, 'A', 34, "ghu"],
    [20210101, 'B', 180, "xyz"],
    [20210102, 'B', 123, "kqt"]
]
    ).toDF("txn_date", "txn_type", "txn_amount", "other_attributes")
)

Each date has multiple transactions of each of the different types. My task is to compute the standard deviation of the amount for each record (for the same type and going back 30 days).
The most obvious approach (that I tried) is to create a window based on type and include records going back to past 30 days.
days = lambda i: i * 86400
win = Window.partitionBy("txn_type").orderBy(F.col("txn_date").cast(LongType())).rangeBetween(-days(30), 0)
df = df.withColumn("stddev_last_30days", F.stddev(F.col("txn_amount")).over(win))

Since some of the transaction types have millions of transactions per day, this runs into OOM.
I tried doing it in parts (take only few records for each date at a time) but this leads to error prone calculations since standard deviation is not additive.
I also tried 'collect_set' for all records for a transaction type and date (so all amounts come in as an array in one column), but this runs into OOM as well.
I tried processing one month at a time (I need at a minimum 2 months data since I need to go back 1 month) but even that overwhelms my executors.
What would be a scalable way to solve this problem?
Notes:

In the original data, column txn_date is stored as long in "yyyyMMdd" format.

There are other columns in the data frame that may or may not be same for each date and type. I haven't included them in the sample code for simplicity.
Answer: Filtering
It's always good to remove data which is not needed. You said you need just last 60 days, so You could filter out what's not needed.
This line would keep only rows with date not older than 60 last days (until today):
df = df.filter(F.to_date('txn_date', 'yyyyMMdd').between(F.current_date()-61, F.current_date()))

I'll not use it now in order to illustrate other issues.
Window
The first simple thing, if it's already in long format, you don't need to cast to long again, so we can remove .cast(LongType()).
The other, big thing, is that your window's lower bound is wrong. Look, let's add one more line to the input:
[19990101, 'B', 9999999, "xxxxxxx"],

The line represents the date from the year 1999. After the line was added, running the code, we get this:
# +--------+--------+----------+----------------+------------------+
# |txn_date|txn_type|txn_amount|other_attributes|stddev_last_30days|
# +--------+--------+----------+----------------+------------------+
# |20210101|       A|       103|             abc|               1.0|
# |20210101|       A|       102|             def|               1.0|
# |20210101|       A|       101|             def|               1.0|
# |20210102|       A|        34|             ghu|34.009802508492555|
# |19990101|       B|   9999999|         xxxxxxx|              null|
# |20210101|       B|       180|             xyz|  7070939.82553808|
# |20210102|       B|       123|             kqt|  5773414.64605055|
# +--------+--------+----------+----------------+------------------+

You can see that stddev for 2021 year lines was also affected, so 30 day window does not work, your window actually takes all the data it can. We can check what is the lower bound for date 20210101:
print(20210101-days(30))  # Returns 17618101 - I doubt you wanted this date as lower bound

Probably this was your biggest problem. You should never try to outsmart dates and times. Always use functions specialized for dates and times.
You can use this window:
days = lambda i: i * 86400
w = Window.partitionBy('txn_type').orderBy(F.unix_timestamp(F.col('txn_date').cast('string'), 'yyyyMMdd')).rangeBetween(-days(30), 0)
df = df.withColumn('stddev_last_30days', F.stddev('txn_amount').over(w))

df.show()
# +--------+--------+----------+----------------+------------------+
# |txn_date|txn_type|txn_amount|other_attributes|stddev_last_30days|
# +--------+--------+----------+----------------+------------------+
# |20210101|       A|       103|             abc|               1.0|
# |20210101|       A|       102|             def|               1.0|
# |20210101|       A|       101|             def|               1.0|
# |20210102|       A|        34|             ghu|34.009802508492555|
# |19990101|       B|   9999999|         xxxxxxx|              null|
# |20210101|       B|       180|             xyz|              null|
# |20210102|       B|       123|             kqt| 40.30508652763321|
# +--------+--------+----------+----------------+------------------+

unix_timestamp can transform your 'yyyyMMdd' format into a proper long-format number (UNIX time in seconds). From this, now you can subtract seconds (30 days worth of seconds).


Getting the contents of a xml tag NOT INSIDE A XML DOCUMENT

Question: NOT USING A XML PARSER
I want to get the contents of the XML tag so my XML file looks like ( Includes more than XML this is just the XML part )
<ns1:CarrierServiceId>PD_PRIVATPAKKER_W</ns1:CarrierServiceId>
<ns1:LabellingReturned>0</ns1:LabellingReturned>
<ns1:DeliveryDate>2012-07-27T23:00:00Z</ns1:DeliveryDate>
<ns1:DespatchDate>2012-07-26T22:59:59Z</ns1:DespatchDate>
<ns1:International>0</ns1:International>
<ns1:ClearanceDocumentation>0</ns1:ClearanceDocumentation>
<ns1:ExceptionCode>0</ns1:ExceptionCode>

How would i get the contents of CarrierServiceId so i only get PD_PRIVATPAKKER_W and nothing else i've managed to get the line using below, but i need the contents
FOUND=$(awk '/<ns1:CarrierServiceId>/ {print $1}' TEST.tmp)

echo "${FOUND}"

Answer: Use the angle brackets as delimiters and print the field.
FOUND=$(awk -F '[<>]' '/<ns1:CarrierServiceId>/ {print $3}' TEST.tmp)

Answer: With grep:
grep -Po '(?<=CarrierServiceId>).*?(?=<)' inputFiles


Why are there so many primes in the convergents of Pi?

Question: Recently, I was looking into fractional approximations of pi, such as $\frac{22}{7}$ or $\frac{355}{113}$. I found that there was a name for these approximations, 'convergents' of pi, and I found a list of the first 100. Upon inspection of list, it seemed that a lot of the fractions had odd numerators or denominators. Furthermore, it seemed a lot of the fractions had prime numerators or denominators!
A quick check with mathematica confirmed this:
91 out of the first 100 convergents had prime numerators or denominators! Even more astounding, 6 out of the 9 that didn't hold this property occured within the first 11 convergents.
This seems like it can't simply be a coincidence, but I couldn't find anything about it online. Why is this true?
Comment: Without thinking about it very hard, it occurs to me that if you reduce a fraction to lowest terms, so that the numerator and denominator are relatively prime, the odds that at least *one* of them is itself prime go up significantly, surely.
Comment: And obviously, all of them have to have at least an odd numerator *or* and odd denominator (or both).
Comment: Are you able to produce convergents on your own?  If so, it might be interesting to try it for some (quasi-)randomly chosen number, and see if the same proportion holds, or if the difference is significant.
Comment: @BrianTung I also thought along those lines, but I feel that argument would only work if I was saying odd, not prime. Then again, when a number is odd it also has a better change of being prime, but this seems to be on too large a scale (91%) to just be a question of fully simplified fractions.
Comment: ...and of course, the convergents thus produced will always be in lowest terms.
Comment: @brian-tung: your point stands well if you say almost the same but taking Jim's answer in account. The concentration of primes drop with the numbers growing and thus having a convergent with both numerator and denominator prime is vanishing.
Comment: @PinhollowEuri: Aha!  I had thought about that, but didn't really think through how that would affect the proportion.
Answer: Your data seems to be exactly backwards.  Of the first 100 convergents of pi, only 9 of them have either a prime numerator or a prime denominator, with 91 of them having both composite numerator and composite denominator.  Moreover, 6 of the 9 that have a prime numerator or denominator are in the first 11 convergents.
Here's the Mathematica code that I used:
<code>In[1]:=   c = Convergents[Pi, 100];

In[2]:=   test = PrimeQ[Numerator[#]] || PrimeQ[Denominator[#]] &;

In[3]:=   Length[Select[c, test]]
Out[3]:=  9
</code>
According to Mathematica, these 9 are
$$
3,\;\; \frac{22}{7},\;\; \frac{355}{113},\;\; \frac{103993}{33102},\;\; \frac{833719}{265381},\;\; \frac{4272943}{1360120},\;\;
\frac{411557987}{131002976},\;\; \frac{2646693125139304345}{842468587426513207},
$$
$$
\frac{7809723338470423412693394150101387872685594299}{2485912146995414187767820081837036927319426665}.
$$
Comment: I love how Mathematica has a function for everything ... even convergents.
Comment: So the reason for the many primes is that primes and composite numbers were confused?
Comment: @gnasher729 Yes.  91 primes out of 100 is fairly surprising, but 91 composite numbers out of 100 is about what you would expect.


Don't allows user to move from directory in VB.NET

Question: Someone knows if it is possible, in an openDialog or saveDialog, don't allow to the user to move to another directory? I am not able to find nothing about this.
An example, if I click a button and open de fileDialog in a specific folder, I don't want allow to the user to move to another directory. I am doing this to save and load files that are going to be in a specific directory and I don't want the user moves to other directories.
Thanks
Comment: You can't prevent the user navigating to a folder that they have permission to access (as far as I know, not with managed code anyway) but you can certainly validate the path of the selected file and make sure that it is in the folder you want.  Of course, if you don't want the user to see files in any other folder then you can always build the UI yourself and not use the standard dialogues.
Answer: The answer is simple. If you don't want a user to be able to change folders then don't give them the ability to do so. In other words, don't use the openDialog and saveDialog.
The above dialogs are to allow you to select a folder and name, you appear to not need that.
If you still want the user to be able to specify a filename then you could simply have a textbox or an inputbox for the filename alone. Then do your relevant load/save error checks as you would normally.


Need idea how to go about fingerprint access control system in C#

Question: I am developing membership software with C#, but I need to implement access control and also searching for members with fingerprint technology. Any ideas how I could go about that?
Answer: SourceAFIS works well and is C#. I know a Java shop that wrote their client code in C#  primarily so they could use it.
Comment: Thanks Very much. what database are the image stored to.
Comment: I don't think it implements any storage, only fingerprint matching and analysis algorithms.


Migration via phpmyadmin

Question: We are moving to a Drupal distribution. From all the research we have done, there is no easy path to do Drupal 8 to Drupal 8 migration.
We are looking to copy some content via phpmyadmin.
Data we need to move are:

Taxonomy
Nodes
Comments

Which tables do we need to copy from one database to the new?
We did copy the users successfully.
Now wondering if we can do the content as well.
Comment: `there is no easy path to do D8 to D8 migration.` Have you looked at the Migrate API + Migrate Plus module?
Answer: if migration sounds too hard ...
install and config
https://www.drupal.org/project/default_content_deploy
then
<code>"drush default-content-deploy:entity-list"
</code>
to see a list then:
<code>"drush default-content-deploy:export taxonomy_term"
"drush default-content-deploy:export node"
"drush default-content-deploy:exportcomment"
</code>
copy the exported files to your next site, make sure the nodes taxonomy_terms and comments have same field structure (can do this by exporting / importing config or look at features module)
after that import your content.
<code>   "default-content-deploy:import"
</code>
again make backups of your "new" database before this though !
extra bonus use version control on the exported content .. that way developers / live sites can share the content easily.

Migration via phpmyadmin


Prove that for all $x\in\mathbb{R}\enspace \lvert x\rvert + \lvert x-6 \rvert\geq 6$

Question: I am beginning proofs in analysis. I am reading Kane's book, but I am not sure if this proof counts as a proof in analysis. I have tried proving by contradiction, but I failed. I also tried using the triangle inequality
($\lvert x - 6\rvert\leq\lvert x\rvert + \vert\text{-}6\rvert$), but it just does not follow from it. The farthest I got was showing that $\lvert 2x -6\rvert\leq \rvert x\lvert + \rvert x - 6\lvert$ by letting $y = x-6$ and using the triangle inequality. If anybody has any idea how to proceed, I would greatly appreciate some guidance. Thanks.
Comment: Use triangular inequality: $a,b\in\mathbb{R}\;\implies\;|a+b|\le |a|+|b|$, and notice that $6=x+(-x+6)$
Comment: @ÁngelMarioGallegos Thanks for your comment. It was what I needed to figure it out.
Answer: Use the triangle inequality $$|a|+|b|\geq |a+b|$$ and the fact that $|-a|=|a|$.
Comment: Thanks for your help. This part helped at the end of the proof.


How can I output tables to a PDF file with the iPhone SDK?

Question: I want to output a PDF using UIKit's PDF creation methods.  I see plenty of information on the web about creating a graphic context in a PDF, but I want to create smart text tables whose cells the user can later copy and paste into other applications (Word, Excel, etc.).  How do I do this?
Answer: Unfortunately, that's not trivial. I recommend you the libharu PDF library for iPhone as a good point to start from.


re-fitting linear regression after data cleaning

Question: I don't have a good knowledge of statistic but i have looked for 3 hours without resolving the problem so i think to ask
I have two variables and i have made a linear regression

my question is: if i would like to remove same values to have a more fitting line how should i proceed?
i tried removing high values of mean, difference and sd but as i later discovered i was doing it wrong
Comment: Why do you want to remove values to improve the fit of the line? If that's your data, and that's the line that your regression has fitted to the data, then what is your motivation for trying to change it?
Comment: While it may be nicer looking, it's also misleading to present a regression line fitted to a full data set, alongside a set of data points subsequently handpicked to be closer to that regression line. If there are data points there that reflect outliers then it may be worth removing them, following a defensible approach, but then you'd refit the regression it.
Comment: Whatever you do, take a principled approach. If you try multiple ways of messing with the data until it looks nice, your conclusions would be misleading and invalid. Any further statistics you run would be over-optimistic because they wouldn't take into account the earlier procedures that were done explicitly to get a good fit. https://en.wikipedia.org/wiki/Data_dredging
Comment: Yeah, that's possible. You should read the material IcannotFixThis has suggested. If that is the case though then you really should reconsider removing those items. That nice looking graph will be statistically dishonest.
Comment: we are doing a study about a comparison between self-reported (questionnaire, subjective) and objective measurement of physical activity and to show a preview of the study before going to check all the answer in the questionnaire they asked me to hide values too far from the regression
Comment: Why did they tell you to remove to remove those points though? There are different reasons, both good and bad, for why you might want to remove data points.
Comment: it is just to have a "nice looking" preview of the final result before the summer holiday, i know it is not a good reason
Comment: there are a lot of outliers (i will just do two examples: many people didn t understand a frequency question and thought that they had to ansked in hours instead of minutes or they answered about only the past day and not about all the week)
Comment: If you know which people were responding in hours rather than minutes then you could probably justify converting their response into minutes. If you don't then you probably have to assume that everyone responded in minutes and then remove those data points that are statistical outliers (e.g., those >2 SDs from the mean). Similarly, if you know who was thinking only of the past day then remove them a priori, otherwise keep them in unless they're statistical outliers.
Comment: i know but my question is different, for example you say to remove >2 SDs from the mean, but is possible that this person have a >2 SDs from the mean even in the other measurement, so at the end it is correct!
Comment: wow i have just finished to remove the >2 SDs of both the variables and i didn't remove anyone of the "bad" results.... any idea how to proced would be apreciate
Comment: Inspect the model residuals for outliers, and remove any data point associated with an outlier residual (i.e. > 2 SDs). If none of the residuals are outliers then your regression is doing a decent job of modelling your data and you're possibly not justified to remove any data points
Answer: I guess, sometimes you even Need to learn how to lie with statistics. I am not going to comment or judge. The easiest way would be to look at the residuals and kill the point with the largest residual before  repeatind fitting and checking residuals.
The plot looks as if it was made with R, so here is a hint to R: The linear regression is done using lm() and that Returns an object with a slot "residuals". The number of the point with the largest residual can be found with which.max:
<code>x<-rnorm(100)
y <- 2*x+5+rnorm(100)

which.max(lm(y~x)$residuals)
</code>
Comment: I agree with the spirit of the answer, but this approach would only remove one data point. The question did talk about removing values, plural. Perhaps there should be an iterative process?
Comment: @Ian_Fin I had an iterative process in mind, when I wrote "before repeatind fitting and checking residuals". Once one point is removed, all the residuals are not correct anymore and I propose to compute them anew before proceeding. Thanks for the clarification!
Comment: Well, @Bernhard, your comment above is not 100% correct - I fear you are confusing residuals and "influence". High residual points might or might not have big influence
Comment: @IcannotFixThis Fear not! First: High residual points might have big influence and therefore my recommendation stands, to remove one at a time and calculate residuals every time anew. If it was not influencial, than little is lost. Second: the OP wants to improve the visual impression of his regression line. An influential point may very well be situated very close to the regression line and therefore it seems logical, not to remove it just because it is influential. I must admit though, that I am neither trained nor experienced in massaging data into linear form to make for a nice plot.
Answer: Without getting in the dispute concerning whether removing points from your data set might or might not make sense, I think it might be useful clarify the difference between outliers, leverage and influence
I believe such framework would help you understand the role played by each point to your regression.
You can have read here.
Comment: thanks for the paper, i didn't read everything yet, but i think mine are both leverage and outliers, anyway i don't think i will find there how to solve my problem
Comment: Well, I honestly don't think that there's a "right" (in the string meaning of the term) answer to your question. The deck at least covers the theory though.
Comment: @IcannotFixThis The OP asked for a procedure and you answered with the clarification of related terms. You should really clarify, what your answer to the question is.
Answer: In some cases and if your problem is to better "present" rather than interpret it could make sense to log transform data.  It must be done with care and need to ensure that the data are still interpretable. As example

However, if the correlation is poor I don't see how you can get it "nice" looking


Finding orthogonal matrix that maps one vector to another

Question: Let $w, v \in \mathbb{R}^k$ be two known vectors such that $||w|| = ||v||$ ($|| . ||$ is the usual Euclidean norm). My questions are related with the problem of finding $Q$ orthogonal such that $v = Q w$. I know that if we take $u = (w-v)/||w-v||$, then $Q = I-2uu'$ (Householder reflection) will be a solution.
My questions are:
1) Is there any explicit way to characterize the solutions $Q$? For instance, what would be another concrete "formula" for a solution $Q$, different than the Householder reflection above?
2) Say that we have a matrix with eigenvalues $\Lambda$. If $Q_1$ is another solution (i.e. $Q_1$ is orthogonal and $v = Qw = Q_1 w$, $Q_1 \neq Q$), is it true that $Q D Q' = Q_1 D Q_1'$?
3) In essence, does the restriction $Q w = v$ for $Q$ orthogonal "tie down" the structure/properties of $Q$ in any interesting way?
Any reference or pointers are much appreciated... Thanks in advance!
Answer: Assume for simplicity that $\|v\|_2=\|w\|_2=1$. First, we characterize all $Q$ such that
$$\tag{1}v=Qw$$
without assuming any structure of $Q$. The equation (1) can be considered as a linear system with the unknown matrix $Q$. Using the Kronecker product, we can rewrite (1) as
$$\tag{2}
(w^T\otimes I)\mathrm{vec}(Q)=v.
$$
If we set $Q=vw^T$, (1) clearly holds. Hence any $Q$ satisfying (1) can be written as $Q=vw^T+Q_0$, where $Q_0$ is such that
$$\tag{3}(w^T\otimes I)\mathrm{vec}(Q_0)=0.$$ Noticing the structure of $w^T\otimes I$, (3) holds if and only if each row of $Q_0$ is orthogonal to $w$ and hence, if $W$ is a matrix such that its columns contain an orthonormal basis of $\mathrm{span}(w)^\perp$ (that is, $W^TW=I$ and $W^Tw=0$), any $Q_0$ satisfying (3) can be written as $Q_0=VW^T$ for some $(k-1)\times k$ matrix $V$ (combining in an arbitrary way the columns of $V$ to form $Q_0$). Consequently,
$$
\{Q:\;v=Qw\}=\{vw^T+VW^T:\;V\in\mathbb{R}^{(k-1)\times k}\}.
$$
Notice that $Q=vw^T+VW^T$ can be written as $Q=[v,V][w,W]^T$, where the matrices on the right-hand side are square. Since $[w,W]$ is orthogonal by the choice of $W$, $Q$ is orthogonal if (and only if) $[v,V]$ is, that is, $V^TV=I$ and $V^Tv=0$. If we somehow fix both $V$ and $W$, the term $VW^T$ in $Q$ can be replaced as well by the term $VUW^T$ for any orthogonal $(k-1)\times(k-1)$ matrix $U$. Consequently, (denoting $O(k)$ the set of $k\times k$ orthogonal matrices) we have the following:

Let $v,w\in\mathbb{R}^k$ be such that $\|v\|_2=\|w\|_2=1$ and let $V$ and $W$ be $(k-1)\times k$ matrices such that $V^TV=W^TW=I$, $V^Tv=W^Tw=0$. Then
  $$
\{Q\in O(k):\;v=Qw\}=\{vw^T+VUW^T:\;U\in O(k-1)\}.
$$
Comment: there does not seem to be any need for Kronecker product. am i right in assuming that $rank(V) = k-1$ and $V^Tv = 0$ are sufficient.
Comment: @abel True, there's no need for that. I just wanted to write (1) as a "proper" system of equations for $Q$. The matrix $V$ has orthonormal columns to ensure that $Q$ is orthogonal.
Comment: How do you ensure that your solution is in SO(k)?


Angular 2 em produção

Question: Eai galera,tenho uma aplicação em Angular 2 que quero publicar ele no site www.umbler.com servidor de hospedagem mas não consig, tem colocar no servidor em NodeJs mas não funciona.
Não sei como funciona a parte de build, etc.
Alguem poderia me ajudar?
Comment: Não entendi muito bem sua pergunta, mas vc deve fazer sua aplicação local a depois subir para o servidor. Da mesma forma que usar NPM, Bower, Composer, etc.
Comment: Com angular 2 quando gero o build é criado a pasta /dist onde devo colocar em um servidor tomcat por exemplo, como eu faria isso la?
Answer: Para colocar em um servidor nodejs:
1- Se você estiver usando angular cli execute o comando para gerar os bundles:
ng build --prod

Isso irá gerar a pasta dist com todo o código estatico do site.
2- Baixe o Node js em sua máquina de desenvolvimento
3- Após instalar digite no prompt (no diretório um nível acima da pasta dist):
npm init

Responda as perguntas do console (deixando o script de inicialização como index.js)
4- Crie um arquivo index.js:
const express = require('express');

const app = express();

app.set('port', 80);

app.use(express.static(__dirname + '/dist'));

app.get('*',function (req, res) {

  res.sendFile(__dirname + '/dist/index.html');

});

app.listen(app.get('port'), function() {

  console.log('Running on port ', app.get('port'));

});

Sua estrutura de diretório deve ficar:
home
    -dist/
      --arquivos html
    -index.js
    -package.json

Para testar local execute:
npm start

Para executar um site em cada rota:
Considerando que haverão dois diretórios dist: dist_site1 e dist_site2
const express = require('express');

const app = express();

app.set('port', 80);

// Servir estático os dois dists em rotas separadas:
app.use('/site1',express.static(__dirname + '/dist_site1/'));
app.use('/site2',express.static(__dirname + '/dist_site2/'));

// Direcionar para um dos sites caso entre com o link do dominio
app.get('/',function (req, res) {

  res.redirect('/site1');

});

// Direcionar todas as rotas não interceptadas pelo estático para o angular
app.get('/site1/*',function (req, res) {

  res.sendFile(__dirname + '/dist_site1/index.html');

});

// Direcionar todas as rotas não interceptadas pelo estático para o angular
app.get('/site2/*',function (req, res) {

  res.sendFile(__dirname + '/dist_site2/index.html');

});

app.listen(app.get('port'), function() {

  console.log('Running on port ', app.get('port'));

});

Comment: Isso funcionou, obrigado, mas apenas para uma aplicacao, tentei fazer o node gerenciar 2 aplicacoes enviando cada uma atraves do path, mas nesse caso nao da certo, sabe me dizer como resolver?
Comment: Adicionei um exemplo no descritivo da solução com os comentários em cada linha chave.

Angular 2 em produção


How to merge items in a clojure list by types?

Question: I am trying to merge items within a Clojure list by the data type of items.
I tried using conj, merge, into to merge items. But they seem to work perfectly when the list items have the same data types.
(def args '({:a 1} "Heading"))
(def args2 ({:b 2}))
(conj args args2)

The output I get is ({:b 2} {:a 1} "Heading"). Instead I would like to achieve ({:b 2 :a 1} "Heading").
Comment: What if both inputs contain a string?
Comment: For my specific case it multiple strings are there they are the part of the list. Final result can look like `({:b 2 :a 1} "Heading 1" "Heading 2")`. But it would be great if I could specify if I want to concat them or not.
Answer: It is a little bit complicated, but since you asked for flexibility, there you go:
(defmulti conjoin first)

(defmethod conjoin :default [[k v]] v)
(defmethod conjoin (class {}) [[k v]] [(apply merge v)])
(defmethod conjoin (class []) [[k v]] [(vec (apply concat v))])

(let [a '({:a 1} "Heading" [a])
      b '({:b 2} "Fake" [b])]
    (->> (concat a b)
         (group-by class)
         (mapcat conjoin)))
=> ({:a 1, :b 2} "Heading" "Fake" [a b])

Comment: Thanks a lot. This seems really clean and adaptable for future use cases. Realized it more after porting it for my own use case.
Comment: Though this works perfectly in case of Clojure. But when I start using it in ClojureScript I get the error `Use of undeclared Var ad.base.typography/class`. I modified the `class` method to `type` and it worked like a breeze.


Two people listening to the same song, yet they hear two different guitar chords that sound right played. Why?

Question: So two of my friends are learning a song on guitar:

The song begins with an instrumental intro and kick right into the chorus with vocals. The chorus, which is sung and played twice, is the main issue.
One friend (A) hears: Amin, Fmaj, Dmin, Amin, Fmaj, Dmin, Gmaj, Emaj, then this progression repeats
Second Friend (B) hears: Amin, Fmaj, Gmaj, Cmaj, Amin, Fmaj, Gmaj, Emaj, then repeat.
They both sound right to me, oddly, so I suggested loading the song into Riffstation, a popular program that calculated the chords for songs. And see what it would give. The results were from Riffstation:
Amin, Fmaj, Gmaj, Cmaj, Amin, Fmaj, Gmaj, Cmaj. which is also slightly different from both.
My questions are why do both of their progressions sound correct, is it a play on the ear? or can this be explained using theory? Also, what is the true progression for the chorus.
Comment: I hear it as Am F G C Am F Dm E, going more from the bass line than anything, although it's useful to determine whether a chord is maj or min, and tricky to decide between maj and relative min. The sequence is a fairly common one. Either works for the song, but technicalities come into play, as in bar 8 is probably E, the V of Am.
Comment: Are all these chords being played w/o inversion, and w/o any 7ths or other added notes?  Your friends may be hearing 2 or 3 pitches and "filling in" their preferred pitch to build the chord they expect.  I don't know how Riffstation works, so am suggesting you try doing a pure FFT to see what peak frequencies show up (if Riffstation doesn't do that)
Comment: I'm with @Tim, simply listen to the bass line! (Along with some filling in by listening to the rest of the band of course). You don't need Riffstation. It is unequivocally Am F G C Am F Dm E.
Answer: To answer the 'why' in the main question, this phenomenon has to do with tone relationships in the respective scales in classical theory.
The best example is where you equate Am & C above... A natural minor and C major are essentially the same scale with a different tonic - there are no sharp or flat tones in either. So, the C triad - C, E, G - contains the 3 & 5 of the Am triad - A, C, E.
Answer: So, Dm and G are confused, Am and F,also Am and C. I think a clearer listen is in order. Of course, any of these chords COULD fit, possibly in different orders, but the poser is why.
The answer may be shared notes, with C and Am, there's common C and E. With F and Am there's A and C. Dm aganst G is trickier, but if it was G7, a common V chord in the key, there's  D and F.
When there is more than one player, each may not be playing all the notes from a chord, and often the bass player, especially in stuff like this, will be putting down the root note each bar. That's my go-to for working out chord sequences, and it usually works. There are occasional chords - like Em and Cmaj7, which are very similar in make up, so the bass note will define them. Each of the queries involves a choice between a major or minor chord. Just being able to hear the difference will sort that out.
Answer: They are both hearing it imperfectly.  Very likely they are both mixing what they DO hear and what their grasp of theory suggests as a possibility.   (It's quite hard to 'hear' a chord that is beyond your 'theory' horizon.  For instance, if you only know major and minor triads, a dim7 or a dom7(b9) is likely to confound you.)
Yes, there will usually be several equally possible chord sequences that fit the same song.
And although Riffstation can be a useful resource, it is by no means authoritative!


Question about bayesian theory with mixed discrete and continuous variables

Question: For Bayes' rule, $P(a|x)=\frac{P(x|a)P(a)}{P(x)}=\frac{P(x|a)P(a)}{P(x|a)P(a)+P(x|a^c)P(a^c)}$, if we have $P(x|a)$ and $P(a)$ are discrete (e.g. both equal to 0.5), and $P(x|a^c)$ is continuous (e.g uniform(0,1)), could I just set $P(x|a^c)=1$? If not, how could calculate $P(a|x)$?
Answer: Define $Y=I_A$ and let $f_{X\mid Y}$ denote be the conditional density of $X$ given $Y$. Then,
$$
  \Pr(A\mid X=x) = \Pr(Y=1\mid X=x) = \frac{f_{X\mid Y}(x\mid 1)\cdot\Pr(A)}{f_{X\mid Y}(x\mid 1)\cdot\Pr(A) + f_{X\mid Y}(x\mid 0)\cdot\Pr(A^c)}.
$$
Comment: I don't quite understand, here $f_{X|Y}(x|1)$ and $Pr(a)$ still equal to $0.5$ in my example, and $f_{X|Y}(x|0)$ still follows uniform (0, 1), it does not change anything.
Comment: $Y=1$ means that the event $A$ "happened", while $Y=0$ means that it didn't. You have to specifiy the conditional density of $X$ for the two possibilities. For intance, you may say that $X\mid Y=1\sim\mathrm{N}(0,1)$ and $X\mid Y=0\sim\mathrm{N}(2,1)$. If $X\mid Y=y\sim\mathrm{U}[0,1]$ always (meaning, for $y=0,1$), then the problem is trivial: $\Pr(A\mid X=x)=\Pr(A)$. Check if you're reading your problem statement correctly.


Are these two 2-manifolds homeomorphic?

Question: I have a 2-Sphere with a finite number $k$ of points removed (at least 3), and I want to equip it with a Riemannian metric of constant negative curvature.
My first thought was to take a free subgroup $A_k$ of rank $k$ of the isometries of the hyperbolic plane $H$, which acts fix point free and discontinuous on $H$ (such a group exists).
If I take the quotient $H/A_k$, I end up with a complete manifold of constant negative curvature, with isomorphic fundamental group to the punctured Sphere, but are those two manifolds homeomorphic? Maybe this is a terribly clumsy way to try to accomplish this task.
Answer: That's not necessarily the case.
Take the sphere with 3 punctures: it has fundamental group free on two generators, right?
Now take the torus with one puncture (which has indeed a hyperbolic structure). You can easily see this has the same fundamental group.
So, to answer your question you probably need to know which group you are using (simply because we have just seen that it depends on that).
To endow the thrice punctured sphere with a hyperbolic metric, you should start with two identical copies of an ideal triangle in $\mathbb{H}^2$, and glue them together. If you are careful you can check that this gives you a hyperbolic surface homeomorphic to $S^2-\{p_1,p_2,p_3\}$.
Do you see how you could generalize this to, say, a 4-punctured sphere?
For another approach, non explicit:
A thrice punctured sphere $S$ can be embedded (continuously) in $\mathbb{C}$, as $\mathbb{C}$ with two holes. So, $S$ has a riemann surface structure. By the riemann mapping theorem, it is holomorphically covered by $\mathbb{H}$, the upper half plane (since we removed at least 2 points). This allows you to put a hyperbolic metric on $S$. For details and references, check the wikipedia article, http://en.wikipedia.org/wiki/Uniformization_theorem#Geometric_classification_of_surfaces
Comment: Ah, right, thanks! I didn't think it through there. Still I want to try to avoid any explicit construction of this punctured sphere in hyperbolic space if possible. Could there be a way to use the uniformization theorem? If I can rule out the possibility for the punct. sphere to be a quotient of $\mathbb{R}^2$ by a nice group of euclidean isometries, then there has to be a metric of constant negative curvature on this sphere (the positive curvature case is ruled out because the fundamental group is not finite). Maybe it's not possible to find such a "nice" free subgroup of eucl. isometries?
Comment: Well, do you just need to show that it exists, or do you need to find one?
Comment: Also, I'd like to "nudge" you towards an explicit construction. This has the advantage that you realize how many different hyperbolic structures you can endow it with, and also makes you realize what makes them distinct.
Comment: Yeah, you're probably right, but for now it would suffice for me to know, that there exists one. The only way I can think of doing this, is by showing that there is no free subgroup (of rank at least 2) of the isometries of $\mathbb{R}^2$, which acts freely and properly discontinuous. I don't really know if this makes sense.
Comment: Well, it's easy to see that the euclidean isometries of $\mathbb{R}^2$ are just traslations and rotations, and that any subgroup acting freely and discretely must be either trivial, infinite cyclic generated by a traslation, or $\mathbb{Z}^2$ generated by two independent traslations.
I added a bit at the end of the answer, concerning the use of the mapping theorem.
Answer: I think we are saying the same thing but in a slightly different order. Isn't it true that a sphere with at least 3 points removed is homeomorphic to an open set in a compact Riemann surface of genus $\geq2$?
If so, the latter can be endowed with a constant negative curvature because it is uniformized by the hyperbolic plane.
Comment: I think it is implied that riemannian metric of constant negative curvature means that it is complete.

Are these two 2-manifolds homeomorphic?


Is there such a thing as "pop music theory"?

Question: There is large body of work describing the theory of Western classical music;
I'm unaware of the state of analysis of popular music forms.
Are the differences between, say post-50's American pop-music (i.e. rock and similar),
and classical western music significant enough to claim that pop music
can/should be described in terms of a different theory than classical music?
Who in the musicology world have made claims on whether a different or the
same theoretical descriptions apply to both popular and classical music?
Answer: All I can say is that the more recent the music is, the less scholarly work has been done to study it. This goes along with the timeless principle that "theory follows practice."
But music theory is music theory, and yes, you can study any piece of Western pop music and analyze it according to the established principles of Western functional harmony and form and analysis.
Just look at all the guitar and keyboard magazines published in the last 40 years that provide transcriptions and analysis of this or that famous guitar or keyboard solo, in jazz and rock. They all use the same Western musical notation and music theory and form and analysis that is applied to classical music. That is enough to prove the point that the theory still applies and is useful in studying and playing this music.
There are academic journals that publish scholarly papers on music written in the last sixty years.
A lot of musicological work has also been done in the area of testimony for copyright infringement lawsuits, although that work doesn't get widely published.
There are some books out there on analyzing music of the last 60 years. One that springs to mind is The Beatles as Musicians series by Walter Everett.
Comment: I totally agree with Wheat Williams, just wanted to add that both 'new' music, jazz and pop are well known for throwing a curve ball at traditional ways of doing music--hence expanding and challenging theoretical boundaries and limits.
Comment: Since electronic music is relatively new, at least under 100 years old, it's going to take some time to develop a theory on how it is/was practiced. Also, since this form is so technology driven, the vocabulary is a moving target. Terms like VCA, VCO, and Ring Modulator now longer have the same relevance they had in the 1970's when I was a graduate student learning analog manual patching on a Moog and Buchla. However the same acoustics apply, so perhaps like then most electronic composers embrace the more physical aspects of the sound.
Comment: I've often noted to myself that traditional music theory isn't at all adequate to describe electronic music because there isn't any formal vocabulary to deal with the timbres of soundscapes, sound design, synthesis, non-tonal forms of musical expression like "glitch" and so on and so forth. It may be that these elements in music will always defy theory and analysis to some degree or other.


H to check if proportions in two small samples are the same

Question: I have two small samples denoting if something occured or not. For example:
$s_1=\left(1,1,1,0,0,1\right)$
$s_2=\left(1,0,0,0,0,1,1,0\right)$
I want to test if proportions of occuring $1$ are the same. I cannot use proportion test for two samples, as samples are too small. Can I use Chi square homogenity test and get the robust outcome?
Comment: *Of course* you can use a test of proportions on these data. Your real problem is that you haven't enough data to discern any but the very largest differences between proportions. Unfortunately--in this case--that problem will not go away by choosing another test. You have only three kinds of options: (1) make some strong assumptions (such as a prior distribution); (2) collect more data; (3) accept the results of the test of proportions (which would be not to reject the null hypothesis of equal proportions). Which of these options are ones you would consider?
Comment: Take a look at fischers exact test. Also, you can find a calculator on the net to do the calculations
Comment: Fishers exact test (or indeed any of the other myriad alternatives) will simply not solve the problem that there's no power to reject any but extremely large differences in proportion.
Answer: The reason your sample size is a problem for a chi-square test or a test of proportions that relies on approximation to a Normal distribution is that these tests use asymptotic results. In other words, they are built on the assumption that you have "large enough" sample size.
A test that is similar to the chi-square test, but does not rely on asymptotic assumptions, is Fisher's exact test described here: http://en.wikipedia.org/wiki/Fisher%27s_exact_test
Comment: No, that's exactly my point: there is not enough evidence here. To obtain a significant result (at $\alpha=0.05$) with two samples of sizes six and eight, conditional on observing seven positives, you would need *all* elements of the smaller sample to be positive. And you could determine that using the standard test of proportions (but, as you point out, only by applying the exact sampling distribution,  because an asymptotic approximation would be suspect).
Comment: Unfortunately, exact implementations of tests of proportions will not make these data appear significantly different (at least for any reasonably small test size).  For instance, even an exact one-sided test has a p-value of $\sum_{i=4}^{4+3}\binom{6}{i}\binom{8}{4+3-i}/\binom{6+8}{4+3}=1016/3432\approx 30\%$.
Comment: Agreed - but it does allow you to perform a test. It is conceivable that there is enough evidence to reject a null hypothesis of equal proportions with the sample sizes Misery gives.


CSS: how to get scrollbars for div inside container of fixed height

Question: I have the following markup:

Title

    ..blah blah blah...

The CSS looks like this:
.FixedHeightContainer
{
  float:right;
  height: 250px;
  width:250px;
}
.Content
{
  ???
}

Due to its content, the height of div.Content is typically greater than the space provided by div.FixedHeightContainer. At the moment, div.Content merrily extends out of the bottom of div.FixedHeightContainer - not at all what I want.
How do I specify that div.Content gets scrollbars (preferably vertical only, but I'm not fussy) when its height is too great to fit?
overflow:auto and overflow:scroll are doing nothing, for some bizarre reason.
Answer: setting the overflow should take care of it, but you need to set the height of Content also. If the height attribute is not set, the div will grow vertically as tall as it needs to, and scrollbars wont be needed.
See Example:
http://jsfiddle.net/ftkbL/1/
Comment: If you give `Content` a fixed height, you shouldn't give `FixedHeightContainer` a fixed height, because you can't know how high your title will be, so `Content` may be pushed out. See: http://jsfiddle.net/ftkbL/2/ You should set the fixed height **only** on the element with `overflow: scroll`. See http://jsfiddle.net/ftkbL/3/ or http://jsfiddle.net/ftkbL/4/
Comment: Okay - thanks. So I need to specify a height for div.Content? I assumed it would work out from the container that it wouldn't fit, and apply scrollbars on that basis.
Comment: I see your point (from the first link) but the title text is known and is too short to break over a line unless the user inflated the text to an impractical size.
Comment: Is there a way to have a short height and hide the scroll-bar at the same time?  this way when users drag down on mobile devices, they will see that they are scrolling down, but without the cumbersome of seeing 2 scroll-bars on their browser?
Comment: @blackhawk - Not that I know of. You may need to use Javascript for this. Specifically I am thinking of the jQuery Draggable library: http://jqueryui.com/draggable/ -one thing to consider is... how will desktop users know to scroll?
Comment: Thanks @Dutchie432.  I will use CSS media queries to display this feature only on small width devices, then on desktop sizes, show it differently.
Comment: `height: auto;` not working in this case. any suggestions?
Comment: The height has to be hard-coded to a numeric value. If you specify "height:auto" the container should grow in height to fit the content, thus eliminating the need for scrollbars in the first place.
Answer: FWIW, here is my approach = a simple one that works for me:

blah blah blah

html, body {
   height: 100%;
   margin: 0em;
}

#outerDivWrapper, #outerDiv {
   height: 100%;
   margin: 0em;
}

#scrollableContent {
   height: 100%;
   margin: 0em;
   overflow-y: auto;
}

Answer: This is a great question because the answer is not immediately obvious, for such a simple task. The problem has hit me several times over the years, and It seems I always have to think about it before I get it right.
Below is my solution which uses (only) two CSS-classes, 'innerDiv' and 'outerDiv'.

     A
 B
 C
 D

Why does that work, and why do I say the answer is "not immediately obvious"?
In the example above both the outerDiv and innerDiv have fixed height because we set the height: -property for both of them.
We have the innerDiv height less than the height of the outerDiv. Therefore innerDiv "fits" into the outer-div and therefore the outer div should not get a scrollbar, right? Yes except if the innerDiv has so much content that it does not fit into the fixed height of the innerDiv but overflows. Therefore the question is how can we prevent the inner div from "overflowing"?
The way to prevent the innerDiv from overflowing is to give it the attribute "overflow: auto", or "overflow-y: auto" if we just want to prevent vertical overflow.
The reason the solution is "not immediately obvious" is this: The default value of 'overflow' is NOT 'auto', but 'visible'.
One might (erroneously) assume that the default value of overflow is  "auto", because then everything should be auto-matically taken care of, right? Not so fast, the default of overflow is 'visible', NOT 'auto'.
So if you remove the "overflow-y:auto" from the example above there will be no scrollbars. If you remove it but add it to the outerDiv, the outer div will have the scrollbar, which is NOT what we want. If you have it in both then only innerDiv will have it.
Answer: Code from the above answer by Dutchie432
.FixedHeightContainer {
    float:right;
    height: 250px;
    width:250px;
    padding:3px;
    background:#f00;
}

.Content {
    height:224px;
    overflow:auto;
    background:#fff;
}

Answer: HTML

Title

    ..blah blah blah...

CSS
.FixedHeightContainer
{
  float:right;
  height: 250px;
  width:250px;
  overflow-y: scroll;
}

/*SCROLLBAR MODIFICATIONS*/

.FixedHeightContainer::-webkit-scrollbar {
    width: 8px;
}

.FixedHeightContainer::-webkit-scrollbar-thumb {
    background: #909090;
    border-radius: 8px;
}
.FixedHeightContainer::-webkit-scrollbar-track {
    background: #FFFFFF;
}

Answer: 1)Set the height and width of div.Content smaller or equal as per design(preferably smaller). You may use px or pcnt as unit.
2)Now bring overflow-y property and set the value auto.

CSS:

HTML:

Title

    Any length/size of content.