Gauss-Seidel Method in MATLAB

Question: I am trying to implement the Gauss-Seidel method in MATLAB. But there are two major mistakes in my code, and I could not fix them:

My code converges very well on small matrices, but it never converges on large matrices.
The code makes redundant iterations. How can I prevent from redundant iterations?

Gauss-Seidel Method on wikipedia.
N=5;
A=rand(N,N);
b=rand(N,1);
x = zeros(N,1);
sum = 0;
xold = x;
tic
for n_iter=1:1000
    for i = 1:N
        for j = 1:N
            if (j ~= i)
                sum = sum + (A(i,j)/A(i,i)) * xold(j);
            else
                continue;
            end
        end
        x(i) = -sum + b(i)/A(i,i);
        sum = 0;
    end
    if(abs(x(i)-xold(j))<0.001)
        break;
    end
    xold = x;
end
gs_time=toc;
prompt1='Gauss-Seidel Method Time';
prompt2='x Matrix';
disp(prompt2);
disp(x);
disp(prompt1);
disp(gs_time);

Answer: First off, a generality. The Gauß-Seidel and Jacobi methods only apply to diagonally dominant matrices, not generic random ones. So to get correct test examples, you need to actually constructively ensure that condition, for instance via
A = rand(N,N)+N*eye(N)

or similar.

Else the method will diverge towards infinity in some or all components.

Now to some other strangeness in your implementation. What does
if(abs(x(i)-xold(j))<0.001)

mean? Note that this instruction is outside the loops where i and j are the iteration variables, so potentially, the index values are undefined. By inertia they will accidentally both have the value N, so this criterion makes at least a little sense.
What you want to test is some norm of the difference of the vectors as a whole, thus using sum(abs(x-xold))/N or max(abs(x-xold)). On the right side you might want to multiply with the same norm construction applied to x so that the test is for the relative error, taking the scale of the problem into account.

By the instructions in the given code, you are implementing the Jacobi iteration, computing all the updates first and then advancing the iteration vector. For the Gauß-Seidel variant you would need to replace the single components in-place, so that newly computed values are immediately used.

Also, you could shorten/simplify the inner loop
xold = x;
for i = 1:N
    sum = b(i);
    for j = 1:N
        if (j ~= i)
            sum = sum - A(i,j) * x(j);
        end
    end
    x(i) = sum/A(i,i);
end
err = norm(x-xold)

or even shorter using the language features of matlab
xold = x
for i = 1:N
    J = [1:(i-1) (i+1):N];
    x(i) = ( b(i) - A(i,J)*x(J) )/A(i,i);
end
err = norm(x-xold)

Answer: %Gauss-seidal method for three equations
clc;
x1=0;
x2=0;
x3=0;
m=input('Enter number of iteration');
for i=1:1:m
    x1(i+1)=(-0.01-0.52*x2(i)-x3(i))/0.3
    x2(i+1)=0.67-1.9*x3(i)-0.5*x1(i+1)
    x3(i+1)=(0.44-0.1*x1(i+1)-0.3*x2(i+1))/0.5
    er1=abs((x1(i+1)-x1(i))/x1(i+1))*100
    er2=abs((x2(i+1)-x2(i))/x2(i+1))*100
    er3=abs((x3(i+1)-x3(i))/x3(i+1))*100

    if er1<=0.01
       er2<=0.01
       er3<=0.01
        break;
    end
end

Gauss-Seidel Method in MATLAB


Starting violin with previous music experience

Question: I have a question with probably not such a straightforward answer, but I'd love some guidance.
I'm 25 years old, and I started learning the piano when I was 11 years old. After 5 years of learning I quit taking music classes, which I actually regretted doing, so about two years ago I bought myself a piano and started practicing again on my own, and this year (from september on) I have started taking piano classes again.
I progressed quite fast when I was younger I'd say, and I feel right now the same is happening. I'm quite dedicated and practice every day, I don't have a rigorous planning but I think I do atleast half an hour per day.
And now to come to my actual question: I have actually always thought about learning the violin besides piano because I find it such a beautiful instrument, and recently I've been thinking I'm in a good place right now to start learning it. The one thing that scares me is the idea that it would take me years to learn just the absolute basics (just producing clean notes, lets say). So I was wondering how long it usually takes to get past the first stage of learning the violin where it sounds just horrible (lets be honest), keeping my experience in mind. By the way, I would definitely take classes for it if I would start playing.
Thanks in advance!
Answer: "The one thing that scares me is the idea that it would take me years to learn just the absolute basics"
This is more of a psychological issue than a musical issue.  I'll share some of my experiences and maybe that will help.  I started on the violin at age 5.  I was good right away and excelled fast in private lessons.  At about 8 or 9 (I don't remember) I started private lessons on electric guitar and forgot about the violin.  Later in high school I wanted to enter an accelerated music program at a college prep academy and guitar was not on the list of allowed instruments.  So I thought I'll just pick up the violin again and it will be no problem to get my chops back.  That never happened.  It was a disaster.  I started on the upright bass and took to that so fast I was doing gigs within 6 months.  However!  I continued with classical bass lessons for about 6 years and personally feel like I never mastered the bow (regardless of whether my instructor gave positive feedback), but I persisted.  I tried different quality of bow, weights on the bow etc.  As a music major I had to "learn" piano (take piano lab) and hated it.  I have no problem using both hands on the classical guitar but could never coordinate both on the piano (I will likely never understand why).  At this point in my life I've played guitar professionally for many years and decided last year to take classical voice lessons.  I have gotten better but it was a real crush to my ego that I didn't take to it naturally, as with guitar and bass.  But it is what it is.
If you're looking for someone to tell you that violin is easy or not that is not a fair question since it depends on the person.  For me it was easy then impossible as I was a different person 5 years after quitting.  I will tell you a couple things about any member of this family of instruments.
(1) They are fretless.  Unlike the piano which is an equal tempered instrument the violin, cello, and bass are fretless so there is no guide for being in tune except the ear.  You will need to get used to that and that is good for all musicians in my opinion.
(2) The bow can be a challenge to master for some.  But that depends on the quality of the bow and the teacher.
I think you should get a teacher from the start if you want to seriously explore the violin.  Also, you are going to just have to try and see how it goes.  So what if it takes 5 years to get good?  That is what it takes.  You are either committed or not.  If you put a time constraint on it before you start you have already quit.  I'd say just try and see if you like it.  See if it fits your body and personality.  Then you will devote the time and the technique will evolve.
I think that most people that pursue the arts have some natural talent in one area or another.  This is good but can backfire in that when things take actual effort it's a turn off.  The things that take effort are definitely worth doing.
Answer: Well, you practice for about at least 30 minutes a day on piano, so if you're willing to do the same on the violin, I'm pretty sure you can learn it. It probably won't take you too long to get past the basics, considering that you practice every day. You might want to ask your violin teacher (if you get one) about that because it really depends for everybody. For some people, it might take a few years, but for others, it might take only a few months. Some tips...

Listen to the violin. A LOT. When you listen to it, you can try to find the beat or maybe just listen to it for fun. It might speed up the process from the "beginner stage". Whatever book you are using (I use Suzuki), listen to the soundtracks. That'll probably make the process faster.
Ask questions. Since you said that you'll get a teacher, I agree with @ggcg on getting a teacher at the start if you're serious about learning about it. If you're confused, don't be afraid to ask questions. (You can ask questions on here and to your teacher)

You probably will have tapes on your violin as guidance as to where your fingers go. Gradually, as you get better at it, the tapes might be removed from you violin. And also, don't be nervous! You'll be just fine. You should probably talk to your teacher about it.
I hope you decide to take on learning the violin!
~ CatbusTotoroDustbunnies07


Checking the uniform convergence of a sequence of functions.

Question: $f_n(x)= \frac{1+x^n}{2+x^n},$    $x\in[0,1)$
Clearly, the pointwise limit function is the constant $\frac{1}{2}$  function.
I found $|f_n(x)-f(x)|$ for uniform convergence:
Let $g_n(x)=|f_n(x)-f(x)|$.
$g_n(x)=\frac{x^n}{4+2x^n}$.
I noticed that $g_n((\frac{1}{2})^\frac{1}{n})= \frac{1}{10} \neq 0$.
Therefore we don't have uniform convergence in the given interval. I have here 2 questions. Does my work seem good and acceptable? Also, what if we want to change the domain of definition to $[a,1)$ or $[0,a]$ for some $a$? How should we choose this $a$ to avoid having $x's$ like the one I mentioned before in the interval?
Regards.
Answer: The problem (for the uniform convergence) is not near $x=0$, but near $x=1$.
(Indeed the sequence $(1/2)^{1/n}$ converges to $1$.)
You can see this fact also in this way.
If you consider the functions $f_n$ defined in $[0,1]$, instead of $[0,1)$, then the sequence converges pointwise to the function
$$
f(x) =
\begin{cases}
1/2, &\text{if}\ x\in[0,1),\\
2/3, &\text{if}\ x = 1,
\end{cases}
$$
hence $(f_n)$ cannot converge uniformly to $f$ in $[0,1]$ (for otherwise $f$ must be continuous, being the $f_n$ continuous).
But you see in a moment that
$$
\sup_{x\in[0,1]} |f_n(x) - f(x)| = \sup_{x\in[0,1)} |f_n(x) - f(x)|,
$$
so the convergence cannot be uniform also in $[0,1)$.


Using singleton class in Sharepoint2010 environment - webapplication wide?

Question: Let's say I have one web-application with n SiteCollections. Now I have a feature, which includes a singleton class. Does every SiteCollection have their own context and their own Instance of this class or is the singleton "Web-Application-Wide"?
Answer: Okay, I found it on my own: Yes, the singelton is the same for every site-collection, because there are all running in the same iis application-pool.


Is this safe on a production server?

Question: I have a database application (or search engine) which is called Solr.
I connect to it via port 8983.
I do this from PHP code, so I add and remove records from it via php.
On my server I have a firewall.
I have set this firewall to only allow connections to and from this port (8983) from the ip address of my own server. In other words, only allow servers IP to access this port.
Is that safe? Or am I thinking all wrong here? Will others be able to "simulate" my ip address and act as the server?
This is because otherwise others may add/remove records as they want from their own IP addresses...
Thanks
Comment: you're good, in general others cannot "simulate" your IP address.
Comment: @Artefacto - Unless they're able to bounce a packet off of a compromised computer.
Comment: @amphetamachine: Thats called a "martian" packet. Its when a packet that claims to be a local address comes in on a remote interface. Most systems helpfully drop them automatically, and it can be configured to log that instance.
Comment: @amphetamachine: Or they exploit a remote vulnerability in the firewall. Or they bypass the firewall because it tries to detect nondeterministic behavior, or the firewall has a bug and you can bypass a basic IP/Port block rule on it, etc, etc, etc. Point is, firewall has nothing to do with weather your server is secure or not.
Answer: It might be a good idea to also block all outgoing traffic from port 8983 on the server to anywhere but your own server's IP address. This, in addition to dropping any packet to that port not from your server, will doubly ensure that, even if someone is somehow able to modify the daemon listening on port 8983 on the server, allowing it to mirror traffic to another host, it would be dropped before it leaves your computer.
Answer: Yes, you are safe as long as no one gains control of your local server.
You can also cause Solr to bind to the "localhost" or "127.0.0.1" adapter as opposed to "0.0.0.0", which would have a similar effect. It never hurts to layer the firewall above that just in case the configuration is messed up.
Answer: You would not be safe if you are worried of tampering from the same network. There are many situations where the real threats are from inside the network, not from some script kiddie a continent away.
I agree with theatrus to use only localhost.
If you are deployed on multiple hosts there are several ways to create a secure tunnel, e.g
ssh -l 8983:localhost:8983 solr.server
this will create a secure tunnel. (Although it takes non trivial CPU when the bandwidth is high). There are also other solutions.
An additional advantage is that for a developer you can use a sample solr server locally and your code in your IDE, and it will just work with the same config as in production. The less that needs to be changed when deploying, the better.
Answer: This is safe.  The ip address used in a TCP connection because of the three way handshake.  This is a good firewall rule-set,  but you should always test your rulesets with nmap.
What you do have to worry about is running an open proxy server on the server that is executing the PHP.

Is this safe on a production server?


melody analysis according to chord or structural tones?

Question: I have come across two ways of studying melody.
First way is with structural tones(melodic outline) and second way is with chords(chord/none-chord tones).
I assume considering melodic outline is only when chords are not available.
Then, when there are chords present in a lead sheet, considering melodic outline really necessary?
Comment: Chords in a lead sheet are not a harmonic analysis. They're instructions for accompaniment. If there's a chord symbol F and the melody has an E note - if that makes the accompanist play a simple F major triad and a soloist a simple E note, you effectively get an Fmaj7 chord. The whole chord-tone vs non-chord-tone question is subjective and genre dependent. An acquired taste.
Comment: This may not fully answer your question, but I suggest reading it and then updating your question based on what you learn: [Can two songs share the same melodic notes but still have distinct melodies?](https://music.stackexchange.com/q/132443/70803).
Answer: Theory on melody is not as well covered, and easily available, as theory about harmony. So, if you want to get into studying melody, take time to build up a good collection of resources.
However, this much should be said about your question: it is not an either or case. Rather that choose analysis by chord tones or structural tones, as if those are mutually exclusive, you want to use both analysis tools.
When you say "lead sheet" I assume we are taking about melodies in a homophonic style. That is, melodies that could be harmonized with chords. If you are analyzing such a melody and there happens to be no chords given, you can, and probably should in many cases, analyze the melody according to its harmonic implications.
Answer: Analyze the melody. Lead sheets are not that good in general. If you get the melody analyzed, you may find more than one possible harmony.
A good (very old but the principles still apply and there is a free pdf) book on melody is Geotschius' "Exercises in Melodic Writing." It's good for most songs. Once you get the basics, you can expand as your ear tells you.


Incluir registro e retornar botão desabilitado sem refresh

Question: Eu fiz um botão adicionar amigo, como nas redes sociais, só que uso formulario pra fazer isso, ele manda o usuário para uma outra pagina. Queria alguma forma dinâmica sem atualizar a pagina, ao clicar no botão é feito o insert no banco de dados e retorna o botão desabilitado não deixando o usuário acionar o botão mais de uma vez.
Esta é a página  onde o usuário adiciona um usuário:
amigos.php
<a class="btn btn-primary" amigo_id="<?php echo $id; ?>">Adicionar amigo

Ao clicar no botao teria que executar esse arquivo em segundo plano sem atualizar a página.
adicionar-amigo.php
<?php

session_start();

$conn = new mysqli(DB_HOST, DB_USER, DB_PASS, DB_NAME);
$conn->set_charset("utf8");

if ($conn->connect_error) {
    die("A conexão falhou: " . $conn->connect_error);
}

$sql = "INSERT INTO amigos (id_usuario,id_amigo,data_adc)
VALUES ('$_SESSION[UsuarioID]', '$_GET[id]', now() )";

if ($conn->query($sql) === TRUE) {
    echo "Amigo adicionado!";
} else {
    echo "Erro: " . $sql . "
" . $conn->error;
}

Answer: Primeiro coloque uma class no botão para identificá-lo para o javascript:
<a class="btn btn-primary botao-adicionar-amigo" amigo_id="<?php echo $id; ?>">Adicionar amigo

Agora utilize essa class como seletor no evento click:
    $(".botao-adicionar-amigo").click(function() {

      var amigo_id = $(this).attr('amigo_id'); //Capturando o atributo amigo_id

     //Post ajax para sua página php, sem refresh
     $.ajax({
       type: "GET",
       url: "adicionar-amigo.php",
       data: {id: amigo_id},
       success: function(resposta){
          console.log(resposta);
       }
     });

      //Esconde o botão
      $(this).css('display','none');

    });

Aqui tem a documentação do post via ajax https://api.jquery.com/jquery.post/
Na sua página php continue fazendo a mesma coisa. Ela não mudará nada.
Talvez você precise tratar o retorno de alguma forma específica, no exemplo acima está sendo devolvido ao console do navegador, mas você pode fazer da forma que preferir.
O botão, após executar a função click receberá o atributo hidden do css. Você também pode trocar para aquele que for melhor para seu cenário.


Evaluate $ \int^{ \pi/2}_{- \pi/2} \frac {1}{ 1+e^{\sin x} }dx $

Question: Evaluate $ \int^{\pi/2}_{-\pi/2} \frac {1}{ 1+e^{\sin x} }dx $
Solution:

I think odd, even functions are of no use here.
Also we get nothing by taking $e^{\sin x} $ common in denominator.
Also rationalizing  the denominator is of no use.

Really I have no idea how to solve this question.
Please help.
Comment: I must say that your comment about odd/even functions caused me to try what I did below in my comment. Good one.
Answer: Put
$$f(x)=\frac1{1+e^{\sin x}}\implies f(-x)=\frac1{1+e^{\sin(-x)}}=\frac1{1+e^{-\sin x}}=e^{\sin x}f(x)=1-f(x)$$
so that we have
$$\int\limits_{-\pi/2}^{\pi/2} f(x)\,dx=\int\limits_{-\pi/2}^0f(x)\,dx+\int\limits_0^{\pi/2}f(x)\,dx=$$
$$\int\limits_0^{\pi/2}f(x)e^{\sin x}dx+\int\limits_0^{\pi/2}f(x)\,dx=\frac{\pi}2$$
Comment: Great job. Thanks a lot
Comment: Believe me, when I got this result I was pretty surprised, and I even did something I do very few occassions: I resourced to WA to verify the result!
Comment: Very neat! Not entirely sure how you solved the final integrals on the last line though. I might just be having a slow moment.
Comment: @john, since $\,f(x)e^{\sin x}=1-f(x)\,$ , in the last line we're left only with the integral of $\,1\cdot dx\,$ ...
Comment: ahh, of course. Long day...
Answer: $$ \int^{\frac \pi2}_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx $$
$$ =\int^0_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx+\int^{\frac \pi2}_0 \frac 1{ 1+e^{\sin x} }dx $$
$$\text{Now, putting } y=-x \text{  in } \int^0_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx,$$
$$\text{ we get } \int^0_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx=\int^0_{\frac \pi2}\frac1{1+e^{\sin(-y)}}(-dy)$$
$$=\int^0_{\frac \pi2}\frac{e^{\sin y}}{1+e^{\sin y}}(-dy)\text{ as }\sin(-y)=-\sin y$$
$$=\int^{\frac \pi2}_0\frac{e^{\sin y}}{1+e^{\sin y}}dy \text{ as } \int^a_bf(x)dx=-\int^b_af(x)dx$$
$$=\int^{\frac \pi2}_0\frac{e^{\sin x}}{1+e^{\sin x}}dx \text{ as } \int^a_bf(x)dx=\int^a_bf(y)dy$$
$$\implies  \int^{\frac \pi2}_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx=\int^0_{-\frac \pi2} \frac 1{ 1+e^{\sin x} }dx+\int^{\frac \pi2}_0 \frac 1{ 1+e^{\sin x} }dx $$
$$=\int^{\frac \pi2}_0\frac{e^{\sin x}}{1+e^{\sin x}}dx+\int^{\frac \pi2}_0 \frac 1{ 1+e^{\sin x} }dx$$
$$=\int^{\frac \pi2}_0\frac{1+e^{\sin x}}{1+e^{\sin x}}dx=\int^{\frac \pi2}_0dx$$
Comment: it is somewhat like above answer. But you explained in detail . good work.
Comment: @rst, if you observe carefully, my answer has evolves around the manipulation triggered by the limits, whereas the other answer evolved around the property of the function $f$
Comment: Yeah,you can say that
Answer: This answer is a slight generalization of @DonAntonio's.
Any function can be decomposed into even and odd functions,
$$f(x) = f_{+}(x) + f_{-}(x),$$
where
$$f_\pm(x) = \frac{1}{2}(f(x)\pm f(-x)).$$
Note that $f_\pm(-x) = \pm f_\pm(x)$,
so $f_+$ is even and $f_-$ is odd.
Thus,
$$\begin{eqnarray*}
\int_{-a}^a f(x)dx
&=& \int_{-a}^a f_{+}(x)dx + \underbrace{\int_{-a}^a f_{-}(x)dx}_{0} \\
&=& 2\int_{0}^a f_{+}(x)dx
\end{eqnarray*}$$
for any function.
For $f(x) = 1/(1+e^{\sin x})$ we have
$f_+(x) = 1/2$.
Therefore,
$$\int^{ \pi/2}_{- \pi/2} \frac {1}{ 1+e^{\sin x} }dx
= 2\int_{0}^{\pi/2} \frac{1}{2} dx
= \frac{\pi}{2}.$$
Answer: Generalization:
Use $\displaystyle I=\int_p^qf(x)\ dx=\int_p^qf(p+q-x)\ dx$
so that $\displaystyle I+I=\int_p^q[f(x)+f(p+q-x)]dx$
For $\displaystyle I=\int_{-a}^a\dfrac1{1+c^{g(x)}}$ where $g(x)$ is any odd function i.e., $g(-x)=-g(x)$ like $\sin x$
$f(p+q-x)=\dfrac1{1+c^{g(-x)}}=\dfrac{c^{g(x)}}{1+c^{g(x)}}$
$f(x)+f(p+q-x)=1$
The rest should be easy.


Data conversion failed while converting Time from CSV file

Question:
Problem:in mentioned Screenshoot data cannot able to convert into Time format column in SQL , this data is coming from csv file and executed by SSIS package  only these column data from csv as string cannot loaded into database table
Error1:[OLE DB Destination [76]] Error: There was an error with OLE DB Destination.Inputs[OLE DB Destination Input].Columns[Recurring Start Time] on OLE DB Destination.Inputs[OLE DB Destination Input]. The column status returned was: "The value could not be converted because of a potential loss of data.".
Comment: did you try CONVERT( TIME, '10:00:22' ) in your query? write your insert query in question if you can.
Comment: @hosseinzakizadeh not explicitly , but in connection manager on advanced tab changed this column type as 'DT_DBTIME' but still no use, NULL are coming in place of dates in DB  when i ignore failure
Comment: Try replacing your SQL Server destination with an OLE DB destination.  SQL Server destinations are type-specific and do not allow casts, implicit or not (as far as I know).
Comment: I wrote my comment as the answer. please check my answer to be clear what is the solution.
Comment: Load the data into a staging table where all he values are strings.  Then do the conversion in the database.
Answer: Try replacing your SQL Server destination with an OLE DB destination. SQL Server destinations are type-specific and do not allow casts, implicit or not (as far as I know).


A question about cache of filesystem

Question: When I read a large file in the file system, can the cache improve the speed of
the operation?
I think there are two different answers:
1.Yes. Because cache can prefetch thus the performance gets improved.
2.No. Because the speed to read from cache is more faster than the speed to read from
  disk, at the end we can find that the cache doesn't help,so the reading speed is also
 the speed to read from disk.
Which one is correct? How can I testify the answer?
[edit]
And another questions is :
What I am not sure is
that, when you turn on the cache the bandwidth is used to
1.prefetch
2.prefetch and read
which one is correct?
While if you
turn off the cache ,the bandwith of disk is just used to read.
If I turn off the cache and randomly access the disk, is the time needed comparable with the time when read sequentially with the cache turned on?
Answer: 1 is definitely correct. The operating system can fetch from the disk to the cache while your code is processing the data it's already received. Yes, the disk may well still be the bottleneck - but you won't have read, process, read, process, read, process, but read+process, read+process, read+process. For example, suppose we have processing which takes half the time of reading. Representing time going down the page, we might have this sort of activity without prefetching:
Read
Read
Process
Read
Read
Process
Read
Read
Process

Whereas with prefetching, this is optimised to:
Read
Read
Read     Process
Read
Read     Process
Read
         Process

Basically the total time will be "time to read whole file + time to process last piece of data" instead of "time to read whole file + time to process whole file".
Testing it is tricky - you'll need to have an operating system where you can tweak or turn off the cache. Another alternative is to change how you're opening the file - for instance, in .NET if you open the file with FileOptions.SequentialScan the cache is more likely to do the right thing. Try with and without that option.
This has spoken mostly about prefetching - general caching (keeping the data even after it's been delivered to the application) is a different matter, and obviously acts as a big win if you want to use the same data more than once. There's also "something in between" where the application has only requested a small amount of data, but the disk has read a whole block - the OS isn't actively prefetching blocks which haven't been requested, but can cache the whole block so that if the app then requests more data from the same block it can return that data from the cache.
Comment: Yes, but now I have to get my eldest son changed after swimming. Back later :)
Comment: @Jon Skeet..:But is the time to process the last piece of data comparable with the time to prefetch the next piece of data? And another questions is :If I turn off the cache and randomly access the disk, is the time needed comparable with the time when sequentially read with the cache turned on?
Comment: @Jinx: That entirely depends what you're doing with the data! If you're doing some complex encryption, you may end up spending more time processing than reading. If you're just counting lines of text, the processing time will be smaller than the IO.
Comment: If you need to access the disk *randomly* (i.e. not reading a file from start to finish, but jumping around) then the cache is less likely to be able to help.
Answer: First answer is correct.
The disk has a fixed underlying performance - but that fixed underlying performance differs in different circumstances.  You obtain better real performance from a drive when you read long sections of data - e.g. when you cache ahead.  So caching permits the drive to achieve genuine improvement its real performance.
Answer: In the general case, it will be faster with the cache. Some points to consider:

The data on the disk is organized in surfaces (aka heads), tracks and blocks. It takes the disk some time to position the reading heads so that you can start reading a track. Now you need five blocks from that track. Unfortunately, you ask for then in a different order than they appear on the physical media. The cache will help greatly by reading the whole track into memory (lots more blocks than you need), then reindex them (when the head starts to read, it probably will be anywhere on the track, not on the start of the first block). Without this, you'd have to wait until the first block of the track rotates under the head and start reading -> the time to read a track would be effectively doubled. So with a cache, you can read the blocks of a track in any order and you start reading as soon as the head arrives over the track.
If the file system is pretty full, the OS will start to squeeze your data into various empty spaces. Imagine block 1 is on track 5, block 2 is on track 7, block 3 is again on track 5. Without a cache, you'd loose a lot of time for positioning the head. With a cache, track 5 is read, kept in RAM as the head goes to track 7 and when you ask for block 3, you get it immediately.
Large files need a lot of meta-data, namely where the data blocks for the file are. In this case, the cache will keep this data live as you read the file, saving you from a lot more head trashing.
The cache will allow other programs to access their data in an efficient way as you hog the disk. So overall performance will be better. This is very important when a second program starts to write as you read. In this case, the cache will collect some writes before it interrupts your reads. Also, most programs read data, process it and then write it back. Without the cache, a program would either get into its own way or it would have to implement its own caching scheme to avoid head trashing.
A cache allows the OS to reorder the disk I/O. Say you have blocks on track 5, 7 and 13 but file order asks for track 5, 13 and then 7. Obviously, it's more efficient to read track 7 on the way to 13 rather than going all the way to 13 and then come back to 7.

So while theoretically, reading lots of data would be faster without a cache, this is only true if your file is the only one on the disk and all meta-data is ordered perfectly, the physical layout of the data is in such a way that the reading heads always start reading a track at the start of the first block, etc.
Answer: Jon Skeet has a very interesting benchmark with .NET about this topic. The basic result was that prefetching helps, the more processing per unit read you have to do.
Answer: If the files are larger than your memory, then it definitely has no way of helping.
Answer: Another point: Chances are, frequently used files will be in the cache before one is even starting to read one of them.

A question about cache of filesystem


É possível saber qual o Container JEE está sendo usado durante a inicialização do Dispatcher Servlet?

Question: Depois de algumas dificuldades com o uso do Spring Security e JBoss usando anotações com base na API Servlet 3 (Servlet 3.1 mais especificamente), descobri que ao usar o Spring com JBoss, em especial o JBoss EAP 6.1+, não se pode fazer o mapeamento do Dispacher Servlet para a URL "/" simplesmente, é preciso que tal mapeamento seja feito para a URL "/*".
Para resolver tal problema cheguei a criar um fork para estudar tal problema, e este se encontra no link: https://github.com/carlosdelfino/spring-security/tree/rb4.0.1.RELEASE-JBoss-patch-1, onde já testei o exemplo "Insecure MVC" e tal correção resolve o problema.
Porém estou desenvolvendo uma aplicação que será executada em diversos containers, e receio que possa a vir ter problemas com outros recursos, e assim eu precise parametrizar a inicialização do servlet.
Neste caso o Spring me oferece algum recurso para identificar em qual container está sendo executado?
Como fazer isso?
Comment: Este problema está relacionado com: http://pt.stackoverflow.com/questions/68110/qual-a-programa%C3%A7%C3%A3o-deve-ser-feita-para-usar-o-spring-security-com-jboss-com-anot
Answer: Você pode criar uma classe para inicializar o WebApp que implemente a interface WebApplicationInitializer.
Essa interface tem um único método que recebe um ServletContext e pelo ServletContext você pode pegar o nome do servidor.
public class WebAppInitializer implements WebApplicationInitializer {

    @Override
    public void onStartup(ServletContext context) throws ServletException {
        System.out.println(context.getServerInfo());
    }

}

Saída no console:
Comment: Adelmo, perfeito funcionou, como estou usando a classe `AbstractAnnotationConfigDispatcherServletInitializer`, sobrepus o método e chamei o super, funcionou perfeitamente. https://www.dropbox.com/s/g8v49v91olo3of5/Captura%20de%20tela%202015-07-10%2015.04.58.png?dl=0


Change point detection

Question: I have a specific question about the formulation of offline multiple change point detection given in Burg and Williams.

Where the change points are denoted $\{\tau_i\}$, and the slice of a time series from $a$ to $b$ is $\textbf{y}_{a:b}$. $\ell$ is a loss function, and $P$ is a penalty on the number of change points.
My question is: why is it written as $\ell(\textbf{y}_{\tau_{i-1}:\tau_{i}-1})$ and not $\ell(\textbf{y}_{\tau_{i-1}:\tau_{i}})$? In other words, why is there a second $-1$ term in the loss function?
Comment: This notation is fully explained two paragraphs before the appearance of the formula.  Plugging in some numbers for the change points (take $n=2$) will make it obvious.
Answer: So $\textbf{y}_{\tau_{i-1}:\tau_{i}}$ are the elements of the time series going from changepoint number $i-1$ going up to changepoint number $i$.
Then $\textbf{y}_{\tau_{i-1}:\tau_{i}-1}$ (notice the additional -1 is not a subscript of the $\tau$ but rather a subscript of the $\textbf{y}$). So this time series starts at changepoint number $i-1$ and goes up to one element before changepoint number $i$ ($\tau_i-1$).
This is because you want to calculate the loss function from one changepoint to the next with no overlap.
Comment: Of course, it's because you don't want to include the first element of the next partition. Thanks!

Change point detection


How to show '\n' and '\t' characters in VS Code?

Question: I'm debugging a laravel app and I need to know if any of these characters ( \n \t ) are present.
Previously VS Code did show them, now it does not:

Is there any way to enable this again?

I'm using Xdebug + VS Code.
Comment: There is a setting to `Render Whitespace` - it doesn't seem to show anything for newlines though.
Comment: Related: [Show whitespace characters in Visual Studio Code](https://stackoverflow.com/questions/30140595/show-whitespace-characters-in-visual-studio-code).
Answer: It seems Visual studio code "delegated" this escape to the debugger, I have spoken to the developer of xDebug and they told me they aren't escaping anything on their side, it seems a problem of vs code and this should be an option for the end-user not the debugger.
there's no current way of doing this atm on this date
Answer: This might help?
  // Controls how the editor should render whitespace characters.
  //  - none
  //  - boundary: Render whitespace characters except for single spaces between words.
  //  - selection: Render whitespace characters only on selected text.
  //  - all
  "editor.renderWhitespace": "selection",

https://code.visualstudio.com/docs/getstarted/settings
Comment: Just tried this a while ago but it doesnt work, \n characters are still invisible
Answer: Gremlin Tracker plugin for VS Code is great for finding invisible/weird characters in combination with the "Render Line Endings" plugin.
Gremlin Tracker: https://marketplace.visualstudio.com/items?itemName=nhoizey.gremlins
Render Line Endings:
https://marketplace.visualstudio.com/items?itemName=medo64.render-crlf


How could a non-striker on 99* win the game and reach 100, if 1 ball remains?

Question: A batsman who is at 99 not out and at non-striker needs 1 run to win off the last ball. How is it possible that they win the game and also that the batsman reaches his century?
Answer: I think the only way to do it is by way of penalty runs.  Most straightforwardly, the batsmen could try to steal a run during the bowler's run up.  As soon as they cross, the umpires would call dead ball and award 5 penalty runs to the fielding side.  The batsmen return to their original ends.  Now the batting side needs 6 to win off one ball.  The bowler could then bowl a no ball or a wide allowing the batsmen to cross, which would score 2 runs.  Now the guy on 99 is on strike with one legal ball to go and needing 4 to win.
Comment: may be this thing could be possible , but are you sure the penalty is awarded if the batsman tries to run during bowler's run up.Has there been such a case before.Or is that a written rule.
Comment: Yes, it's in the laws.  It never happens in practice because it's not allowed.  There are also other ways the fielding side could get 5 penalty runs, but they are more difficult to provoke on the spot (e.g., repeatedly damaging the pitch).  Also, intentionally provoking a penalty would likely cause the players to be subject to sanctions by the league or other governing body, so don't try this at home. ;-)
Answer: The batsman attempt another run before the final ball is bowled (once bowler has commenced his run up), cross, and the other batsman gets run out.
The batsman who is on 99 is now on strike and hits a single off the one remaining delivery.
Comment: I think the question assumes that the ball is dead, in which case they cannot attempt a run.
Comment: Actually you might be right and the answer may be a combination of our answers. If a run out, from a dead ball can result in the batsmen changing ends, it could be set up for the guy on 99 to score a run from the last delivery, without penalties... ???
Comment: When the ball is dead, they cannot run or attempt to run or be run out.
Comment: You can't be run out once the ball is dead, but you could be run out in the act of the ball becoming dead. What I should've said was, "if a run out occurs and the result is a dead ball".
Comment: So it's back to my original answer. Batsman attempt to run as the bowler runs in (ball is live once he commences his run up). Batsman would need to cross, and bowler would affect the run out at the bowlers end. The guy on 99 is then on strike and away he goes... ;-)
Comment: Stealing a run during the bowler's run up is not allowed.  See law 42.16 and my answer.
Comment: Yes, I'm clearly barking up the wrong tree.
Answer: When the batsman going for a run , one of the either is obstructed by a fielder in which a run is not completed...in this case the umpire will call dead ball and the non striker who is on 99 will take strike...( as the batsman has crossed each other)
Comment: Which part of the [Laws of Cricket](https://www.lords.org/mcc/laws-of-cricket/laws/) would allow an umpire to make this call? I see nothing in [Law 23](https://www.lords.org/mcc/laws-of-cricket/laws/law-23-dead-ball/) which supports this answer.
Answer: There are two other cases that haven't been raised here - either a no ball or wide being bowled.  In both cases the ball is rebowled and the batsman are able to run a single (or three for that matter).
Comment: This is incorrect; the game is over as soon as the no ball or wide is bowled, so there would be no opportunity for the non striker to score a run.


What constitutes a "game-winning drive" statistically?

Question: After the Buccaneers @ Patriots game - in which the Buccaneers won - it was revealed by statisticians that it was Tom Brady's 50th career "game-winning drive".
What constitutes a "game-winning drive" statistically?
Coming from the Bucs-Pats game, it cannot be taking the lead on the final drive of the game, as the final drive of the game contained three successive QB kneels by the Bucs, after the missed FG attempt by the Pats. It also cannot be taking the lead on the last drive that team makes in the game, for the same reason. Unless of course, a drive of QB Kneels does not count.
It could be defined as a drive that takes the lead from the opponent, to which the opponent does not recover the lead, but then surely every win contains a game-winning drive, and Brady would have significantly more than 50.
I guess it could also be that it's the last scoring drive of the game, that also takes the lead - but let's say that drive happens in the first quarter, and the rest of the game is a shutout - would that still be considered a game-winning drive? I would imagine in most people's eyes a game-winning drive would be in the latter stages of the 4th quarter - in typical Brady-esque fashion.
Answer: So - one bit you're missing, is that it was Tom Brady's 50th career game-winning drive in the fourth quarter or overtime.  That's frequently, but not always, said with that sentence, but it's always a component even when unsaid.
Specifically, a game winning drive is a score in the fourth quarter or overtime by the offense with the team tied or down by less than a score, which leads to the team to win the game without the lead changing after that.
You can see this post on PFR for more details on how they calculate it - it's not an "official" stat exactly so it's up to the individual site to calculate it, but it seems pretty standardized for the most part. You can also see their list of QBs ranked by game-winning drives which includes, of course, Tom Brady near the top.


Pandas - How can I change the values of a column based on a given non-python dictionary?

Question: I have a dataframe that looks like this (i've filtered it down to the important parts):
              INC_KEY               PTOCCUPATIONALINDUSTRY
96       170000016620                       Other Services
127      170000016651                       Other Services
170      170000016694                        Manufacturing
181      170000016706                         Construction
268      170000016793                       Other Services

I was also given a 'dictionary' (not a python dictionary), that looks like this:

My task is to convert the values in PTOCCUPATIONALINDUSTRY to the numbers that you see in the dictionary. Is there an easy way to do this? Or do I need to manually do if statements for each value?
The reason I'm looking for a shortcut is because I need to do this exact task over ~30 different columns, so it would take forever to do it manually.
Comment: Im still not clear what is the expected output
Comment: in what form is the dictionary presented? Is this a txt file?
Comment: Presumably you'd need to turn that text file/document/structure into a python dictionary. Then the standard pandas `map` replace` operations would work as normal.
Answer: Create your dict :
map_dict = {"Finance, Insurance, and Real Estate": 1, "etc...}

Then, simply do :
df["PTOCCUPATIONALINDUSTRY"] = df["PTOCCUPATIONALINDUSTRY"].map(map_dict)


How add custom node in a View (Nested View)?

Question: Currently I have simple view that is used on the main page. In includes nodes of custom type. That's great. But now I need to do something different.
Consider I have following structure:
-----View-----
-------Node 1--------
-------Node 2--------
-------Node 3--------
-------Node 4--------
-----End View-----
But I want to have the structure like that:
-----View-----
-------Node 1--------
-------Node 2--------
-------Node 3--------
---------View Nested in the node ---------
------------Nested Node 1-------------
------------Nested Node 2-------------
---------End Nested View ---------
-------Node 4--------
-----End View-----
So I need nest another view in a top level node.
How can I implement this?
Is there any way to do this using core functionality?
Otherwise I need to change the content of a node before data is passed to a template, but I don't which hook to use?
Please suggest how to deal with this task.
Comment: The structure is possible in core with a reference field for the nested nodes. If you want a view in a node to have more control on how the nested nodes should be displayed you can use https://www.drupal.org/project/eva.
Comment: I only suggested EVA, because this would result exactly in the structure you want, including the second View for the nested nodes. As said, you don't need this or a hook, because displaying nested nodes is core functionality.
Comment: @4k4 thank you so much could you please suggest what hooks can be used to modify node content, the plugin you've suggested it great, but I don't need to include it only for this task, I think it would be better for performacnce just to use a hook.
Comment: @4k4 Thank you so much could you post your comment as an answer
Answer: You can use the Views Field View to show the Nested Nodes (that should be a kind of Node Reference)

There are a lot of cases in views where you want to embed a list
  inside each row. One example could be, you have a list of groups, and
  for each groups you want to list the first and last name of all of
  their members.

For modify the content of a node you can use the template_preprocess_node() maybe you will need to Check content type of node in preprocess hook.
Comment: Thanks for the answer, but this module has not been update for a long time, so I think it is really better to use EVA
Comment: @ketazafor You can use EVA too, or a better way is implement your own pseudo-field and create a custom view_mode. https://www.webomelette.com/creating-pseudo-fields-drupal-8 to show the pseudo-field
Comment: Yeah, it was my initial idea to create field for example and type there my view name, and than handle it in the code, is it possible to do like that ?
Comment: @ketazafor, you can create a new view mode for your content type and a new pseudo-field with the list of nodes and show only this pseudo-field en the view mode, and in the view you can show the view_mode, or you can create a new custom field for the view https://www.webomelette.com/creating-custom-views-field-drupal-8


Negative t-value

Question: I have run a test on a treatment where the mean (of particular performance) after treatment is greater than that before and standard deviation has decreased. My t value is in the -22 area and I have not found a negative t value before. Should I just use the absolute value and check this agains the t-table, as in compare 22 against the critical value according to my DF and confidence level?
Does a negative t reflect a positive or negative effect?
Comment: The $t$ statistic will have precisely the same sign as the difference between the means. If the first mean is 42 and the second is 666 the difference is negative; if you flip them round it's positive. There is no meaning to the order in which you present the means. Doesn't your software produce a P-value automatically?
Answer: The t distribution is two-sided and centered around 0, so it is possible to have a negative t statistics. What was the null hypothesis?

Negative t-value


Positioning CSS errors

Question: Here is the CSS code for a webpage I am developing using Django.
The problem I am currently running into is that the width, left and margin-left within inputlabel and input area do not have an effect on the position.
inputlabel fits the content within it, as does input area.
Neither of the two seem to change no matter the values I change.
However, #id_description works as expected.
I'm new to CSS and am not 100% sure what it is I'm missing.
container {
    position: relative;
}
inputlabel {
    width: 200px;
    position: absolute;
    left: 1000px;
}
inputarea {
    margin-left: 200px;
}

#id_description {
    resize: none;
    width: 500px;

}
body {

}

Comment: `inputlabel` and `inputarea` (and `container`) (TY) should either be classes (start with a dot) or IDs (start with a hash #). It will depend on your HTML which you have not provided.
Comment: ^ and `container`, I would suggest reading up on [selectors](http://www.w3.org/TR/CSS2/selector.html)
Comment: http://www.codecademy.com/courses/css-coding-with-style/0/1
Answer: you're missing '.' and '#'
if container is a class,then you need to use .container
if container is a id,then you need to use #container
Same applied for all.
Answer: To define css for a class, use .container{}, .inputlabel{}, etc. You are missing the .'s
Answer: You should use next:
.container {
    position: relative;
}
.inputlabel {
    width: 200px;
    position: absolute;
    left: 1000px;
}
.inputarea {
    margin-left: 200px;
}

Html code would be so:

or
<input class='inputlabel' ...>

Also you can use next:
#container {
    position: relative;
}
#inputlabel {
    width: 200px;
    position: absolute;
    left: 1000px;
}
#inputarea {
    margin-left: 200px;
}

Code would be so:

or
<input id='inputlabel' ...>

So # before style name means you should link this style with tag via id. . symbol means you have to link this style with tag via class.
But: body part in your css code might be ok. You can use body or input names, so it will affect on all tags with such names.
There is an article where you can read about the difference between id and class styles: click me

Positioning CSS errors


jquery slidetoggle hide previous element

Question: I'm working on a slideToggle function on click, I want to hide the previous element that was clicked when a new one is clicked, I've used this same set of code previously and its worked however it's no longer working now and I'm stumped at why its no longer works:
My code is as below or view a jsfiddle
js/js.js
$('.togglesources').hide();
$('.captionsource').on('click', function () {
            $(this).next('.togglesources').slideToggle();
            $(this).parent().siblings().children().next().slideUp();
                      return false;
        });

index.html

Nokia:

3100

Most popular mobile phone of the year

Click to toggle source

Wikipedia (Data Source)

GSM Solution (Image Source)

Lord of the Rings:

Return of the King

Highest Grossing Film

Click to toggle source

Wikipedia

IMDB (Image Source)

would appreciate some help in regards to this.
Why aren't the other '.togglesources' divs closing when I open a new one?
Comment: Fiddle is Working fine
Comment: @ling.s its not hiding the previous source
Comment: Previous source means what?
Comment: @ling.s He wants to close all the other .togglesources divs that he's opened. In other words if you click on the first "Click to toggle source" and then the second "Click to toggle source" the first sources div should close while the second one opens.
Answer: Try this:
    $('.togglesources').hide();
    $('.captionsource').on('click', function () {
    $(this).closest('.mostpopular').siblings().find('.togglesources').slideUp();
            $(this).next('.togglesources').slideToggle();
            $(this).parent().siblings().children().next().slideUp();
    return false;
        });

Working Fiddle
Comment: thanks that worked exactly how I wanted it to. Is there a particular reason as to why my example wasn't working as expected?
Comment: There was no code for hiding other toggling elements if they are already shown. in above code we ensure that all other toggling elemets gets slideup except the one on which you have clicked,


Comparison between Cox regression and parametric regression using AIC

Question: In a survival analysis, is it possible to use a likelihood based measure like AIC to compare the fit obtained from a Cox model and a parametric regression model? I have seen different views, from (1) no you can't to (2) finding a variety of published papers that do so!
While the more I think about it the more I wonder what is the AIC in the Cox model (given that you do not assume a model for the distribution, how would you calculate a likelihood to begin with?), I was hoping that someone could provide a definitive answer to whether this AIC comparison is possible or not and, if not, what are the suggested ways to choose between the two approaches.
Many thanks in advance for any feedback sent this way.
Comment: A specific quote from that site that is directly on topic: "AIC cannot be used to compare parametric and semi-parametric models, however, since parametric models are based on observed event times and semi-parametric models are based on the order of event times."
Comment: This might help - https://www.mailman.columbia.edu/research/population-health-methods/time-event-data-analysis. Good site for investigation of your question
Comment: @DavidHall Many thanks - very useful resource
Answer: The AIC can only be used to compare results derived from the same sample for the same outcome estimated from the same procedure with differing independent variables. The AIC is based on maximum likelihood estimators intrinsic to a given model. One cannot compare between models as they do not share MLEs. For example, one can generate AIC for linear regression and Poisson regression models, which are not comparable. That Cox regression and parametric regression measure time-to-event make them no more comparable.
Comment: I can not locate a citation which firmly suggests that AIC can be compared for two distinct procedures. The following CV post suggests comparison is not feasible in this case. https://stats.stackexchange.com/questions/139201/model-selection-can-i-compare-the-aic-from-models-of-count-data-between-linear
Comment: This answer is incorrect. One can compare AIC across models with different responses (e.g. Poisson vs Gaussian), with our without the same covariates. What you can't do is compare models which have used different data.
Comment: @Tiago Marques: I am with Todd here. For comparability, the likelihoods must be defined with respect to the same dominating measure, so discrete (Poisson) and continuous cannot be compared


Arc length of epicycloid when small radius goes to 0

Question: So I've been working out on epicycloids and I got the arc length formula, let's say for a lap around the big circumference. Big radius is $R$ and small radius going around is $r$. Then that formula is $8(R+r)$. Now, intuitively, one can argue that this length should be the length of the big circumference when the radius $r$ goes to 0, because the curve is approaching that. However, it is clear that $\lim_{r\to 0} 8(R+r) = 8R$, which is clearly not equal to $2\pi R$, the length of the circumference. Hence, intuition failed to work. I would appreciate if anyone could give me an explanation for why they are different. My guess is that every little arc is becoming steeper when approaching radius 0, not smoother, so it won't approach $2\pi R$, but that's too vague. Thanks.
Answer: I think I understand your dilemma now. Say that $m=a+b$ is fixed and we let $b=1/n$, where $n$ is an integer, so that we satisfy the condition that $m/b$ is indeed an integer. Now, what happens when we let $n\to \infty$? We can never actually achieve $b=0$. It's sort of like Zeno's paradox; we just can't get there. I demonstrated this numerically, i.e., for any value of $n$ the arc length is $L=8m$. If you plot this for large $n$ it looks like a circle, but it isn't. There are a large number of cusps and adding up the total length always gives the same result. If you try to force $b=0$ then you are confronted with the problem that $m/b$ is not really defined. At the same time, we can look at the area enclosed, which I believe to be $A=\pi m(m-b)$, we see that area of the epicycloid approaches that of the circle of larger radius for large $n$.
Comment: I guess it makes sense. Infinite stuff gets messy. The idea that $m/b$ won't be an integer so we can't make $b=0$ also makes sense. I think that was what you were trying to tell me before  but I didn't understand it then. It's also quite interesting that the area does approach that of the circumference. Thanks for your insights.
Answer: According to A Catalog of Special Plane Curves, J. Dennis Lawrence, Dover, 1972, the equations for the epicycloid are
$$x=m\cos t-b\cos mt/b\\
y=m\sin t-b\sin mt/b
$$
where $t\in[-\pi,\pi]$ and $m=a+b$ (i.e., the radii).
However, it states that $L=8m$ (if $m/b$ is an integer). Therein lies your problem.
Comment: Excuse me, I don't see how that helps my problem. Let's say a=3, b=1. Then I know the arc length for the epicycloid is $8*(3+1)=32$, but I was trying to figure out why wouldn't this arc length approach the arc length of the 3 radius circumference when the other one approaches radio 0. In other words, I would like to know why doesn't it equal $2πa$ in the limit as b goes to 0, as one would think intuitively, the epicycloid would fit the shape of the big circumference.


find all nested array in single array in python WITHOUT FLATTEN or any inbuilt function

Question: Hi I need a single list of nested lists in python without any inbuilt function. I did a lot of research but not able to find without flatten or ravel function().
for ex:
input = [1,2,[2,3,[4,5]],[6,7],[8,[9]],10]
output = [1,2,2,3,4,5,6,7,8,9,10]

Comment: [The first solution here](https://stackoverflow.com/questions/952914/how-to-make-a-flat-list-out-of-list-of-lists) doesn't use any built-in functions.  Is this acceptable?
Comment: You can use recursion to implement your own `flatten` function.
Comment: @DarrylG This was not working
Comment: duplicate of https://stackoverflow.com/questions/2158395/flatten-an-irregular-list-of-lists
Comment: You're right, that only works when all elements are lists.
Answer: You can use recursion to implement your own flatten function. Here's a nice tutorial explaining the concepts of recursion.
The idea here is that you loop through each element in your list and if you encounter a sublist you recursively call the same method with the current list. If it's a valid element you add it to to your output list.
Example:

def flatten(input_list, output_list=[]):

  for i in input_list:
    if isinstance(i, list):
      flatten(i, output_list)
    else:
      output_list.append(i)

  return output_list

input_list = [1,2,[2,3,[4,5]],[6,7],[8,[9]],10]

print(flatten(input_list))

Outputs:
[1, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Comment: Would suggest changing function signature to: def flatten(input_list, output_list = []).  That way the user does not have to create or pass output on call.
Comment: @DarryIG, good spot. Updated the answer.
Answer: I found a solution on my own. Below is the solution:
c = []
def test(a):
    for val in a:
        if type(val) == list:
            test(val)
        else:
            c.append(val)
return c

a=[1,2,[2,3,[4,5]],[6,7],[8,[9]],10]
   print test(a)


hyperbolic geometry and affine transformations

Question: A very famous geometer recently commented to me that "hyperbolic geometries are the only geometries invariant under affine transformations". It is unclear to me what this comment even means. Can someone clarify this for me. Some type of reference would also be nice.
Comment: This statement does make much sense, I think you should ask the "very famous geometer" for a clarification.
Comment: OK, that makes sense.
Comment: @DietrichBurde:  meant "does not make much sense".
Answer: Here is a guess:

First, we need to define "geometry": I will assume that it means a complete simply-connected Riemannian manifold of constant curvature $0$ or $\pm 1$. (This, of course, excludes many other geometries.)
Next, given a Riemannian manifold $(M,g)$ with the Levi-Civita connection $\nabla$, a diffeomorphism $f: M\to M$ is called affine if it preserves $\nabla$. Similarly, $f$ is projective if it sends unparameterized  geodesics to unparameterized geodesics. In particular, each affine transformation is projective.
Now, among all n-dimensional  geometries, both the Euclidean space and the unit sphere admit projective transformations  which are not isometric, but the hyperbolic n-space, does not admit such. On the other hand, each affine transformation of $S^n$ is an isometry. Thus, my conjecture is that "famous geometer" meant projective instead of affine and what he had in mind is:

If $(M,g)$ is a complete simply-connected Riemannian manifold of constant curvature, for which the group of projective transformations equals the group of isometric transformations, then $(M,g)$ has negative curvature; in other words, it is a hyperbolic space.
Edit. As requested, here is a sketch of a proof that every affine transformation of the round sphere $S^n$ is an isometry ($n\ge 2$). First of all, let $P(S^n)$ denote the group of projective transformations of $S^n$, i.e. maps sending (unparameterized) geodesics to (unparameterized) geodesics. Let $PGL(n+1,R)$ denote the projectivization of the general linear group $GL(n+1,R)$; this group is acting on the projective space $R P^{n}$ by projective transformations (where the Riemannian manifold $R P^{n}$ is the quotient of $S^n$  by the antipodal involution). According to the Fundamental Theorem of Projective Geometry (FTPG),   $PGL(n+1,R)$ is exactly the group of projective transformations of Riemannian manifold $R P^{n}$. Each element of $PGL(n+1,R)$ lifts to a self-diffeomorphism of $S^n$; I will use the notation $G$ to denote the group consisting of such lifts: Algebraically speaking it is a central extension of $PGL(n+1,R)$:
$$
1\to {\mathbb Z}_2\to G\to PGL(n+1,R)\to 1.
$$
With a bit of work, using FTPG, one proves that $G$ is exactly the group of projective transformations of $S^n$. Thus, the problem reduces to proving that the only affine transformations of $RP^n$ are the isometries of $RP^n$, i.e. the subgroup $PO(n+1)<PGL(n+1,R)$. We will use the SVD, the singular value decomposition: Every element of   $PGL(n+1,R)$ can be written as a product $kak'$, with $k, k'\in PO(n+1)$ and $a$ the projectivization of a diagonal matrix with positive diagonal entries. Thus, the problem reduces to proving that if such $a$ is an affine transformation of $RP^n$ then $a=id$. Every such $a$ preserves some projective lines in $RP^n$, projectivizations of coordinate 2-planes in $R^{n+1}$. If $a\ne id$ then it acts by a non-identity map on one of these projective lines $L$, fixing precisely two points $p, q$ on $L$. But, regarding $L$ as a parameterized geodesic, it is clear that $a: L\to L$ cannot send such a parameterized geodesic to a parameterized geodesic (the speed of $a(L)$ at $p$ will be different from the speed of $a(L)$ at $q$).
Comment: Do you have a simple reason or a reference for the stated claim that only affine transformations of the sphere are isometries?
Comment: The geometer might have as well been referring to projective transformations in the same loose way Dietrich seemed to be referring  to the projective transformations between points in the disk boundary (the real projective line) in Klein-Beltrami projective model of hyperbolic space as the invariance of the 2-dimensional disk to affine transformations of the disk to itself. So depending on the dimensionality one wishes to concentrate on the projective transformations can be called affine transformations in n+1.
Comment: @Walterscott: You as well could be right, but in my answer I was trying to explain where did "the only geometries" come from in the statement by a "famous geometer". I am well aware of the fact that the Lorentzian model provides a way to regard hyperbolic isometries as linear transformations, but this does not explain "the only" statement. Isometries of both euclidean and spherical geometries are clearly affine, thus, thus "afineness" in this sense cannot be a distinguishing feature of the hyperbolic geometry.
Comment: And your explanation is correct. I just suggested a way in wich the claim in the question could actually be referring(misleadingly in its use of the term "affine") to projective transformations as it is the only way the "famous geometer" could be right. The fact is that projective transformations are almost always explained in the context of projective spaces associated to a vector space. Even hyperbolic geometry doesn't scape this and it is generally represented in scale invariant  models like the Klein or the hyperboloid models that are ultimately based on  linearity rather than intrinsic.
Comment: @ViditNanda Not simple, but it is an application of the Fundamental Theorem of Projective Geometry.
Comment: @ViditNanda: See the edit.
Comment: @moishekohan this is great, thank you!
Answer: One can define hyperbolic geometry as the study of points and chords of a disk $D$ invariant under affine transformations of $D$ to itself; see Klein's disk model of non-Euclidean geometry. However, there are also others models of hyperbolic space, e.g., the Poincare model.
The remark could be related to Klein's disk model of hyperbolic geometry, but I am not sure.
The "very famous geometer" might not agree.
One would need a bit more context, I think. A good reference here are excellent articles and lecture notes of W. Goldman on projective, affine and hyperbolic geometry on manifolds.
(link:http://www.math.umd.edu/~wmg/ and more in particular  http://www.math.umd.edu/~wmg/pgom.pdf )


Como preencher uma coluna de um DF Pandas usando, como comparação, uma coluna específica entre este e outro DF?

Question: Tenho um dataframe Pandas, no qual preciso adicionar uma nova coluna chamada codprojeto. Para tal, criei esta nova coluna e inseri zeros, para que a mesma tenha o tipo int64, da seguinte maneira:
df['codprojeto'] = 0

Esta nova coluna deve ser alimentada com o codprojeto presente em outro dataframe. Esta coluna é do tipo int64, motivo pelo qual fiz o passo anterior.
Como comparação, estou usando a coluna CNPJ, com valores em ambos DFs. Caso seja igual, deve preencher em df['codprojeto'] o valor de df2['codprojeto'].
Tentativa 1:
for i in range(len(df['CNPJ'])):
    for j in range(len(df2['CNPJ'])):
        if df2.loc[j, 'CNPJ'] == df.loc[i, 'CNPJ']:
            df.loc[i, 'codprojeto'] ==  df2.loc[j, 'codprojeto']

Retorna:
Erro: KeyError: 4

Tentativa 2:
for index, row in df.iterrows():
for index2, row2 in df2.iterrows():
    if str(row['CNPJ']) == row2['CNPJ']:
        df.loc[index,'codprojeto'] = df2.loc[index2,'codprojeto']

Retorna: execução eterna; processamento pendente.
Conjunto de dados (amostra):
DF:
CNPJ,DATA,codprojeto
00000000000123,2020-12-02 00:00:00 UTC,0
99900000000123,2020-12-02 00:00:00 UTC,0
00000000000123,2020-12-02 00:00:00 UTC,0
00000000000123,2020-12-02 00:00:00 UTC,0
00000000000145,2020-12-02 00:00:00 UTC,0
00000000000123,2020-12-02 00:00:00 UTC,0
00000000000167,2020-12-02 00:00:00 UTC,0
00000000000167,2020-12-02 00:00:00 UTC,0
00000000000167,2020-12-02 00:00:00 UTC,0
00000000000167,2020-12-02 00:00:00 UTC,0
00000000000101,2020-12-02 00:00:00 UTC,0
00000000000122,2020-12-02 00:00:00 UTC,0
00000000000144,2020-12-02 00:00:00 UTC,0
00000000000123,2020-12-02 00:00:00 UTC,0
00000000000155,2020-12-02 00:00:00 UTC,0
00000000000155,2020-12-02 00:00:00 UTC,0
00000000000155,2020-12-02 00:00:00 UTC,0
00000000000166,2020-12-02 00:00:00 UTC,0
99900000000123,2020-12-02 00:00:00 UTC,0
99900000000123,2020-12-02 00:00:00 UTC,0

DF2:
"codcliente";"nome";"CNPJ";"codprojeto"
1;"CLIENTE 1";"00000000000123";1234
2;"CLIENTE 1";"00000000000145";5678
3;"CLIENTE 1";"00000000000167";9012
4;"CLIENTE 1";"00000000000189";3456
5;"CLIENTE 1";"00000000000101";7890
6;"CLIENTE 1";"00000000000122";11
7;"CLIENTE 1";"00000000000133";22
8;"CLIENTE 1";"00000000000144";33
9;"CLIENTE 9";"00000000000155";44
10;"CLIENTE 10";"00000000000166";55

O DF e DF2 originais possuem, respectivamente, 635939 e 1054 linhas.
Para os casos que não houver CNPJ compatível, deixar como 0 o 'codprojeto' mesmo.
Como posso corrigir isso?
Comment: Marlos, boa tarde! Consegue disponibilizar o conjunto de dados? Abraço!
Comment: Sem os dados é difícil responder, mas te adianto que 1) Essa solução é extremamente lenta, use `np.where` em vez de nested loops O(n^2); 2) Na última linha o correto é `=` e não `==`; 3) O erro possivelmente está ocorrendo pq 4 não está no index de `df` ou `df2`
Comment: Inclui o conjunto de dados na pergunta. Sobre o '==' na última linha, já fiz a correção. Sobre o KeyError 4, é estranho, pois não estou buscando por valor 4, e sim comparando entre 2 valores apartados.
Comment: Uma dica para conseguir sempre respostas objetivas e que te atendam é disbonibilizar amostras de dados que contemplem todos os problemas que você esta passando. Com os dados que você disbonibilizou você recebeu 2 respostas diferentes que apitaram 2 erros diferentes. Existe um tópico exclusivo que mostra como criar um [exemplo minimo, completo e verificavel](https://pt.stackoverflow.com/help/minimal-reproducible-example)
Comment: Tranquilo, obrigado pela dica, Terry.
Answer: Minha sugestão é fazer um LEFT JOIN usando a função merge(). E para ela funcionar corretamente não é necessário criar a coluna codprojeto com zeros no DataFrame 'DF'
# aqui vou excluir a coluna 'codprojeto' de df,
# mas acredito que no seu projeto seja necessário
# apenas comentar a linha onde ela é criada
df = df.drop(columns= ['codprojeto'])

df = df.merge(df2[['CNPJ', 'codprojeto']], on = 'CNPJ', how = 'left').fillna(0)
#para não ter valores duplicados use
#df = df.merge(df2[['CNPJ', 'codprojeto']].drop_duplicates('CNPJ'), on = 'CNPJ', how = 'left').fillna(0)

#para converter a nova coluna para int
df['codprojeto'] = df['codprojeto'].astype('int64')

#saida:
    CNPJ    DATA                    codprojeto
0   123     2020-12-02 00:00:00 UTC 1234.0
1   999...  2020-12-02 00:00:00 UTC 0.0
2   123     2020-12-02 00:00:00 UTC 1234.0
3   123     2020-12-02 00:00:00 UTC 1234.0
4   145     2020-12-02 00:00:00 UTC 5678.0
5   123     2020-12-02 00:00:00 UTC 1234.0
6   167     2020-12-02 00:00:00 UTC 9012.0
7   167     2020-12-02 00:00:00 UTC 9012.0
8   167     2020-12-02 00:00:00 UTC 9012.0
9   167     2020-12-02 00:00:00 UTC 9012.0
10  101     2020-12-02 00:00:00 UTC 7890.0
11  122     2020-12-02 00:00:00 UTC 11.0
12  144     2020-12-02 00:00:00 UTC 33.0
13  123     2020-12-02 00:00:00 UTC 1234.0
14  155     2020-12-02 00:00:00 UTC 44.0
15  155     2020-12-02 00:00:00 UTC 44.0
16  155     2020-12-02 00:00:00 UTC 44.0
17  166     2020-12-02 00:00:00 UTC 55.0
18  999...  2020-12-02 00:00:00 UTC 0.0
19  999...  2020-12-02 00:00:00 UTC 0.0

Comment: Terry, deu certo! Era exatamente isso que eu precisava. Muitíssimo obrigado!!!
Comment: Mesma questão, Terry: coluna `codprojeto` em df fica zerada.
Comment: Acredito que o tipo das coluna CNPJ não é o mesmo em ambos os dataframes.  Tente converter essa coluna para número usando o comando : `df['CNPJ'] = df['CNPJ'].astype('int64')`. Faça o mesmo no df2 e depois faça o merge
Comment: Terry, aí acontece o erro:  ValueError: invalid literal for int() with base 10: '00.000.000'
Comment: Pelo visto em um dos seus DataFrames o campo CNPJ possui pontos, o que é um problema para converter para o tipo numérico. Pesquise ou crie uma nova pergunta no SO-PT sobre como transformar números com pontos para int64 no pandas, dando exemplos e como deve ser a saida final
Comment: Terry, deu certo! Tinham três linhas de teste com '00.000.000'. Quando as removi, rodou. Porém, armazenou como float (inclusive no seu exemplo acima). Da pra converter pra int? Além disso, foram acrescentadas mais 14.773 linhas no dataframe, enquanto o esperado é manter os 636.042 registros.
Comment: se não tiver nenhum valor nulo da pra converter para int, desta forma: `df['codprojeto'] = df['codprojeto'].astype('int64')`
Comment: Sobre aumentar o número de linhas, entra o que eu comentei na resposta do @imonferrari, sugeri usar o `drop_duplicates` junto com o `map`e você disse que não poderia usa-lo porque cada CNPJ tinha informações únicas dentro do DF2
Comment: Editei minha resposta com os comandos mostrando como não ter dados duplicates e como converter para int a coluna `codprojeto`
Comment: Não entendi como usar o drop dentro do map. Poderia me explicar, por gentileza?
Comment: Eu coloquei um exemplo na minha resposta que evita a criação de linha extras, é só dar uma olhada no [update](https://pt.stackoverflow.com/posts/484952/revisions)
Answer: Importando o pandas
import pandas as pd

Carregando os arquivos de teste
df = pd.read_csv('./df.csv')
df2 = pd.read_csv('./df2.csv', sep = ';')

df
    CNPJ        DATA              codprojeto
0   123  2020-12-02 00:00:00 UTC      0
1   123  2020-12-02 00:00:00 UTC      0
2   123  2020-12-02 00:00:00 UTC      0
3   123  2020-12-02 00:00:00 UTC      0
4   145  2020-12-02 00:00:00 UTC      0
5   123  2020-12-02 00:00:00 UTC      0
6   167  2020-12-02 00:00:00 UTC      0
7   167  2020-12-02 00:00:00 UTC      0
8   167  2020-12-02 00:00:00 UTC      0
9   167  2020-12-02 00:00:00 UTC      0
10  101  2020-12-02 00:00:00 UTC      0
11  122  2020-12-02 00:00:00 UTC      0
12  144  2020-12-02 00:00:00 UTC      0
13  123  2020-12-02 00:00:00 UTC      0
14  155  2020-12-02 00:00:00 UTC      0
15  155  2020-12-02 00:00:00 UTC      0
16  155  2020-12-02 00:00:00 UTC      0
17  166  2020-12-02 00:00:00 UTC      0
18  123  2020-12-02 00:00:00 UTC      0
19  123  2020-12-02 00:00:00 UTC      0

df2
    codcliente  nome        CNPJ    codprojeto
0   1           CLIENTE 1   123     1234
1   2           CLIENTE 1   145     5678
2   3           CLIENTE 1   167     9012
3   4           CLIENTE 1   189     3456
4   5           CLIENTE 1   101     7890
5   6           CLIENTE 1   122       11
6   7           CLIENTE 1   133       22
7   8           CLIENTE 1   144       33
8   9           CLIENTE 9   155       44
9   10          CLIENTE 10  166       55

Mapeando a Serie de df para atribuir os valores à correspondência no df2
df['codprojeto'] = df['CNPJ'].map(df2.set_index('CNPJ')['codprojeto']).fillna(0)

Imprimindo
df

Saída:
    CNPJ             DATA           codprojeto
0   123   2020-12-02 00:00:00 UTC   1234
1   123   2020-12-02 00:00:00 UTC   1234
2   123   2020-12-02 00:00:00 UTC   1234
3   123   2020-12-02 00:00:00 UTC   1234
4   145   2020-12-02 00:00:00 UTC   5678
5   123   2020-12-02 00:00:00 UTC   1234
6   167   2020-12-02 00:00:00 UTC   9012
7   167   2020-12-02 00:00:00 UTC   9012
8   167   2020-12-02 00:00:00 UTC   9012
9   167   2020-12-02 00:00:00 UTC   9012
10  101   2020-12-02 00:00:00 UTC   7890
11  122   2020-12-02 00:00:00 UTC     11
...

Comment: Parabéns pela solução. Você pode explicar o que a função map está fazendo nesse caso? Par mim não é óbvio
Comment: Segue, Imonferrari:

`df['codprojeto'] = df2['CNPJ_EMPRESA'].map(df.set_index('CNPJ_EMPRESA')['codprojeto']).fillna(0)`

Retorna o seguinte:

`InvalidIndexError: Reindexing only valid with uniquely valued Index objects`
Comment: Rodou, mas preencheu a coluna "codprojeto" do df com "0.0", ao invés de inserir o "codprojeto" do df2.
Comment: É como se não encontrasse nada e preenchesse toda a coluna com 0, ao invés do valor presente em df2['codprojeto'].
Comment: Segue preenchendo a coluna 'codprojeto' com zeros...

codprojeto
0.0
0.0
0.0
0.0
0.0
Comment: [Veja](https://i.ibb.co/s97z9gD/pandas.png)
Comment: Sim, vejo que pra você funciona, mas aqui continua preenchendo com zeros. Será que é pelo falo de o CNPJ, no df estar como int64 e no df2 como object, se perdendo na hora de realizar o map? Ou seja, em um dataset é um número, no outro uma string.
Comment: É uma possibilidade, aqui eles ja são convertido em int64 pelo proprio pandas,. Você pode testar fazendo `df2['CNPJ'] = df2['CNPJ'].astype('int64')` na hora que carregar o data frame.
Comment: ValueError: invalid literal for int() with base 10: '00.000.000'
Comment: Me envia seus 2 datasets inteiros. Hospeda eles no google drive e cola o link aqui.
Comment: Imon, são dados internos, não posso colocar publicamente. Tem alguma maneira de entrarmos em contato particularmente?
Comment: Enviei um e-mail.
Comment: Pela mensagem de erro me parece que usar um `df.drop_duplicates('CNPJ_EMPRESA')` antes de setar a coluna como index (dentro do `.map`) resolve, você consegue testar?
Comment: Terry, não posso dropar os CNPJs, pois cada linha possui informações únicas, por mais que o CNPJ se repita.


C++ what is "->"?

Question:
Possible Duplicate:
What is the arrow operator (->) synonym for in C++?

I just have a very basic question. Couldn't search it on google... I don't know what -> is and what it does in c++, for example:
render = Surface->Logic->Scale;

Comment: It is the arrow operator. [Possible Dublicate of this question:](http://stackoverflow.com/questions/221346/what-is-the-arrow-operator-synonym-for-in-c)
Comment: @net5tebiu - why the -neg rep (btw, I didn't do that). It's probably because the answer to your question is brutally and absolutely obvious in any C or C++ book, and either a) you are not reading your book, or b) you don't have a book (either case making your question a terrible and sloppy one.)
Comment: I'm learning C++ at school. It's not our main course. That's why we don't have any books. We didn't come to the point where we use this. But I've seen this operator(?) in many scripts and didn't know what it does. And those books are awfully expensive in my country... And I don't want to download illegal versions too...
Comment: @ne5tebiu, I get you man, it's not very straightforward when you're looking for operators like -> or @ in a language if you don't know what they are, at least. like @ in C# meaning a string literal, for e.g. And I do know some folks from Lithuania and I totally agree with you, books in your country are too expensive. But you can consult a number of free tutorials all over the internet which go through the basics and into some more complicated things. Have a look at [C++ Tutorial](http://www.cplusplus.com/doc/tutorial/) and read on a bit ;)
Comment: Why -1? Did I forget to mention something?
Comment: @Comptrol: You need to use `[description](link)` in the comments.
Comment: @Cat Plus Plus, thanks for that info :))
Comment: I agree that this question is rather simple and could be solved with a google search but he did mention that he couldn't search it on google. Now my question is, if he couldn't find it on google or he couldn't use google to begin with.
Comment: @Hallaghan I couldn't find it...
Comment: @Hallaghan Thanks, I'll read that :)
Comment: How can somebody be learning (or, worse, teaching) C++ without a book, even if it's not the main course. It's mind numbing.
Comment: By the power on the internet maybe? :)
Answer: In C++ -> has 2 contexts:

With respect to pointers
With respect to objects

It dereferences the pointer, so basically it brings to memory location where the variable is stored at certain offset. Thus,
Surface->Logic->Scale;

is equivalent to,
(*Surface).Logic->Scale;  // same applies for 'Logic' too

With respect to object, you can mock this operator by overloading it as,
struct B { void foo(); };
struct A {
  B *p;
  B* operator -> () { return p; }
};

Usage will be,
A obj;
obj->foo();

Comment: Note that, above is pseudo code, you may have to initialize `B *p;` to a valid object  (like `p = new B;`). I haven't mentioned this, as it will complicate the demo.
Answer: Do you have a book in C++? If the answer is no, then why not.
If the answer is yes, I'm sure that this is covered in it.
Anyways, the '->' symbol is the pointer de-referencing operator.
Answer: The -> operator dereferences the pointer and accesses the member. It is similar to writing
(*(*Surface).Logic).Scale.
So in the above, we first deference the Surface pointer with
*Surface
Once the pointer is dereferenced we can access the Logic member
(*surface).Logic
Since Logic is also a pointer you need to dereference the pointer
*(*Surface).Logic
And now you can access the Scale member
(*(*Surface).Logic).Scale
As you can see this is much more cumbersome than using the -> operator. Which deferences the poiner and accesses the member in one.
Answer: Well, the operator arrow(->) in c++ is used to access the members of a struct using a pointer. for example say you have:
struct A {
int x;
};

A* mp  = new A;

then to access the element x and assign to some other element say y you have to write:
int y = mp -> x;

For further explanation you may look here:
http://www.cplusplus.com/doc/tutorial/structures/

C++ what is "->"?


Where is the descending tritone in "I Hope I Get It" from "A Chorus Line"?

Question: I am looking for reference songs for intervals, and I am told that there is a descending tritone in "I Hope I Get It" from "A Chorus Line". The trouble is, I have no idea where in the song the descending tritone is. Can anyone tell me?
The source of the information is the following website, but it doesn't say where in the song, it just gives the song and musical:
https://www.earmaster.com/products/free-tools/interval-song-chart-generator.html
Answer: The best place to hear it is in the instrumental parts. For example, in the original cast recording (below), it can be heard in the piano at 0:34 (both an ascending and descending tritone) and in the trumpets at 0:49 (again, both ascending and descending).
Written out, with the tritone in red, the part of the music you're looking for goes like this:

Also helpful might be this post about interval songs.

Reference songs for interval training (esp. A5/d5; M6; m6)


Bulk import module

Question: I've created a custom content type and I need to enter 30 or so records. The type has around 5 fields.
Is there a module that provides a table for me to enter these details? At the moment, I assume I need to enter them one by one, or spend time writing some kind of mySQL script.
Answer:
Feeds - Import or aggregate data as nodes, users, taxonomy terms or simple database records.
Node import - Node import allows users to import content (node, user, taxonomy) from CSV or TSV files.
Node export - This module allows users to export nodes and then import it into another Drupal installation, or on the same site.
Migrate - The migrate module provides a flexible framework for migrating content into Drupal from other sources

I am recommend for simple case node import/export using. If importing happen eventually and it's simple, then Feeds. If import make hard data and big count of records, then Migrate.

Bulk import module


Put PHP opening and closing tags into variable

Question: I am trying to make my PHP code write another PHP file and need the opening <?php and closing ?> tags included when writing the file.  How would I go about doing that?
Example:
$Data = '<?php $keywords = "Testing"; ?>';

Full file writing code, which does not work:
<?php
$File = "hello.php";
$Handle = fopen($File, 'w');
$Data = '<?php $keywords = "Testing"; ?>';
fwrite($Handle, $Data);
print "Data Written";
fclose($Handle);
?>

Comment: Did you try that? What happens?
Comment: Tested here on PHP 5.4.21 - your code works fine. Make sure you have permissions to write the file.
Comment: It doesn't seem to write the file, but if I do something like `$Data = '$keywords = "Testing";';` it does.
Answer: Check out this sample. You are on the right path
$string = '<?php

$dbhost = "'. $_POST["dbhost"]. '";

$dbuname = "'. $_POST["dbuname"]. '";

$dbpass = "'. $_POST["dbpass"]. '";

$dbname = "'. $_POST["dbname"]. '";

$prefix = "'. $_POST["prefix"]. '";

$user_prefix = "'. $_POST["user_prefix"]. '";

$dbtype = "'. $_POST["dbtype"]. '";

?>';

$fp = fopen("config.php", "w");

fwrite($fp, $string);

fclose($fp);


Probability of occurence of alphabets

Question: A circular disc with 3 sectors marked as alphabet O,A,B has
circumference l. The alphabet coming infront of marker is noted
down aftr each spin (as shown in diagram). Assume boundary
of sectors do not come in front of marker. It is spun 6 times around
its centre and resulting alphabet which comes infront of marker
are noted down in order. lf the probability that; the word. BAOBAA
to be formed is maximised, than (where x,y,z respectlvely denotes
the probability of occurrence of alphabet O,A,B)
W have to maximixe zyxzyy
So I used AM>GM
That is $xy^3z^2 < (27)(4)/(6^6)$
But how how to proced
Answer: It sounds like you're almost there. We want to choose $x,y,z\geq 0$ with $x+y+z=1$ such that $xy^3z^2$ is as large as possible. Applying AM-GM to the numbers $x,\frac y3,\frac y3,\frac y3,\frac z2,\frac z2$, we get $$\sqrt[6]{\frac{xy^3z^2}{108}}\leq\frac 16,$$with equality if and only if $x=\frac y3=\frac z2$. Since in that case $x+y+z=6x=1$, the probability is maximised exactly when $x=\frac 16$, $y=\frac 12$ and $z=\frac 13$.
Answer:
$p(a)=1-p(b)-p(c)$,
$p(cbacbb)= \frac{p(c)^2 p(a)^3p(a)}{{6 \choose 3}{3 \choose 2}}=\frac{1}{60}p(c)^2 p(b)^3p(a) = \frac{1}{60}p(c)^2 p(b)^3(1-p(c)-p(b))$
$f(y,z)=60 p(cbacbb) = p(c)^2 p(b)^3(1-p(b)-p(c))  = y^2z^3(1-y-z)$
$\frac{\partial f(y,z) }{\partial y} = y^2z^2(3-4y-3z)\\
\frac{\partial f(y,z) }{\partial z} = y^3z(2-2y-3z)$
$$\frac{\partial f(y,z) }{\partial y}=\frac{\partial f(x,y) }{\partial z}=0 \iff (y,z)\in \left\{\left(\frac{1}{2},0\right),\left(\frac{1}{2},\frac{1}{3}\right)\right\}$$
If we take $(y,z)=\left(\frac{1}{2},0\right)$, then $p(cbacbb)=0$, so we will skip this case.
$\frac{\partial^2 f(y,z) }{\partial y^2}=6yz^2(1-2y-z)\\
\frac{\partial^2 f(y,z) }{\partial z^2}=2y^3(1-y-3z)\\
\frac{\partial^2 f(y,z) }{\partial y\partial z}=y^2z(6-8y-9z)$

$$H(y,z)=\left| \begin{array}. 6yz^2(1-2y-z) & y^2z(6-8y-9z) \\
y^2z(6-8y-9z) & 2y^3(1-y-3z)\end{array} \right|$$
$\frac{\partial^2 f(y,z) }{\partial y^2}|_{(y,z)=\left(\frac{1}{2},\frac{1}{3}\right)}=-\frac{1}{9}<0\\
H\left(\frac{1}{2},\frac{1}{3}\right)=-\frac{1}{8}\frac{1}{72} <0$
so in point $(y,z)=\left(\frac{1}{2},\frac{1}{3}\right)$ function $f$ have the local maximum.
The probability is them maximized, if $(p(a),p(b),p(c))=\left(\frac{1}{6},\frac{1}{2},\frac{1}{3}\right)$

Probability of occurence of alphabets


Unable to add new Tomcat in Eclipse

Question: Can't properly add a new version of Tomcat Server.
IDE: Eclipse Luna (64 bit)
Server: Tomcat 8.0.30 (64 bit)
Java: jdk1.8.0_65 (64 bit)

Comment: I think u have preconfigured tomcat 8 before so first please remove it
Answer: I did the following step and Eclipse allowed me to add new tomcat.
Step 1: In {workspace-directory}/.metadata/.plugins/org.eclipse.core.runtime/.settings delete the following two files:
          i. org.eclipse.wst.server.core.prefs
         ii. org.eclipse.jst.server.tomcat.core.prefs
        iii. Restart Eclipse

Step 2:  Delete the tomcat: Server > right click delete.
         i. New server installation:
        ii. Server > right click > tom 8.0 > next > finish.
       iii. Start the server.

Answer: Go to Windows->preference->servers->Runtime Envirorments.
In this location remove all the pre-configured tomcat 8 and later try to add new one
Comment: I have removed previous tomcat-8 entry from Runtime Environments but still unable to add new tomcat-8.

Unable to add new Tomcat in Eclipse


Large number of positive labels in classifier when actual population has few

Question: I have been tasked to help with a sort of classifier. In the make up of the problem the set we want to identify as "Positive" is know to be very very small. However the training set I have been given is the complete opposite, it is almost all positive numbers.
For example an estimate of the real world split would be 1% positive and 99% negative(This is likely an over estimate of the number of positives). The training Data I have been given is a hand labeled set that is 91% positive and 9% negative. The data has both text features and some categorical features.
At first glance I think the training set is bad, because of it being so positive. I also realize that simply labeling a random entry positive 91% of the time will give  a real good classifier in my training but an awful one in an actual application.
I am reasonably new to my field and would like to find a way to justify my concerns more so than "this doesnt look good". Am I correct in thinking that a training set split 90/10 when realistically the population is about 1/99 is bad training set to begin with?
Answer: Yes it is indeed bad if you have a training set that is not only imbalanced, but also doesn't represent the true distributions of real world examples.
In your case it would be really good if you could get more data for the negative class. That shouldn't be too difficult since you said it is the most common class in the real world.
Also think about what is the most costly class to misclassify. Have a look at the literature about cost-sensitive learning.
As I already wrote in another thread, usually most classifiers work better if you have access to the same number of data for all classes, so you have a balanced dataset. This is not usually the case so the problem of imbalanced learning has been already studied quite in depth.
Check this paper for a good review of learning with imbalanced datasets.
One way of dealing with the problem is to do artificial subsampling or upsampling in the training set to balance the datasets.
I think it is usually better to have a balanced training set, since otherwise the decision boundary is gonna give too much space to the bigger class and you are going to misclassify too much the small class. This is usually bad. (think of cancer detection where the smaller class is the most costly, namely having a tumor).
If you don't want to use sampling methods, than you can use cost based methods, where you weight the importance of every sample so that the loss function has more contribution from the samples of the most important class. In cancer detection, you would weight more the cost coming from training samples of hte positive class (having a tumor).
Finally, remember that if the test set is very unbalanced classification accuracy is not a good measure of performance. You would be better off using precision/recall and the f-score, easily computed from the confusion matrix. Check this paper for references on classification performance measures for a lots of different scenarios.
Also another good read on the topic is this one.


Python Scatter Plot Edgecolors by Label

Question: I'm trying to figure out how to dynamically set the edgecolor of a datapoint.
The facecolor can be set by using c=[array of numeric labels], but not so for edgecolors.
What im trying to achieve is a plot where the facecolor equals the predicted label of an algorithm for an instance, and the edgecolor equals the true label of that instance.
This way, i could visually compare the results (important for further work).
Any suggestions? This is my code so far:
fig, axs = plt.subplots(2, 2)
axs[0, 0].scatter(X_train[:,0], X_train[:,1], c=y_train)
axs[0, 0].set_title('org.Data with true labels')
axs[0, 1].scatter(X_train[:,0], X_train[:,1], c=y_pred)
axs[0, 1].set_title('org.Data with LOF labels')
axs[1, 0].scatter(data_PCA[:,0],data_PCA[:,1], c=y_pred)
axs[1, 0].set_title("PCAplt, labels LOF: org.Data")
axs[1, 1].scatter(data_PCA[:,0],data_PCA[:,1], c=y_pred_PCA)
axs[1, 1].set_title('PCA.plt, labels LOF: PCA')

which gives me the following plot

I appreciate any suggestions.
Thank you!
Comment: See [this post](https://stackoverflow.com/questions/59810599/how-to-create-a-scatter-plot-with-two-colors-per-dot/59831737#59831737) about how to create a bicolored scatter plot.
Answer: Sometimes the answer comes after laying stuff to rest for some time. I found a solution and wanted to share in case anyone ever needs this:
In short: I precomputed a list of "true label colors" using a very simple list comprehension, and used another list of length 2 with the same colors for the facecolors.
The code:
colors = ["green","red"]
true_labels_color_encoded = ["red" if label == 1 else "green" for label in y_train]

fig, axs = plt.subplots(2, 2)
axs[0, 0].scatter(X_train[:,0], X_train[:,1], c=y_train, cmap=ListedColormap(colors), edgecolors=true_labels_color_encoded)
axs[0, 0].set_title('org.Data with true labels')
axs[0, 1].scatter(X_train[:,0], X_train[:,1], c=y_pred,cmap=ListedColormap(colors),edgecolors=true_labels_color_encoded)
axs[0, 1].set_title('org.Data with LOF labels')
axs[1, 0].scatter(data_PCA[:,0],data_PCA[:,1], c=y_pred,cmap=ListedColormap(colors),edgecolors=true_labels_color_encoded)
axs[1, 0].set_title("PCAplt, labels LOF: org.Data")
axs[1, 1].scatter(data_PCA[:,0],data_PCA[:,1], c=y_pred_PCA,cmap=ListedColormap(colors),edgecolors=true_labels_color_encoded)
axs[1, 1].set_title('PCA.plt, labels LOF: PCA')

so true_labels_color_encoded looks something like = ["red","green","green","green"]
(note: in my case, a label in y_train = 1 means anomaly, and 0 means normal)
which gives me the following plot:
See my plot here
I got even better results using other reduction techniques like ISOMAP or t-SNE.

Python Scatter Plot Edgecolors by Label


How to compute number of combinations with placement restrictions

Question: There are five buckets labeled 0 (AC), 1 (AB), 2 (AC), 3 (BC), 4 (BC), as well as four marbles of type A, A, C, C. Marbles may only fit in a bucket that includes it's type (i.e. A can go into AC or AB, but not into BC). Each bucket can only hold 1 marble.
In how many different ways can I place the marbles in the buckets?
I solved this problem manually by listing all the possible combinations (7):
0-A, 1-A, 2-C, 3-C
0-A, 1-A, 3-C, 4-C
0-A, 1-A, 2-C, 4-C
0-A, 2-A, 3-C, 4-C
1-A, 2-A, 0-C, 3-C
1-A, 2-A, 0-C, 4-C
1-A, 2-A, 3-C, 4-C
How can I mathematically (perhaps using combinatorics) arrive at this answer? I have to deal with considerably more complex problems where listing out the solutions is not feasible.
Comment: You didn't say that each bucket can only hold one marble, but your solution seems to indicate that.  Did you intend that?
Comment: @RossMillikan Yes, each bucket can only hold one marble. Thanks for making that explicit.
Comment: Your wording indicates that the order in which marbles are placed should matter, your enumeration indicates that it does not.  (That is, if A marbles go into 0 and 1, does it matter whether you place them (0,1) or (1,0)?)  Which is it?
Comment: @btilly Sorry for the ambiguity. Order does not matter.
Answer: The interlocking conditions make this hard to do systematically.  I would start with the As.  If they go in bins $0,2$, the Cs only have one choice.  If they go in bin $1$ and either of the others ($2$ ways) the Cs have ${3 \choose 2}=3$ choices, so the total is $1 \cdot 1 + 2 \cdot 3=7$
Answer: There are two generic ways to do this.  Dynamic programming, and Inclusion-Exclusion.  If you have several of these to do, I would personally take the programming approach because I know how to.
Dynamic programming would require that you write a program that walks through the buckets and produces.  For each bucket you would construct a data structure from the vector of remaining marbles to how many ways you could have arrived at it.  The answer for no remaining marbles at the end is your answer.
The equivalent to dynamic programming can also be done with a top-down recursive function with memoization to speed it up.  This may be easier if you know how to program but haven't done dynamic programming before.
As for https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle the approach is to count the number of ways to place with possibly multiple, subtract off ways to place with at least one bucket with 2, add ways to place with at least 2 buckets, subtract ones with at least 3, and so on.  In this case:

possibly multiple: ${3\choose 2} {4 \choose 2} = 18$ (put 2 A's in 3 buckets, 2 C's in 4 buckets)
at least 1: ${2 \choose 1} {2 \choose 1} {3 \choose 1} = 12$ (choose the multiple bucket, place remaining A in choice of 2 places, remaining C in choice of 3 places)
At least 2: ${2 \choose 2} {1 \choose 0} {2 \choose 0} = 1$ (choose both multiple buckets, place no remaining A's and B's anywhere.
All higher terms: there is no way to generate more collisions than 2, so they are all 0.

So we get $18 - 12 + 1 = 7$.
Note that as you get to more complex cases, inclusion-exclusion will have more and more terms.  In fact in general you'd want a program to figure it out...which is why I'd write the program first. :-)
Comment: This is the more informative answer, although unfortunately it reveals that there is a fair number of calculations required for complex scenarios.
Comment: @Andrés Would it help if I wrote a simple program for a general version of this, just to see how to do it?


Can someone explain me the sentence about ideals?

Question: Can someone explain me the sentence:
"If $R=K[x]$ the prime ideals are $\langle f(x)\rangle $ where $f(x)$ is an irreducible polynomial in $K[x]$ and $\langle 0\rangle $, and again $\langle f(x)\rangle  $ $\ f(x)$ is also maximal."
?
Comment: Could it read "... and again $(f(x))$ is also maximal."?
Comment: @user159870, isn't there some comma or period or something in "and again $\;\langle f(x)\rangle\;f(x)\;$ is also..."
Comment: @HagenvonEitzen I formatted it with LATEX.
Comment: @HagenvonEitzen I found it in that way in my lecture slides. Isn't it correct formulated?
Comment: @Timbuc I found that in that way. How should it be?
Comment: @user159870, most probably as Hagen suggested: "...and again, $\;\langle f(x)\rangle\;$ is also maximal" (meaning, $\;f(x)\;$ is irreducible iff $\;I:=\langle f(x)\rangle\;$ is prime iff $\;I\;$ is maximal. This is true **only** if $\;K\;$ is a field, and in general it is not true that prime $\;\implies\;$ maximal).
Comment: @Timbuc So to apply this in an exercise which demands to show that the ideal $I=$ is maximal, where $f(x)$ is irreducible, we have to show first that $K$ is a field and then that $I$ is prime?
Comment: @Timbuc Ok!!!!!
Answer: The notation $\langle a_1,\dots,a_n \rangle$  for $a_i$ in a ring $R$ sometimes means the ideal generated by $a_i$. So $\langle f(x) \rangle$ is the same as $(f(x))$ in other notation, or simply the set $\{f(x)g(x): g(x) \in K[x]\}$. The statement is saying a prime ideal of $K[x]$ is either the zero ideal, or a maximal one which is of the form $\langle f(x) \rangle$ for $f(x)$ irreducible.
Comment: My answer assumed that $K$ is a field to begin with, otherwise the statement in your question is not true. To show $\langle f(x) \rangle$ is maximal, one way would be to show $K[x]/\langle f(x)\rangle$ is a field (not the same as $K$ typically). But it's probably easier to just show $f(x)$ is irreducible.
Comment: Assuming $K$ is in fact a field, yes, if you show $f(x)$ is irreducible, then $\langle f(x) \rangle$ is maximal.
Comment: So to apply this in an exercise which demands to show that the ideal $I=$ is maximal, where $f(x)$ is irreducible, we have to show first that $K$ is a field and then that $I$ is prime?
Comment: So we show that $f(x)$ is irreducible in $K[x]$ and we are done. Correct?