Why does VI ii°7 V7 i sound so bad in minor?

Question:
I am currently working on subdominant 7th chords. In some exercises I'm asked to voice lead typical subdominant to dominant progressions. Most the progressions I'm asked to write are in major. I'm trying to do some in minor just to get a different idea of what it sounds like and I came up with this progression which is supposed to be very standard chord progression but for some reason it doesn't sound very good. Or at least not as good as in major. I don't know why because the voice leading seems okay. Is it the dissonant interval in the bass line that makes it sound not as good as it does in major? Was this chord progression used in classical harmony at all or was ii65 the go to chord?
the progression in my image is in c#
Comment: Key signature and clefs would help
Comment: You have full diminished seventh in the title but half dim in the score, and I think you meant `B` natural, not sharp, in the score.
Comment: @MichaelCurtis - quoted C#minor thus G and F clefs. And leading note should be B# in that cey.
Comment: @Tim It would be much more clear to have the clef and sharps next to the clef written out in the score.
Answer: There are two "rules" broken in the given progression:

No leaps by tritone;
No leaps into a dissonance.

The reason these rules came into being is exactly what you've discovered: they create harsh(er) sounds.
The bass's leap from A to D# is itself harsh sounding (within the aesthetic expectations of tonal theory), and the D# is dissonant against the A in the ii7 chord.
The difference with using ii65 is that the bass makes a consonant leap from the VI chord's A to the ii chord's F#, and there are no dissonances in the ii chord against the F#: there are a minor sixth, a minor third, and a perfect fifth.
Comment: "Classical music" encompasses quite a lot of music. Off the top of my head, I don't know an example of root-position ii7 - V in. minor, but I'll update if/when I stumble onto one.
Comment: @MichaelCurtis Key specified in the OP is C# minor.
Comment: @armani, if you want to avoid these dissonances, just arpeggiate the bass from the `A` up to `E` before it moves to `D#`.
Comment: So was this chord progression not used in minor at all? according to some of my books ii°7 was used a lot so how was it approached? From scale degree 6 then is a no-no according to what you say and this is the sxale degree that most often precedes a ii chord.
Comment: @armani Another "solution" might be III instead of VI or between VI and ii
Comment: did you mean to say: no dissonances in the ii65 chord?
Comment: Todd III to ii°7 you think will work?
Comment: I assume this has a key sig of three flats for C minor. Did you mean `A` flat to `D` natural?
Comment: @armani I didn't say it wasn't used; only that out of the hundreds of thousands of classical pieces, I didn't happen to automatically know of a specific example. And, no, I didn't claim there are no dissonances in the ii65 chord. There are no dissonances between the bass and any other individual voice.
Comment: Ah, I totally missed that.
Comment: Michael, in real world composition I will keep that tip in mind thanks!


Change background color when clicked textbox in C#

Question: I like for the textbox to change the background when the textbox has focus (if clicked with a mouse, tabbed into, etc).
I don't like the textchange method since it won't change until one character has been added.
private void txtFoo_OnGotFocus(object sender, EventArgs e)
    {
        txtFoo.BackColor = Color.LightYellow;
        txtBar.BackColor = Color.White;
    }

This doesn't seem to work for me.  What am I doing wrong?
Answer: Did you also subscribe the event handler?
Answer: Like Ben said... did you subscribe to the event handler?
Is your code even executing? You can put a break point in there and check pretty easily.
Answer: You could try using the Enter event instead
private void txtFoo_Enter(object sender, EventArgs e)
{
    txtFoo.BackColor = Color.LightYellow;
    txtBar.BackColor = Color.White;
}

Answer: txtWFileNo.Style.Add(HtmlTextWriterStyle.BackgroundColor, "silver");


cross ratio and homogeneous coordinates

Question: the wikipedia article on cross ratio
https://en.wikipedia.org/wiki/Cross-ratio#Definition_in_homogeneous_coordinates
says:
Definition in homogeneous coordinates
If four collinear points are represented in homogeneous coordinates by vectors $a, b, c, d$ such that $c = a + b$ and $d = ka + b$, then their cross-ratio is $k$.
I am wondering about this if the vectors  $a$ and $b$ are not collinear then the points $a, b, c, d$ are not collinear to start.
and also I doubt the rest. can somebody check?
(also i thought we had a cross ratio tag , is it gone?)
Answer: A collection of distinct points in projective space $\mathbb{P}^n$ are colinear if the vectors representing each point in $\mathbb{R}^{n+1}$ lie in the same plane.
Answer: Collinearity of points in projective space corresponds to coplanarity of the vectors representing them (since the plane is what collapses to the line in projective space).
For example, it is always the case that two linearly independent vectors are coplanar: this corresponds to the fact that there is always a projective line through the two points they represent. If you meant that the vectors $a,b$ were linearly dependent at the start (instead of  "collinear", however one interprets that for vectors), then you're right: the two vectors represent the same projective point, and you didn't have a full set of four points in projective space at the start.
All definitions I am aware of rely on the four projective points being distinct (i.e. no two of the representing vectors are linearly dependent), so this should no be an issue.
Comment: Thanks for your answer I learned that I don't know a lot about homogeneous vectors ,is it okay with you if I add my questions on them to this question? +
Comment: @Willemien if you have a little set of questions you want to ask about homogeneous coordinates, it's probably better to ask them separately at this point.

cross ratio and homogeneous coordinates


$f_{n}$ uniformly converges to $f$ and $|f'_{n} (x)| ≤ C $, is $f$ necessarily differentiable?

Question: Assume $f(x)$ is the uniform limit of real differentiable functions $f_{n}(x)$ on $[−1,1]$. Assume that $|f'_{n} (x)| ≤ C $ for some $C$ independent of $n$ and $x ∈ [−1, 1]$. Is the function $f(x)$ necessarily differentiable?
Comment: See for example https://math.stackexchange.com/a/2420628 or https://math.stackexchange.com/a/2316364
Answer: No. Choose $f_n(x) := \sqrt{x^2 + \frac{1}{n}}$. Then, for all $x \in [-1, 1]$:
$$
\left \lvert \sqrt{x^2 + \frac{1}{n}} - \lvert x \rvert \right \rvert = \sqrt{x^2 + \frac{1}{n}} - \lvert x \rvert \leq \lvert x \rvert + \frac{1}{\sqrt{n}} - \lvert x \rvert = \frac{1}{\sqrt{n}} \overset{n \rightarrow \infty}{\longrightarrow}0
$$
Therefore
$$
\sup_{x \in [-1, 1]}\left \lvert \sqrt{x^2 + \frac{1}{n}} - \lvert x \rvert \right \rvert \leq \frac{1}{\sqrt{n}} \overset{n \rightarrow \infty}{\longrightarrow}0
$$
and thus uniform convergence. The limit, i.e. the absolute value is not differentiable in $0$. Furthermore, for all $n \in \mathbb{N}$, $x \in [-1, 1]$:
$$
\lvert f_n'(x) \rvert = \left \lvert \frac{x}{\sqrt{x^2 + \frac{1}{n}}} \right \rvert = \frac{\sqrt{x^2}}{\sqrt{x^2 + \frac{1}{n}}} \leq \frac{\sqrt{x^2 + \frac{1}{n}}}{\sqrt{x^2 + \frac{1}{n}}} = 1
$$
Answer: For every $x,y \in [-1,1]$ and for every $n$ there exists $z_n \in (x,y)$ such that $f_n(x) - f_n(y) = f'(z_n)(x - y)$, so we have:
$$|f_n(x) - f_n(y)| \leq C |x - y|$$
for every $x,y \in [-1,1]$ and for every $n$, passing to the limit we get:
$$|f(x) - f(y)| \leq C |x - y|$$
for every $x,y \in [-1,1]$, so $f$ is Lipschitz and so is differentiable for $\mathcal{L}^1$ a.e. $x \in [-1,1]$.
But we can't conclude that $f$ is differentibale evreywhere, here there is a counterexample:
$$f_n(x) := (x^2)^{\frac{n}{2n -1}}$$
observe that $\lim_{n \to \infty}f_n(x) = |x|$ which is not differentiable at $x = 0$, the derivative is uniformly bounded wrt $n$ and the convergence is uniform


Acessar horas e minutos com Angular Timer

Question: Quero pegar os dados de horas e minutos do Angular Timer e jogar em uma variável para salvar no banco, como faço?
Answer: Interrompa o Timer com um broadcast:
$scope.$broadcast('timer-stop');

Ao mesmo tempo, monitore o evento timer-stopped, e capture o objeto contendo o estado do timer:
$scope.$on('timer-stopped', function (event, data){
    console.log(data);
});

O objeto conterá as seguintes propriedades:

days
hours
  millis
minutes
  seconds
  timeoutId
Comment: Deu certo, mas usando o ng-click tenho que executar duas vezes para exibir no console e atribuir em outra variável. Pode ser algum problema com o ng-click ou assincronismo? Obrigado.


A little pedal newbie

Question: so i have a question, as much I search about it  i can't find any answer,it's about the distortion and overdrive pedal,soo if i have a distortion pedal and use it for rythmic parts and when its  time to solo i turn the overdrive,does the overdrive give me a boost?,does it work?or i just unleashe the chaos of feedback? (I use clean channel from the amp)
I have a distortion pedal from Behringer um 300 specifically,if it works what overdrive pedal do you suggest me? Thank you all for your patience,(sorry for my bad English)
Comment: " ...the chaos of feedback...", great name for a band.
Answer: There's a lot to unpack in your question, but the key concept to understand here IMO is compression. Whenever you have clipping, you have compression. This is because clipping is a limiting of the dynamic range of the signal, more or less by definition.
So, when you have the distortion on you are creating a compressed signal that goes to the amp. So if you put an overdrive in front of the distortion (between the guitar and the distortion), then you're not likely to experience a boost in volume because the distortion pedal cannot produce any more volume only more severe clipping.
But if you put the overdrive after the distortion (between the  distortion and the amplifier) then it probably will work to boost the signal and maybe do some softer clipping of its own, too.
In a way there's a spectrum of different effects depending on the characteristics of the clipping. But all of these effect involve some kind of amplification component.
<code>
 Compressor   zero to soft clipping
  Overdrive   soft to medium clipping
 Distortion   medium to hard clipping
       Fuzz   hard clipping
</code>

A little pedal newbie


Why this nginx rewrite rule add trailing slash?

Question: I've the following rewrite rule on location /, and it enforces trailing slash!
location / {
        rewrite ^(\/.*) https://example.com/ar$1 permanent;
}

each, time I curl mydomain.com, I got the following response:
HTTP/1.1 301 Moved Permanently
Content-Type: text/html
Connection: keep-alive
Location: https://example.com/ar/

I do not want the trailing slash at the end of the redirected URL!
Comment: A url always _starts_ with / so `$1` will also with the config in the question.
Comment: Thank you, but any link/resource for that?
Comment: Look at the location block. You are matching a `/` and then inside the block too, you are matching a `/`. If you need to eliminate `/`, you need to ignore it somewhere. Probably, you are looking for the following solution... `location / { rewrite ^/?(.*) https://example.com/ar$1 permanent; }`.
Comment: @Pothi This one worked! Thank you, can you add it as answer?
Comment: Sure. Glad to know it worked.
Answer: Please look at the location block. You are matching a / and then inside the block too, you are matching a /. If you need to eliminate /, you need to ignore it somewhere. Probably, you are looking for the following solution...
location / { rewrite ^/?(.*) https://example.com/ar$1 permanent; }.


Intuition about whether to switch in box problem

Question: I ran across an apparent paradox which I then located in the paper The Box Problem: To Switch or Not to Switch as such:

Imagine that you are shown two identical boxes. You know that one of
  them contains. \$b and the other \$2b. Picking one at random and opening
  it, you must decide whether to keep it (and its contents), or exchange
  it for the other box.

In short, when you find $x dollars in a box, the expected value of the other box is .5*.5x + .5*2x = 1.25x, meaning that it's always better to switch. This appears to violate the symmetry of the problem and the fact that you still know nothing meaningful about either box.
The paper goes another direction with it, talking about how having prior knowledge of expected values gives a more meaningful analysis (along with other discussions). However, what if there's no prior knowledge, and we have the original problem as stated. Can anyone give me some intuition to make sense of this?
EDIT: Someone found this question which asks a slightly different formulation, but an identical problem. The accepted answer there just points to a paper, and I'm having difficulty understanding the paper. It explains away the paradox by noting the expectation is based on an infinite sum, and the value depends on the order the sum is evaluated. I'm not familiar with how the order of a sum can change a value, and I also don't see talking about a different way to evaluate the expectation explains the strange result outlined above. My math understanding is primarily based on reading textbooks as a hobby, and I haven't yet worked up to fully understanding math academic papers, so a simpler explanation would be helpful.
Comment: Edited to note that I have now seen this duplicate question and how I don't follow the paper given as an answer. Thanks for the pointers!
Comment: See http://en.wikipedia.org/wiki/Envelope_paradox (a closely related problem in which you don't open the first box before deciding whether to switch).
Comment: The accepted answer there just points to a paper. After reading the paper, I still have no intuition here. I apologize, but my math knowledge is that of a hobbyist, and I would still appreciate a simple explanation.
Comment: That doesn't change the fact that this is a duplicate of that question. To make it a new question, you could edit it to point out what you're missing in the answer to that question -- as specifically as possible, but if the problem is that you don't understand that answer well enough to be able to describe what you're missing in it, you could just ask for an explanation in simpler terms -- in that case it would help if you describe as specifically as possible what sort of level of explanation you're looking for.
Comment: And not a free article either.
Comment: What's wrong with the (not accepted, but more highly voted) other answer to the duplicate question?
Comment: I'm not totally sure how that answer is relevant (as in, I'm clueless, not that there's necessarily anything wrong with the answer). I'm guessing that it's talking about the values of $b, but even if you pick some distribution, you still have the paradox, since the paradox isn't using the distribution.
Answer: You asked for "intuition to make sense of this" and that's what I hope to provide. I state that there is no paradox and, despite the symmetry of the situation, it IS better to switch.
Suppose you walk into a not-for-profit casino where there is no house advantage and place a double or nothing bet for \$x. There is a 0.5 probability that you will win and recieve \$2x (your stake plus \$x winnings) a 0.5 probability that you will lose and walk away empty-handed. The expected winnings are 0.5*\$2x + 0.5*\$0 = \$x. Thus the game is perfectly 'fair' and there is no statistical indication for whether or not you should play. It's just a matter of how lucky you feel...
However, what if the casino offered to give you back half your winnings if you lose? Then you'd be crazy not to play. You'd have a 0.5 probability of winning and getting \$2x and a 0.5 probability of losing and getting back only \$x/2. The expected winnings now are
0.5*\$2x + 0.5*\$x/2 = \$1.25x. It's just like the problem you describe.
Hopefully one can intuit now that a double-or-halve bet is better than a double-or-nothing, and since the double-or-nothing is a fair bet then the double-or-halve one is clearly stacked in your favour. Hence you should take it if offered and that's why it is always good to swap the boxes in the question as originally posed.
This explanation, of course, is only valid for a constant value of x so cannot be applied to the original formulation of the 'Two Envelopes' question, a discussion of which is beyond the scope of my evening. I added this disclaimer because I'm wary of being accused of providing an irrelevant 'lay perspective'.
Comment: This suddenly seems intuitive, thank you!!
Answer: This page may shed some lights:
https://en.wikipedia.org/wiki/Two_envelopes_problem
Comment: But 50% x/2 + 50% 2x = 1.25x .
Comment: Sure, but that statement is unrelated to the problem. It may be an attempt at confusing or creating an illusion of an apparent paradox.
Comment: What do you mean it's unrelated? You said in your answer that the expected money is 50% x/2 and 50% 2x. This equals 1.25x.
Comment: Sorry to be obtuse, but why can't you simply add the multiplied values? I get that they're exclusive, but that's normally exactly the condition where you would multiply and add. Are you saying that for some reason it's impossible to calculate the expected value in terms of x? Why?
Comment: An expected value is vaguely what the outcome per event would be if we carried out the experiment an infinite number of times. So with infinite number of boxes each having random .5x and 1x dollars you would expect 1.5x per box. Also in your equation 2x/(x/2)=4 so this is giving the wrong ratio of money in the boxes. Money in one box should be twice as many according to the problem, not 4 times as many. With p=50% then the expected outcome is .5*2x+.5*x=1.5x after one trial.
Comment: I think you're explaining the expected value over a long period of time where x is known? The question is about where x is unknown and whether you should switch after looking in a box. This is why my math differs: if you see x, you don't know if the other box has x/2 or 2x.
Comment: Considering uniform distribution and that in reality money is limited there will always be a limit to the max $ per box. So you will be able to make predictions on the maximum after a number of trials and then based on that decide which box to pick subsequently for higher gain. So the question may be split in two; whether there is a limited amount of money available to draw from(reality) or infinite money available?
Comment: Given that this is an abstract math problem, I think it's a fair interpretation that there is no largest number and that x could be any value, but I don't think the particular value of x is what matters here (for simplicity, let's assume that x is finite). And again, the point is not to find patterns after a number of trials. The question is about the apparent paradox after a single drawing and wondering whether to switch.
Comment: The first problem is that 50% x/2 + 50% 2x is wrong! It gives ratio of money 4:1! The ratio of $ in the two boxes is always 2:1. The second problem is that this question is not clear and precise as I indicated earlier. I don't see any paradox.
Comment: You misunderstand the problem. If you open a box and see \$x, you know that the other box has either \$2x or \$.5x. This is because you don't know if you opened the bigger or smaller box to begin with. Thus, there actually is a a 4:1 ratio between the two possibilities.

I'm aware that there probably doesn't exist a paradox, I just don't see how the expected value of switching is always greater than what you have. This isn't like Monty Hall where you've gained some information.
Comment: There are only two possibilities after you've opened the first box and now have x dollars. First possibility is that you find 2x. In this case 2x/1x = 2:1 everything is good. Second possibility is that the second box has .5x, x/.5x=2:1 again the same. Those two events are EXCLUSIVE they cannot happen simultaneously so you cannot find both twice the money and half the money to get 4:1 ratio. Give me an example with a fixed dollar amount that the ratio is 4:1?
Comment: It's not that the other box is 4 times what you have. The difference between the two possible values is 4:1. Let's say you open a box and find \$10. This means the other box either contains \$5 or it contains \$20. \$20 is 4 times \$5. As there is .5 probability of either case, the expected value is .5 * \$20 + .5 * \$5 = \$12.50. My question is essentially for an intuition of why you always expect the other box to be greater.
Comment: I fully understand that the events are exclusive. That's exactly why I calculate the expectation as the sum of values times probabilities. There's 50% of \$5 and 50% of \$20 and 0% of both \$5 and \$20.
Comment: Can you precisely define what you call the expected value?
Comment: You are confusing the definition of a trial. It consist of opening two boxes. What you call a trial is having already opened one box then what is the expected value of the second box only. Your trial consist of only the second box. So you are calculating the expectation value of the second box only. Is there a link to the original problem defined?
Comment: I'm using the standard definition of expected value as "a weighted average of all possible values" (http://en.wikipedia.org/wiki/Expected_value). The original problem is linked in the question (http://www.jstor.org/stable/2691373?seq=1) and if that link isn't available (JSTOR is quirky based on what network you're on), google "The Box Problem: To Switch or Not to Switch". I'm not talking about infinite trials; this is a probability problem, not a statistics problem. And yes, I'm considering the conditional expectation of the second box, given the first box being open.
Comment: It is explained in the paper why it seems that based on the expectation value you would chose the second box. The issue is that the prob distribution is not stated. So you cannot assume 50% chance. The consequence of this is also asymmetry giving more prob weight to the higher value because .5x is only .5x from x, but 2x is 1x distance from 1x, hence you get 1.25x for the expected value. I don't see anything outstanding. The rest of the paper goes then into what happens when different prob distributions are assumed.http://econ.as.nyu.edu/docs/IO/9393/RR90-26.pdf
Comment: So if we modify the question to assume a uniform distribution, we can then assume a 50% chance? It seems like that brings the paradox back in.
Comment: Yes, then the apparent paradox is that from the expectation value you would guess that choosing the second box is better. Think about what happens if it were that you either get 1/2x or 3/2x? The expectation value is just x. How about 1/2x or 5/4x, then you expect .875x. Now it's better to chose the first box, even though the prob distribution is the same? The expectation value does not necessarily indicate the way to a better game.
Comment: I don't follow your examples. If you found .5x, the other box would have .25x or 1x. .5*.25x + .5*1x = .625x. Then we note that .625x > .5, so the other box still appears better. Multiplying by a scalar changes nothing. I don't think you're thinking things through very well. Sure you have a lower expectation on the second box, but that corresponds to an even lower value on the first box (but since these are in terms of an unknown, nothing is meaningfully different). Finally, that's exactly what expectation means in the context of a game. Are you familiar with expectation in probability?
Comment: No, I meant what if you changed the game so the second box is either half or 3/2 etc...and I probably do don't you think?
Comment: Ok given that, I see what you're getting at. I guess the crux of the issue then is why is expectation suddenly meaningless? In the couple probability and game theory texts I've read, expectation was always exactly the way to, well, predict what value you'd expect. It seems like the expectation is no longer doing that.
Comment: It's not the case, the line of reasoning is wrong for the expectation. Look here: https://en.wikipedia.org/wiki/Two_envelopes_problem which explains exactly what I've been trying to tell you. The issue is you are mixing different instances of the variable in the expected value equation. Also:http://en.wikipedia.org/wiki/Expected_value
Comment: Your explanation is wrong as noted in that wikipedia page. Look at the 'alternative interpretation' section where examining the first envelope is considered: "the proposed 'common resolution' above breaks down, and another explanation is needed." However, that wikipedia page's alternative explanations work well. Since you found that page, I would accept an answer that pointed there.
Comment: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/9704/discussion-between-alian-and-akroy)


Plist vs SQLite vs Core Data for a rss reader type application?

Question: I'm developing an application similar to an RSS news reader. I need some mechanism to store the parsed XML locally. I tried using Plist but it soon became enormously complicated as the complexity of the application grew.
I'd like to know what would be the best method to store data for an application in which one frequently needs to read/write data to the device ?
Answer: Plist should be used for simple and limited data needs requirements that are somewhat static.  They are good in PickerViews for example.  SQLite and CoreData offer true data access and manipulation, and if data is changing, data integrity.  Data models can be normalized, and dynamic relationships maintained between entities.  In addition, complex queries can be performed with relational joins.
I have published applications where I used SQLite, and while more because of iOS SDK (pre-Core Data), the standard SQL syntax made the effort less taxing.
My latest applications, SureSafe Home Inventory completely uses CoreData.  CoreData generates and uses a SQLite data base.  Core Data is a layer above the SQLite code I had to write in my earlier App.  While the initial learning curve can be a bit tough, it is well worth the change over!!  I spent maybe 2 months reading everything and playing with Apple sample code to become somewhat proficient.  But it takes care of so much of the headache of good database coding.  There is even database model versioning, so if you have an upgrade later you can add new db fields and migrate existing customer data.
Long Winded...  but if you are starting new, go with CoreData.  I have no affiliation with Apple.
Steve S.
Answer: I have found both CoreData and SQLite to be fairly simply to use while allowing for the type of complications you have hinted at.  Depending on what you want to store, one will be better than the other.  For what you are doing, I would guess SQLite would probably be best.  See these questions for further discussion between the two:

Core Data vs SQLite 3
Core Data vs. SQLite for SQL experienced developers
Answer: Using a plist is actually the simplest form of the 3 you mentioned. If you plan on having a large amount of data to store, I would suggest using a plist, versus CoreData and SQlite.
Alternatively, you can download the XML filesto your Documents directory and parse them accordingly.
Comment: Really, for a large amount of data you'd suggest a plist instead of Core Data or SQLite?
Comment: Maybe I read the question wrong. The complexity of CoreData and SQlite versus Plist is greater. Storing a large amount of data I would go with CoreData.


Hardness of discrete log in additive group

Question: Why is discrete log considered to be hard in multiplicative group but not in additive group?
Answer: It's a matter of how field elements are represented. Elements of large finite fields are usually represented as vectors over a prime field with components in ${\mathbb Z}/(p)$. So addition is easy and is just addition mod $p$. Multiplication is also not difficult and just involves working modulo a minimal polynomial.
The discrete log problem is to express a given element (represented as a vector) as a power $w^k$ of a primitive element of the field $F$, where $0 \le k < |F|-1$. This is believed to be difficult (i.e. not solvable in polynomial time), although there has been so much work done on algorithms and fast implementation, that you need to go up to very large fields before it starts to become noticeably slow.
For smaller finite fields, elements are often represented as $w^k$, in which case discrete log and multiplication are very easy, but addition is harder.
Comment: For finite fields with small characteristic the discrete logarithm happens to be less difficult than thought before, see [A quasi-polynomial algorithm for discrete logarithm in finite fields of small characteristic](http://eprint.iacr.org/2013/400).
Comment: Somehow you forgot to mention that the "discrete log" in the additive group is otherwise known as "division".


KM versus Cox model

Question: This question is in general, but I would also like to know how the answer pertains to building the model in R.
I have some time to event data and want to do some survival analysis for it. For sake of example imagine that the response is some time to death value and that I have three groups A, B, and C.  Now if I plot a KM curve for this data, given the fact that I have 3 groups, I will get three curves (one for each group).
No I am fitting the following model in R
model = coxph(Surv(time,censor) ~ group, data=death.data)

where group is a categorical variable with 3 levels pertaining to the three different groups: A, B, or C.
I am using the following command to plot the survival curves
plot(survfit(model,xlab ="Time",ylab="Survival"))

however, this only plots one curve.  Is this what I should expect?  Really I wanted three curves but am only getting one.
Does this imply that I need to build three survival models?  One for each group, for example one model for group A versus other, one model for group B versus others, etc.?  And if so, could I plot them all on the same graph and this would be equivalent to the KM curve representation?
Answer: What survfit() does if you do not specify a newdata argument is to try to give you a survival curve of an "average" individual. This is usually a bad idea because it is not obvious what you get out of it.
In your case you should specify this way:
plot(survfit(model, newdata = data.frame("group" = c("group1", "group2"))) and in that case you will get two survival curves, one for an individual with group = "group1" and one for an individual with group = "group2".
For example I generated some data:
times <- rexp(30)
status <- rep(1, 30)
groups <- rep(c(1,2,3), 10)

dat <- data.frame(groups, times, status)
dat[dat$groups==1, "times"] <-  dat[dat$groups==1, "times"]+6

Then I fit the Cox model and i try to get the survival plots:
mod1<- coxph(Surv(times, status) ~ groups, dat)

plot(survfit(mod1)) # an "average" individual, in this case the same as someone from group 2
plot(survfit(mod1, newdata = data.frame("groups" = 1)))
plot(survfit(mod1, newdata = data.frame("groups" = 2)))
plot(survfit(mod1, newdata = data.frame("groups" = 3)))

# Plot all three survival curves in one plot
plot(survfit(mod1, newdata = data.frame("groups" = c(1,2,3))))

Comment: Thanks for this. What if you also had other covariates in your model, but you just wanted to get an overall predicted survival curve for the groups? In the help file for survfit.coxph() it says: "a data frame with the same variable names as those that appear in the coxph formula."
Comment: For the sake of argument let's say I had the above model plus a factor covariate with 4 levels and an additional continuous covariate. How would I specify a newdata data.frame in order get an 'average' individual on those other covariates, but still create 3 different survival curves for group? I hope this makes sense. I can't find the answer anywhere and i'm not sure if this warrants a separate post.

KM versus Cox model


New to java: I'm not sure how to store the objects in the arrays after it is created. The program gets rid of it, making it null

Question: The main objective was to have a lake and add fish objects to it, which have details about their weight etc. But however, every time a new fish is created, the previous object appears as null instead of its location, which I could use my getName to get the name.
package com. company;

public class Main {

    public static void main(String[] args) {
        Lake l = new Lake("jersey", 1, 2, 3,4 );
        l.addFish("carp", 20, 1);
        l.addFish("josh",20,1);

        Lake n = new Lake("york",1,2,4,20);
        n.addFish("catfish",20,30);
        n.addFish("tot",20,30);
        n.addFish("salmon",20,30);
        n.addFish("shrimp",20,30);
        n.addFish("swordfish",20,30);

    }
}

package com. company;

import java.lang.reflect.Array;
import java.util.Arrays;

public class Lake {
    private String name;
    private double length;
    private double width;
    private double depth;
    private final int amountOfFish;
    private int currentFishAmount = 0 ;

    Lake(String name, double length, double width, double depth, int amountOfFish)
    {
        this.name = name;
        this.length = length;
        this.width = width;
        this.depth = depth;
        this.amountOfFish = amountOfFish;
    }

    public void addFish (String name, double length, double weight)
    {
        Fish fishInLake []  = new Fish[amountOfFish];
        fishInLake[currentFishAmount]= new Fish(name, length, weight);
        System.out.println("Lake: " + this.name +  "|" + "Fish:" + ""+ fishInLake[currentFishAmount].getName() + fishInLake[currentFishAmount].getLength());
        System.out.println(currentFishAmount);
        currentFishAmount++;
        System.out.println(Arrays.toString(fishInLake));
    }

}
package com.company;

import java.lang.reflect.Array;

public class Fish {
    private String name;
    private double length;
    private double weight;

    Fish (String name, double length, double weight){
        this.name = name;
        this.length = length;
        this.weight = weight;
    }

   public String getName(){
        return this.name;
   }

   public double getLength(){
        return this.length;
   }

   public double getWeight(){
        return this.weight;
   }

}

The main objective was to have a lake and add fish objects to it, which have details about their weight etc. But however, every time a new fish is created, the previous object appears as null instead of its location, which I could use my getName to get the name.
Comment: In your own words, at the top of the `Lake` class, where you write `private String name;`, what do you think that does? Can you think of a way to use the same technique to make the `fishInLake` stay around?
Comment: In your own words, when you `import java.lang.reflect.Array;` and `import java.util.Arrays;`, what do you think these are for? What happened when you tried omitting them?
Comment: The `fishInLake` array is created every time you add a new fish. With arrays you need to know up front how many items you would like to store in them. Try an ArrayList instead and add the fishes to it.
Comment: @KarlKnechtel thanks for the help. But I really can't think of how to do that. I tried to work around it but It still resulted in the same result.
Answer: First off, to make your output a bit more readable, add a toString() method in your Fish class:
 public String toString(){
    return this.name;
  }

Next, since you want the array to be saved every time a new fish is added to the lake, we want to make the array a private instance variable, and then initialize it in our constructor:
package com. company;

import java.lang.reflect.Array;
import java.util.Arrays;

public class Lake {
    private String name;
    private double length;
    private double width;
    private double depth;
    private final int amountOfFish;
    private int currentFishAmount = 0 ;
    private Fish[] fishInLake;

    Lake(String name, double length, double width, double depth, int amountOfFish)
    {
        this.name = name;
        this.length = length;
        this.width = width;
        this.depth = depth;
        this.amountOfFish = amountOfFish;
        this.fishInLake = new Fish[amountOfFish];
    }

    public void addFish (String name, double length, double weight)
    {
        fishInLake[currentFishAmount]= new Fish(name, length, weight);
        System.out.println("Lake: " + this.name +  "|" + "Fish:" + ""+ fishInLake[currentFishAmount].getName() + fishInLake[currentFishAmount].getLength());
        System.out.println(currentFishAmount);
        currentFishAmount++;
        System.out.println(Arrays.toString(fishInLake));
    }
}

Comment: And if you have another question feel free to post another question and just let me know here so I get notified (can't guarantee I'll be able to answer it tho lol)
Comment: Thanks man, here's my other question: https://stackoverflow.com/q/68673063/16602632
Comment: I posted my answer :)
Answer: better to create a static method or a static Array, otherwise every time that tries to insert data, the previous data will be disappearing, with a static Array the whole data will be there until program execution finish.


Time Dummies and Time Trend in the same equation

Question: Can we run this regression:
$Y_{it} = a + bX_{it} + c_2Time_2 + ... + c_TTime_T + ht + U_{it}$
$i = 1,2,..., N;$ $t = 1,2,3...,T$
Where

$Time_T$ are time dummies
$t$ is the time trend.

In other words, can we run a regression with both time dummies and and a time trend? I suspect that we cannot, but am looking for a more formal explanation.
Thank you!
Answer: The way you have written the model some parameter will be unidentified hence the answer is: NO.
To see this consider the model without covariates  $x_{it}$ without loss of generality
$$(1) \ \ y_{it} = a + \sum_{t=2}^T c_t T_t + h \cdot t + \epsilon_{it}$$
for $t=1,...,T$.
For $t=1$ the term $th = h$ which we can add to the constant $a$ to get $b:=(a+h)$ hence the model can be rewritten
$$ y_{it} = b + \sum_{t=2}^T c_t T_t + h \cdot (t-1) + \epsilon_{it}$$
such that the time trend $h \cdot (t-1)$ is now $0$ for time period 1 for which the model includes no dummy. For the other periods the value of the time trend $h \cdot (t-1)$ is simply added to the dummy coefficient $c_t$ to get $\tilde c_t := c_t + h \cdot (t-1)$ allowing us to rewrite model
$$y_{it} = b + \sum_{t=2}^T \tilde c_t T_t  + \epsilon_{it}.$$
Because the model with time trend has been shown equivalent to a model without the timetrend the timetrend itself is not identified.
You can easily simulate this in R to see what happens in a model where you simulate (1) and then tries to estimate it
set.seed(1)
library(data.table)
T <- 5
N <- 1000
a <- 1
c <- c(0,abs(rnorm(T-1)))
h <- 2

id <- sort(rep(1:N,T))
time <- rep(1:T,N)

dt <- data.table(id=id,time=time)
setkey(dt,id,time)

dt[,y:=a+c[time]+h*time,by=id]

# pooled panel data regression
lm(y~as.factor(time)+time,data=dt)
a + h
c[2:T] + h*(2:T-1)

resulting in the output
Call:
lm(formula = y ~ as.factor(time) + time, data = dt)

Coefficients:
     (Intercept)  as.factor(time)2  as.factor(time)3  as.factor(time)4
           3.000             2.626             4.184             6.836
as.factor(time)5              time
           9.595                NA

> a + h
[1] 3
> c[2:T] + h*(2:T-1)
[1] 2.626454 4.183643 6.835629 9.595281

where the coeffient on time is not estimated and the constant term is $a+h$ and the time dummy coefficients are given by the formula $c(t) + h(t-1)$.
(note: I have simulated the model without error $\epsilon_{it}$ but the result are the same with error terms)
Answer: Yes you can and probably should. The trick is to properly identify a minimally sufficient model that may include events, level shifts, seasonal pulses AND ARIMA structure.
Comment: right but the order of the various tests one does matters. that's why I think it's best to forget about time dummies at the start and first figure out the issue of trend stationary, difference stationary or just plain stationary ? – mlofton yesterday
Comment: the tests for one versus the other are done simultaneously ... thus the "most probable solution "  via F tests is selected .. .. potentially leading to a mix'n match .
Comment: yeah, my "style" is just to look at things one at a time rather than throwing everything in at once but that's just me. statistically, you're right that things should be tested simultaneously. good point.
Comment: AUTOBOX does look at things/options/improvements one at at a time and then identifies the best of these options and proceeds.
Comment: Hi: When you put $h \times t$ in the model, you're assuming that there's a time trend in addition to the effect of the regressors $X_{i}$. I  would google for "trend stationary versus difference stationary" and possibly test that assumption of trend stationarity. This response doesn't cover the issue of the time dummies which is a different discussion altogether
Comment: I think that you may be missing the subtlety that trends don't always..start at the beginning there may be multiple trends which can be found via Intervention Detection schemes.........
Comment: Possibly but I don't think that means that you automatically include a trend term ? does it ?
Comment: Only if significant ..
Comment: Prepare a new question that has two  two  time trends an i will respond. report your  data


Symfony controller not redirecting

Question: I have this in controller
$this->redirect($this->generateUrl('success_template'));

return 'other template stuff...'

routing.yml
success_template:
    path: /success
    defaults:
        _controller:  FrameworkBundle:Template:template
        template:     'MyDefaultBundle:Default:success.html.twig'

Everytime i go to home page my nomral template appears instead of redirecting to success template which is simple hello world template
Answer: You have to return (the response for redirect):
return $this->redirect($this->generateUrl('success_template'));

Symfony controller not redirecting


Output printed to terminal even with ob_start() on when script is executed from command line

Question: I wrote a little command line script to process a document [a markdown file with lilypond musical notation inserts, just for completeness' sake].
#!/usr/bin/env php
<?php

$body = "";

...
// text gets processed here and stored in $body
...

ob_start();
include 'template.php';
file_put_contents(
    __DIR__ . '/' . str_replace('.md', '.html', $argv[1]),
    ob_get_flush()
);

template.php
<!DOCTYPE html>

  ...

  <?php echo Markdown($body); ?>

When I call:
$ ./phlily source.md

File gets generated properly, but the template contents are printed to console too:
GNU LilyPond 2.14.2
Processing `/Users/.../phlily/ly/4add05a74d249f34b3875ef6c3c1d79763927960.ly'
Parsing...
Converting to PNG...
<!DOCTYPE html>

...

and it's annoying because I want to read errors and warnings from the LilyPond script, being them buried behind the html wall in terminal.
Long story short, is it possible to shut the output buffer up in CLI environment?
Comment: What's inside `template.php`?
Answer: I think you want ob_get_clean() instead of ob_get_flush():
ob_get_clean:

ob_get_clean — Get current buffer contents and delete current output buffer

ob_get_flush:

ob_get_flush — Flush the output buffer, return it as a string and turn off output buffering

In this case, "flush" means "send to stdout".
file_put_contents(
    __DIR__ . '/' . str_replace('.md', '.html', $argv[1]),
    ob_get_clean()
);

Comment: Thank you! So easy to find, should have figured myself.


Layer KML usando OpenLayers3

Question: Estou iniciando em Openlayers e estou com um problema na visualização de polígonos em mapas.
Tenho um conjunto de polígonos sendo retornados de uma consulta no banco de dados (postgres/postgis). Este conjunto de polígonos é passado via json por uma requisição ajax para o frontend onde eu pretendo usar a biblioteca OpenLayers3 para exibir esses polígonos em um mapa.
Exemplo do retorno como KML:
{st_askml=<MultiGeometry><Polygon><outerBoundaryIs><LinearRing><coordinates>-53.460269936027402,-24.954734829872194 -53.460072335178054,-24.954744520125182 -53.460093771307605,-24.955104793997403 -53.460291372725941,-24.955095103672289 -53.460269936027402,-24.954734829872194</coordinates></LinearRing></outerBoundaryIs></Polygon></MultiGeometry>}
{st_askml=<MultiGeometry><Polygon><outerBoundaryIs><LinearRing><coordinates>-53.459896169834188,-24.955114484062349 -53.459874734273619,-24.9547542101181 -53.459701833437066,-24.954762688648493 -53.459723268499758,-24.955122962655768 -53.459896169834188,-24.955114484062349</coordinates></LinearRing></outerBoundaryIs></Polygon></MultiGeometry>}

Qual seria a melhor forma de criar a camada vetorial com esses polígonos utilizando OL3?
Salvando o conjunto de polígonos em um arquivo e utilizando este arquivo como source para a camada?
Usando o próprio json como source para a camada? (existe esta possibilidade?)
e como ficaria a sintaxe dessa layer em OL3?
Answer: Vamos assumir que seus polígonos sejam compostos por listas de coordenadas lat x lng, por exemplo:
var a =[
    {
        "lng": 106.972534,
        "lat": -6.147714
    },
    {
        "lng": 106.972519,
        "lat": -6.133398
    },
    {
        "lng": 106.972496,
        "lat": -6.105892
    }
]

A primitiva geom.Polygon() aceita coleções onde o último item da lista é equivalente ao primeiro (para que seu polígono seja propriamente fechado) e onde latitude e longitude são definidas em uma coleção de valores primitivos ([lng, lat]).
Você precisa fechar o seu polígono, usando as mesmas coordenadas iniciais como finais:
a.push(a[0]);

Após isso, transforme a coleção inicial de membros complexos para coleções primitivas:
var b = a.map(function(item) {
    return [item.lng, item.lat];
});

Após isso, crie o objeto via inicialização:
var polygon = new ol.geom.Polygon([b]);

Finalmente, para mostrar o polígono você deve utilizar um wrap do tipo feature() e adicioná-lo a uma layer de vetores, que é então adicionada ao mapa:
var feature = new ol.Feature(polygon);

var vectorSource = new ol.source.Vector();
vectorSource.addFeature(feature);

var vectorLayer = new ol.layer.Vector({ source: vectorSource });

map.addLayer(vectorLayer);

(Adaptado de https://stackoverflow.com/questions/27210362/open-layers-3-how-to-draw-a-polygon-programmically)

Layer KML usando OpenLayers3


How can I solve a Bessel equation with Reduction of order?

Question: If $y_{1}(x) = \frac{\sin(x)}{\sqrt(x)}$ is one solution of the differential equation $$x^2y'' +xy' + (x^2-\frac{1}{4})y = 0$$
find the second solution $y_{2}(x)$.
My effort using Wronskian
The general form of the homogeneous linear 2nd order differential equation is :
$$  y'' + p(x)y' +q(x)y =0$$
The given differential eqaution is a Bessel equation with $n =\frac{1}{4}$ :
$$x^2 y'' +xy' + (x^2 - \frac{1}{4})y = 0$$
Rearranging :
\begin{align*}
x^2 y'' +xy' + (x^2 =\frac{1}{4})y = 0 \\
 y'' +\frac{x}{x^2}y' + \frac{(x^2 -\frac{1}{4})}{x^2}y = 0 \\
  y'' +\frac{x}{x^2}y' + \frac{(x^2 -\frac{1}{4})}{x^2}y = 0 \\
   y'' +\frac{1}{x}y' + (1 - \frac{\frac{1}{4}}{x^2}) y = 0 \\
      y'' +\frac{1}{x}y' + (1 - \frac{1}{4x^2}) y = 0 \\
\end{align*}
Therefore $$p(x) = \frac{1}{x}$$
Using Abel's Theorem we have :
$$y_{2}(x) = y_{1}(x) \int \left( \frac{ e^{-\int p(x)dx}}{ y_{1}^2(x)} \right) dx$$
Substituting :
$$\begin{align*}
y_{2}(x) &= \frac{sin(x)}{\sqrt(x)} \int \left( \frac{ e^{-\int \frac{1}{x}dx}}{ \left(\frac{sin(x)}{\sqrt(x)}\right)^2} \right) dx \\
&= \frac{sin(x)}{\sqrt(x)} \int \left( \frac{ e^{-log(|x|)}}{ \left(\frac{sin^2(x)}{\sqrt(x)^2}\right)} \right) dx \\
&= \frac{sin(x)}{\sqrt(x)} \int \left( \frac{ |x|}{ \left(\frac{sin^2(x)}{x}\right)} \right) dx \\
&= \frac{sin(x)}{\sqrt(x)} \int  \frac{ x}{ sin^2(x)|x|}dx \\
&= \frac{sin(x)}{\sqrt(x)} -  \frac{ x cot(x)}{|x|} \\
&= \frac{sin(x)}{\sqrt(x)} -  \frac{ x cos(x)}{|x|sin(x)} \\
&= \frac{cos(x)}{\sqrt(x)}
\end{align*}$$
The Wronskian matrix :
$$W = \begin{bmatrix}
y_{1} & y_{2} \\
y_{1}'  & y_{2}'
\end{bmatrix}$$
becomes :
$$W = \begin{bmatrix}
 \frac{\sin(x)}{\sqrt{x}}& \frac{\cos(x)}{\sqrt{x}} \\
\frac{\sin'(x)}{\sqrt{x}}  & \frac{\cos'(x)}{\sqrt{x}}
\end{bmatrix} \neq 0$$
Thus the determinant is :
\begin{align*}
W(x) &= \frac{\sin(x)cos'(x)}{\sqrt{x}} - \frac{\sin'(x) cos(x)}{\sqrt{x}} \\
&= - \frac{sin^2(x) + cos^2(x)}{\sqrt(x)} \\
&= - \frac{1}{\sqrt(x)}
\end{align*}
With Reduction of order
Given that $$y_1(x) = \frac{\sin(x)}{\sqrt{x}}$$ is a solution to the Bessel equation $$x^2y'' + xy' + (x^2 - \frac{1}{4})y = 0$$, i need to find the second linearly independent solution, denoted as $y_2(x)$.
I'll use the method of reduction of order. Assuming $y_2(x) = v(x)y_1(x)$, where $v(x)$ is a function to be determined.
I have:
$$ y_2(x) = v(x)y_1(x) = v(x)\frac{\sin(x)}{\sqrt{x}} $$
Now, I find $y_2'(x)$ and $y_2''(x)$:
$$ y_2'(x) = v'(x)y_1(x) + v(x)y_1'(x) $$
$$ y_2''(x) = v''(x)y_1(x) + 2v'(x)y_1'(x) + v(x)y_1''(x) $$
Substituting the above into the given differential equation:
$$ x^2y_2'' + xy_2' + (x^2 - \frac{1}{4})y_2 = 0 $$
i have :
$$ x^2(v''(x)y_1(x) + 2v'(x)y_1'(x) + v(x)y_1''(x)) + x(v'(x)y_1(x) + v(x)y_1'(x)) + (x^2 - \frac{1}{4})v(x)y_1(x) = 0 $$
I substitute $y_1(x) = \frac{\sin(x)}{\sqrt{x}}$ and its derivatives:
$$ x^2(v''(x)\frac{\sin(x)}{\sqrt{x}} + 2v'(x)\frac{\cos(x)}{\sqrt{x}} - v(x)\frac{\sin(x)}{2x\sqrt{x}}) + x(v'(x)\frac{\sin(x)}{\sqrt{x}} + v(x)\frac{\cos(x)}{\sqrt{x}}) + (x^2 - \frac{1}{4})v(x)\frac{\sin(x)}{\sqrt{x}} = 0 $$
Now, i want to choose $v(x)$ such that the terms involving $v(x)$ cancel each other out. I can do this by setting the coefficient of $\frac{\sin(x)}{\sqrt{x}}$ to zero. This gives me the following differential equation for $v(x)$:
$$ x^2(v''(x) + 2v'(x)\frac{1}{x} - v(x)\frac{1}{2x}) + x(v'(x)\frac{1}{\sqrt{x}} + v(x)\frac{1}{\sqrt{x}}) + (x^2 - \frac{1}{4})v(x) = 0 $$
How I proceed further ? Any help ?
Comment: You didn't differentiate $\dfrac{\sin(x)}{\sqrt{x}}$ correctly.
Comment: @CheeHan where? can you be more specific?In which method ?
Comment: You claimed that $y_1' = \frac{\cos x}{\sqrt{x}}$ but you need to use the Product Rule to compute $y_1'$.
Answer: Let $y_2=\frac{\sin(x)}{\sqrt x}u$ be a solution. Then
$$ y_2'=\frac{2 x \sin (x) u'(x)+u(x) (2 x \cos (x)-\sin (x))}{2 x^{3/2}}$$
and
$$ y_2''=\frac{u(x) \left(\left(3-4 x^2\right) \sin (x)-4 x \cos (x)\right)+4 x \left(x \sin (x)
   u''(x)+u'(x) (2 x \cos (x)-\sin (x))\right)}{4 x^{5/2}}. $$
Now putting them in the equation gives
$$ x^{3/2}(2\cos(x)u'+\sin(x)u'')=0 $$
which gives
$$ \frac{u''}{u'}=-\frac{2\cos(x)}{\sin(x)}. $$
Integrating both sides gives
$$ u'=\frac{c_1}{\sin^2(x)} $$
from which one has
$$ u=-c_1\cot(x)+c_2. $$
Choose $c_1=-1,c_2=0$ and then $u=\cot(x)$. So the second solution is
$$ y_2=\frac{\sin(x)}{\sqrt x}\cot(x)=\frac{\cos(x)}{\sqrt{x}}. $$
Comment: Sorry, it is a small mistake. Thanks
Answer: $\textbf{Hint:}$ Rewrite the differential equation using
$$v = \sqrt{x}y \implies \begin{cases} xy' = \sqrt{x}v' - \frac{1}{2\sqrt{x}} v \\ x^2y'' = \sqrt{x^3}v'' -\sqrt{x}v' + \frac{3}{4\sqrt{x}}v\end{cases}$$
which simplifies the differential equation to
$$x^2y'' + xy' + \left(x^2 - \frac{1}{4}\right)y = \sqrt{x^3}v'' + \sqrt{x^3}v = 0$$
or $$v'' + v =0$$
This is a more easily solvable equation now.


Using values from one data file to select rows from another data file

Question: I have data from multiple participants and I am trying to use information from one set of files to aggregate data in another set of files.
I have one data file (videodata.csv) with four columns: Event.Type (video name), Name (start or end), Row (row number), and Source (the participant id code). Set up like this:
videodata <- data.frame(
  Event.Type = c("Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3",
                 "Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3",
                 "Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3"),
  Name = c("start", "end", "start", "end","start", "end",
           "start", "end", "start", "end","start", "end",
           "start", "end", "start", "end","start", "end"),
  Row = c(1, 10, 11, 20, 21, 30,
           8, 16, 19, 28, 1, 7,
           2, 8, 21, 26, 9, 20),
  Source = c("012", "012", "012", "012", "012", "012",
             "013", "013", "013", "013", "013", "013",
             "014", "014", "014", "014", "014", "014")
)

I have like 80 participants, so each of them have another data file in csv format (id_data.csv), with two columns: time and score. Each row corresponds to one second, so the time column is simply numbered from 1 to somewhere around 1,800 or whatever it goes to. score is a binary code of 0 or 1.
For participants 012, 013, and 014 data would look like this:
`012_data` <- data.frame(
  time = c(1:30),
  score = c(0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
)

`013_data` <- data.frame(
  time = c(1:30),
  score = c(0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
)

`014_data` <- data.frame(
  time = c(1:30),
  score = c(0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
)

I want to use the information from videodata.csv to slice up id_data.csv and save them into individual files corresponding to Event.Type.
Specifically, I would look in the videodata.csv file to find the id number. For a given id number, I would start with the first Event.Type (eg Movie1) and find the start row and end row. I would use this information to go to the data file for that id number and select those rows of score and save them to a file called movie1.csv. I would repeat for each participant until movie1.csv contained data from all participants.
For example, for participant 012, I would like to slice up the id_data.csv file into three chunks corresponding to Movie1, Movie2, and Movie3. To do this I would:

I would select rows 1-10 (based on Row which indicates the Start and End) by rounding up to the nearest row number
Save the score column to a file called Movie1 with the column header 012 from the Source column
Repeat for participant 013 and append their score column next to the previous row.
Then repeat for participants 014 and so on through 80.

The resulting file for Movie1 might look like this:

It includes values of score from rows 1-10 (for 012) 8-16 (for 013) and 2-8 (for 014).
I would then have another file corresponding to Movie2 (by selecting rows 11-20, 19-28, and 21-26), and so on for each participant id.
Each participant might have a slightly different number of rows.
This is just beyond my skill and I would appreciate any help as to how I can accomplish this. Thanks!
Comment: @aborruso only R please. It needs to fit into my existing work flow.
Comment: Only r way, or is it possible to suggest some cli tool answer?
Answer: Assuming you have your id data in a directory data\id, and that they all have the same 2 columns time and score, we can read them in all at once with readr::read_csv() while specifying the id argument which will save the file path as a column. We can then split that file path and keep just the part we want to get a Source column that matches the video data.
We can then join the data on the Source column, but only after pivoting both wide. With the data joined, we can then pivot them long and define a new Event.Type column by finding the time with the start and end parameters. Finally we select only the columns we want, and pivot them wider again to get the desired output.
This will still contain all the movies, but you can easily add a dplyr::filter(Event.Type == "Movie1") to get unique files.
library(tidyverse)
library(here)

# Here I take your dummy data and write them as csv's

`012_data` <- data.frame(
  time = c(1:30),
  score = c(0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
) %>% write_csv(here::here('data', 'id', '012_data.csv'))

`013_data` <- data.frame(
  time = c(1:30),
  score = c(0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
) %>% write_csv(here::here('data', 'id', '013_data.csv'))

`014_data` <- data.frame(
  time = c(1:30),
  score = c(0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
) %>% write_csv(here::here('data', 'id', '014_data.csv'))

# Read them back in all at once into a single tibble

id_data <- here::here('data', 'id') %>%
  list.files(full.names = T) %>%
  readr::read_csv(id = 'Source') %>%
  dplyr::mutate(Source = stringr::str_split_i(Source, pattern = "\\/", i = -1),
                Source = stringr::str_remove(Source, pattern = "_data.csv")) %>%
  tidyr::pivot_wider(names_from = time, values_from = score)

videodata <- data.frame(
  Event.Type = c("Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3",
                 "Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3",
                 "Movie1", "Movie1", "Movie2", "Movie2", "Movie3", "Movie3"),
  Name = c("start", "end", "start", "end","start", "end",
           "start", "end", "start", "end","start", "end",
           "start", "end", "start", "end","start", "end"),
  Row = c(1, 10, 11, 20, 21, 30,
          8, 16, 19, 28, 1, 7,
          2, 8, 21, 26, 9, 20),
  Source = c("012", "012", "012", "012", "012", "012",
             "013", "013", "013", "013", "013", "013",
             "014", "014", "014", "014", "014", "014")
)  %>%
  tidyr::pivot_wider(names_from = c(Event.Type,Name), values_from = Row)

df <- dplyr::left_join(videodata, id_data) %>%
  tidyr::pivot_longer(cols = 8:37, names_to = "time", values_to = "score") %>%
  dplyr::mutate(
    time = as.numeric(time),
    Event.Type = dplyr::case_when(
      time >= Movie1_start & time <= Movie1_end ~ 'Movie1',
      time >= Movie2_start & time <= Movie2_end ~ 'Movie2',
      time >= Movie3_start & time <= Movie3_end ~ 'Movie3',
      T ~ as.character(NA)),
    score = ifelse(is.na(Event.Type), NA, score)) %>%
  dplyr::select(Event.Type, Source, score) %>%
  tidyr::drop_na() %>%
  dplyr::group_by(Event.Type, Source) %>%
  dplyr::mutate(id = row_number()) %>%
  tidyr::pivot_wider(names_from = Source, values_from = score)

df
# A tibble: 32 × 5
# Groups:   Event.Type [3]
   Event.Type    id `012` `013` `014`
   <chr>      <int> <dbl> <dbl> <dbl>
 1 Movie1         1     0     0     1
 2 Movie1         2     0     1     0
 3 Movie1         3     0     0     0
 4 Movie1         4     0     0     0
 5 Movie1         5     1     1     1
 6 Movie1         6     0     0     0
 7 Movie1         7     0     0     1
 8 Movie1         8     0     0    NA
 9 Movie1         9     1     1    NA
10 Movie1        10     0    NA    NA
# … with 22 more rows
# ℹ Use `print(n = ...)` to see more rows

Update: My updated answer takes inspiration from @asd-tm. Their answer is overall better, but perhaps you could use bits from both such as file reading from me, and data cleanup from them.
Comment: If you have more than, say 10 movies... Your `case_when` is going to be very heavy to use...
Comment: Yeah, I also realized this, it's a bit quick and dirty. I am happy to edit the answer with any suggestions on how to solve this.
Comment: @Cullen.Molitor your resulting output has NAs at the top row. But the first row (line 1) should be the `score` from row 1 (for 012 data), row 8 (for 013 data) and row 2 (for 014 data). That is, the 'start' row should be the first row for any given Movie. Since each id would have varying lengths of rows, the NAs need to be appended at the end.
Comment: Updated my code to get you there, but @asd-tm has much cleaner logic overall.
Answer: As I assume that the numer of rows in each of the _data.csv files might differ I would suppose to combine the data in the following way:
data_list <- list()
data_list[["012_data"]] <- `012_data`
data_list[["013_data"]] <- `013_data`
data_list[["014_data"]] <- `014_data`

I attract your attention that here is a list not a dataframe because of possible variations of data structure and row count. The same effect can be achieved while reading files in a lapply function applied to list.files.
Then we create the "long" data.frame with data from such files.
rm(dl)

library(dplyr)
library(tidyr)

for (filename in names(data_list)) {
  if (exists("dl")) {
    dl <- dl %>%
      union_all(
        data_list[[filename]] %>%
          mutate(fileName = strsplit(filename, "_")[[1]][1]) # Here we use base R function
      )
  } else {
    dl <- data_list[[filename]] %>%
      mutate(fileName = strsplit(filename, "_")[[1]][1])
  }
}

All these steps can be done at file upload.
dl is long and thus handy for further manipulations:
> dl
   time score fileName
1     1     0      012
2     2     0      012
3     3     0      012
4     4     0      012
5     5     1      012
...

All the rest is done in a single pipe:
videodata %>%
  pivot_wider(names_from = Name, values_from = Row) %>%
  left_join(dl, by = c("Source" = "fileName")) %>%
  filter(time >= start& time <= end) %>%
  select(-c(start, end, time)) %>%
  group_by(Event.Type, Source) %>%
  mutate(id = row_number()) %>%
  pivot_wider(names_from = Source, values_from = score)

You can see the intermediate results executing the pipe stepwise. The pipe returns the following output:
# A tibble: 32 × 5
# Groups:   Event.Type [3]
   Event.Type    id `012` `013` `014`
   <chr>      <int> <dbl> <dbl> <dbl>
 1 Movie1         1     0     0     1
 2 Movie1         2     0     1     0
 3 Movie1         3     0     0     0
 4 Movie1         4     0     0     0
 5 Movie1         5     1     1     1
 6 Movie1         6     0     0     0
 7 Movie1         7     0     0     1
 8 Movie1         8     0     0    NA
 9 Movie1         9     1     1    NA
10 Movie1        10     0    NA    NA
11 Movie2         1     0     0     0
12 Movie2         2     0     0     0
13 Movie2         3     0     0     0
14 Movie2         4     0     0     0
15 Movie2         5     0     0     0
16 Movie2         6     0     0     0
17 Movie2         7     0     0    NA
18 Movie2         8     0     0    NA
19 Movie2         9     0     0    NA
20 Movie2        10     0     0    NA
21 Movie3         1     0     0     0
22 Movie3         2     0     0     0
23 Movie3         3     0     0     0
24 Movie3         4     0     0     0
25 Movie3         5     0     0     0
26 Movie3         6     0     0     0
27 Movie3         7     0     0     0
28 Movie3         8     0    NA     0
29 Movie3         9     0    NA     0
30 Movie3        10     0    NA     0
31 Movie3        11    NA    NA     0
32 Movie3        12    NA    NA     0

The other advantage of this code is that it does not contain any references to movie names and thus scalable without modifications as in other answer. It is robust to differences in number of items selected (see number of elements in Movie1 == 10 and Movie3 == 12). It will also work propwerly if the number of elements for 012 is less then for rightwards columns 013, 014 etc
Answer: I managed to generate the files as you wanted. Please check the comments in the code for a deeper explanation.
I think the last block (the double loop) may not be the most efficient solution in terms of time, but it is the "cleanest" I could come up with. Sometimes is better to sacrifice some performance for the sake of understandability.
### load necessary packages
library(data.table)

### read file from disk (I used your data)
videodata = fread("videodata.csv")

### read individual files from disk. If not exactly 80, adjust
### the number. If the filename pattern doesn't match exactly,
### adjust accordingly. (I used your data)

individualFileNames = paste(sprintf("%03d", 1:80), "data.csv", sep = "_")

### files are read into a **list**
individualData = lapply(individualFileNames, fread)

### we need this because of your naming convention :'(
### In general, it's bad idea to name files starting with a number
names(individualData) = paste(sprintf("%03d", 1:80), "_data", sep = "")

### a function to pad with NAs the vectors of different lengts
padNAs = function(x){
  n = max(sapply(x, length))
  for (i in 1 : length(x)){
    length(x[[i]])= n
  }
  as.data.table(x)
}

### This double loop will go over each event type (movie) and
### each source in it, and create a table of the scores (which
### are obtained from each individual data table). The table
### is then exported as a csv file with the name of the movie.

for (i in videodata[, unique(Event.Type)]){
  j_score = list()
  counter = 1
  for (j in videodata[, unique(Source)]){
    filename = paste0(i, ".csv")
    startRow = videodata[Event.Type == i & Source == j & Name == "start", Row]
    endRow = videodata[Event.Type == i & Source == j & Name == "end", Row]
    j_score[[counter]] = individualData[paste0(j, "_data")][[1]][startRow : endRow, score]
    names(j_score)[counter] = j
    counter = counter + 1
  }
  fwrite(padNAs(j_score), file = filename)
}

Comment: @socialresearcher did this solution work for you?
Comment: @socialresearcher did this solution work for you?


How to use Linq to SQL to update certain fields only of entity?

Question: How to use Linq to SQL to update certain fields only of entity?
ASPX Page:
    Dim pdf As New Dashboard.Process_PDF()
    pdf.ID = ID
    pdf.Label = rtbLabel.Text
    pdf.IsShared = cbxIsShared.Checked
    pdf.AccountID = AccountID

    If RadUpload1.UploadedFiles.Count > 0 Then  '<-- Uploading a new file is optional in Update Form but is required for Insert (Insert works fine)'
        Dim imageFile As UploadedFile = Nothing
        imageFile = RadUpload1.UploadedFiles(0)
        Dim bytes() As Byte = New Byte(imageFile.InputStream.Length - 1) {}
        imageFile.InputStream.Read(bytes, 0, CInt(imageFile.InputStream.Length))
        pdf.FileName = imageFile.FileName
        pdf.FileBytes = bytes
        pdf.FileSize = imageFile.ContentLength
        pdf.ContentType = imageFile.ContentType
        pdf.UploadedBy = RUserName
    End If

    pdf.Update()

Process_PDF.vb - version1
If I have this (below) then nothing happens.  No error.  No update.
Public Sub Update()
    dc.Process_PDFs.Attach(Me)
    dc.SubmitChanges()
End Sub

Process_PDF.vb - version2
If I have this (below) then I get an error on the Attach line:
An entity can only be attached as modified without original state if it declares a version member or does not have an update check policy.
Public Sub Update()
    dc.Process_PDFs.Attach(Me,True) '<--- ERRORS OUT HERE'
    dc.SubmitChanges()
End Sub

Process_PDF.vb - version3
If I have this (below) then it works perfectly IF I update the file also but if I don't update the file I get an error on the SubmitChanges line (no matter what the Update Check settings are set to):
Cannot insert the value NULL into column 'FileName', table 'MyDbName.dbo.Process_PDFs'; column does not allow nulls. UPDATE fails. The statement has been terminated.
Public Sub Update()
    dc.Process_PDFs.Attach(Me)
    dc.Refresh(System.Data.Linq.RefreshMode.KeepCurrentValues, Me)
    dc.SubmitChanges() '<--- ERRORS OUT HERE IF NO FILE UPDATE'
End Sub

Comment: Is your DataContext a singleton, or what's the scope of it? And please specify what errors do you get.
Answer: Well, the cleanest way is to retrieve the entity, update the fields that need to be updated, and call SubmitChanges(). (Creating a new entity, and copying the attributes has the disadvantage that if new fields are added to the entity than you need to update the code, so it's not easy to maintain)+ Anyways LINQ needs to original entity so that it can determine the fields that were updated...
Comment: How do you suggest that I do this using the current code that I posted?  The original entity is already detached and I can't ***not*** detach it because my business and my presentation layer are separate.
Comment: If you have the detached entity in place, than you can use Attach(updatedEntity,originalEntity).
Comment: I accidentally gave you the bounty (which is fine), but your suggestion did not work.  You have all of my code above so I'm not sure what else to give you.  I'm hoping someone can point me to a simple (yet complete) code example of how to update an entity in the scenario that I describe in my initial request.  If I don't have every property in the entity, I still want to be able to update the entity without have to do a one by one reassignment from the current entity to a new entity.  It's not practical when you have 100 fields in your db table so there has to be a way to do this.


Automatic scroll to top doesn't work in UITableView

Question: usually when tapping the top of the screen the tableview scrolls all the way to the top. For some reason this doesn't work in one of my view controllers.
The hirarchy is as follows:
UIView
-> WebView
-> TableView
----->SearchBar
SearchDisplayController.
I think I have everything hooked up correctly (datasource, delegate, ...). I have a similar view controller where everything works. The only difference seems to be WebView, which is missing in the view controller where the tap-and-scroll-to-top works...
Any ideas?
Best regards,
Sascha
Answer: You have probably added some view to your view hierarchy that is a UIScrollView or contains scroll views (e.g. UIWebView or UITextView). When you tap the status bar, iOS searches for the topmost scrollview with scrollsToTop set to YES and scrolls to its top.
Add this to your class:
- (void) disableScrollsToTopPropertyOnAllSubviewsOf:(UIView *)view {
    for (UIView *subview in view.subviews) {
        if ([subview isKindOfClass:[UIScrollView class]]) {
            ((UIScrollView *)subview).scrollsToTop = NO;
        }
        [self disableScrollsToTopPropertyOnAllSubviewsOf:subview];
    }
}

…and call
[self disableScrollsToTopPropertyOnAllSubviewsOf:myAddedSubview];

to solve the problem and let your table view scroll to top again.
Comment: Thank you so much! I couldn't find the problem why my tableview isn't scrolling up as I had only two tableViews, and only one enabled to scrollToTop. Using this function, I found out, that there was a third scrollview, which I was adding as a subview. I totally forgot about that one! :) Thanks!!!!
Comment: This is really helpful for finding the problematic subviews, thanks!
Answer: You need to disable scrollsToTop on all but one of the scrollable views.  Since UITableView is a descendant of UIScrollView, it's easy to do.  However, if you want to disable it on the webview, it's a little trickier.  This seems to work:
[webView.subviews objectAtIndex:0].scrollsToTop = NO;

But that is a little iffy since it's poking around in undocumented parts of the webview.  It may stop working if Apple decides to rearrange the subviews.
Comment: As of iOS 5, the -scrollView property of UIWebView [is exposed](http://developer.apple.com/library/ios/documentation/uikit/reference/UIWebView_Class/Reference/Reference.html#//apple_ref/occ/instp/UIWebView/scrollView).
Comment: Couldn't I simply enumerate all subviews, check which one is a subclass of a scrollview and use the code above on all appropriate subviews?
Comment: Here is what I do (which should even continue to even if Apple decides to rearrange the subviews):  for (int i=0; i < [webView.subviews count]; i++)
 {
  id subView = [webView.subviews objectAtIndex:i];
  if ([subView isKindOfClass:[UIScrollView class]])
  {
   ((UIScrollView *)subView).scrollsToTop = NO;
  }
 }
Answer: Don't know what Apple were smoking but by design, the behaviour on iPhone is distinctly different from iPad.  Apple's documentation does include a "Special Condsideration" note as follows...

On iPhone, the scroll-to-top gesture has no effect if there is more
  than one scroll view on-screen that has scrollsToTop set to YES.

So on iPhone you have to ensure that there is only ever one UIScrollView (or UITableView/UICollectionView/UIWebView etc) that has its scrollsToTop = YES.
Since scrollsToTop defaults to YES you have to explicitly set scrollsToTop = NO for the UIScrollView's you do not want to scroll when the user taps the status bar.
Answer: Apparently if you have more than one scrolling view (in your case the WebView and TableView) in a view controller, the "tap status bar to scroll to top" is disabled.
http://twitter.com/drance/status/2448035250438144
(Matt Drance is a former iPhoneOS developer evangelist and current iOS development rockstar)
Answer: Swift variant of opyh's answer. Upvote him!
func disableScrollsToTopPropertyOnAllSubviewsOf(view: UIView) {
    for subview in view.subviews {
        if let scrollView = subview as? UIScrollView {
            (scrollView as UIScrollView).scrollsToTop = false
        }
        self.disableScrollsToTopPropertyOnAllSubviewsOf(subview as UIView)
    }
}

Answer: I just wanted to add to this, if you are using a UITextView, it isn't the subviews that need the scrollsToTop adjusting, it's the actual UITextView, in short you need this code:
[myTextView setScrollsToTop:FALSE];

This should fix the problem (additionally, the 'disableScrollsToTopPropertyOnAllSubviewsOf' function doesn't work on the UITextViews).
Hope this helps someone!
Answer: I had this problem with just a tableview in the NIB. Very frustrating until I remembered I was programmatically adding a UITextView as the table view header. scrollsToTop=NO on the UITextView did the trick.
Answer: Set scrollsToTop = NO on all scroll views in the view, except for the one you want to scroll to top. That helps the system find the correct scroll view.


How to solve linear congruences as a newbie?

Question: Please, before referring to another problem on the site or giving a link, I would like to say I read most of them such as: This, This, and many others but due to the (subtle) difference in my question, I'm having a really hard time to apply the methods used in other questions.
We yesterday had our first lecture about Number Theory. I have been trying to work on this problem since yesterday evening.
$$16a + 17b + 18c \equiv 19\pmod{100}$$
It's given that $ 1 \leq a,b,c \leq 99$ and that $a = 95\ and\ c=11$. We want to know what $b$ is.
I know that the answer is $53$ ( I used the naive way to get the answer) but I fail to do it according to the official methods. Can someone please explain this?
I tried this: ( i filled both a and c in)
$1718 +17b \equiv 19\ (mod\ 100)$
$17b \equiv 19-1718\ (mod\ 100)$
$17b \equiv -1699\ (mod\ 100)$
$$ x \equiv \frac{-1699}{17} \equiv \frac{-1699}{17} \equiv \frac{1}{17}\ i\ got\ stuck\ here$$
I also tried to work with it simplified:
$$17b \equiv 1 (mod\ 100)$$ I thought i was almost done here since i was able to write it like this:
$$b \equiv \dfrac{1}{17} (mod\ 100)$$
So i thought there is some $b$ number if i divide it by a $1/17$ i get $n$ as an answer and $100$ as rest but even doing so yielded in a wrong answer. Can someone please help?
Comment: See https://math.stackexchange.com/questions/25390/how-to-find-the-inverse-modulo-m
Comment: @labbhattacharjee I just applied the extended Euclidean algorithm such that $1s+100t = 1$ where t,s are integers. I found s = -99 , t =1. So that $1b \equiv 1 \pmod {100}$ but it was wrong for some reason
Comment: I don't think there are special ways for newbies, and no official ways. The (extended) Euclidean algorithm is just the most handy tool.
Comment: @ProfessorVector Could you please show me how The (extended) Euclidean algorithm will apply here. I tried and failed as i commented above
Comment: Sure. [Extended Euclidean algorithm](https://www.wolframalpha.com/input/?i=ExtendedGCD%5B17,100%5D) will give you the result $\{1, \{-47, 8\}\}$, meaning the GCD is $1$, and $(-47)\cdot17+8\cdot100=1$. And that means $1/17=-47=53\pmod {100}$.
Comment: @ProfessorVector I finally understand how the algorithm works and i was able to understand everything you did till " And that means ". May i ask how did you conclude that?
Answer: HINT.-As a newbie you could act as follows:
First at all, note that $17$ is invertible modulo $100$ because it is not divisible by $2$ nor by $5$ so $17b \equiv 1 (mod\ 100)$ has solution.
Since $17n=17+17\cdots+17$ (n times), you could add successively $17$  plus $17$ until you get a number of the form $100x+1$ (this requires to be some patient having into account you know the answer is $53$).
Another shorter way is to add successively $100$ adding $1$ and dividing by $17$ until you have an exact quotient which is precisely your answer.This way requires just $9$ times instead of $53$.


SVN 1.7.1 Issue

Question: I have just upgraded to SVN 1.7.1 and I have had nothing but problems trying to get my netbeans and everything else to work with it. I have tried 'svn upgrade' but I get an error which is as follows:

C:\wamp\Projects>svn upgrade BMPortal
svn: E155019: Can't upgrade 'C:\wamp\Projects\BMPortal' as it is not a
  pre-1.7 working copy directory
  svn: E150000: Missing default entry

Could anyone suggest a way to fix this. The reason why I am trying to upgrade is because my netbeans says:

The Path 'C:\wamp\Projects\BMPortal' appears to be part of a
  Subversion 1.7 or greater working copy. Please upgrade your Subversion
  client to use this working copy.

As I have upgraded my SVN Client to 1.7, I don't know why it is still moaning at me.
Just FYI:
The SVN Server is on an Arch Linux box running UberSVN.
The client (my computer) is Windows 7 64bit.
Answer: I actually had the very same problem (or at least the same error message!).
After trying everything you have said earlier and not getting any results, I realise that I had copied some files to the project that contains svn files from an older (1.6) version. I delete the 1.6 folders and the everything worked fine again.
I hope this could help some other people too!
Best regards,
Andrés
Comment: how did you find the old folders?
Answer: Your Netbeans installation seems to use another SVN-Client as the one when you type svn on the console.
So you have to upgrade your Netbeans-Subversion plugin too.
Comment: Thanks for the help, it was a mixture between yours and the above post that helped! If I could accept two I'd accept your post as well ^_^
Answer: Sorry to waste your time guys, I have finally found a work around for this, http://netbeans.org/projects/versioncontrol/pages/Subversion1_7
But thanks again all the same! It's much appreciated!
Answer: SVN v1.7.0 and above uses a new working copy file format, so you need to upgrade your working copy to the new format.  If you right-click the folder in Windows Explorer, the TortoiseSVN context menu items should show only "SVN Upgrade working copy" - click that to upgrade it.
Warning: If you use any other SVN apps as well as TortoiseSVN on the same PC, they may not work with the new working copy file format.
It's worth upgrading in my opinion though - must faster, better merging, better dialogs and error reporting, etc!
Comment: Sorry, misread your question - the error you're getting appears to say that your working copy is *already* in the v1.7 file format...
Comment: Could it be that your netbeans can't communicate to the newer Tortoise?  Is there a newer version of netbeans you could try?
Comment: Okay so if my working copy is already 1.7 then why does it say I need to upgrade my svn client? Im just so confused now ¬_¬
Comment: I'm reinstalling NetBeans now, It's not the best solution, I don't like having to reinstall things, but oh well. i'm sure It's not the end of the world. I'll let you know how it goes

SVN 1.7.1 Issue


how can I prove this rigorously by using Mathematical analysis? $(|f'(x)| \le1 \iff |f(s)-f(t)| \le |s-t|)$

Question:
For given $f$ which is differentiable in $\mathbb R$,
(a) $\forall x \in \mathbb R, |f'(x)| \le 1$
(b) $\forall s,t \in \mathbb R, |f(s)-f(t)| \le |s-t|$
Show that (a) and (b) is iff.

How can I show that (a) and (b) is iff by using mathematical analysis? I can prove this by using MVT, but I want to know the rigorous proof.
Comment: what is the difference between "I can prove this" and a "rigourous proof"? And its not an iff, consider $f(x)=|x|$
Comment: (a) implies (b) uses the mean-value theorem. (b) implies (a) is generally false unless you assume $f$ is differentiable everywhere.
Comment: IThat is my fault, sorry.
Comment: I was not familiar with MathJax, so there was a mistake in the process of writing.
Answer: (b) implies (a).
Assume
(i) $f$ is differentiable at $x$
(ii) $\forall s,t \in \mathbb R, |f(s)-f(t)| \le |s-t|$.
Let $h \in \mathbb R$.  By (ii)
$$
\left|\frac{f(x+h)-f(x)}{h}\right|
=\frac{|f(x+h)-f(x)|}{|(x+h)-x|} \le 1
$$
and by (i)
$$
|f'(x)|=\left|\lim_{h\to 0}\frac{f(x+h)-f(x)}{h}\right|
=\lim_{h\to 0}\left|\frac{f(x+h)-f(x)}{h}\right| \le 1
$$


Match element with specific element not working JList ActionListener

Question: I'm working on a program that takes an array of contacts and I'm trying to return an integer of how many people are online. I already have a button called refresh that is implemented. I press the "refresh" button on a JFrame and it will refresh the JLabel onlineCountNum which is a empty string at first. After pressing refresh, it shows up 1 to include the user that was online. However, when I log in another user and pressed the refresh button, it doesn't increment but stays at 1 for that user. The JList listContacts has elements with either the name or name(Online). I'm trying to count how many of those have name(Online). If the user is the only one on the server, it should show 1 as the result. If there are two people on the server, it should show 2 for both people and both would see another person as name(Online) on a pane.
private JLabel onlineCountNum;     //label gets refreshed by button
private JLabel onlineCountText;    //"Online Count: " label
private JButton onlineCountUpdate; //refresh button

onlineCountText = new JLabel("Online Count: ");
onlineCountNum = new JLabel("ERROR");
onlineCountUpdate = new JButton("Refresh");
onlineCountNum.setText("");

onlineCountUpdate.addActionListener(new MyButtonListener6());

class MyButtonListener6 implements ActionListener{
    public void actionPerformed(ActionEvent e){
        int count = 1;
        ListModel list = listContacts.getModel();
        for(int i = 0; i < list.getSize(); i++){
            if(list.getElementAt(i).equals(list.getElementAt(i)+"(Online)")){ //possible source of error
                count++;
            }
        }
        onlineCountNum.setText(String.valueOf(count));
    }

}

Answer: if(list.getElementAt(i).equals(list.getElementAt(i)+"(Online)"))

Yes, this is the source of error...
Now, suppose:
list.getElementAt(i) == "someName(Online)"

What you check for, in the if statement, is:
list.getElementAt(i) == list.getElementAt(i)+"(Online)"

Which is like checking:
"someName(Online)" == "someName(Online)(Online)"

So it will always return false.

What you have to do is:
if(list.getElementAt(i).endsWith("(Online)"))

Comment: I edited my code accordingly and I'm presented with this error, cannot find symbol, when compiling. It refers to the .endsWith method.
Comment: Actually, I found my mistake. I forgot to specify the type of JList. However, your code still works! Thanks!


Utilizar Redis na prática

Question: Estou com dificuldades em perceber na prática onde usar o Redis em um ecommerce, por exemplo.
Estou lendo um livro mas não consigo perceber o que tenho que levar em conta na hora de decidir usar Redis ou não.
O Redis deve ser usado para dados estáticos?
No caso do ecommerce, utilizar Redis para gravar dados do login ou carrinho de compra e depois gravar definitivamente em um banco como MongoDB ou MySql, etc...?
Answer: Amigo, eu utilizo Redis em um ecommerce e acho que posso te dar uma ideia bem simples e prática da sua utilização. Antes de usar o Redis eu utilizava memcached então vamos tentar entender em que ponto isso me ajudou.
1º Minha loja tinha muito acesso e eu não poderia toda vez que o usuario fosse no catalogo ou na página de detalhe carregar o produto pelo banco de dados então eu criei uma implementação de "cache" bem simples e comum que é:
1º O usuario pede a pagina de produto de id 14325
2º Meu adapter de dados da minha aplicação ve se existe um chave no redis chamada product_id_14325 (o nome eu estou definindo), caso exista eu retorno o conteudo do redis, ou seja, não consulto no banco.
3º Caso a chave não exista no redis então eu busco no meu banco de dados e o retorno eu passo para o usuario, serializo ele e persisto no redis. Pronto todo o usuario que acessar essa pagina de detalhe ela fica cacheada.
Existem outras abordagens como por exemplo criar um processo para gravar todos seus produtos no cache e deixar com expiration de 1 dia, e pequenas atualizações para produtos modificados, mas isso depente da sua estratégia de cache, o que importa é que o redis tem uma seria de utilidades.
Comment: Obrigado pela dica, não percebo porque alguém deu -1, entendo pouco de Redis ainda mas a dica é válida sim, as informações do produto não se altera constantemente.
Comment: Sabe de algum cenário onde se usa o Redis para manter estados de autenticação? Ou seja, o usuário loga e algo é persistido e lido no Redis nas próximas operações? Tenho um cenário que quero fazer autenticação com JWT e usar o Redis mas não sei se se aplica ao caso.
Answer: Bancos NoSQL, entende-se "Not only SQL":

Esses surgiram da necessidade de escalar bancos de dados relacionais com propriedades ACID em projetos web de alta disponibilidade que operam em larga escala. Suas principais características são alta performance, escalabilidade, fácil replicação e suporte a dados estruturados.

Este rompimento com os padrões SQL causa sempre grande repercussão e muitas discussões carregadas de sentimentos e emoções, mas a verdade é que os bancos de dados relacionais ainda servem para resolver muitos problemas que nem sempre (veja bem, nem sempre) poderão ser resolvidos com bancos NoSQL, como por exemplo:

Necessidade de forte consistência de dados, tipagem bem definida, etc;
Pesquisas complexas que exigem um modelo relacional dos dados para realizações de instruções e operações de junção, por exemplo;
Dados que excedam a disponibilidade de memória do servidor, por mais que possamos utilizar swap, ninguém quer prejudicar a performance neste caso.

NoSQL Redis:
Armazenamento de sessões de usuários

Este é um modelo muito simples de como utilizar o Redis para salvar as informações da sessão de um usuário.
Para cada sessão, gera-se uma chave que é gravada no cookie do navegador. Com essa chave, o sistema tem acesso a um hash com informações desta sessão: status do login, produtos e publicidades clicadas, preferências de idioma e outras configurações temporais, que perdem a validade após algumas horas.

O benefício de não guardar tais informações de sessão diretamente no cookie é evidente: ganhamos a segurança de integridade dos dados, não correndo o risco de algum usuário malicioso modificá-los. Com o Redis, utilizamos operações simples de get/set para acessar estes dados diretamente da memória do servidor (ou servidores, caso exista mais de um), sem desperdício de recursos, graças ao eficiente sistema de expiração promovida por este NoSQL.
O algoritmo de expiração não monitora 100% das chaves que podem expirar. Assim como a maioria dos sistemas de cache as chaves são expiradas quando algum cliente tenta acessá-la. Se a chave estiver expirada o valor não é retornado e o registro é removido do banco.
Em bancos que gravam muitos dados que perdem a validade com o tempo, como neste exemplo, algumas chaves nunca seriam acessadas novamente consequentemente elas nunca seriam removidas. Essas chaves precisam ser removidas de alguma maneira, então a cada segundo o Redis testa um conjunto randômico de chaves que possam estar expiradas.  O algoritmo é simples, a cada execução:

Testa 100 chaves com expiração setada.
Deleta todas as chaves expiradas.
Se mais de 25 chaves forem inválidas o algoritmo recomeça do 1.

Esse lógica probabilística continua a expirar até que o nosso conjunto de keys válidas seja próximo de 75% dos registros.
Fonte de pesquisa: Link NoSQL Redis
Comment: Agradeço o seu tempo mas já tinha lido isso, o que eu queria era na prática saber como aplicar em um ecommerce, esse texto acima é como estudar teoria para montar uma bicicleta, na prática você vai cair!
Answer: Eu administro uma plataforma de desenvolvimento de sites e nela usamos Redis e MySQL.
O MySQL, obviamente, armazena todos os dados do sistema, inclusive os dados que renderizam os sites. Mas por quê então eu tenho um banco redis junto com esse MySQL? Simples: velocidade.
Por fornecer sites, eu sempre quero que o site abra da forma mais rápida possível, tanto para melhorar o SEO quanto para melhorar a experiência dos usuários, e pra alcançar essa marca, eu utilizo um banco redis pra armazenar os dados que renderizam os sites(que no caso são todos em JSON), já que por ser um banco que roda em memória, o acesso é muito mais rápido do que se o sistema tivesse que, de alguma forma, consultar o sistema de arquivos em busca desses dados. Esse banco redis é atualizado sempre que o cliente faz alguma atualização em algum conteúdo que seja relevante para a renderização do site ou caso a nossa VPS seja reiniciada.
Resumindo, o Redis é extremamente útil, como foi meu caso, para armazenar dados que precisem ser entregues de forma rápida.
Agora imagine essa mesma situação, para um e-commerce.
O cliente vai e cadastra as mercadorias, e esses dados são enviados pra um banco em memória pra ser consumido pelo seu client. Isso, entre muitas coisas, melhoria a experiência dos visitantes do site pois reduziria bastante o tempo de carregamento dos dados de produto, já que ao invés de ter que consultar o sistema de arquivos ou uma api, os dados já estariam disponíveis na memória do servidor. Isso deve ser muito levado em consideração, já que sistemas mais lentos tem chances maiores de perder acessos por conta de carregamentos demorados, que podem causar erros de timeout, por exemplo.

Utilizar Redis na prática


Find area of a crescent

Question: Find the area of the shaded region that is enclosed by a full circle and a quarter circle inscribed in a square.

My math teacher gave me this question as a challenge. But I can't seem to solve it. Can you guys help? Please tell me the steps to solve it.
Thanks.
Comment: Welcome to MSE. Please read this text about [how to ask a good question](https://math.meta.stackexchange.com/questions/9959/how-to-ask-a-good-question).
Comment: You need to present this as a better question rather than a rough diagram in an image.
Answer:
Assume unit square. The area is $A_{square}-A_{purple}-A_{green}-2A_{grey}$, i.e.
$$Area
=\frac34 - \frac{3\pi}{16} -2A_{grey} \tag1
$$
where $A_{grey}$ can be obtained by evaluating the areas of the circular sectors ABP, OPE, and the triangles AFP and FPO; then subtracting them from the half square, as done below.
Let the vertex A being the origin, the polar equation of the two circles are $r=1; \space \space r(\sin\theta + \cos\theta)=r^2+\frac{1}{4}$. Eliminate $r$ to obtain $\sin\beta = \frac{5+\sqrt{7}}{8}$ and then
$\sin\alpha =\frac{\sqrt{7}-1}{4}$. The area of two triangles AFP and FPO is
$$A_q = \frac{1}{4}\sin\beta +  \frac{1}{4}\left(\frac{1}{2}-\cos\beta \right)=\frac{2+\sqrt{7}}{16}
$$
and the area of the two circular sectors is
$$A_c=\frac{1}{2}\left( \frac{\pi}{2}-\beta\right) + \frac{\alpha}{8}
=\frac{\pi}{8}-\frac{1}{8}\sin^{-1} \left(\frac{23\sqrt{7}+67}{128}\right). $$
As a result,
$$A_{grey}=\frac{1}{2}-A_c-A_q=\frac{6-\sqrt{7}-2\pi}{16}+\frac{1}{8}\sin^{-1} \left(\frac{23\sqrt{7}+67}{128}\right) $$
Substitute into (1) to obtain
$$Area = \frac{\pi+2\sqrt7}{16} - \frac{1}{4}\sin^{-1} \left(\frac{23\sqrt{7}+67}{128}\right)$$
Comment: Impressive! BTW, I got the same result as yours
Answer: Assuming unitary square, the demicircles have equations
$$y=\frac{1}{2} \left(2 \sqrt{x-x^2}+1\right);\;y=\sqrt{1-x^2}$$
Intersection point is at $x_1=\frac{1}{8} \left(5-\sqrt{7}\right)$
Intersections with the diagonal are at $$x_2=\frac{1}{\sqrt{2}};\;x_3=\frac{1}{4} \left(\sqrt{2}+2\right)$$
The requested area is twice the area colored in the picture below that is
$$A/2=\int_{{x_1}}^{{x_2}}\left(\frac{1}{2} \left(2 \sqrt{x-x^2}+1\right)-\sqrt{1-x^2}\right)\,dx+\int_{{x_2}}^{{x_3}}\left(\frac{1}{2} \left(2 \sqrt{x-x^2}+1\right)-x\right)\,dx$$
$$A=\frac{1}{16} \left(2 \sqrt{7}-5 \pi +16 \arcsin \left(\frac{1}{8} \left(5-\sqrt{7}\right)\right)+8 \text{ arcsec}\left(\frac{2 \sqrt{\sqrt{7}+5}}{3}\right)\right)\approx 0.14638$$

$$..$$

Find area of a crescent


Recognize outliers in a set as data collected

Question: Suppose I am aggregating data at multiple granularities, where each key is associated with the number of counts collected during the time interval.
EX:
1 minute {
x:2,
y:5,
z:3,
a:312
}
This is a small example, the real set will have anywhere from 1000-6000 unique keys associated with the number of hits collected in a time interval.
Is there a method that will immediately allow me to recognize that a member(s) of the set is an outlier? If not, what are some efficient algorithms to recognize outliers for data of this sort?
Comment: Can you give more details about the null hypothesis? i.e. What do you know about the "typical" distribution of counts?
Comment: This is somewhat tricky, as I am unable to assume anything about the data, or how the counts are aggregated - For instance, the aggregation function could be a function of time elapsed in the interval. I am attempting to create a tool to handle data aggregation for realtime streams, and provide various interfaces into the data - In this case, reporting a set free of outliers, or reporting a maximum which is not an outlier, within a reasonable time frame, as this is meant for realtime visualization.
Comment: For others that may encounter a problem such as this... Have a look at:https://www.tulips.tsukuba.ac.jp/dspace/bitstream/2241/106312/1/LNCS_5181.pdf
Comment: If you have no assumptions, what could constitute an outlier?
Comment: Glen_b: it's obvious: observations that don't follow the pattern of the bulk of the data! @Jordan Dillon Chapian: so this is 4-variate data, right?
Answer: Your data comes from an unknown distribution $f:\mathbb{Z}_+^d \rightarrow \mathbb{R}$ where d>1000. If you cannot assume anything about this distribution then there is no way to detect outliers.
One weak and relatively standard assumption you can make is that the distribution has bounded finite differences. i.e. if $|\bf x_1 - \bf x_2|$ is small then $|f({\bf x_1}) - f({\bf x_2})|$ must be small.
In that case, it is possible in principle to use nonparametric density estimation such as Kernel Density Estimation to estimate the density at each new data point (given all the previously recorded points) and give an outlier alert if this density lies below a threshold. This approach is easy to implement and needs only a reasonable choice of distance metric, kernel function and kernel width. The width can be estimated via cross-validation and the choice of kernel usually does not matter too much.
For a much simpler procedure, you can just compute the distance to the nearest sample. If this distance is large then you have an outlier.
However, given that the dimension is high, you probably won't get good results using both of these methods due to the curse of dimensionality - accurate nonparametric density estimation requires a number of samples exponential in the dimension.
I can think of several solutions to this problem:

Use nonparametric density estimation on each key separately. Create an outlier alert if one of the keys is an outlier.
Use nonparametric density estimation on each key separately. Estimate the density using the product of marginal densities and alert if this result is low. (i.e. take the product of estimated densities for all keys)
Apply a dimension reduction algorithm such as PCA to go down to 2-3 dimensions and use nonparametric density estimation in that space.


Get-GPO: Value does not fall within the expected range

Question: I've been trying to count how many GPO files in DCs From DC-A to DC-B ( They are in a trusted domain) . I got this error:
Get-GPO : Value does not fall within the expected range.
At C:\Temp\HealthCheck\ADReplication.ps1:70 char:18
+     $gpoCount = (Get-GPO -all -Server $dc).count
+                  ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Get-GPO], ArgumentException
    + FullyQualifiedErrorId : System.ArgumentException,Microsoft.GroupPolicy.Commands.GetGpoCommand

it works with other dcs, but few of them getting this error. I wish to know why it happens and to have a solution for this.
Thank you!
Comment: Are the working and non-working DCs in different domains?
Comment: I tried to run this in different domain and I got this error. When I run it in the same domain the it works just fine. Do you think it obstructed by some credential needed?
Comment: Are you saying that you are getting errors when using `DomainA\User` to run the `Get-GPO` command on a DC for `DomainB`? If so, is it in a shared forest, or between a trusted pair of domains? Please update the question with this info.
Comment: @CoryKnutson You'r right, Edited!. I also got an answer that it's not able to query the GPO from the Domain A to Domain B.
Comment: Are you able to view Group Policy objects from the other domain through the GUI console?
Comment: @CoryKnutson Yes, I do
Answer: This is reproducible in my environment to a trusted domain. I do not have a direct fix, but would this be usable for you? This would require having credentials to the remote forest, but I would assume that you would have them.
Invoke-Command -Credential $cred -ComputerName $server -ScriptBlock {Get-GPO -All}

Comment: Thanks. Due to security issue, I don't really want to address any credential :).
Answer: The only exception to this rule is cross-domain GPOs where a GPO is defined in another domain, but is linked in such a way that clients from neighboring domains need to apply them (a Site-level GPO is an example, since Active Directory Sites can span multiple domains).  In the case of a cross-domain GPO, the client will need to pull content from a DC in the neighboring domain which can be a very slow process.  For this reason, cross-domain GPOs are not generally recommended - From TechNet