Must there be an invertible matrix $Q$ such that $Q^{-1}MQ=N$?

Question: Let $M$ be a complex square matrix such that it has the following:
Minimal polynomial: $\mu(\lambda)= (\lambda-3)^2(\lambda+2)^3(\lambda-5)$
Characteristic polynomial: $\chi(\lambda)=(\lambda-3)^4(\lambda+2)^5(\lambda-5)^2$
$\dim(E_3)=2$ and $\dim(E_{-2})=3$, where $E_\lambda$ is the eigenspace for $\lambda \in \mathbb{R}\,$.
Now, my question is this:
If $N$ is another complex square matrix with the same characteristic polynomial, minimal polynomial, and geometric multiplicities as stated above, must there be an invertible matrix $Q$ such that $Q^{-1}MQ=N$ ?
If this is true, I would like to understand why. And if not, a counter-example would be much appreciated.
Answer: All of the information that you have given can be interpreted as a description of the Jordan normal form of $M$.  In particular:
Characteristic polynomial:

$M$ has size $\deg(\chi) = 11$
The sum of the sizes of the Jordan blocks associated with $\lambda = 3$ (i.e. the number of times $3$ appears on the diagonal) is $4$. This sum is $5$ for $\lambda = -2$ and $2$ for $\lambda = 5$.

Minimal polynomial:

The largest block associated with $\lambda = 3$ has size $2$. Respectively, $3$ for $\lambda = -2$ and $1$ for $\lambda = 5$.

Algebraic multiplicity:

There are $2$ Jordan blocks associated with $\lambda = 3$ and $3$ for $\lambda = -2$.

Let $J_k(\lambda)$ denote the Jordan block of size $k$ associated with $\lambda$, and let $\oplus$ denote a direct sum.  Verify that with the information given above, the Jordan form of $M$ must be (up to a permutation of the blocks)
$$
J = [J_2(3) \oplus J_2(3)] \oplus [J_3(-2) \oplus J_1(-2) \oplus J_1(-2)] \oplus [J_1(5) \oplus J_1(5)].
$$
Two matrices are similar if and only if they have the same Jordan form. However, any matrix $N$ that has all of the listed characteristics must also have this Jordan form. It follows that for any such $N$ is similar to $M$, i.e. there is a matrix $Q$ for which $
Q^{-1}MQ = N$.
Comment: @FireFenix777 It says that the characteristic polynomial and minimal polynomial being the same isn't **always** enough, which is why we had to interpret the information given to make sure that it really is enough in our case.
Comment: @FireFenix777 In other words, we could find pairs of non-similar matrices that have the same characteristic and minimal polynomials and the same geometric mulitplicities, but we can't find two non-similar matrices that have the particular polynomials and multiplicities listed in your question.
Comment: I'm confused, doesn't the post you linked Hanno say that this need not be true for $n >6$?
Answer: Warning: This answer is incorrect, as currently given.
Correction is underway.
If you are given both the characteristic and the minimal polynomial of a matrix, then you know its Jordan canonical form (which is usually not unique because permutations of Jordan blocks are still allowed).
A matrix $M$ and its Jordan normal form $J$ are similar, which means $Q^{-1}MQ=J$ for some invertible matrix $Q$.
Your question is answered in the positive, because $M$ and $N$, sharing characteristic and minimal polynom, then also share the Jordan normal form, and both $M,N$ are similar to it. Now recall that similarity is an equivalence relation.
If you strive for uniqueness, then look after the Frobenius normal form of a square matrix.
Comment: Actually the minimum polynomial does not fix the Jordan form but only the size of the largest Jordan block for each eigenvalue (as told in Ben Grossmann's answer).
Comment: @Hanno Your answer is wrong. See [this post](https://math.stackexchange.com/a/577948/81360).
Comment: @Ben Grossmann Thanks for the feedback. Will reconsider it tomorrow.


Interbase 2009: complete removal?

Question: Are there any tools for completely removing an Interbase 2009 SMP installation? I found an article describing how to remove IB6 (http://edn.embarcadero.com/article/28131), but I'm reluctant to delete anything in the registry according to a document which applies to another version of IB. Are there any equivalent descriptions for IB2009?
Answer: I guess you've run the uninstall and just want to clean up the registry.
You could try CCleaner. select registry and scan for Issues.
If its the first time you've run it there will probably be many unused entries.
If you're worried about changes then unselect all and just choose the ones related to IB.
Make sure you choose "Back up changes to registry"
Comment: Thanks 10bithacker. It doesn't seem to pop up with any loose interbase settings, though. I'm afraid I have to scan the registry "manually".
Answer: Try RevoInstaller. A very good program to cleanup register and files after an uninstall. Note that you should run this program directly when uninstalling. It calls the applications uninstall routines and the cleanup afterwards.

Interbase 2009: complete removal?


J. S. Bach's place in musical history

Question: Was there any reason Bach didn't follow the trends of the times? He was surely very much in touch with contemporary composers, and knew of Handel's and Scarlatti's works specifically. Amazingly, he composed not a single opera in his life. Did his influences lead him to focus on perfecting the existing High Baroque style, or were there other reasons for not following the rapidly changing style? I am tempted to say that as a very cerebral man, the Baroque style (in particular the fugue) better suited him, but I know few of the musicological facts here. Also notable is the high volume/proportion of religious music Bach composed, when the tendencies were increasingly secular everywhere. (He composed many great masses, despite the lacks of operas.)
J. S. Bach was positioned at an interesting period in musical history, without doubt. Born when the Baroque style was still predominant and flourishing, most of his later life represents the beginning of the Classical period. In fact, his birth year of 1685 was shared with both G. F. Handel and Domenico Scarlatti, the latter in particular who adopted (and indeed helped develop) the early Classical style. It is perhaps less surprising then that Bach's style was often deemed "turgid" and somewhat outdated in his own day. There is no doubt he was a "culminator" rather than an "initiator" of styles however. His true genius was recognised and his works acclaimed from around the start of the Romantic period.
Comment: I swapped your paragraphs around to get to crux of the question. This edited version sounds more "Q&A"-like and less soliloquy. Feel free to rollback if you don't agree.
Comment: @Robert: I did ramble a bit, so fair enough! I'm happy to leave it this way. (There's a valid question in there either way, but perhaps it's more obvious now.)
Answer: Much of Bach's education in composition came from him making hand copies of many musical scores, as music was not mass-printed at the time.  Having mastered the Baroque notions of counterpoint and developing motives, combined with his known skill at improvising in the style, why would he seek to give up mastery of a style to be mediocre at a new one?
As to the nature of his works, such as not writing operas, Bach spent much of his life employed by Lutheran churches, where he was obligated by his position to supply service music.  Bach went above and beyond at this--at his Leipzig post alone, he wrote cantatas for every Sunday service, as well as for every church holiday.  Much of his secular music such as the Brandenburg Concertos originated while he was instead serving as a court music director for the Prince of Anhalt-Köthen, who used simpler music for worship services due to being Calvinist, leaving Bach to focus on the secular end of things during that and other similar periods.  Based on the religious beliefs of both Bach and his employers, it is also probable opera was considered far too worldly.
So between religious and teaching obligations and just being so good at what he was already doing, I rather think Bach didn't really take the time to ever worry about what was "in style."  He certainly wanted to continue his musical growth, exploring different techniques and learning from other musicians, but I don't think he spent any real effort at following trends to the extent of some other long-lived composers.
Comment: I think there's some truth in this, indeed. Bach was no doubt an intellectual and probably followed his mind and spirit more than his job or the fashion. (Not that he was ever really poor.) As far as I know however, Bach is generally only considered to have become the ultimate master of counterpoint in his later life's works (post 1720-ish)... But yeah, some useful info in this answer, so +1 and thanks.
Comment: Bach could express himself perfectly well through his style; the new galant style didn't offer anything to him.  If you compare the music of the galant or early classical period vs the music of the baroque, history is on Bach's side.
Answer: If you take a look at the St John's Passion by Bach, it is surprisingly operatic with its use of turba choirs and the kind of introductory chorus and the dying scenes.
It is true that Bach was employed by churches for large stretches of his time (as opposed to his time in Köthen, for example), including his final years.
But if you take a look at his magnum opus, the last completed major work before his death, it was the Mass in B Minor, a mass in old Catholic Rite, utterly unsuitable for performance in either Protestant (his own faith!) or Catholic churches of the time.  He was dead longer than he had been alive before it was actually performed in full.
So when he wrote a work close to the end of his life that was absolutely without commercial interest (since it was unperformable for his life time and any foreseeable future), it still was a deeply religious work, with at best loose connections to the churches of his time.
True, the next large work he started (and never finished) was "Die Kunst der Fuge", not inherently a religious work.
But he clearly loved the music he was doing (and had a lot of trouble with his employers when stepping over the lines of musical conventions while doing his job).  And it was clear that to him, the potential of that music was still not exhausted.  Many musicians keep doing weaker copies of the stuff they became famous for when they age.  Bach did some of his deepest works right before his death.
Answer: You could say the same about Beethoven.  Rossini was off inventing 19th-century opera, and Weber was applying those operatic innovations to instrumental music, and what was Beethoven doing?  Putting fugues in his major works.  Beethoven's late style was completely uninfluenced by the new trends, and even became a dead end until it influenced later composers like Brahms and Bruckner.
On the other hand, take a look at Luigi Cherubini.  Listen to his first Requiem (1816).  Strictly old-fashioned.  Cherubini spent the next 20 years thwarting Hector Berlioz in every possible way.  Now listen to the second (1836).  It sounds like a copy of the Berlioz Requiem.  Clearly Cherubini saw a reason to follow the trend of the times, if Bach and Beethoven did not.
Many of the composers we respect most highly do not bring their music to our level.  We must go to their level.
Comment: I'm not one to down-vote answers my own questions, but I'm afraid this doesn't really answer the question! Also, I have to disagree about Beethoven. Beethoven was a superb culminator *and* initiator. He was one of *the* founders of the Romantic style in music. Also, he heavily influenced Schubert, a contemporary of his later life. Mozart would be a better example of a pure culminator.
Comment: The answer to the question would have to be "We'll never know why Bach didn't follow the trend of the times.  It's too late to ask him."
Comment: Most great people don't follow trends. Einstein, Edison, you name it.
Comment: Also, Mozart and Beethoven, while both having unique styles (obviously), were more or less in agreement with the contemporary musical styles.
Comment: Unless he was already asked and it was written down. At minimum I would expect there to be some discussion of this in a musicological publication.  Even if it was educated speculation, it's better than "I don't know."

J. S. Bach's place in musical history


Given initial positions and velocities of two boats, do they collide?

Question: This is a homework question from a precalculus class that I'm a TA for.

Boat $A$ is initially at position $(1,4)$ and moves at a constant velocity $\langle 3,5 \rangle$. Boat $B$ is at position $(7,2)$ and moves at a constant velocity of $\langle 1,10 \rangle$. Do the paths of the boats ever cross? If so where? Will the boats collide? If they don't collide, what's the closest the boats get to each other?

I wanted to write up a thorough solution to this exercise for my class, and figured I'd post it online to help anyone else who may wander across it.
Comment: What is it that you want your students to practice? The simplest solution (I think) is to move to a frame in which one of the boats is at rest at the origin and see if the resulting forward ray from the other boat reaches the origin, i.e., that the difference in positions is a negative multiple of the difference in velocities.
Comment: @amd That's probably the simplest solution computationally, but I don't think it's very intuitive. In the situation *both* boats are moving, so within the calculations both boats should be moving too. The students are struggling to become comfortable thinking in terms of vectors anyways, and I don't think they are ready to make the mental leap of shifting the frame of reference from the origin to one of the boats. I'll probably share this thought with any of the students who have a good grasp on working with vectors though, because it is a very useful thought.
Answer: Let $P_A(a)$ denote the position of the boat $A$ at time $a$, and let $P_B(b)$ denote the position of boat $B$ at time $b$. From the initial positions and velocities given, we have:
$$
\begin{align}
  P_A(a) &= (1,4) + a\langle 3,5 \rangle
  &\qquad
  P_B(b) &= (7,2) + b\langle 1,10 \rangle
  \\
  &= (3a+1, 5a+4)
  &\qquad
  &= (b+7,10b+2)
\end{align}
$$
Now these equations give the paths of the boats starting at time $a=b=0$. The paths of the boats cross only if at some time $a$ and some time $b$ after each starts moving they have the same position. In terms of those equations, the paths of the boats will cross if there are positive times $a$ and $b$ such that $P_A(a) = P_B(b)$. Now the boats collide if not only is there a location where their paths cross, but if they are at that location at the same time. So the boats collide if $P_A(a) = P_B(b)$ for some positive $a$ equal to $b$. So we can proceed by setting $P_A = P_B$:
$$
  (3a+1, 5a+4)
  = (b+7,10b+2)
\implies
\begin{cases}
3a+1=5a+4
\\
b+7=10b+2
\end{cases}\
\implies
\begin{cases}
3a-b=6
\\
5a-10b=-2
\end{cases}\,,
$$
This, being a system of linear equations, has at most a single solution, which we can calculate to be $a = \frac{62}{25}$ and $b = \frac{36}{25}$. These are both positive times, so the paths of the boats do cross, but since this is the only solution and $a \neq b$, the boats do not collide. To find the actual coordinates  where they do cross will be the location of boat $A$ at time $a=\frac{62}{25}$ (which should equal the location of $B$ at time $b=\frac{36}{25}$ if we've done our calculations correctly), which we can calculate:
$$
P_A\left(\frac{62}{25}\right)
= \left(3\cdot\frac{62}{25}+1, 5\cdot\frac{62}{25}+4\right)
= \left(\frac{211}{25} , \frac{410}{25} \right)\,.
$$
To figure out how close the boats get, we can write a function to represent the distance between the boats at a time $t$ and minimize that function. We now need to consider the boats in the same time-frame and let $a=b=t$. The distance between boat $A$ and boat $B$ is given by
$$\begin{align}
d(t) &= \sqrt{(3t+1-t-7)^2+(5t+4-10t-2)^2}
\\   &= \sqrt{29t^2-4t+40}
\end{align}$$
The minimum of the function $d$ will occur at the minimum of the quadratic $29t^2-4t+40$ since the square root is a strictly increasing function. And the minimum of that quadratic occurs at $t = \frac{4}{2\cdot 29} = \frac{2}{29}$. So the actual minimum distance they achieve is $d(
\frac{2}{29}) = \frac{34}{\sqrt{29}}$ miles apart.
Comment: Yeah, that's a very common beginner mistake with parametric equations. It might also be worth pointing out to your students what happens when you do use a common parameter: the correct conclusion is that the boats don't collide, not that their paths don't intersect.
Comment: I would probably write the functions $P_A$ and $P_B$ originally with $t$ as the argument to both, then switch to $t_1$ and $t_2$ when looking for whether they cross.
Comment: Why not use the same time parameter and see if the positions are ever equal? That's a single equation with only one variable to check, basically equivalent to trying to solve the above system by back-substitution. Using two different parameters seems more appropriate when simultaneity isn't important, such as when you're simply computing the intersection of the two lines. As in my original comment to your question, it comes down to what you're trying to illustrate, which isn't clear to me from the question itself.
Comment: @amd Part of the problems *does* ask where (if) the paths of the boats cross though.
Comment: Ah, yes, missed that part of the question.
Comment: @amd Admittedly I don't like having different time parameters in this question either. I've seen all my students' first attempts at answering this question, and for the ones that set up the situation with vector equations and a common time parameter $t$, many concluded that *there is a time $t$ where the $x$ coordinates of the boats are equal, so the boats must collide, and so obviously their paths cross,* ignoring the fact that the $y$-coordinate don't match (if they even calculated the $y$-coordinate).


symfony - sfDoctrineGuard - restricting user creation based on group credentials

Question: I am currently in the process of developing a fairly large and complex user management system using sfDoctrineGuard
I have created 4 groups, editors, moderators, admins and superadmins.
What I'm looking to do, is restrict certain users in the admin to be able to create/view/edit other users in the sfGuardUser admin module.
So for example a superadmins user can create editors, moderators, admins and other superadmins, but a moderator can only create editors.
Is this possible in sfDoctrineGuard, if so, could someone give me an insight on how I'd achieve this?
Thanks
Answer: First of all you can set credentials in generator.yml to show/hide links to actions and object actions based on credentials. For example:
config:
  list:
    object_actions:
      _delete:
        confirm: Вы уверены, что хотите удалить пользователя?
        credentials: superuser
    actions:
      _new:
        credentails: moderator

Next, configure your forms with custom table methods for doctrine choice widgets of groups:
class sfGuardUserForm extends PluginsfGuardUserForm
{
  public function configure()
  {
    //groups_list
    $this->getWidget('groups_list')->setOption('expanded', true);
    $this->getWidget('groups_list')->setOption('table_method', 'getListForAdmin');
    $this->getValidator('groups_list')->setOption('query', Doctrine::getTable('sfGuardGroup')->getListForAdmin());
  }
}

class sfGuardGroupTable extends PluginsfGuardGroupTable
{
  /**
   * Builds list query based on credentials
   *
   */
  public function getListForAdmin()
  {
    $user = sfContext::getInstance()->getUser();

    $q = $this->createQuery('g');

    if (!$user->isSuperAdmin() && $user->hasCredential('moderator'))
    {
      $q->addWhere('g.name IN (?)', array('editor'));
    }
    else if ($user->hasCredential('editor'))
    {
      $q->addWhere('g.name IN (?)', array('editor'));
    }
    return $q;
  }
}

A couple of enhancements: get rid of singletone call by passing user instance from action (in preExecute) and load group names form app.yml with sfConfig::get instead of hardcoding in it in code.
Comment: If I am logged in as a moderator, will I still see all users, or just the moderatoes and editor users in the list?
Comment: You can make a custom list of users based on credentials with the same approach. Create a sfGuardUserTable::getForList method, and specify it in generator.yml > list > table_method.
Comment: Ok, but that will be called no matter what user I'm logged in as. I have 4 user groups, with let's say 2 users in each group. A super user can see all users. But a moderator should only be able to see moderators and editors and an editor should only be able to see other editors.
Comment: No this will be called depending on what user is loggin in, rest assured. Look at the line: "$user = sfContext::getInstance()->getUser();
". Here you get instance of the logged in user. Although this is not a best practice.
Comment: @Dziamid I have tried your code above, but I cannot get it working. I've logged in with a basic account, i.e. has permissions of an editor, but I can still see all users and all groups. It doesn't seem to call the `getListForAdmin` method. Any ideas/help?
Comment: I looked at implementing it, but I have been on vaccation.
Comment: Thank you! I needed it! An editor can `view/edit/create/delete` other editors ONLY. I also need editors to only be able to filter other editors and no other users. A `moderator` can `create/edit/delete/view/list/search` other `moderators` AND 'editors'. They should only be able to filter/search for `moderators/editors`. Current editor permissons are `Create.Editor`, `View.Editor`, `List.Editor`, `Delete.Editor`, `Search.Editor`. Thank you
Comment: Well, just add your logic to getForAdminList function. I suggest you also configure it in app.yml for convenience. Good luck. Filters are configured the same way as forms.
Comment: I do have the `getforAdminList` defined in my model, but it doesn't seem to get called. I had `public function getListForAdmin(){exit;}`, which doesn't exit the call. Also, what do mean about configuring it in the `app.yml`
Comment: @Dziamid would there be any reason why this method doesn not get called, even when I have followed the code that you provided? Thanks
Comment: It should be called: $this->getWidget('groups_list')->setOption('table_method', 'getListForAdmin'); I've tested it on my mashine (2 weeks ago). You really should install xdebug and track what went wrong.
Comment: Hi. I have put this code in: `lib/model/doctrine/sfDoctrineGuardPlugin/sfGuardUserGroupTable.class.php` and in `lib/model/doctrine/sfGuardUserGroupTable.class.php` both give me: `Unknown method sfGuardGroupTable::getListForAdmin`. This is really annoying me now, as I cannot get it woring! :(
Comment: but it does throw an error if i put the code in `sfGuardUserAdminForm`
Comment: I don't know what is troubling you. I've done it again and uploaded a test project on github here: https://github.com/dziamid/sipher. Hope this will help.


Struts2 date parsing

Question: I have a form with a field that uses a date/time picker which produces a hidden field with the full date & time in the 'British' format dd/MM/yy HH:mm through the use of javascript. This is submitted to a struts2 action whch has getters & setters for the field with operate on java.util.Date.
In the JSP:
<s:form>

  ...
  <input type="hidden" name="myDate" value="08/05/11 16:00" />
  <s:submit />

</s:form>

In the action
public class MyAction
{

   public void setMyDate(Date d)
   {
      this.d = d;
   }

   public Date getMyDate()
   {
      return d;
   }

}

As expected in Firefox when I submit the form struts automatically parses the date and I end up with a Date object in the action with the correct date.
However in Google Chrome when I submit the form the date is parsed as if it is in the US format MM/dd/yy HH:mm and as a result I get 5th August instead of 8th May.
Inspecting the HTTP headers sent by both browsers reveales that despite my OS language set to en_GB google chrome still sends the 'Accept-Language' header as en_US which means struts must be using this to determine what date format to use.
I can only assume this is a bug in Google Chrome however given that I don't have the freedom to fix google chrome is there a way in Struts 2 that I can specify a fixed date format which tells Struts the format it should use to parse dates in form parameters?
Comment: Have you tried using `request_locale` parameter set to the desired lang? This parameter sets a session variable `WW_TRANS_I18N_LOCALE` and, for my application, it overrides 'Accept-Language' set language. Hope it helps... http://struts.apache.org/2.1.2/struts2-core/apidocs/com/opensymphony/xwork2/interceptor/I18nInterceptor.html
Answer: In order to avoid misinterpretation of dates caused by different Locale values, I would go with an unambiguous and well-defined method of representing dates and times. This is what the ISO 8601 standard covers.
Unfortunately I don't think Struts2 can convert to Date given this representation "out of the box". For date conversion, by default, Struts2 uses the SHORT format for the Locale associated with the current request.
I guess you could try finding a plugin that already converts ISO 8601 dates or a simple approach would just be to write your own.
Comment: I have gone with creating a custom TypeConverter for the ISO standard, its a bit disappointing struts doesn't allow you to specify an unambiguous format but the TypeConverter solved my problem.
Answer: I recommend using the Joda Time API. It fully supports the ISO 8601 standard and has an extensive API for locales, timezones, conversions, etc. I use it for all of my date-time manipulations in all of my applications (including Struts 2).

Struts2 date parsing


General principles for extending the Elo system to games in which the margin of victory matters

Question: Note
I am a first-time user of this site, and am not sure what determines how broad a question is. I consciously limited the question to general principles in order to avoid making it too broad (by asking for examples, etc.).
Nevertheless, if it is too broad, let me know in the comments, and do suggest how to narrow it down.
The question
I am aware (through chess) of the Elo rating system and how it works. Basically if you know the ratings of two players, you can calculate the expected score of both of them, and adjust their ratings based on the actual score.
However, I was wondering how this system is adapted to work for games in which the margin of victory matters.
Specifically, I was wondering how the rating system can be extended so that one can calculate not only the expected score but also the expected margin of victory or defeat from the ratings of two players/teams.
I'd also expect the system to take into consideration the actual score as well as the actual margin of victory when adjusting the ratings after a match.
A hypothetical example
The actual details needn't be the very same as what I mention here, but the general idea goes like this:
Consider a rating system where Chelsea and Man City have ratings 2000 and 2100. I'm looking for a rating system which not only predicts the score (around 0.64 for City) but also the margin of victory.
Considering that the rating somehow gives us an expected margin of +3.2 for Manchester City, and the team wins 2-0, I'd also expect the system to reduce City's rating for not winning by a large enough margin.
But I do wonder if two variables (expected score and expected margin of victory) are needed or just one (expected score) does the job.
In short
What are the general principles and methods involved for extending the Elo rating system to games where the margin of victory matters?
Answer: A simple version of ELO can be cast as a logistic regression: for players $i,j$ with ratings $R_i,R_j$,
$$P(i\mbox{ beats }j)=\frac{1}{1+\exp(\beta(R_i-R_j))}.$$
So you could just as easily predict score instead by using a different link function, for example a lorentzian or gaussian:
$$P(\mbox{Game score}=x)=a\exp(-\alpha|\beta(R_i-R_j)-x|^\gamma)$$,
where the game score can be positive (in favor of $i$) or negative (in favor of $j$). So you don't need to calculate the probability of beating and just directly optimize the game score.
Comment: That sounds promising. I thought of something similar, where the expected margin of victory $m_e$ is given by $m_e=A({2S_e}-1)$ and $S_e$ is the expected score, calculated as usual $\left(S_e=\frac{1}{1+10^\frac{-d_R}{400}}\right)$. Then, $\Delta R=K(m_a-m_e)$, where $m_a$ is the actual margin of victory. The constants $A$ and $K$ can, I guess, be obtained by some experimentation.
Comment: Could you edit your answer to elaborate a little on how the ratings are updated once the result of the game is known? I'm not sure if I understand fully.
Comment: @HarryWeasley: The most straightforward way to update result for players $(i,j)$ with a new datapoint is to recalculate $R_i,R_j$ by updating the loss function, via gradient descent.
Comment: Sorry to be so bothersome, but I'm not an expert, and didn't quite understand some of the terms (especially 'loss function' and 'gradient descent') you used in your last comment. Could you rephrase the same in simpler terms? :)
Comment: @AlexR. what is the function represented by β?
Comment: @user160104: that's a learned parameter of the model.
Answer: There are some works intended to include the margin of victory in rating system (e.g. FiveThirtyEight for NFL), but usually ranking systems (e.g. Elo, Glicko, or our rankade - here's a comparison) don't incorporate the margin of victory.
In most sports/games the margin of victory is not significant.
In chess the goal is to checkmate your opponent's king (and it doesn't matter how many pieces you and your opponent have on the board when you're able to do this), in basketball - like in most sports - winning 89-60, or 86-85, or 90-23 gives the team just a victory (and the score doesn't matter - except for mostly unused tiebreaker), and so on.

Consider a rating system where Chelsea and Man City have ratings 2000 and 2100. I'm looking for a rating system which not only predicts the score (around 0.64 for City) but also the margin of victory.
  Considering that the rating somehow gives us an expected margin of +3.2 for Manchester City, and the team wins 2-0, I'd also expect the system to reduce City's rating for not winning by a large enough margin.

Opposite to rugby, in which you get a (little) bonus if you score 4+ tries, in soccer City gets same 3 points even if it wins 8-0 (and probably, while leading 4-0, City coach wants their best players to rest for next matches...). Margin of victory could be significant (showing that there's a big difference between teams), but it also could not, for many reasons. And, in a structure in which the goal is winning (no matter for the score), it's not a good idea to build a ranking system that rewards an 'unuseful' large victory (3 points for championship ranking) and subtracts points for a 1-0 win with last team in ranking (same 3 points!).
Eventually, you can somehow reward a bigger than expected win, but you can't 'punish' a team for not winning by a large enough margin. They won, so they did their job.
Sure there are (few) games in which the margin of victory matters, but soccer (and nearly all sports, both with round robin or brackets) is not in this list.
Comment: Thanks for your answer (+1)! I found it quite informative, and liked the idea of the ree algorithm. While I do agree that the margin is insignificant in many games, my question was about the games in which it *actually* matters.
Comment: You're welcome! On rankade you can easily manage this issue (i.e. incorporating margin of victory) using *weight* feature (e.g. *normal weight* for matches with normal margin, *mid-light weight* for low margin, *heavy weight* for high margin), tuning point rewards (that will be always given to the winner, anyway).


por que o g++ não compila o codigo que o visual studio compila?

Question: ja faz alguns projetos que testo e meu g++ não compila de jeito nenhum meus projetos do visual studio, erros que aparecem no g++ não existem no visual studio... alguem sabe o por que? e como voces acham que da pra resolver isso?
output do G++
<code>Loader w Version Checker.cpp:35:2: error: 'b' does not name a type
}   w h i l e   ( P r o c e s s 3 2 N e x t ( S n a p S h o t ,   & p r o c E n t r y ) ) ;
</code>
^
Enquanto no visual studio...
Comment: Não posso responder com certeza porque nunca usei o visual studio, mas pode ser algo relacionado com a linkagem das bilbiotecas. Talvez o visual studio faça isso automaticamente pra você com alguma biblioteca que você esteja usando, enquanto que pelo g++ você teria que fazer isso de maneira explícita.
Comment: pode ser tambem mas a minha solução tambem funcionou então não sei
Answer: O visual studio usa uma plataforma para compilar diferente do g++, focada no windows a MSVC : Microsoft C++ Compiler
enquanto o g++ usa a Compilador c++ baseado na nix ou nix-based C++ compiler
Pra compilar um projeto c++ feito no visual studio teria que baixar o Visual C++ 2015 Build Tools e não o g++


Why am I getting this error?

Question: I am getting this error.

Call to undefined function feeds_node_prepare() in dkan_datastore_node_update() (line 307 of /dkan/dkan/profiles/dkan/modules/dkan/dkan_datastore/dkan_datastore.module).

What could be causing the error? Could it be a module interfering with function?
Comment: Any feedback to the answer?
Answer: This issue is referenced in Dkan developpers Google group.
You probably have updated Feeds modules in Dkan profiles folder? Read the above post carefully and consider trying proposed solution :

Keep the Dkan profile original code in place
Download Feeds 7.x-2.0-beta4 to sites/all/modules/contrib
Rebuild registry

EDIT : I browsed both Feeds and Dkan feeds.module history. I couldn't find any implementation of feeds_node_prepare() in Feeds, while Dkan contrib feeds.module does implement it.
Seems that Feeds in DKAN profile contrib modules directory is a fork, not official release.
Comment: So, I have to put the feeds module in that directory and that should do it?
Comment: Nope. 3 steps : first revert Feeds to Dkan version in Dkan profile, then put last Feeds release in contrib folder, last rebuild registry.

Why am I getting this error?


Como organizar datas no ggplot2

Question: Tenho um gráfico feito com ggplot2, mas na hora de exportá-lo, as datas no eixo X saem totalmente ilegíveis, como nesse exemplo:

Como posso arrumá-las? Código que estou usando:
suppressPackageStartupMessages({
  library(tidyverse)
  library(reshape2)
  library(patchwork)
  library(gghighlight)
  library(zoo)
})

#### Import data ####

url <- httr::GET("https://xx9p7hp1p7.execute-api.us-east-1.amazonaws.com/prod/PortalGeral",
                 httr::add_headers("X-Parse-Application-Id" =
                                     "unAFkcaNDeXajurGB7LChj8SgQYS2ptm")) %>%
  httr::content() %>%
  '[['("results") %>%
  '[['(1) %>%
  '[['("arquivo") %>%
  '[['("url")

dados <- utils::read.csv2(url, stringsAsFactors = FALSE, fileEncoding = "latin1")
dados$data <- lubridate::as_date(dados$data)

dados <- dados %>%
  filter(data > as.Date("2020-02-29"))

rm(url)

dados %>%
  mutate(mm7dCasos = rollmean(casosNovos, 7, fill = list(NA, NULL, NA), align = "right")) %>%
  filter(data > "2020-03-14") %>%
  ggplot() +
  geom_col(aes(x = data, y = casosNovos), na.rm = TRUE, color = "black", fill = "#ADD8E6") +
  geom_line(aes(x =data, y = mm7dCasos), color = "#FF6347", size = 0.75) +
  labs(x = "", y = "") +
  ggtitle("Variação diária de novos casos confirmados - Brasil", "(MM7d)") +
  coord_cartesian(ylim = c(0, 500)) +
  facet_wrap(~estado, nrow = 3)

Comment: Tente `theme(axis.text.x = element_text(angle = 60, hjust = 1))`.
Comment: Não consegui compreender como funciona o ```scale_x_date```.
Answer: Você pode usar o scale_x_date ou alterar o ângulo das datas.
Usando o scale_x_date:
scale_x_date(date_labels = "%b %d", date_breaks = "2 week")

Alterando o ângulo das datas (neste caso, em 45˚ graus):
theme(text = element_text(size = 10), axis.text.x = element_text(angle = 45, hjust = 1))

Código novo:
dados %>%
  mutate(mm7dCasos = rollmean(casosNovos, 7, fill = list(NA, NULL, NA), align = "right")) %>%
  filter(data > "2020-03-14") %>%
  ggplot() +
  geom_col(aes(x = data, y = casosNovos), na.rm = TRUE, color = "black", fill = "#ADD8E6") +
  geom_line(aes(x =data, y = mm7dCasos), color = "#FF6347", size = 0.75) +
  labs(x = "", y = "") +
  ggtitle("Variação diária de novos casos confirmados - Brasil", "(MM7d)") +
  coord_cartesian(ylim = c(0, 500)) +
  scale_x_date(date_labels = "%b %d", date_breaks = "2 week") +
  theme(text = element_text(size = 10), axis.text.x = element_text(angle = 45, hjust = 1)) +
  facet_wrap(~estado, nrow = 3)

Como organizar datas no ggplot2


A derivative problem

Question:
Let $a<c<b,  f:(a,b) \to \mathbb{R}$ be continuous. Assume $f$ is differentiable at every point of $(a,b) \backslash \{c\}$ and $f'$ has a limit at $c$. Then which of the following are true?
1)$f$ is differentiable at $c$.
2)$f$ need not be differentiable at $c$.
3)$f$ is differentiable at $c$ but $f'(c)$ is not necessarily  $\lim_{x \to c}f'(x)$.
4) $f$ is differentiable at $c$ and  $\lim_{x \to c}f'(x)=f'(c)$.

Since $f$ is given to be continuous everywhere and is differentiable at every point except at $c$, I feel option 1) is correct, hence 2) is false.
How to look at 3) and 4)?
Comment: @Naman: The answers to the question that I linked to show that (4) holds. – Also here: https://math.stackexchange.com/q/221273/42969 and here: https://math.stackexchange.com/q/169157/42969 and here: https://math.stackexchange.com/q/257907/42969
Comment: With "$f'$ has a limit point at $c$", you mean that $\lim_{x\to c}f'(x)$ exists?
Comment: but....can the limit $\lim_{x \to c} f'(x)$ be different from $f'(c)$?
Comment: @Hagen von Eitzen yes....I corrected the same
Answer: Note that corners and cusps are out of the question because either would produce a jump discontinuity (of finite length) in the image of $f'$. The remaining options of both a removable discontinuity or a hyperbolic portion are both plausible since they both preserve the notion of at limit at $c$ [keep in mind that the hyperbola would preserve the limit if of odd degree]. This should help you answer your question.

A derivative problem


Lack of sustain on guitar

Question: I am playing a Gibson Les Paul Standard and a Fender Stratocaster through my pedal board into my amp.
I have problems with my sustain on both guitars. I was able to have a nice sustain (for sure with my Les Paul) but recently my guitar fades away almost immediately.
My pedalboard chain looks like this:

Tuner --> Wah --> Whammy --> Tubescreamer --> Overdrive --> Delay --> Reverb

I have tried several combinations, but even with only my OCD enabled (which has a lot of gain), I barely have sustain which annoys me a lot.
Sometimes it just fades away, sometimes it converts into feedback.
Any tips on how to have a much longer sustain or what my problem could be?
Comment: Do any of your effects have a built in noise gate?
Comment: I can't comment yet, so... Just wondering, are *all* your cables good? I've had some funny things happen (including what you describe) with cables that had shorts.
Comment: I have the best of cables. But I guess that wouldn't be a bad idea to test my gear with only my OCD overdrive and see what that does.
Answer: If it's for both guitars, I guess it's probably not the guitars themselves.
Having said that, old strings can cause loss in sustain or if they've both got damp or something. Or muck on the bridge, or if you've changed gauge of strings to lighter ones.
Do they both play with good sustain unplugged ? ie, same as before ?
Assuming the guitars are ok, and your effects board hasn't changed to cause the poor sustain ..

Try swapping out the leads. Sometimes leads do weird things on their way out as the resistance or conections become unreliable. A muting /muffling effect isn't uncommon in my experience.
Don't forget this could also be the patch leads between effects
Check the voltage to your effects board. If it's lower than it should be, this can cause odd thigns to happen too. This is more likely if you use a universal power supply capable of several voltage ranges. Is the issue still there when not connecting through your pedalboard?
Amp. I don't know what kind of amp you have but I have noticed a particular kind of fault where you play a chord or note, it's loud (as usual) for maybe a quarter of a second, then it dies prematurely. Next note played seems to 'peak' through the issue but dies again soo after. I've found this ona  few amps as jam nights and one of my own. I don't know the cure unfortunaely- probably something for a technician to look into.  I guess the test would be to try someone else's amp, ideally one similar to your own.
Have you changed your technique lately ? Maybe it's finger-error ? This seems unlikely but I'm just trying to think of everything

If you want more sustain, a compressor will do that. They smooth the sound out very well ! But I guess that doens't help you find the real cause.
Comment: Does the problem occur when plugged in direct? That would be my first test.
Comment: I have changed my pedal board layout recently (a few times) but I had the problem prior to that. That's why I changed the layout.
Furthermore, my guitars are perfect in shape, I didn't change my playing technique, I use 0.11 strings, they are pretty new and I haven't changed the thickness of them.
For my voltage, I use the Voodoo Lab Pedal Power which should work perfectly. I will however check the voltage.
Comment: I agree with VarLogRant - that, then if no luck try a different amp.
Answer: I would take a trial and error approach. Plug your guitar directly into your amp and test sustain without the effects. Then add in each pedal until you notice your sustain lacking. As Matthew suggested, it's highly probable that one of your pedals has a noise gate built into it that you didn't realize. I bought a multi effects pedal a long time ago, and when I first started using it, it destroyed my sustain until I figured out how to mess with the noise gate. I doubt it's the guitars that cause the problem as you have tried more than one. Last resort, Try a different amplifier if you have one, or can use a friends. Maybe you have some tubes on their way out.
Comment: The tube thing is interesting. And indeed. I will first do the trial and error approach.

Lack of sustain on guitar


post-hoc analysis for interaction terms in logistic regression: emtrends(), type I error

Question: I built a logistic regression, including the followed variables. And I tried to compare effects of temp among group and habitat using emtrends() in emmeans R package.
I have 3 questions about the followed results.
First, it feels like that there was type I error in the results. How can I figure out there is type I error or not?
Second, I wonder why df is Inf in my results of emtrends().
Third, I ran emtrends() seperately for group variable and for habitat variable. My model included the interaction term between temp and habitat but not a habitat variable. Then, should I ran like this emtrends(fit_1, ~habitat|group, var = "temp")?
> str(dataset)
'data.frame':   1440 obs. of  6 variables:
$ habitat    : Factor w/ 4 levels "A","B","C",..: 2 2 3 2 1 1 1 1 1 1 ...
 $ group      : Factor w/ 4 levels "red","skyblue","indigo",..: 3 4 4 4 3 4 4 4 1 1 ...
$ temp       : num  15.5 13.9 13.9 15.4 15.5 ...
 $ event      : Factor w/ 2 levels "0","1": 1 1 1 1 2 1 1 1 1 1 ...
$ weight     : num  3 3 2.73 3 10 ...
 $ rain       : Factor w/ 2 levels "0","1": 2 2 2 2 2 2 2 2 2 2 ...

> fit_1 <- glm(event ~ group*temp + rain + habitat:temp + rain:temp
               , weights = weight,family=binomial, data=dataset)

> summary(fit_1)

Call:
glm(formula = event ~ group * temp + rain + habitat:temp +
 rain:temp, family = binomial, data = dataset,
 weights = weight)

Deviance Residuals:
 Min       1Q   Median       3Q      Max
-3.6849  -1.7761  -1.1416  -0.2509   8.5246

Coefficients:
                 Estimate Std. Error z value Pr(>|z|)
(Intercept)         -2.51354    0.63168  -3.979 6.92e-05 ***
groupskyblue         2.06381    0.85254   2.421  0.01549 *
groupindigo          8.48504    0.97759   8.680  < 2e-16 ***
groupblack           4.09039    0.83798   4.881 1.05e-06 ***
temp                -0.06882    0.05273  -1.305  0.19187
rain1                2.25311    0.89997   2.504  0.01230 *
groupskyblue:temp   -0.24329    0.06062  -4.013 5.99e-05 ***
groupindigo:temp    -0.59809    0.06905  -8.662  < 2e-16 ***
groupblack:temp     -0.35243    0.06355  -5.546 2.92e-08 ***
temp:habitatB        0.20523    0.01928  10.644  < 2e-16 ***
temp:habitatC        0.20808    0.01925  10.807  < 2e-16 ***
temp:habitatD        0.17938    0.01955   9.173  < 2e-16 ***
temp:rain1          -0.17861    0.06167  -2.896  0.00378 **
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

 Null deviance: 9483.6  on 1439  degrees of freedom
Residual deviance: 8604.1  on 1427  degrees of freedom
AIC: 8630.1

Number of Fisher Scoring iterations: 6

> emt_fit_group <- emtrends(fit_1, ~group, var = "temp")
> summary(emt_fit_group)
group    temp.trend     SE  df asymp.LCL asymp.UCL
red          0.0903 0.0866 Inf   -0.0794    0.2601
skyblue     -0.2686 0.0719 Inf   -0.4096   -0.1276
indigo      -0.5297 0.0816 Inf   -0.6897   -0.3698
black       -0.2192 0.0803 Inf   -0.3765   -0.0619

Results are averaged over the levels of: rain, habitat
Confidence level used: 0.95

> emt_fit_habitat <- emtrends(fit_1, ~habitat, var = "temp")
> summary(emt_fit_habitat)
habitat  temp.trend     SE  df asymp.LCL asymp.UCL
A            -0.380 0.0547 Inf    -0.487   -0.2727
B            -0.175 0.0534 Inf    -0.279   -0.0701
C            -0.172 0.0534 Inf    -0.277   -0.0672
D            -0.201 0.0534 Inf    -0.305   -0.0960

Results are averaged over the levels of: group, rain
Confidence level used: 0.95

Comment: In your third question, you ask about `emtrends(fit_1, ~alt|group, var = "temp")` but there is no variable named `alt` in the model -- so this is not a valid call. What variable did you mean `habitat` rather than `alt`?
Comment: @RussLenth Oh, I wrote it down wrong. The `alt` is corrected with `habitat`. Thank you.
Answer: It's not clear why you think that there is "Type I error" (false positives) in your results. If the assumptions underlying the binomial model are correct, then you shouldn't be having false-positive results unless your model is overfitting the data. If you have more than 15 or so cases in the minority outcome class per coefficient that you are estimating (as you seem to), you are probably OK. You can check by repeating the modeling on multiple bootstrap samples.
For the degrees of freedom (df), the maximum-likelihood method used to fit the binomial logistic regression is based on asymptotic theory that holds in the limit of an infinite number of observations. Comparisons are then based on a normal distribution of coefficient estimates, the limit of the t distribution as the degrees of freedom approach infinity.
For your model, you are asking for trouble when you omit the individual coefficient for a predictor (habitat) that you are including in an interaction. You might get away with that if there's only 1 interaction term in the model, but you have several. I'd be reluctant to give advice on how to set up the emtrends in this situation.
Comment: Thank you! I decided to include a habitat variable in my model after reading the link you cited. And can you explain the method, repeating model anaysis on bootstrap samples, in detail?
Comment: @pineapple159 [this answer](https://stats.stackexchange.com/a/373431/28500) outlines the process of the "optimism bootstrap," a good way to evaluate overfitting. Frank Harrell's notes on [Regression Modeling Strategies](https://hbiostat.org/rmsc/) explain the bootstrap in Section 5.2 and show its use in model validation and calibration in Section 5.3.5. The `validate()` and `calibrate()` functions in Harrell's [`rms` package](https://cran.r-project.org/package=rms) in R can do that for you if you use that package's `lrm()` function for logistic regression.


Expected winnings from a game

Question: A game of lucky dip consists of drawing 2 chips without replacement
from a box containing 1 green chip, 2 blue chips and 40 white chips.
You are paid according to the type of chips drawn. The green chip
carries a prize of $20 and a blue chip 10. A white chip carries no
prize. Find the expected winnings from one game.
I am a bit confused whether do i multiply or add the 2 expected winnings of the chips for each possible combination?
Answer: Suppose random variable X can take value $x_1$ with probability $p_1$, value $x_2$ with probability $p_2$, and so on, up to value $x_k$ with probability $p_k$. Then the expectation of this random variable X is defined as,

Here X is the winning from one game. X can take any value from ${0,10,20}$.
Now $$P(X=20) = \frac 1 {43}$$
$$P(X=10) = \frac {2} {43}$$
$$P(X=0) = \frac {40} {43}$$
Hence the expected value of X (wiki),
$$E(X) = 20P(X=20) + 10P(X=10) + 0P(X=0)$$

Expected winnings from a game


Why to combine Huffman and lz77?

Question: I'm doing a reverse engineering in a Gameboy Advance's game, and I noticed that the originals developers wrote a code that has two system calls to uncompress a level using Huffman and lz77 (in this order).
But why to use Huffman + lzZ7? Whats the advantage to this approach?
Comment: The output of LZ77 (lengths, distances, literal symbols, ...) is often not uniformly distributed (some occur more frequently, some less). You can use variable-length codes (such as Huffman) to code them more efficiently, gaining better compression.
Comment: The [DEFLATE](http://en.wikipedia.org/wiki/DEFLATE) algorithm uses both Huffman and LZ77 (for the same reasons Dan Mašek mentioned). Is it possible that the developers are actually using DEFLATE, simply to be able to re-use tested and debugged software rather than writing something new from scratch?
Comment: @DavidCary Yeah, this is possible. Maybe the developers just re-used a software that already uses these algorithm. Could you write an answer bellow, in order to me approve, please? I want to close this question.
Comment: @DanMašek Same for you, please. Could you write an answer bellow, in order to me approve, please? I want to close this question.
Answer: Using available libraries
It's possible that the developers are using DEFLATE (or some similar algorithm), simply to be able to re-use tested and debugged software rather than writing something new from scratch (and taking who-knows-how-long to test and fix all the quirky edge cases).
Why both Huffman and LZ77?
But why does DEFLATE, Zstandard, LZHAM, LZHUF, LZH, etc. use both Huffman and LZ77?
Because these 2 algorithms detect and remove 2 different kinds of redundancy in common to many data files (video game levels, English and other natural-language text, etc.), and they can be combined to get better net compression than either one alone.
(Unfortunately, most data compression algorithms cannot be combined like this).
details
In English, the 2 most common letters are (usually) 'e' and then 't'.
So what is the most common pair? You might guess "ee", "et", or "te" -- nope, it's "th".
LZ77 is good at detecting and compressing these kinds of common words and syllables that occur far more often than you might guess from the letter frequencies alone.
Letter-oriented Huffman is good at detecting and compressing files using the letter frequencies alone, but it cannot detect correlations between consecutive letters (common words and syllables).
LZ77 compresses an original file into an intermediate sequence of literal letters and "copy items".
Then Huffman further compresses that intermediate sequence.
Often those "copy items" are already much shorter than the original substring would have been if we had skipped the LZ77 step and simply Huffman compressed the original file.
And Huffman does just as well compressing the literal letters in the intermediate sequence as it would have done compressing those same letters in the original file.
So already this 2-step process creates smaller files than either algorithm alone.
As a bonus, typically the copy items are also Huffman compressed for more savings in storage space.
In general, most data compression software is made up of these 2 parts.
First they run the original data through a "transformation" or multiple transformations, also called "decorrelators", typically highly tuned to the particular kind of redundancy in the particular kind of data being compressed (JPEG's DCT transform, MPEG's motion-compensation, etc.) or tuned to the limitations of human perception (MP3's auditory masking, etc.).
Next they run the intermediate data through a single "entropy coder" (arithmetic coding, or Huffman coding, or asymmetric numeral system coding) that's pretty much the same for every kind of data.

Why to combine Huffman and lz77?


Is there a program to apply played MIDI velocity to preexisting notes?

Question: I have a song that has some complex notes in it that are difficult to play. I have the sheet music written out as a MIDI sequence. Now I need to apply velocity to the notes.
Is there a program that can take incoming MIDI velocity input from a keyboard, and apply that to prewritten notes for humanization purposes?
Edit: After doing some more research, it looks like I'm looking for some sort of step sequencer. Anyone know of a sequencer with this function?
Comment: I know that FL Studio has a way to record input which can be applied as a mute track that manipulates velocity. But as far as how to do it, I'm a bit lost on that subject. It might also not fit with what you're attempting to do.
Comment: Interesting -- I'll keep that approach in mind! One pitfall I predict from your description is that I'll have to play it more or less in time with the prewritten notes, to get the notes in sync before the velocity kicks in.
Comment: That sounds promising... Could you elaborate and/or point me in the right direction? I'm not very familiar with FL Studio, and Google isn't helping.
Comment: It's certainly an advanced topic. If I recall, the procedure goes something like recording input onto the track. If an instrument is applied you then remove the instrument from the track and leave the track as a blank "sample" track with the MIDI input still saved to it. Then, you map the output of this track to manipulate the velocity value (there's a "knob") via a controller that can be applied to the track that is the one you want as an "effect". But as for the specifics, I've forgotten most of that. I haven't done FL Studio in years!
Comment: I should add that if your aim is to do it real-time, once you complete the setup, you can just start from a blank input and do it again
Answer: A reverse approach can be more rewarding.  Play the piece into a sequencer, aiming for feel rather than the right notes!   Play ANY notes, but with the right phrasing.
Now, start editing.  It's easy to correct the wrong notes.  When you've 'fluffed' at a more basic level, take a passage that DID go well as your model, copy the note positions and velocities.  This will be easiest in a 'piano roll' editing page. This shows the Cubase one. Pitch, position and length is shown in the main display.  Velocity for a selected note is shown both numerically (at the top of the page) and graphically (at the bottom). All the good MIDI sequencer programs have something similar.  Even the ones much cheaper than Cubase!
Comment: Not a bad idea, and also you'll get rhythmic nuance too.  What you would really want I suppose would be something that would map *any* note you play on a midi controller to the notes in the piece.  That way you could play whatever, and it would come out sounding like the song, and you could actually "interpret" the music in real time (rather than playing atonally with the correct rhythm and dynamics and just *imagining* the piece.
Comment: This is basically what I've been doing so far. Record the track in MIDI, then copy the velocity information manually to the preexisting notes. Tedious, but it works.
@Some_Guy, that's exactly what I need. If anyone has any recommendations, please contribute!
Comment: The technique is most useful for tidying up a piece of music that you can ALMOST play correctly!   Beyond that, I suggest you put your energy into improving your playing skills, rather than into covering up the lack of them.
Answer: As far as I know, most DAW software allows for track automation, and I believe that track automation can be applied to MIDI tracks as well, though I have only ever used it for audio tracks.
For audio, I usually record my guitar clean signal as the track; and any MIDI footpedal presses/midi expression pedal movement on an automation roll attached to the track. The automation then gets passed to the the VST sims to manipulate VST parameters.
Automation can be armed and recorded after the fact as well, and can be adjusted manually via mouse. In fact you can "draw pictures" with the mouse if that suits you or even copy image histograms etc.
Comment: Yeah, I've used automation for MIDI velocities before. The problem is translating the MIDI information coming from the *physical keyboard* into a velocity automation roll. I have yet to find any software with this feature. Someone mentioned FL studio might be able to do something along those lines -- I'll examine further.
Comment: I think most DAW (Digital Audio Workstation) software will support it. Looks like Reaper also has a MIDI velocity automation: ( https://www.reddit.com/r/Reaper/comments/3yzkn8/automating_midi_velocities/ ) It is (or used to be at least) fully functional 60 day trial-ware that did not deactivate after the trial, so you can take your time learning it.


Match number of positives in unbalanced data set

Question: I am dealing with a very unbalanced binary classification problem: 1% positives, 99% negatives. Training set is around 10 million rows, 40 columns. I choose the decision threshold (cutoff) on the training set as to match the number of positives in-sample. However, when I go out of sample, I underpredict the number of positives by around 20%. What are some things I can do to fix this problem? I would like to match the number of positives out of sample.
My AUC is decently high, around 94%, but the precision and recall are around 30% (these numbers hold both in-sample and out-of-sample). I am currently not using intercept in my model (to reduce risk of overfitting) - should I change this?
I am using binary cross entropy loss for training.
I don't know if I am missing something, perhaps to look into getting better features to improve precision and recall, or it would be hard to get a good match because the problem is so unbalanced? Would it make sense to try randomized (stochastic) threshold, would that even make sense or solve anything? What other ideas can I try?
Most of my features (columns) are discrete one-hot encodings, and I can clearly see that for some of them even in sample I am not matching the number of positives well, e.g. there's one bin that overestimates and another that underestimates, is there anything I can change related to the training (e.g. the loss function) or something else as to nudge the model to match the counts in each bin (predicted vs actual) during training?
Thank you
Update: In some comments below, some people suggest I don't even need a threshold. Then, how do I obtain a binary 0 vs 1 decision, which is my ultimate goal, without using a threshold? What strategies and other approaches are there for the decision-making process?
Comment: Because at the end of the day, I need a binary decision, 0 or 1. This is the business (application) need for this project.
Comment: Too bad your boss is engaged in dichotomous thinking.  Optimal decisions require understanding gray zones and probabilities in the middle where the best decision is no decision.
Comment: @FrankHarrell: I only use probabilistic rules in my work but in many cases, people are forced to make a black-vs-white call. Yes, when we plan ahead it is obviously wrong to ignore grey zones but for a specific unit-of-analysis (patient, house, car, whatever) we other "treat" or not.
Comment: @Dave: hm. I do see it, both here and on the bounties tab. I plan on awarding it in two or three days and keep the thread visible until then.
Comment: Why do you need a threshold at all?
Comment: Needing to make a binary decision has little to do with using a threshold during analysis.  And the awful effects of tampering with the sample sizes in the sample have been discussed extensively on this site.  See [this](https://fharrell.com/post/classification) and other articles on the blog.
Comment: I see. Can you please share some other resources, what else can I do other than using a threshold? In other words, what are my steps forward after I have trained a logistic regression model, what can I do with it to arrive at a binary decision?
Comment: For another resource, he wrote a book called *Regression Modeling Strategies*. I believe that a substantial proportion of data science questions posted on here (and elsewhere on the internet) discuss issues that are addressed and clarified in the book. Additionally, he has [other blog posts](https://www.fharrell.com/post/class-damage/) related to this topic. // I do not understand your logic about omitting the intercept. Could you please clarify? Do you just mean that you want to lower the parameter count and are willing to sacrifice the intercept to do so?
Comment: Yes, that was the idea, to have fewer params. I am considering adding it back, do you think it has any chance of helping materially with this problem.
Sorry, I do not have that book handy at the moment. I will paraphrase my original question and try to get more input so I can make progress on my project before acquiring that book.
Comment: Question for @FrankHarrell: I read your blog post here: https://www.fharrell.com/post/classification/
Can you please explain what you mean by "(2) recalibrating the intercept (only) for another dataset with much higher prevalence" and how can I apply it to my situation? I am still at a loss and not sure how to go about getting binary decision, i.e. how to establish a decision-making process that meets my criteria described above.
Comment: Asking about that quote from the blog would be a wonderful question to post! // The gist of several of Frank Harrell's blog posts is that it is best to get out of the binary world and think in terms of accurate estimates of event probability. This is extremely different from how much of machine learning is presented, yes, and Harrell addresses that on his blog, too.
Comment: Yes, I went over some of his thoughts. They are great, sadly my application is such that I have to return to my boss a list of 0s and 1s, it comes down to that. I would really appreciate some input on how to do that based on my situation described above.
Comment: [Of possible interest](https://stats.stackexchange.com/q/464636/247274)
Comment: That is not the case in general.  Decisions are delayed.  Decisions are implemented on a provisional basis.   Doctors do 1-month trials of drugs or start the patient on a lower dose when there is uncertainty.  Most of all people delay decisions until they get more data when probabilities are in the middle.  You have oversimplified how decisions are handled.
Comment: I recommend that you clearly, cleanly and consciously draw a distinction between the probabilistic statistical modeling aspect and the subsequent decision, which takes as inputs probabilistic predictions, but also costs of decisions. Note I am not writing "costs of misclassifications". See here: [Reduce Classification Probability Threshold](https://stats.stackexchange.com/q/312119/1352)
Comment: @StephanKolassa I don't mean to be the arrogant punk I warn against being in my answer, but did you mean to award a bounty?
Comment: @Dave: so I did. It seemed appropriate.
Comment: @StephanKolassa I don't see a bounty. Perhaps something went awry because I edited the post while you set up the bounty?
Comment: @StephanKolassa That makes sense. I thought something went wrong because I edited the answer with Tim's link as you set up the bounty, but I see your point about keeping this visible in the bounty tab as long as possible. (And thanks for the points! I've learned a lot of this because of you!)
Answer: Your question indicates a number of common errors in regression modeling.

Omission of the intercept in order to have fewer parameters and less overfitting

Particularly in an imbalanced problem where you want the model to be skeptical about membership in the minority class, the intercept is important. I understand the idea of model parsimony in hopes of avoiding overfitting, but the intercept plays a unique role in the model by affecting the outcome in the same way no matter what the features are.
No other parameter in the model can do this, so if you're going to do away with a parameter, that isn't the one you want to lose unless you have an excellent reason for doing so (such as knowing that the outcome is zero when the features all are zero).

Pure linearity

You don't state anything about this in the question, but you are allowed to have nonlinear features like splines in your generalized linear model. By allowing the probability to increase, decrease, and then increase again (for instance) as a feature increases, you allow for better predictions than you would get from forcing the straight-line fit that I suspect you are using.

Parsimony

In one of his videos, the same Frank Harrell as in the comments mentions that parsimony is the enemy of predictive accuracy. My answer here gets into some of why that might be the case and links to additional material (by Frank Harrell). If you're worried about overfitting but want to have a lot of variables, interactions, and nonlinear features (e.g., splines), use some kind of penalized estimation, such as a ridge penalty. This fights overfitting without getting rid of any of the parameters.
All of these and more are discussed in Frank Harrell's Regression Modeling Strategies textbook.
Finally, by having a good model that considers all of these issues, you can, if you must, apply a threshold to make dichotomous decisions based on the known costs of mistakes, and that is how you should pick your threshold. I would argue that, if you don't know the costs of your mistakes, you have no business making classifications and should only give predicted probabilities.
In the extreme, where it is unacceptable to mistake a $0$ for a $1$ or a $1$ for a $0$, the best dichotomous decision might be to classify every instance as one class so that you never make the unacceptable mistake.
Additionally, don't be surprised when you find yourself unable to make accurate dichotomous decisions for instances whose predicted probabilities are near the threshold. The best decision in such a situation might be no decision. This is related to what Kolassa means when he writes that "a cost optimal decision might also well include more than one threshold!"
If I may pontificate at the end of my answer, your post suggests that you face a common issue for data scientists to face. You don't have to knuckle under to every bad idea someone has. You're the expert; act like it. This doesn't mean being an arrogant punk, but you're allowed to explain why someone is wrong. This is scary the first time you have to do it, and it is when you are most valuable as a statistician or regression expert. (It also get less scary as you do it more and more.)
Comment: +1 Btw, in #1 it is worth mentioning this thread that discusses it in detail https://stats.stackexchange.com/questions/7948/when-is-it-ok-to-remove-the-intercept-in-a-linear-regression-model


Should this SE be renamed?

Question: Is "expats" a suitable name for this site? Expats is just a politically correct word for immigrant. It mostly exists because immigrant has negative connotations and white people don't want to be called that.
Keeping the name just re-enforces this. The site is about questions that immigrants have and shouldn't use harmful politically correct language. It's harmful because it perpetuates the idea that there is some difference between an expat and an immigrant.
Comment: Just wanted to comment that there have been previous attempts to have a StackExchange site for 'Immigration' and related topics, but those sites never made it long into their Beta stages. There are probably numerous reasons for that - sometimes it's just luck or bad timing. For whatever reason, when the Expats proposal went through, it got some real traction with users. Perhaps it was the name that appealed to a wider audience. Not sure, but all long-term visa topics ended up here as well as topics on citizenship and residence. It's just the way things have worked out.
Comment: "It mostly exists because immigrant has negative connotations" - Maybe to you. But not to me. And not to many people I know, including the vast majority of my neighbors and colleagues, most of whom are native born English. Got a link to that assertion? "white people don't want to be called that." I see. You mean like English immigrants, Polish immigrants, Canadian immigrants, Romanian, etc...? I think this is again an idea that exists in your head and no one else's.
Answer: "Expatriates" covers the case of someone living temporarily abroad, whereas "immigrants" does not. This site covers many cases that are not "immigrants". For example in the US system short term work visas like H1B, student F visas, company transfer L and exchange visitor J are all non-immigrant visas, and people living in the US under their terms are not called "immigrants" in normal speech. We answer questions about them.
"Expatriates" is not just a politically correct name for immigrants. "Expatriates" as a word has been around for hundreds of years before "political correctness" was even thought of.
It would be weird to expect a user to come to a site called "Immigration" to ask about non-immigrant circumstances.
No it should not be renamed. Or at least not to "Immigration" or anything similar.
However I will say that "Expatriates" is not a common word and may not be understood by people who have English as a second language. It's possible that a change to "Living Abroad" or something similar might be helpful.

Should this SE be renamed?


How to translate variable containing html with the i18n_variable module?

Question: I am trying to translate a variable with i18n_variable module. I am using the following code:
/**
 * Implements hook_variable_info().
 */
function MY_MODULE_variable_info($options) {
  $variables['pm_email_notify_subject'] = array(
    'title' => t('My variable title'),
    'description' => t('The description', array(), $options),
    'default' => t('A good default value', array(), $options),
    'type' => 'string',
    'localize' => TRUE,
    'access' => 'administer checkout',
    'group' => 'my_variable_group',
  );
  return $variables;
}

This works as expected: I can choose the variable to be translated at /admin/config/regional/i18n/variable and when I do so, I can find its value via /admin/config/regional/translate.
But the problem is, that the variable contains HTML markup and although this works with the original content, the markup in the translated string gets converted to HTML entities. what am I missing?
Answer: A solution I used when I faced a similar problem in a different context is to preventively replace tag-markers by the corresponding entity in the content before it is processed.
Adapated to your case, it might look like this:
/**
 * Implements hook_variable_info().
 */
function MY_MODULE_variable_info($options) {
  $my_variable_content = preg_replace_all(
    '/<([^>]*)>/', '<$1>',
    'A good default value with some <tags> inside'
  );
  $variables['pm_email_notify_subject'] = array(
    'title' => t('My variable title'),
    'description' => t('The description', array(), $options),
    'default' => t($my_variable_content, array(), $options),
    'type' => 'string',
    'localize' => TRUE,
    'access' => 'administer checkout',
    'group' => 'my_variable_group',
  );
  return $variables;
}

This way, the finally rendered text keeps its "<" and ">" intacts.
But caution: it might be not appropriate to your case, so try it carefully.
Comment: You can't put variable content into the t-function.
Comment: @user5950. Sorry, you're right! I read your question too fast, so I focused to the "A good default value" as if it was the content where you wanted to allow tags. Then I felt in a second error: even in that case, my solution was wrong and should have been `'default' => t(@var, array(var => $my_variable_content), $options)`, which becomes absurd!
Answer: The simple answer is, not to use the interface for string translation of the i18n module under admin/config/regional/translate/translate, but instead go to the admin path, where the original (not translated) variable is set. In my use case this was /admin/config/messaging/privatemsg. As I patched the privatemsg module, to make it's variables translatable, on the module's admin form appeared a language selection option:

If you put in the translation there, all HTML is preserved. I guess this would be the general approach to translate variables, which are set via the admin interface. If you have to translate a variable, which is not set via the admin interface, I guess you have to provide the translation in code.


pundit_user: undefined method `current_user' for #

Question: I have two layouts Admin and Domain. And I don't need any extra configuration in Admin layout. but if user tries to access Domain layout they must be in their valid domain.
This means that, I need to customize all of my Domain policy to include both current_user as well as current_domain. I found this can be done with UserContext and pundit_user... so here is what I have done:
application_controller.rb
class ApplicationController < ActionController::Base
  include Pundit
  rescue_from Pundit::NotAuthorizedError, with: :user_not_authorized

  # Prevent CSRF attacks by raising an exception.
  # For APIs, you may want to use :null_session instead.
  protect_from_forgery with: :exception

  def pundit_user
    UserContext.new(current_user, current_domain)
  end

  def after_sign_out_path_for(resource)
    root_path
  end

  def current_domain
    @current_domain ||= Domain.where(name: requested_domain).first
  end
  helper_method :current_domain

private

  def requested_domain
    return request.env["SERVER_NAME"]
  end

  def user_not_authorized
    # reset_session
    flash[:alert] = "You are not authorized to perform this action"
    redirect_to(request.referrer || root_path)
  end
end

Note that, when I access Admin layout, current_domain will be nil and if I visit any routes of Domain layout, then current_domain will set to currently accessing domain.
user_context.rb
class UserContext
  attr_reader :current_user, :current_domain

  def initialize(current_user, current_domain)
    @current_user = current_user
    @current_domain = current_domain
  end
end

PROBLEM
Suppose I have this policy:
user_policy.rb
class UserPolicy < ApplicationPolicy
  attr_reader :user, :scope

  def initialize(user, scope)
    @user = user
    @scope = scope
  end

  def index?
    binding.pry # debugging
    current_user.admin? ||
    current_user.domain == current_domain
  end

private

  def current_user
    # return user.is_a?(User) ? user : user.current_user
    user.current_user
  end

  def current_domain
    # return user.is_a?(User) ? nil : user.current_domain
    user.current_domain
  end

end

when application runs current_user and current_domain must available in UserPolicy as per documentation(https://github.com/elabs/pundit#additional-context).
But I am getting
undefined method `current_user' for #<User:0x007fcefbc2b150>

That means, still I have user object in it, not user.current_user and user.current_domain
Please let me know, if you need further description. What am I missing here?
Comment: what version of Pundit are you using?
Comment: it's `pundit (1.0.1)`
Answer: It was my own dumb mistake.
PROBLEM
I had a before_filter call in domain/base_controller.rb something like:
class Domain::BaseController < ApplicationController
  before_action :authenticate_user!
  before_action :domain_exists?
  before_action :verify_domain!

private

  def verify_domain!
    # PROBLEM: this line was updating pundit_user again to user object
    raise Pundit::NotAuthorizedError unless DomainConsolePolicy.new(current_user, current_domain).authorized?
  end

  def domain_exists?
    if current_domain.blank?
      redirect_to root_path, alert: 'Domain that you provided is not valid or is permanently removed!'
    end
  end
end

SOLUTION:
I have used headless policy for this because now I have both current_user and current_domain set with pundit_user in application_controller
domain/base_controller.rb
class Domain::BaseController < ApplicationController
  before_action :authenticate_user!
  before_action :domain_exists?
  before_action :verify_domain!

private

  def verify_domain!
    # SOLUTION
    authorize :domain_console, :has_access?
  end

  def domain_exists?
    if current_domain.blank?
      redirect_to root_path, alert: 'Domain that you provided is not valid or is permanently removed!'
    end
  end
end

policy/domain_console_policy.rb
class DomainConsolePolicy < Struct.new(:user, :domain_console)

  def has_access?
    user.current_user.admin? ||
    user.current_user.domain_id == user.current_domain.id
  end

end

Thanks


Differentiating a function inside a function.

Question: Let's say that there's a function $f(x) = g(x^2 + 5x)$, and $f'(0) = 10$. What would $g'(0)$ be here?
Because this is a function composition I tried using the chain rule in this way -
(I took the inside function $g(x)$ as u)
\begin{align}
& \ \ \ \ \ \ \ f'(x) = \frac{dg}{du}.\frac{du}{dx}\\[1em]
& \Rightarrow f'(x) = g'(x^2+5x).(2x + 5)\\[1em]
& \Rightarrow f'(0) = g'(0).(2*0 + 5)\\[1em]
& \Rightarrow 10 = g'(0).(2*0 + 5)\\[1em]
& \Rightarrow g'(0) = 2
\end{align}
Comment: Looks correct to me.
Comment: "Differentiating a function inside a function" is precisely what the chain rule is about.  Your solution and you work are both spot on.
Answer: To preface, derivatives are not quotients.
Given the limit definition of a derivative (where $f$ is some differentiable function for all numbers in the domain of $x$) (and also $\Delta x$ is some small change in $x$)
$$\frac {df}{dx} = f'(x) = \lim_{\Delta x \to 0} \frac{f(x+\Delta x) - f(x)}{\Delta x} = \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta x} $$
This "limit of a quotient" cannot be a "quotient of the limits" because both numerator and denominator go to zero, so the derivative is not a quotient, but it gives intuition to go down this path anyway. There's also nonstandard analysis and all that, but it is kind of irrelevant.

If $f$ is defined as the composition of function $h$ meeting the same criteria as $f$ (a composition seen as $f(h(x))$)
Also given the definition of the chain rule with such a function $f$
$$ \frac {df}{dx} = \frac {df}{dh} \cdot \frac {dh}{dx} $$
We can work with these definitions to see how the derivatives might be treated as quotients
$$ \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta x} = \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta h} \cdot \frac{\Delta h}{\Delta x} $$
$$ \hspace{3cm} =  \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta h} \cdot \lim_{\Delta x \to 0} \frac{\Delta h}{\Delta x} $$
$$ \hspace{4.4cm} =  \frac{\lim_{\Delta x \to 0} {\Delta f}} {\lim_{\Delta x \to 0} {\Delta h}} \cdot \frac{\lim_{\Delta x \to 0} {\Delta h}}{\lim_{\Delta x \to 0} {\Delta x}} $$
$$ \hspace{2cm} = \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta h} \cdot \frac{\Delta h}{\Delta x} $$


Quando usar sistema de cache no PHP?

Question: Andei lendo um pouco sobre o uso do cache em PHP, suas vantagens e desvantagens. Surgiram algumas dúvidas e etc.
Se eu tiver um sistema que só mostra textos com atualização (re-edição) dos mesmos, muito pequena, é aconselhável usar CACHE? Se sim, qual deveria usar e como?
Comment: Acho que a pergunta vale para qualquer plataforma, não apenas para PHP.
Answer: Cache é algo meio abstrato, existem várias otimizações que você pode fazer em um sistema que se encaixam bem nessa palavra, como:

Cache de páginas estáticas
Cache de consultas ao banco de dados
Cache de respostas de serviços
Cache de opcodes

Acho que em boa parte dos casos, tanta preocupação com otimização é desnecessária, embora algumas façam parte das boas práticas de programação.
Tudo depende da carga do seu sistema. Se você está tendo problemas de performance (muito uso de banda, processamento, memória, espaço em disco, etc.), comece a fazer otimizações no sistema partido das mais simples para as mais complicadas.
Existem algumas dicas genéricas e bem fáceis de se implementar como:

Sempre fique atento para os dois maiores vilões da performance: acesso a rede e acesso a disco.
Algumas técnicas como o armazenamento em cache de arquivos estáticos no navegador do cliente usando cabeçalhos podem aliviar seus problemas com excesso de uso de banda.
Importar bibliotecas comuns de CDNs de terceiros também pode ajudar um pouco com banda (Exemplo: https://developers.google.com/speed/libraries/devguide)
Evitar o uso de SELECT * ou adotar tabelas ARCHIVE ou MEMORY num sistema baseado em MySQL pode agilizar consultas.

E por aí vai...
Comment: Eu acho que "meio" é humildade. Cache é algo, de fato, abstrato. Fora isso, +1. ;)
Answer: Sim, deve usar.
Em PHP, para um uso simples, como um website que roda em uma única máquina, use o APC, de preferência passando por um objeto seu que gerencie a leitura e gravação, como por exemplo, o Componente Cache do ZF2.
Cada texto seu deve ser um id único que identificará o cache (ex: texto_id_776482. No momento em que você exibe pela primeira vez o texto 776482,
verifica se ele já está em cache e, se sim, exibe-o a partir do que recuperou do APC ou, se não, exibe e grava no APC o que recebeu do banco de dados.

Quando usar sistema de cache no PHP?


How to prove a function of ordered pairs is injective

Question: $f: (\mathbb{Z}^+{\times}\mathbb{Z}^+) \to \mathbb{Z}^+$ where $f(a,b)=2^a3^b$
How would I show that $f$ is injective?
Comment: Note that two positive integers are the same if and only if they have the same prime factorization.
Comment: A function is injective if, for any two distinct inputs, the outputs are distinct. So  you must show that given _any_ two distinct inputs, the resulting outputs are not the same. In the given problem, this follows from the fact that $2$ and $3$ are coprime. (Why?)
Answer: Suppose that $f(a,b)=f(c,d)$. Then $2^a3^b=2^c3^d$. These numbers have the same prime factorization so $a=c$ and $b=d$. Thus shows that $(a,b)=(c,d)$, which means that $f$ is $1-1$.


Enviar array para o Controller e inserir na DB, com Ajax e Codeigniter

Question: Olá,
Não estou a conseguir receber (no controller) o array que é enviado de uma View através de ajax.
Já testei no console.log e o array é enviado mas no controller nada acontece.
o input, "numdias" define o numero de inputs com a data a ser inserido (o array a ser enviado para o controller).
Segue o código correspondente:
View:

                                Numero de dias a adicionar:
                                <input type="number" class="form-control" name="numDiasAnuais" id="txtNumDias" min="0"/>

                            <template id="tmplLinha">

                                <input type="text" name="data[]" id="data" readonly="readonly" class="inserir_data form-control datasIndisponiveis"/>

                            </template>

                                <button type="submit" name="submitAddDatasAnuaisIndisponiveis" id="submitAddDatasAnuaisIndisponiveis" class="form-control">Adicionar</button>
                                <button type="button" class="form-control " data-toggle="modal" data-target="#modaldataindisponiveis">Listar Datas</button>

Ajax:
//dATAS INDISPONIVEIS
$("#submitAddDatasAnuaisIndisponiveis").click(function () {
    event.preventDefault();

    var dataIndisponivel = new Array();

    $("input.datasIndisponiveis").each(function(){
        dataIndisponivel.push($(this).val());

    })

    console.log(dataIndisponivel);

    jQuery.ajax({

    type: "POST",
    url: "<?php echo base_url('sistema/addDatasIndisponiveis'); ?>",
    dataType: 'json',
    data: {dataIndisponivel},
    success: function(res) {
    if (res)
    {
      $('#mensagensErro').html(res);
      $("#error-dialog").modal("show");
      //$("input#data").val('');

    }
    }
    });
});

Controller:
// #40
public function addDatasIndisponiveis()
{
    if ($this->session->userdata('logged_in') == true && $this->session->userdata('nivel') == 0) {
        // $this->load->library('form_validation');
        // $this->form_validation->set_rules('data[]', 'Datas', 'required|is_unique[tbldataindisponivel.data]');
        // if ($this->form_validation->run() == false) {

        //     $res = '
'.validation_errors().'
';
        //     echo json_encode('$res');
        //     exit;

        // } else {
            $datas = $this->input->post('data[]');
            for ($i = 0; $i < count($datas); $i++) {
                $inserir = $this->sistema_model->addDatasIndisponiveis($datas[$i]);
            }
            if ($inserir) {
                $res = '
Adicionado com sucesso
';
            }else{
                $res = '
Erro
';
            }

            echo json_encode($res);
            exit;
       // }

    } else {

        redirect('inicio/index', 'refresh');
    }
}

Comment: Não podes usar assim `data: {dataIndisponivel},` uma vez que `dataIndisponivel` é uma array. Testa `data: {datas: dataIndisponivel},` e depois `$datas = $this->input->post('datas');`. Funciona?
Comment: Sergio, obrigado funcionou, :D
Answer: No ajax tens data: {dataIndisponivel}, que é sitaxe inválida.
Um objeto tem de ter uma chave e um valor {chave: valor} e neste caso falta-te a chave.
Assim podes mudar para data: {datas: dataIndisponivel}, no JavaScript e mudar no controlador para $datas = $this->input->post('datas');.
Assim passas a mesma chave no ajax como a que estás a selecionar no controlador.


Icon element disturbing the flow of button on mouse hover event, how to I rearrange the icon on the button to make sure this doesn't happen?

Question:
When I hover on the "message" part of the button, the hover animation works, but when i hover over the icon, the hover animation doesn't happen and this makes it a poor button.
Here's the HTML

    message
    <button className="button-icon-message ">Message</button>

Hope this information is enough to understand my problem, comment below if you need more information, thanks!
It's React code, hence the "className".
CSS Code for icon
.profile-user-button-message-icon{
    position:absolute;
    left:0px;
    top:20px;
}

Fiddle
Answer: you can write button:before {content: ''; display: inline-block; width: 10px; height: 10px; background: url('your bg-image');}
Comment: All ingenious is simple
Comment: or use glyphicon, fontawesome for icon
Comment: for example div i{background: red;}
div:hover i{background: blue;}
Comment: I'm using material icons by Google.. Can't I use absolute positioning or something? Not setting the background image for the button. Just creating a span with the button class of material icons and placing it on the button
Comment: give me a fiddle
Comment: problem was in cursor?
Comment: Ahhhh! Thanks a lot :D
I didn't think of nesting it inside the button, you saved a lot of my time.


How to add a new property in json structure existing in nvarchar(max) type column in SQL Server

Question: I have data like this in an employee table in a column of type nvarchar(max):
{
   "ID":101,
   "FirstName":"Vatan",
   "LastName":"Soni"
}

Now I need to update all rows with FullName property as well, so result should be
{
    "ID":101,
    "FirstName":"Vatan",
    "LastName":"Soni",
    "FullName":"Vatan Soni"
}

Please help and provide the SQL update script for it.
Thanks in advance.
Answer: Starting with SQL 2016 you have JSON types and functions: JSON_MODIFY:
UPDATE

  SET <column> = JSON_MODIFY(<column>, '$.FullName',
  JSON_VALUE(<column>, '$.FirstName') + ' ' + JSON_VALUE(<column>, '$.LastName'))

Comment: Its like

Company ID EmployeeData
501                 { "ID":101, "FirstName":"Vatan", "LastName":"Soni" }
501                 { "ID":102, "FirstName":"Ratan", "LastName":"Jain" }
502                 { "ID":103, "FirstName":"Patan", "LastName":"Sharma" }


Java NIO: What does IOException: Broken pipe mean?

Question: For some of my Java NIO connections, when I have a SocketChannel.write(ByteBuffer) call, it throws an IOException: "Broken pipe".
What causes a "broken pipe", and, more importantly, is it possible to recover from that state? If it cannot be recovered, it seems this would be a good sign that an irreversible problem has occurred and that I should simply close this socket connection. Is that a reasonable assumption? Is there ever a time when this IOException would occur while the socket connection is still being properly connected in the first place (rather than a working connection that failed at some point)?
On a side note, is it wise to always call SocketChannel.isConnected() before attempting a SocketChannel.write(), and if so, can I also assume that the connection is "broken" and should be closed if both SocketChannel.isConnected() and SocketChannel.isConnectionPending() are both false?
Thanks!
Answer:
What causes a "broken pipe", and more importantly, is it possible to recover from that state?

It is caused by something causing the connection to close.  (It is not your application that closed the connection: that would have resulted in a different exception.)
It is not possible to recover the connection.  You need to open a new one.

If it cannot be recovered, it seems this would be a good sign that an irreversible problem has occurred and that I should simply close this socket connection. Is that a reasonable assumption?

Yes it is.  Once you've received that exception, the socket won't ever work again.  Closing it is is the only sensible thing to do.

Is there ever a time when this IOException would occur while the socket connection is still being properly connected in the first place (rather than a working connection that failed at some point)?

No.  (Or at least, not without subverting proper behavior of the OS'es network stack, the JVM and/or your application.)

Is it wise to always call SocketChannel.isConnected() before attempting a SocketChannel.write() ...

In general, it is a bad idea to call r.isXYZ() before some call that uses the (external) resource r. There is a small chance that the state of the resource will change between the two calls.  It is a better idea to do the action, catch the IOException (or whatever) resulting from the failed action and take whatever remedial action is required.
In this particular case, calling isConnected() is pointless.  The method is defined to return true if the socket was connected at some point in the past.  It does not tell you if the connection is still live.  The only way to determine if the connection is still alive is to attempt to use it; e.g. do a read or write.
Answer: Broken pipe simply means that the connection has failed.  It is reasonable to assume that this is unrecoverable, and to then perform any required cleanup actions (closing connections, etc).  I don't believe that you would ever see this simply due to the connection not yet being complete.
If you are using non-blocking mode then the SocketChannel.connect method will return false, and you will need to use the isConnectionPending and finishConnect methods to insure that the connection is complete.  I would generally code based upon the expectation that things will work, and then catch exceptions to detect failure, rather than relying on frequent calls to "isConnected".
Comment: You can't get a broken pipe without previously having had a pipe. You can't get a failed connection without previously having had a connection. The final paragraph of this answer is correct but irrelevant.
Comment: is this an issue or it will be taken care by computer, please tell me
Answer: Broken pipe means you wrote to a connection that is already closed by the other end.
isConnected() does not detect this condition. Only a write does.

is it wise to always call SocketChannel.isConnected() before attempting a SocketChannel.write()

It is pointless. The socket itself is connected. You connected it. What may not be connected is the connection itself, and you can only determine that by trying it.
Answer: You should assume the socket was closed on the other end. Wrap your code with a try catch block for IOException.
You can use isConnected() to determine if the SocketChannel is connected or not, but that might change before your write() invocation finishes. Try calling it in your catch block to see if in fact this is why you are getting the IOException.
Comment: on such exception, isConnected() returns true no matter what. Read it as "socket got connected at some point in the past"
Comment: Final paragraph of this answer is completely incorrect. isConnected() does not detect this condition.