3blue1brown's Linear algebra

Question: Hey I was going through 3blue1brown's video about linear transformation in space(https://www.youtube.com/watch?v=kYB8IZa5AuE&index=3&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab at 4:00) and was wondering how the unit vectors get transformed such that the component in which they were 0 is now non-zero.
Eg) in the video vector [1,0] becomes [3,-2]
This matrix is adding a y component to the unit vector but how is it giving it a direction that our original never had?
I was curious how unit vector in x direction now gets a y component:
$$
    \begin{pmatrix}
    a & b \\
    c & d \\
    \end{pmatrix} [1,0]'= $$
\begin{pmatrix}
a + 0b \\
c + 0d \\
\end{pmatrix}$$=
\begin{pmatrix}
a  \\
c  \\
\end{pmatrix}
$$
$$
$$
This seems a bit strange to me as the x-component is added c times and made into the y-component.
side question: the matrix [[3,0],[-2,0]] transforms [0,0] to [0,0] and [0,1] or any [0,a] to [0,0] therefore points in this new space are not unique. This is not consistent with the stretching of the axis metaphor in the video right as this collapses a the line [0,a] to [0,0]?
$$
\begin{pmatrix}
3 & 0 \\
-2 & 0 \\
\end{pmatrix} [0,a]'=[0,0]
$$
Comment: I don't understand what is the question here.
Comment: What do you mean "the points are not unique"?  Which two points do you think are the same?  It is true that some transformations send two points to the same point, but that is nothing to be surprised by...
Comment: And how do you get blue2brown from 3blue1brown?
Comment: Please also take a look at our basic [formatting tutorial](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference).  When people encountered a question that lacks simple formatting (you didn't even use a single line break) and seems to have so many mistakes, they can be less inclined to answer.
Comment: made the changes :)
Comment: Some linear transformations are invertible; some aren't.
Answer:
This matrix is adding a $y$ component to the unit vector but how is it giving it a direction that our original never had?

This seems to indicate that you expect vectors cannot change direction under a linear transformation. But this happens all the time: for example, rotations are linear transformations, and most rotations do change the directions of vectors. For example, in $\mathbb R^3$ or $\mathbb R^2$, you can take any two nonzero vectors and line them up to point in the same direction using a rotation.
Vectors that a linear transformation only scales, and doesn't change the direction of, are special: they're called eigenvectors for that transformation.  Sometimes a transformation doesn't have any eigenvectors (for example, a 90 degree rotation in the plane.)  I haven't watched the video link yet, but if it is discussing a transformation stretching certain vectors, it is almost certainly talking about an eigenvector (although it may be hidden in the context.)

therefore points in this new space are not unique.

The linear transformations that map the points uniquely are called nonsingular and again, that's a special property that not all linear transformations have.
Comment: @Rahul Deora, linear does not imply scalar times vector, where did you get that from?
Comment: @Rahul, $A(\alpha \vec v+\beta\vec w) = \alpha A\vec v+\beta A\vec w.$ It even shows you that property in the video, right past 4:00 where you wanted us to look (4:28, to be more precise).
Comment: @RahulDeora perhaps you should rewatch the video from the start, or watch a separate video on the same topic.
Comment: My confusion was the term "linear" implies a scalar times your vector. So a*(1,0) will never have a y component. However here we are somehow getting a y-component through the matrix which no 'linear' combination can get. Can you speak about how in this linearity this is fine.
Comment: Then what does 'linear' imply?
Comment: @rschwieb yes the change in direction part that you spoke about. Look at the animation at 9:08. One basis is changed and then the other can be changed in any way regardless of the first. Isn't this problematic as the spacing of the first would be affected as that is also acting in y direction?
Comment: @RahulDeora What do you mean "spacing"?  Are you talking about the property of linear maps sending parallel lines to parallel lines?  Sorry, I won't have a lot of time to watch the video soon. I'm just working from basic principles.  I can't exactly grasp what you're implying with your verbal description. Perhaps you can come up with a simple toy numerical example about something happening that you don't like?

3blue1brown's Linear algebra


Custom reference field settings in different content types

Question: I have a field called booked user and two types of content types, event-type-A and event-type-B.
I am referring to the same field from both content types. However, along the process I figured out that I want to have multiple values for this field in event type-A and single value for event-type-B. Initially both had multiple values.
Whenever I change the field settings in one content type, it also changes in the other.
How can I overcome this? Considering that I have content created in type-B and I can not really add a new field to replace that one, cause I will lose all the previous field content for that user.
Answer: The solution a colleague proposed is to add the following code in the hook_form_alter():
<code>$language = $form['field_user']['#language'];
$form['field_user'][$language]['#multiple'] = 0;
</code>
If you want to be a bit more aggressive on data consistency you can also hook into the form validator and check that the correct number of values is inserted.


How can I open an iTunes radio station like Beats 1 with AppleScript?

Question: There does not appear to be a direct way to open an iTunes radio station like Beats 1 with AppleScript? Is there one that I'm missing? Is there a workaround? I'm using iTunes 12.2.
Comment: GUI scripting isn't dependent on something being in a menu.
Comment: Normally you could try 'GUI Scripting' (Link: http://dougscripts.com/itunes/category/gui-scripting/), but I'm not sure if it will work because you have to click the play button which is not really in a menu.
Answer: Try this code. iTunes must be in windowed mode for GUI scripting to work unfortunately. The basic idea is: tell iTunes to activate, check for the "radio" tab, when it appears, click on it, then do the same thing for the "Listen Now" button for Beats 1 radio:
tell application "iTunes" to activate
tell application "System Events"
    tell window 1 of application process "iTunes"
        repeat while (radio button "Radio" of radio group 2 exists) is false
            delay 0.5
        end repeat
        click radio button "Radio" of radio group 2
        repeat while (button "Listen Now" of group 2 of group 1 of group 1 of UI element 1 of scroll area 2 exists) is false
            delay 0.5
        end repeat
        click button "Listen Now" of group 2 of group 1 of group 1 of UI element 1 of scroll area 2
    end tell
end tell

Comment: Yep. You can crawl the view hierarchy by using something like `tell application "System Events" to get buttons of window 1 of application "iTunes"` then if you don't find what you're looking for, try `tell application "System Events" to get buttons of UI elements of window 1 of application "iTunes"` and if that doesn't work, keep adding "of UI elements" right after buttons, and eventually you'll find the button you're looking for. That's how I made this script.
Comment: Also, it helps to download Xcode and use "Accessibility Inspector" buried within Xcode's bundle at "/Applications/Xcode.app/Contents/Applications/Accessibility Inspector.app". Get Xcode from the App Store and go there with the Finder by pushing "⌘ + shift + g" and entering that path. It attempts to show you the UI hierarchy needed to get to any UI element you move your mouse over, but it isn't always correct, and doesn't really give you enough information. I use it to determine what "kind" of UI element I'm looking for, which tells me what to put after "get" when I crawl the hierarchy.
Comment: Excellent. Is there a friendly way to inspect the UI elements?
Answer: Rather than try and traverse the iTunes GUI, use Safari and AppleScript to launch iTunes and play Beats 1. The trick is to use Apple's Beats web page at https://itunes.apple.com/us/station/listen-in-apple-music/idra.978194965.
set theClassName to "action listen-in-itunes"
set elementNum to 0

tell application "Safari"
    open location "https://itunes.apple.com/us/station/listen-in-apple-music/idra.978194965"
    do JavaScript "document.getElementsByClassName('" & theClassName & "')[" & elementNum & "].click();" in document 1
end tell

Note: For this to work, you will need to enable Allow JavaScript from Apple Events in Safari's Develop menu.
This helped: http://www.cubemg.com/how-to-click-a-button-on-a-web-page-with-applescript/
Answer: I had a look at the web page that launches Beats One in iTunes and found the URL that gets launched when the button is clicked. I can now launch Beats One with a one-liner AppleScript:
open location "itmss://itunes.apple.com/us/station/listen-in-apple-music/idra.978194965?cmd=AddStation"

Answer: I guess this query resolved anyway I give my modest solution:
I use Alfred , although automator is the same .
I created a "alt + Ienter image description here " hotkey to open a url that copied from iTunes.
https://i.stack.imgur.com/MeKTa.jpg


jqgrid select box option set selected

Question: i am using jqgrid and i am adding a row that contains fields: textbox and selectbox.
After adding a row by selecting the age from the dropdown, when i click on this row for editing, the age dropdown list must mark the selected value by default. How to achieve this ?
This is the code showing the columns data:
colModel:[
{name:'user_id',index:'user_id',  width:150,align:"center",editable:false,editoptions:{readonly:true,size:10}},
{name:'user_name',index:'user_name', width:250, align:"left",editable:true,editoptions:{size:30}},
{name: 'Age', width: 40, editable: true, formatter: 'select',
edittype: 'select', editoptions: { value: age_list },
hidden: true, editrules: { edithidden: true }}
]

where 'age_list' is the array of age values in the form of a javascript array.
Comment: jqGrid really mark automatically the selected value by default in all my tests in both inline or form editing. You should post more code which you use, because the problem is sure outside of the posted code. For example which `age_list` you use? which format has the contain of the 'Age' column? How you fill the grid?
Answer: Use edittype: 'custom' for your column, and then you can create your own html for the control:

More details at jqGrid's documentation wiki: http://www.trirand.com/jqgridwiki/doku.php?id=wiki:common_rules#editable
Comment: For all the details on that please follow the link to jqGrid's documentation and look at the edtittype -> custom section.  Their example is for creating an "input" element, but you can easily customize it to create a "select" element.  I would answer you directly but I'd just be copying their documentation.
Comment: By the way, i have viewed the source code of this demo page: http://www.ok-soft-gmbh.com/jqGrid/DependendSelects2.htm  and i could get a clean solution to my problem. Exactly what i wanted in edit scenario. Still accepting your solution :)
Comment: I could not find a way of setting a default value like as you said above: **display**. I tried with: **editoptions: { value: age_list, defaultValue:"2" }** but that doesn't work. I see that the first option in the dropdown is getting selected by default.
Comment: I'll edit to be more clear.  You have to use the `custom` edittype, you're not doing that yet.
Comment: what does the custom_element and custom_value indicate?

jqgrid select box option set selected


Apache Solr Search Integtation and the Default Drupal Search Form

Question: Apache Solr Search Integtation (apachesolr module) is dependent on the core Search module. I set this up at https://www.test.epcsite.com/search/content. I also setup display suite so that I can theme my search results.
I have a couple questions:

The Default Search form uses Drupal core search. If you go to https://www.test.epcsite.com/search it redirects to https://www.test.epcsite.com/search/content, I don't quite know why. I would like it to stay on "/search".
apachesolor setup a url at /search/site, which puts a tab on the search form. Clicking on the tab takes you to another search form, but it won't use Display Suite to theme the results. The tab itself is confusing and it would be nice to get rid of it.
Going to https://www.test.epcsite.com/search/content displays "Your search yielded no results" but I haven't run the search yet. How do I customize this message not to show? How do I customize the message so that it says what I want.
Answer: I'm assuming Drupal 7.

This is a limitation of the way core Search is implemented in D7. Try the Custom Search module; it can somehow change the way search paths are handled, although I have not used it.
One tab would be for core search, and the other for Apache Solr search. Go to /admin/config/search/settings and disable all search modules except Apache Solr, and also set it as default. Now the only available search path will be the one set up at /admin/config/search/apachesolr/search-pages.
I cannot replicate this on my Apache Solr & Drupal 7 setup. In general, to change strings that are hard coded into modules, use the String Overrides module.


The order of zero at $ z=0$ of $ f(z) = \cos(z^3) - 1 $.

Question: I'm trying to find the order of the zero at $z = 0$ of the function $ f(z) = \cos(z^3) -1 $.
I know I can express $f(z) $ as $$ f(z) = - \frac{z^6}{2!} + \frac{z^{12}}{4!} - \frac{z^{18}}{6!} + \ldots $$
I am unsure how to proceed.
Answer: Now, factor out $z^6$.
So, you get
$$f(z) = z^6g(z)$$
with $g(0)\neq 0$


Mudar a propriedade FormBorderStyle para None sem perder as funções Sizable em C#

Question: Olá, estou como um pequeno problema, pois preciso mudar o design do meu Form referente a sua borda, sendo que a forma que encontrei em fazer isso foi mudando a propriedade FormBorderStyle para None e fazendo as devidas alterações no design, porém ao fazer isso pego as principais funções de quando estar como Sizable, sendo que as funções Minimize, Restore e Close eu ainda consigo faze-la com o uso de imagens ou botões, porém a função de redimensionar o form com o mouse o "esticar" e "encolher" eu não a tenho mais. O que devo fazer para que o form volte a ter-las?

O que eu estou precisando é algo semelhante ao novo design do visual studio pro exemplo, onde ele mantem as mesmas funções, porém com o layout personalizado, como pode ser visto na imagem abaixo:
Comment: Você vai precisar desenvolver isso...
Comment: Tens alguma ideia por onde eu posso começar?
Answer: É possível fazer o redimensionamento do form usando o método WndProc. Faça um teste com o código abaixo:
public partial class Form1 :Form
{
    public Form1()
    {
        InitializeComponent();
    }

    private int borderWidth = 5; //Exemplo apenas para teste
    private new Padding Padding = new Padding(50); //Exemplo apenas para teste (Pode ser especificado direto nas propriedades do Form)

    private WinApi.HitTest HitTestNCA(IntPtr lparam)
    {
        Point vPoint = new Point((Int16)lparam, (Int16)((int)lparam >> 16));
        int vPadding = Math.Max(Padding.Right, Padding.Bottom);

        if (RectangleToScreen(new Rectangle(ClientRectangle.Width - vPadding, ClientRectangle.Height - vPadding, vPadding, vPadding)).Contains(vPoint))
            return WinApi.HitTest.HTBOTTOMRIGHT;

        if (RectangleToScreen(new Rectangle(borderWidth, borderWidth, ClientRectangle.Width - 2 * borderWidth, 50)).Contains(vPoint))
            return WinApi.HitTest.HTCAPTION;

        return WinApi.HitTest.HTCLIENT;
    }

    protected override void WndProc(ref Message m)
    {
        if (DesignMode)
        {
            base.WndProc(ref m);
            return;
        }

        switch (m.Msg)
        {
            case (int) WinApi.Messages.WM_NCHITTEST:
                WinApi.HitTest ht = HitTestNCA(m.LParam);
                if (ht != WinApi.HitTest.HTCLIENT)
                {
                    m.Result = (IntPtr) ht;
                    return;
                }
                break;
        }

        base.WndProc(ref m);
    }

    [SuppressUnmanagedCodeSecurity]
    internal static class WinApi
    {
        public enum Messages : uint
        {
            WM_NCHITTEST = 0x84,
        }

        public enum HitTest
        {
            HTCLIENT = 1,
            HTBOTTOMRIGHT = 17,
            HTCAPTION = 2
        }

    }
}

*Observe que a propriedade Padding determinará a área em que o redimensionamento poderá ocorrer, neste caso, uma área quadrada de 50 pixels no canto inferior direito.
Este trecho de código foi retirado de um framework desenvolvido para criar aplicações no estilo Metro, usando o Windows Forms:
MetroFramework - Modern UI for WinForms (http://thielj.github.io/MetroFramework/)
Caso os forms e os controles que eles disponibilizam não atendam aos seus requisitos, é possível obter o código fonte e fazer as alterações necessárias.
Comment: Funcionou perfeitamente como você descreveu, por aqui já da para me ter uma ideia de por onde seguir para obter o que desejo, vlw mesmo!


"Microsoft Internet Controls" reference sometimes requires re-assignation

Question: Every once in a while, on my same PC or on new PC with the same Windows (7) version and Excel (2010) version, when I try my usual Excel file with a VBA using "Microsoft Internet Controls" reference, debug incorrectly fails with a "Compile error: can't assign to array".
I found that the problem is immediately fixed by just re-assigning the "Microsoft Internet Controls" (uncheck-OK, then again check-OK).
Note: its path is "C:\Windows\SysWOW64\ieframe.dll".
It's like if my functions which use that "Microsoft Internet Controls" library, and I designed to return an array, stop to work because the link to "Microsoft Internet Controls" gets corrupted(??!).
I'm really lost on this.
Any ideas about how this problem can be fixed?
Answer: I solved adding a call to this sub from ThisWorkbook:
<code>Sub Readd_Reference_Microsoft_Internet_Controls()
    Const fullpath = "C:\Windows\SysWOW64\ieframe.dll"
    ActiveWorkbook.VBProject.References.Remove GetReferenceFromPath(fullpath)

    ThisWorkbook.VBProject.References.AddFromFile fullpath
    'ActiveWorkbook.VBProject.References.AddFromGuid "{EAB22AC0-30C1-11CF-A7EB-0000C05BAE0B}", 0, 0 'Microsoft Internet Controls
End Sub

Private Function GetReferenceFromPath(ByVal strSearchFullPath As String) As Object
    Dim oFs As FileSystemObject, oReferenceS As References, oReference As Reference, sRefFileName As String

    Set oFs = New FileSystemObject

    Dim sSearchFileName As String
    sSearchFileName = oFs.GetFileName(strSearchFullPath)
    Set oReferenceS = ActiveWorkbook.VBProject.References
    For Each oReference In oReferenceS
        sRefFileName = oFs.GetFileName(oReference.fullpath)
        If StrComp(sSearchFileName, sRefFileName, vbTextCompare) = 0 Then ' se uguale
            Set GetReferenceFromPath = oReference
            Exit Function
        End If
    Next
End Function
</code>


synchronize two webform select options

Question: In my webform page I have two select options components:
Item number:
20 | 20 ,
35 | 35 ,
40 | 40
Price List:
100 | 100 ,
200 | 200 ,
300 | 300
I need these two select options components synchronized. If one selects Item number (35) the price list also moves to (200).
Is there any module available to do that, or how can I get the synchronization of the two select options.
Comment: Quick hint: I think the easiest way to do it is through JavaScript. Use `.change()` event to change value in the other option set. If noone replies and if you don't know to do this on your own, I'll post a solution later.
Comment: That might be harder for you to do then. Why don't you just merge everything in one list? E.g. "Select item" and for options have: "20|20 - price: 100, 35|35 - price: 200, 40|40 - price: 300"? That would be the easiest solution and you won't have to write code.
Comment: Thank you in advance. I don't know. I am a drupal user not programmer, but I have a xampp server and drupal in my computer, so I can try the codes and put it to the real server.
Comment: The latest version of the webform module has same-page conditionals. You may be able to twist that around enough to do what you want. However, I suspect that, in the end, you are going to just have to write some custom JavaScript.
Comment: @Topsitemaker, If I can merge it looks ok. The list is long list, generated from Webform Views Select (https://www.drupal.org/node/2483637). How to merge the fields?
Answer: I do not need to synchronize, but combine (merge) them in one list.
In Webform Views Select  I can also combine by using Rewrite Results,
field "Item Number", exclude from display
field "Price List",
Rewrite Result:
[value] - price [value_1]
Then I can get in a single select option list:
20 - price 100
35 - price 200
40 - price 300

synchronize two webform select options


the closure of a set in a topological group

Question: This question is a exercise from Folland's Real analysis (chapter 11, question 1).
If $G$ is a topological group and $E\subset G$, then the closure $\bar{E}=\cap\{EV: V$ is a neighborhood of $ e\}$.
Any help would be apreciated!
Comment: Let $x\in\overline{E}$. Then, there is a net $x_i\in E$, for $i\in D$ and $D$ directed, such that $x_i\to x$. Let $V$ be any neighborhood of $e$. Since $x\mapsto x^{-1}$ is continuous, then $x_i^{-1}\to x^{-1}$. Since $(x,y)\mapsto xy$ is also continuous it follows that $x_i^{-1}x\to e$. Therefore, there is $i_0\in I$ such that for all $i>i_0$ we have $x_i^{-1}x\in V$. Therefore, $x=x_ix_i^{-1}x\in EV$. This proves that $\overline{E}\subset \bigcap_{V}EV$ for $V$ open neighborhoods of $e$.
Comment: Conversely. Assume that $x\in \bigcap_VEV$. Then, for each open $V\ni e$ there are $x_V\in E$ and $u_V\in V$ such that $x=x_Vu_V$. Let $I$ be the directed set of all open neighborhoods of $e$ ordered by inclusion. Then $I$ is directed, and $V\mapsto u_V$ is a net. This net converges to $e$ by construction. By continuity of $x\mapsto x^{-1}$ we have that $u_V^{-1}$ also converges to $e^{-1}=e$. Therefore, by continuity of $(x,y)\mapsto xy$, we have that $x_V=xu_V^{-1}$ converges to $xe=x$. Hence $x\in\overline{E}$.
Comment: The fact that $u_V\to e$ consists in checking the definition. For all open $U\ni e$, there is an element of $I$ (that element is $U$ itself) such that if $V\in I$ with $V>U$ (which by our definition of order means $V\subset U$), then $x_V\in U$. This is true because $x_V\in V\subset U$.
Comment: What have you tried?
Comment: @DavidG.Stork I tried use the characterization of $\bar{E}=\{x \in G: V\cap E\neq \empty, \forall V$  neighborhood of $x \}$ but didn't go anywhere
Comment: What is the intersection over? I understand it is Folland's custom to abbreviate notation but in this case it is not clear to me what the right-hand side of the expression is supposed to mean.
Comment: @Math1000 over all neighborhoods of $e$.
Comment: Then what is $e$? Some point of $E$?
Comment: @conditionalMethod Thanks!! A very elegant approach!
Comment: @Math1000 $e$ is the identity element of the (topological) group
Comment: Oh, that should have been obvious. I have been reading Dummit & Foote where they use $1$ for the identity element. My bad.
Answer: I assume that $G$ is Hausdorff and second countable (to avoid nets). Let $F(E)=\cap \{VE$, $V$ is a neighborhood of the identity$\}$. Firstly, we remark that $F$ is closed. Suppose that $x$ is not in $F(E)$, there exists a neighborhood $V$ such that $x$ is not in $EV$. There exists an open subset $U$ such that $U=U^{-1}$ and $UU^{-1}\subset V$, see the reference. $x$ is also not in $EU$, as $EU \subseteq EV$.
We claim that $EU\cap xU=\emptyset$: if not, let $y=eu=xu', u,u'\in U, e\in E$, we have $x=eu{u'}^{-1}$, which implies that $x\in EU$ since $UU^{-1}\subset V$. This implies that the complement of $F$ is open and $F(E)$ is closed.
Let $L$ be any closed subset, we claim that $F(L)=L$. Let $x\in F(L)$, and $U_n$ a sequence of neighborhoods of $e$ such that $\cap_nU_n=\{e\}$. $x=u_nf_n, u_n\in U_n, f_n\in F$. This implies that $f_n=xu_n^{-1}$ and converges towards $x$, we deduce that $x\in F$.
Let $E$ be any subset,  and $L$ a closed subset which contains $E$, for every neighborhood $V$ of the identity, $EV\subset LV$ implies that $F(E)\subset F(L)=L$, this implies that $F(E)$ is closed subset contained in every closed subset which conatains $E$ so it is the adherence of $E$.
Every neighborhood of identity in a topological group contains the product of a symmetric neighborhood of identity.


Remove element from tuple in a list

Question: I've knocking my head against a wall with this:
Basically what I want is to remove " " items from this list of tuples:
[('650', '724', '6354', '', '', ''), ('', '', '', '650', '723', '4539')]

and obtain the following new list:
[('650', '724', '6354'), ('650', '723', '4539')]

any ideas?
Comment: See http://stackoverflow.com/q/6632188/544198
Answer: Tuples in Python are immutable. This means that once you have created a tuple, you can't change the elements contained within it. However, you can create a new tuple that doesn't contain the items you don't want. For example:
>>> a = [('650', '724', '6354', '', '', ''), ('', '', '', '650', '723', '4539')]
>>> [tuple(y for y in x if y) for x in a]
[('650', '724', '6354'), ('650', '723', '4539')]

This uses a list comprehension [... for x in a] to create a new list using the formula in .... That uses a generator expression y for y in x if y to create a new tuple containing the elements of x only if y is true (meaning the value is truthy, or the string is nonblank).
Answer: a = [('650', '724', '6354', '', '', ''), ('', '', '', '650', '723', '4539')]

print [tuple(x for x in y if x) for y in a]

Remove element from tuple in a list


Judgement score regularization problem

Question: Consider the scenario where M performances (eg. singing contest) are being judged by N judges. Each judge awards a score S(m,n) to each performance on a scale of one to one-hundred.
The problem occurs when each judge has an individual style of using the scale. For example Judge A might give the worst performances a minimum of 30 and the best a maximum of 90, while Judge B may handout the minimum as 10 and the maximum as 80.
How do we regularize the scores so as the get the overall correct score? Some justification (link to an article or paper) for the answer would be appreciated.
This is similar to https://stats.stackexchange.com/questions/138973/normalizing-interview-scores, but that question has received no responses.
Answer: It seems like your goal is to ensure that the center and spread of the scores match up so that all centers are the same (i.e. judge a average = judge b average) and all spreads are the same. However, you want the judges scores' to retain their original shapes - that is, if one score was very high and all others were very low, you want to keep that "ranking" and make sure that the top score is still far higher than the very low scores.
This seems to be the perfect situation in which you use standardization. For each individual $i$, performance $m$, judge $n$, you should calculate:
$z_{imn} = \frac{S_i(m,n) - \bar{S}(m,n)}{\sigma(S(m,n))}$
where $z_{imn}$ is the standardized score, $S_i(m,n)$ is the score of individual $i$ for performance $m$ by judge $n$, $\bar{S}(m,n)$ is the average score by judge $n$ for performance $m$, and $\sigma(S(m,n))$ is the standard deviation of the scores by judge $n$ for performance $m$. This will standardize the means and standard deviations such that the mean for each judge will be 0 and the standard deviation for each judge will be 1, but the shapes of the distributions of scores will be retained to account for situations like the "one very high, all others very low" situation detailed above. You would then report the $z_{imn}$ as the standardized scores and use these for your analysis.
Comment: What I understand from your answer is that you are finding the Z-score for each S(m,n). I have two concerns regarding this. 1. If the data is not normally distributed, this method will not preserve the actual distribution. 2. Calculation of Z-score needs true mean and standard deviation, the formula is different if the sample mean is used.
Comment: 1. This is incorrect. The original shape will always be preserved. You're merely changing the location and scale of the distribution - not the shape. If the data are Normal, then the standardized version will be Normal. If the data are not Normal, then the standardized version will not be Normal. 2. This is true, but you could consider these to be the population of scores and thus the true mean and standard deviation. In any case, treating the sample mean and standard deviation as the population mean/SD will yield the desired effect.
Comment: Aah thanks for clarifying and for taking the time to answer.
Comment: This is also assuming each performance m is unique. If they're all the same, then we can use judge n's mean and SD instead of judge n and performance m's mean and SD. But if they're different (i.e. one performance is short and one is longer), we can calculate a different mean and SD for each combination of judge and performance.

Judgement score regularization problem


How to change max_length of a field on a proxy model in Django

Question: I am working on a bulletin board project. I have two Django model classes:
from ckeditor.fields import RichTextField
...

class Bulletin(models.Model):
    # other fields...
    content = RichTextField('Content (Don\'t use this and content image)',
                            max_length=5000, blank=True,
                            null=True)

# other classes...

class QuoteManager(models.Manager):
    def get_queryset(self):
        return super(QuoteManager, self).get_queryset().filter(
            bulletin_type=BulletinType.QUOTE.name)

class Quote(Bulletin):
    class Meta:
        proxy = True

    objects = QuoteManager()

    def save(self, *args, **kwargs):
        self.bulletin_type = BulletinType.QUOTE.name
        super(Quote, self).save(*args, **kwargs)

This allows me to keep all my proxy classes in the same DB and have the same fields as my base "Bulletin" class.
What I want to do is modify the Quote proxy class to have a max_length of say 200. How can I do this?
What I've tried?
I tried adding "content" to the proxy class and this was a name collision.
I tried adding a field called "quote_content" to the proxy class as a workaround, but this also failed as you cannot create fields in proxy classes.
A perfect solution would allow me to keep my proxy class, but still modify the max_length. Is such a think even possible?
Answer: You can try using, forms
I don't think proxy model will allow you to do so
Comment: Hey, I'm not sure what you mean by "use forms." Any chance you could ellaborate?


c# - how to manage CONTROL + digit on a TextBox?

Question: starting by saying I'm a newbie on C#, I would like to know how to manage on a TextBox, events like the CTRL + A, or CTRL + S, or CTRL + some digit.
TextBox has the method KeyDown, and I think I should use this, but I don't understand how to understand when a user presses first CTRL, then presses one random digit (still with pressed CTRL).
Thank you in advance.
Comment: @Sayse People simply downvote questions because it doesn't affect their reputation
Answer: Try this in KeyDown or KeyUp or anywhere
if (Control.ModifierKeys.HasFlag(Keys.Control))
{
    //user is holding control
}

Answer: On the KeyDown Method (or KeyUp or any other one dealing with KeyEventArgs) you can write:
private void textBox1_KeyDown(object sender, System.Windows.Forms.KeyEventArgs e)
{
    if (e.KeyCode == Keys.E && e.Modifiers == Keys.Control)
    {
      //Both CTRL + E were pressed
    }
}

Comment: i'm using this, it's cleare and it works for my goal. thank you.


Create responsive column of square items with flexbox

Question: Hei, im trying to create a simple responsive gameboard layout. Im using flexbox to keep it responsive and managed to to get it to work very nice in the big 2x7 and 7x7 game area. When page gets resized the "squares" get smaller and still maintain the correct their square look very nicely.
I tried to apply the same method to create a sideboard what i would like to be 1x7 squares and also keep their "squarness" but when the width gets smaller the boxes start to expand out sideways.

@import url('https://fonts.googleapis.com/css?family=Black+Ops+One');

body {
    margin: 0;
}

.game-container {
    display: flex;
    flex-direction: row;
    width: 100vw;
    //height: 90vh;
}

.game-area-container {
    border: 2px solid black;
    display: flex;
    flex-direction: column;
    max-width: 75vh;
    width: 75vh;
    flex: 1;
}

.header-container {
    border: 2px solid black;
    display: flex;
    flex-wrap: wrap;
    min-width: 350px;
}

.gameboard-container {
    border: 2px solid black;
    display: flex;
    flex-wrap: wrap;
    min-width: 350px;
}

.progress-container {
    border: 2px solid black;
    display: flex;
    flex-direction: column;
    max-width: min-content;
    flex: 1;
}

.score-container {
    border: 2px solid black;
    color: #acacac;
    text-align: center;
    font-family: 'Black Ops One', cursive;
    font-size: 2rem;
}

.levels-container {
    border: 2px solid black;
    display: flex;
    flex-wrap: wrap;
    flex-direction: column;
    height: 100%;
    max-width: 10vh;
}

.level-box {
    flex: 1;
    margin: 2px;
    background: #E9D2FF;
}

.level-box:after {
    content: '';
    display: block;
    padding-bottom: 100%;
}

.box {
    flex: 1 0 13%;
    margin: 2px;
    background: #E9D2FF;
}

.box:after {
    content: '';
    display: block;
    padding-bottom: 100%;
}

            Score: 90000

Here's a codepen of my current solution: My current solution in codepen
Answer: Here is a modified version of your code that should give you a better approximation. Still not perfect due to the text but you will always have square shapes. You will note the use the min() function, I have also replaced margin with white border to include them in the width and avoid overflow.

@import url('https://fonts.googleapis.com/css?family=Black+Ops+One');
body {
  margin: 0;
}

.game-container {
  display: flex;
  flex-direction: row;
  max-width: 90vh;
}

.game-area-container {
  border: 2px solid black;
  display: flex;
  flex-direction: column;
  flex: 1;
}

.header-container,
.gameboard-container {
  border: 2px solid black;
  display: flex;
  flex-wrap: wrap;
}

.progress-container {
  border: 2px solid black;
  display: flex;
  flex-direction: column;
  max-width: min-content;
  flex: 1;
}

.score-container {
  border: 2px solid black;
  color: #acacac;
  text-align: center;
  font-family: 'Black Ops One', cursive;
  font-size: min(4vmin, 2rem);
}

.levels-container {
  border: 2px solid black;
  display: flex;
  flex-wrap: wrap;
  flex: 1;
  height: 0;
  width: 10.2vw;
  max-width: min(90px, 9vh);
  align-content: flex-start;
}

.level-box {
  width: 100%;
  border: 2px solid #fff;
  background: #E9D2FF;
}

.level-box:after,
.box:after {
  content: '';
  display: block;
  padding-bottom: 100%;
}

.box {
  flex: 1 0 13%;
  background: #E9D2FF;
  border: 2px solid #fff;
}

* {
  box-sizing: border-box;
}

    Score: 90000

If you can update you html, here is another idea using CSS grid:

@import url('https://fonts.googleapis.com/css?family=Black+Ops+One');
.game-container {
  display: grid;
  max-width: 90vh;
}
.game-container > * {
  border: 2px solid black;
}

.header-container,
.gameboard-container {
  display: grid;
  grid-template-columns: repeat(7, 10vmin);
  grid-auto-rows: 10vmin;
}

.score-container {
  color: #acacac;
  text-align:center;
  font-family: 'Black Ops One', cursive;
  font-size: min(4vmin, 2rem);
  grid-area: 1/2;
}

.levels-container {
  display: grid;
  grid-template-columns: 7vmin;
  grid-auto-rows: 7vmin;
}

.level-box,
.box{
  background: #E9D2FF;
  margin: 2px;
}

  Score: 90000

Comment: What a perfect answere, not only showing me how to fix my css but it also gives another way of getting same result. Thank you!


How to run PostgreSQL on CentOS within Docker?

Question: I ran centos:7.1.1503 and then yum install postgresql. Unfortunately, due to systemd problems I cannot run service postgresql start even after installing initscripts:
Redirecting to /bin/systemctl start  postgresql.service
/usr/sbin/service: line 79: /bin/systemctl: No such file or directory
[root@8109c497195b ~]# yum provides systemctl
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos2.hti.pl
 * extras: centos.komster.pl
 * updates: centos2.hti.pl
systemd-208-20.el7.x86_64 : A System and Service Manager
Repo        : base
Matched from:
Filename    : /bin/systemctl

systemd-208-20.el7_1.2.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /bin/systemctl

systemd-208-20.el7_1.3.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /bin/systemctl

systemd-208-20.el7_1.5.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /bin/systemctl

systemd-208-20.el7.x86_64 : A System and Service Manager
Repo        : base
Matched from:
Filename    : /usr/bin/systemctl

systemd-208-20.el7_1.2.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /usr/bin/systemctl

systemd-208-20.el7_1.3.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /usr/bin/systemctl

systemd-208-20.el7_1.5.x86_64 : A System and Service Manager
Repo        : updates
Matched from:
Filename    : /usr/bin/systemctl

[root@8109c497195b ~]# yum install systemd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos2.hti.pl
 * extras: centos.komster.pl
 * updates: centos2.hti.pl
Resolving Dependencies
--> Running transaction check
---> Package systemd.x86_64 0:208-20.el7_1.5 will be installed
--> Processing Dependency: libcryptsetup.so.4(CRYPTSETUP_1.0)(64bit) for package: systemd-208-20.el7_1.5.x86_64
--> Processing Dependency: dbus for package: systemd-208-20.el7_1.5.x86_64
--> Processing Dependency: acl for package: systemd-208-20.el7_1.5.x86_64
--> Processing Dependency: libwrap.so.0()(64bit) for package: systemd-208-20.el7_1.5.x86_64
--> Processing Dependency: libqrencode.so.3()(64bit) for package: systemd-208-20.el7_1.5.x86_64
--> Processing Dependency: libcryptsetup.so.4()(64bit) for package: systemd-208-20.el7_1.5.x86_64
--> Running transaction check
---> Package acl.x86_64 0:2.2.51-12.el7 will be installed
---> Package cryptsetup-libs.x86_64 0:1.6.6-3.el7 will be installed
--> Processing Dependency: fipscheck-lib(x86-64) >= 1.3.0 for package: cryptsetup-libs-1.6.6-3.el7.x86_64
--> Processing Dependency: libfipscheck.so.1()(64bit) for package: cryptsetup-libs-1.6.6-3.el7.x86_64
---> Package dbus.x86_64 1:1.6.12-11.el7 will be installed
---> Package qrencode-libs.x86_64 0:3.4.1-3.el7 will be installed
---> Package tcp_wrappers-libs.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package fipscheck-lib.x86_64 0:1.4.1-5.el7 will be installed
--> Processing Dependency: /usr/bin/fipscheck for package: fipscheck-lib-1.4.1-5.el7.x86_64
--> Running transaction check
---> Package fipscheck.x86_64 0:1.4.1-5.el7 will be installed
--> Processing Conflict: fakesystemd-1-17.el7.centos.noarch conflicts systemd
--> Finished Dependency Resolution
Error: fakesystemd conflicts with systemd-208-20.el7_1.5.x86_64
 You could try using --skip-broken to work around the problem
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
fakesystemd-1-17.el7.centos.noarch has installed conflicts systemd: fakesystemd-1-17.el7.centos.noarch

What can I do now? I'm considering building it from scratch and working this way.
Comment: You might want to uninstall `fakesystemd` before trying to install `systemd`.
Comment: The best thing is to run the postgresql process directly. But, if you want systemd running (if you need multiple services started), check this link: https://developerblog.redhat.com/2014/05/05/running-systemd-within-docker-container/
Comment: "systemd problems"? Looks like docker does something wacky with an incomplete replacement wrapper for systemd that presumably lacks `provides: systemd`
Answer: Don't bother with systemd, run Postgres directly. The following shows how to get Postgres running inside a base Centos7 container:
$ docker run -it centos:centos7 /bin/bash
bash-4.2$ yum install postgresql-server sudo
[..snip..]
bash-4.2$ sudo -u postgres /usr/bin/initdb /var/lib/pgsql/data/
bash-4.2$ sudo -u postgres /usr/bin/pg_ctl start \
   -D /var/lib/pgsql/data -s -o "-p 5432" -w -t 300
bash-4.2$ ps -eo pid,args
  PID COMMAND
    1 /bin/bash
  118 /usr/bin/postgres -D /var/lib/pgsql/data -p 5432
  119 postgres: logger process
  121 postgres: checkpointer process
  122 postgres: writer process
  123 postgres: wal writer process
  124 postgres: autovacuum launcher process
  125 postgres: stats collector process

You can put that into a Dockerfile to make it re-usable.
Answer: https://hub.docker.com/r/centos/postgresql-10-centos7
I using command down here...

--restart="always"

option means always start when docker starting.

docker run -d --name postgresql --restart="always" \
        -e POSTGRESQL_USER=<yourId> \
        -e POSTGRESQL_PASSWORD=<yourPwd> \
        -e POSTGRESQL_DATABASE=<yourDbName> \
        -e POSTGRESQL_ADMIN_PASSWORD=<postgres's Password> \
        -p 5432:5432 centos/postgresql-10-centos7

try it.~
Comment: This solution doesn't help when one want to run off another centos based image. It would be better if you explain and detail how it's done (in the existing image you suggest) so that one can actually implement it in his own image.


Is there a way to control kubectl exec through something like rbac or anything else?

Question: I want to know if I can limit access to developers, we want to only allow developers to list pods and check logs, no ssh into pods, is that feasible ? This is what I tried, but it seems to not work ..
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: developer-role
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: [""]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: developer-role
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: developer-role
subjects:
  - kind: Group
    name: developer_customized

Answer: An empty set of verbs for the resource pods/exec does not grant any access, but also does not deny it. It would depend on other configurations.
To effectively deny access, you should remove the lines for pods/exec
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: developer-role
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list"]

Comment: Thank you for answering, but I did that, and it is not working, is it possible that another cluster rolebinding is interfering ? and in that case do you know how do I find the culprit ? though, I only have binded just 1 cluster role to my group.
Comment: There are some commands you can try: "kubectl get role bindings,customrolebindings --all-namespaces -o yaml" will list those resources and you can review "subjects". There is also the command "kubectl auth can-i create pods --as=". Same thing with list_pods
Answer: Base on your YAML file, you may want to try to add a verb get for pod/exec:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: developer-role
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/log"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: developer-role
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: developer-role
subjects:
  - kind: Group
    name: developer_customized

If it doesn't work, make sure the group membership is correct and doesn't have any conflicting ClusterRole. You can use command kubectl get events for gain more info about your cluster.


Windows refuses to recognize pyspark installation

Question: I installed pyspark using 'pip install pyspark' and it installed successfully. However, running 'spark --version' gives me 'spark' is not recognized as an internal or external command, operable program or batch file.
While 'pyspark --version' gives me The system cannot find the path specified.
Java (java 22 2024-03-19) and Python (3.11) are working fine.
I currently have PYSPARK_PYTHON set to python3, and %SPARK_HOME% set to [userdir]\AppData\Local\Programs\Python\Python311\Lib\site-packages\pyspark, but neither echoes properly. IE echo %PYSPARK_PYTHON% returns %PYSPARK_PYTHON% (same issue with SPARK_HOME). I have tried setting it manually through the advanced system settings and through the command line using setx /m. I see both in the system variables with the correct values. My best guess is that they somehow aren't set properly somehow (hence the bad echo) or the path isn't correct (should it be the bin directory? the cmd file?). Does anyone know the fix for this?
Im trying to do this in vscode if it matters but the issue is the same on the command prompt. I'm only trying to run spark locally so I don't need a full installation.
Comment: Have you added the pyspark location to your `PATH` variable? Also, make sure that pyspark actually includes an executable named `spark`. When you echo an environment variable and it displays as you indicate in your question, it means the variable is not set in your command session. You can check which variables are actually present bye typing `set p` to list all variable starting with that letter.
Comment: It may be that after using `setx` you need to start a new cmd shell for it to take effect.
Comment: Yeah, should have mentioned I already tried starting a new shell (and restarted vs code). Pyspark doesn't have a spark exe but it does have variations of that? spark-shell.cmd, etc.
Comment: Sorry, I have never use this package so cannot offer any further suggestions.
Answer: The problem is that your SPARK_HOME environment variable does not point to spark. PySpark is just the python library on how you can enroll all functionalities of spark or better said: PySpark is an interface for Apache Spark in Python.
You need to download spark from https://spark.apache.org/downloads.html.
Also consider downloading the corresponding winutils and set them as HADOOP_HOME environment variable.
Download the required winutils from https://github.com/cdarlint/winutils.

For all Pycharm users: It might be helpful solving PySpark problems by adding the PYSPARK_PYTHON environment variable. As I setup PySpark now a lot of times also on Visual Studio the way to setup spark in Pycharm is much smoother.
For all others that are not willing to run into installation problems on windows just use docker containers. But this is then another question on how to setup the Dockerfile and its mounting paths properly. :)
Comment: This did not fix the issue for me. I DLed Spark, which I located exactly like you did above, and changed the path to point to it just like in your photo (obv with the version changed). I change PYSPARK_PYTHON to point to the python exe like you have. I DLed the hadoop winutils and dropped the bin into the spark folder, and pointed it there (I assume there should just be a bin directory in that folder, as that's all there was in the git). I get the same error.
Comment: can u share an image on how your filesystem looks like? also an output of the java version in the cmd.
Comment: the winutils should be placed not in the spark folder. It should be placed in the hadoop//bin/ folder.
Also consider https://phoenixnap.com/kb/install-spark-on-windows-10
Comment: For some reason it's not letting me edit the post to add photos, but Java version is:
java 22 2024-03-19
Java(TM) SE Runtime Environment (build 22+36-2370)
Java HotSpot(TM) 64-Bit Server VM (build 22+36-2370, mixed mode, sharing)
I tried writing out the file structure, but it appears you can have newlines in comments, though it sounds like the hadoop location is the anomaly. I recognize it's not the convention, I just didn't want endless folders and since it's looking for the bin I didn't think it would matter, right? I don't have a version of hadoop, do I need one? Will try that link!
Comment: When you click on the link of my post then it shows you the requirements. If you want to use spark 3.5.1 then you need to download hadoop 3.3. or later. Also consider using Java 17. At the end you shoud have folder more or less similiar called like C:/hadoop/hadoop_3.3/ and C:/spark/spark3.5.1. The subfolders are made in case you want to switch maybe on sometime between the versions so you only need to change the environment variables. In the C:/hadoop/hadoop_3.3/bin just copy the winutils content to c:/hadoop/hadoop_3.3./bin/ to it.


Can I force the Linux kernel to use a specific vga adapter for output when booting?

Question: Suppose I have a Linux Xen machine with multiple graphics adapters + an onboard graphics adaptor.
Suppose they are all PCIe adapters.
I am guessing that the Linux kernel decides to use whichever graphics adapter it finds first to use as it's VGA output when booting?
Is there a way to direct this output to a specific adapter through a kernel boot option?
Reason: I'm wanting to keep onboard graphics as the console output for Linux (it is ubuntu server - not running X).  The other graphics adaptors will be passed through to multiple Xen guests.
Note: It doesn't make any difference as to whether a monitor or dummy VGA plug is attached.  Linux grabs one of the cards regardless of the state of any connected device.  What I'm wanting to do is to tell the kernel I want you to use the adaptor at xx:xx.0 pci address or something.
Comment: On second thought, I think this may be controlled by the BIOS
Answer: Usually, you should be able to select a 'BOOT VGA' in the BIOS, there you should be able to select 'onboard' or 'external'.
Usually linux should use the VGA adapter selected by the BIOS as the default.
The other cards can then be used by running Xorg on them later
Answer: Ya I've had a similar problem to this.  You may not like the solution, but I swear it worked for me!!
In my case I ended up using a hardware VGA plug with 3 resistors.  I made it by cutting a standard vga cable about 1.5 inches from one end of the cable.  Then I stripped out the wires and soldered on 75 ohm resistors across the correct wires.  After testing the ohms with a meter I globed hot glue over the whole thing.  I don't have a picture but it's pretty simple.
Here are 2 guides that cover this procedure:
http://soerennielsen.dk/mod/VGAdummy/index_en.php
http://www.geeks3d.com/20091230/vga-hack-how-to-make-a-vga-dummy-plug/
Finally, I remember some random post about certain motherboards having an option for this in the BIOS...mine did not.
Comment: Thanks, but that's not quite what I was after.  I'll edit my question.
Comment: Please don't sign your posts (your sig block gets added automatically anyway) and please stop posting links to your podcast site. The content may be excellent (I don't know, I've never listened), but it's still unsolicited self-promotion a.k.a. spam. Don't do it. If you want to advertise here, you can pay for it like the rest of the advertisers do.


Fetch Drupal nodes as json by url/title

Question: Our Drupal system has the Services module enabled and is able to return nodes as json using the following url format
http://www.example.com/my_endpoint_path/node/123.json
But I want to retrieve a node using the URL alias I set in the URL path settings tab. I hoped the following would work
http://www.example.com/my_endpoint_path/my_pages_URL_alias.json
But it does not. Is there a way to configure Drupal/Services to fetch nodes by that URL rather than the node id?
Answer: In Service Module API, Some hooks are presents to help you.
You need to alter or add your own implementation of this service.
The second link is a file where you have the default service implementation.
You can try to alter this configuration with your own by using "hook_services_resources_alter".
I can't test but you can try and custom this snippet :
function YOUR_MODULE_services_resources_alter(&$resources, &$endpoint) {
    $['node']['operations']['retrieve']['callback'] = '_your_retreive_method';
    $['node']['operations']['retrieve']['args']     = array(
        'name' => 'node_alias',
        'optional' => FALSE,
        'source' => array('path' => 0), // Put your path index here
        'type' => 'string',
        'description' => 'The alias node to get',
    );
}

function _your_retreive_method($node_alias){
    // transform your alias to nid
    $path   = drupal_get_normal_path($node_alias);
    if($path !== $node_alias && ctype_digit($nid = str_replace('node/','',$path))){
        return _node_resource_retrieve($nid);
    }
    return false;
}

Links :

Service API Docs
File with an implementation of a service
Comment: Here's a gist that handles multi-part aliases, provided the slashes are converted in the request: https://gist.github.com/laradevitt/6ab86bf7ebbd3215b20b
Comment: Thanks - this helped get me started. Unfortunately it doesn't work with multi-part aliases, which is a pretty serious limitation. (*my-alias* will work but not *about-us/my-alias*.) I've remedied this by converting the slashes on the client end and then reverting them in my callback function. I would rather send the alias as a parameter in a query, but couldn't implement it successfully.
Answer: Payou's answer worked for me with some additions. In the YOUR_MODULE_services_resources_alter function I also needed to specify a custom access function:
  $resources['node']['operations']['retrieve']['access callback'] = '_YOUR_MODULE_node_resource_access';
  $resources['node']['operations']['retrieve']['access arguments append'] = TRUE;

And add the access function:
<?php
function _YOUR_MODULE_node_resource_access($op, $node_alias)
{
  $path = drupal_get_normal_path($node_alias[0]);
  if ($path !== $node_alias && ctype_digit($nid = str_replace('node/', '', $path))) {
    return _node_resource_access($op, array($nid));
  }
  return services_error(t('Content could not be found.'), 404);
}

Fetch Drupal nodes as json by url/title


How to Configure Server 2008r2 as a NTP server when it is neither on a domain or a DC

Question: Here is the setup: We have Windows Server 2008R2 servers at customer sites. For separation of trust reasons, those machines are not members of the Customers Domain. They are also not acting as a Domain host themselves, they are strictly stand alone instances. This setup can't be changed.
Windows Servers attached to the customer network receive NTP updates through "w32tm". These servers are also attached to a private Stub network with additional machines not on the customer network. I would like to provide NTP service to hosts on the stub to provide better log synchronization.
All of the documentation I can find online seems to assume that the Windows Servers are Domain Controllers.
I only want to provide the NTP Server service to the stub network if possible. It shouldn't be reachable from the customer network.
What is the best way to setup?
Comment: Just a quick update, I haven't had a chance yet to work on this issue. As soon as I'm able to test on a system I will update.
Answer: NTP Server is not a role of a Domain Controller and not even a feature of Windows Server alone.
Windows Time Service (w32time) has both client and server built-in on every Windows computer. The server side is disabled by default and automatically enabled on Windows Server during dcpromo.
The server state is controlled via registry key:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer]
"Enabled"=dword:00000001

Just Run cmd as Administrator and use commands:
reg add HKLM\system\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer \
    /v Enabled /t REG_DWORD /d 0x1 /f
w32tm /config /update

You'll instantly have a NTPv3 compliant Time Server on your Windows machine.
Comment: There was a second part to this question is how to limit it to the Stub network only. the host needs to be able to query out but only provide to the stub.
Comment: There is no ACL control on W32Time, so limitations must be done in a firewall or by network infrastructure. I'd say it is a bit irrelevant: there is no security concerns and certainly no possibilities for resource exhaustion as W32Time client polls very infrequently even if someone manually changes the time server used.
Comment: This a management driven reason. If I am only making changes that only affect the stub network then I just need to meet our internal change control requirements. If I am making a change that opens *any* new services on the customer network then we have to have a software review with the customer and get approval. Like you say its a benign function. I'd like to avoid several hours of explaining to management at another company why this is a good idea.
Comment: It's still a component that is already installed and a service that is already provided, so I'd consider it as a minor configuration change instead of a new service / software. It's like negotiating over a single Windows Update, which actually may have more effect and may be less predictable. I'm truly sorry you have to work under those limitations.
Answer: If you use a time server on the client network for time updates, you won't be able to block NTP service from being accessible from that machine as NTP is a bidirectional protocol using port 123/udp.
If you want to block any machine on the client network from accessing the NTP service on your server, you'll have to choose an external time server; e.g. uk.pool.ntp.org and configure the server firewall to block all access to port 123/udp from the client network.
(This would have been a comment on PaterSiul's answer, but my rep isn't high enough to comment!)
Comment: This question is for windows, however you are partially incorrect. On *nix based NTP implementations you can set server to only respond to requests from specific ip's defined by a mask. NTP servers are not broadcast services. This isn't a firewall function, but a function of the NTP server itself. Where we have a *nix host on the client network I already use them for this functionality. I am looking for similar functionality for the client sites with only windows hosts on the customer network.
Answer: Try techrepublic or, for a bit more detail, this Microsoft blog post.
Update:
Esa Jokinen has the main part of my links in his answer.
To answer the second part, which I first missed:
I don't know of a way to restrict windows time service to an interface or an ip range directly. I'd solve that with a firewall rule:
netsh advfirewall firewall add rule name="NTP" dir=in action=allow protocol=UDP localport=123 remoteip=18.104.22.168,172.16.0.0/16 enable=yes

"remoteip=22.214.171.124,172.16.0.0/16" is just an example, the rest should work for what you wish, assuming you didn't change the default policy from blocking to accept.
Comment: Its preferred when linking content that you provide the link as a source and provide the content in case the link ever becomes inactive.  Why didn't any of this come up before. Also how do you suggest restricting the server access to the Stub network only -- I can't have it accessible on the customer side. I still need to query the customer servers for the same. (http://support.ntp.org/bin/view/Support/WindowsTimeService) shows that it cant operate as a a high stratum, but that is fine since the stub network currently has NO time service and timing is not super critical for this need.
Comment: In the next month I'm finally getting a chance to look into this again. We are setting up another production equivalent test environment to validate some architecture changes around upgrade deployments.


When did Red Hat start shipping PHP 5.3 with 5.x?

Question: Okay this is a PSA more than a question because I know the answer: January 13, 2011.
See: https://rhn.redhat.com/errata/RHEA-2011-0069.html
Colour me surprised though, didn't hear anything about in the blogosphere until I got a Security Errata notice today.
I have been using the REMI repo for this in the past but will switch over to the Red Hat blessed PHP 5.3.
Don't down-vote me bro! I'll select as the best answer the source that broke the news first (other than Red Hat of course). People have wanted this for so long I'm just amazed that it's finally happened!
Comment: Very nice that RedHat has a recent version of PHP, but not really a SF question :-)
Answer: Since 5.6
http://meetbot.fedoraproject.org/fedora-meeting/2009-09-11/fedora-meeting.2009-09-11-21.00.log.html

21:35:27 <derks> I started the IUS project due to the demand that our customers have for the latest and greatest versions of only a small set of packages: PHP/MySQL/python.. etc
21:37:26 <derks> php52, php53, mysql50, mysql51, python26 (plus a few supporting python26 packages)
21:37:32 <derks> all following upstream stable

http://meetbot.fedoraproject.org/fedora-meeting-1/2010-02-27/fedora-meeting-1.2010-02-27-17.12.log.html

19:35:00 <smooge> no 5.6 will have php53


Google keeps indexing /comment/reply URL

Question: With the new update of Google algorithm called Penguin, I think my site was being penalized due to webspam. But of course I don't create post which seems to be spam to Google. It is just I think how Google index my site.
I found out that Google index the URL of my site like:
http://www.example.com/comment/reply/3866/26556
So there are so many comment/reply URL index by Google. I have already added:
Disallow: /comment/reply/
Disallow: /?q=comment/reply/
but still Google still index this URL.
Any idea how to prevent Google from indexing comments?
Here's my .htaccess:
<FilesMatch "\.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$">
  Order allow,deny
</FilesMatch>

Options -Indexes
Options +FollowSymLinks
ErrorDocument 404 /index.php
DirectoryIndex index.php index.html index.htm

<IfModule mod_php5.c>
  php_flag magic_quotes_gpc                 off
  php_flag magic_quotes_sybase              off
  php_flag register_globals                 off
  php_flag session.auto_start               off
  php_value mbstring.http_input             pass
  php_value mbstring.http_output            pass
  php_flag mbstring.encoding_translation    off
</IfModule>

<IfModule mod_expires.c>
  ExpiresActive On

  ExpiresDefault A1209600

  <FilesMatch \.php$>
    ExpiresActive Off
  </FilesMatch>
</IfModule>

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteRule "(^|/)\." - [F]
  RewriteCond %{HTTP_HOST} !^www\. [NC]
  RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]

  <IfModule mod_headers.c>
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.css $1\.css\.gz [QSA]

    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.js $1\.js\.gz [QSA]

    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    <FilesMatch "(\.js\.gz|\.css\.gz)$">
      Header set Content-Encoding gzip
      Header append Vary Accept-Encoding
    </FilesMatch>
  </IfModule>
</IfModule>

Comment: You may get a better respone over at webmasters.stackexchange.com.
Answer: I would try adding
RewriteCond %{HTTP_USER_AGENT} ^Googlebot
RewriteRule ^comment/reply/$ - [G,L]
RewriteRule ^comment/reply/(.*) - [G,L]

to .htaccess or Apache config to inform spiders that the links should be removed.  Just make sure to put it before the catch-all rule at the bottom of the standard Drupal .htaccess.  The problem is coming up with a list of spiders to add as [OR] conditions to the RewriteCond so that they get a 410 but Drupal handles normal requests.
Google Webmaster Tools has a URL removal tool, too, but I have never tried using it with patterns before.
Comment: hi, thank you very much for the information. I don't know where to put it on my .htaccess. please guide me where to put it. I have edited my question to include the .htaccess.
Comment: also i don't get what you mean by "The problem is coming up with a list of spiders to add as [OR] conditions to the RewriteCond so that they get a 410 but Drupal handles normal requests."
Comment: 1: try placing the lines after `RewriteRule "(^|/)\." - [F]`.  2: if Google is indexing the links, it is likely that Yahoo!, Bing, etc, will start indexing them, too.  The `robots.txt` hing should work, but it will take time to heal.
Comment: I just got some problem when using this code in .htaccess. the comment does not save and it redirect me to /comment/reply/ after submitting the comment form. please don't use this code.
Comment: You may need to copy the RewriteCond line to between the RewriteRules so it applies to each of them.
Comment: the best solution to this is adding
Answer: I am facing the same problem at the moment. One possibility is to modify the "reply" link of the main posts, by adding the rel="nofollow" option. This should be doable by writing an external module, or a custom theme function.
The advantage of this method, is that all search engines will avoid following these links, so you do not have to be explicit for google


Git merge diverged directory from branch into a new renamed directory while keeping original

Question: Context: I have a big repo with lots of little creative coding projects in different directories. Sometimes I modify a directory on a branch and it turns into something new I want to keep in addition to the original.
In my repo I have a directory Dir holding a project.
On branch Experiment, Dir has diverged from master. I actually now want to keep both variants of Dir, the original on master and the one on Experiment.
How can I end up on master with both its existing Dir and also a directory ExperimentDir that has the version of Dir currently sitting on branch Experiment?
In effect, I'm forking a directory within a single repo. I could just copy and paste, but I want to keep the history of changes to these files from both branches.
I tried on branch Experiment to git mv Dir ExperimentDir and then merge into master. But I'm left with a convoluted merge conflict as git is trying to merge in the rename to avoid duplication.
Answer: I found a solution by adapting jthill's answer https://stackoverflow.com/a/66609162. That question and answer address a more general question which is why I couldn't find it. It feels worth answering this specific scenario: merging a single directory into a renamed directory without combining it with the original.

On branch Experiment, delete everything apart from the divergent directory. Move contents of that directory to the root and delete the now empty directory. Commit.

In git:

git checkout master
git merge --no-commit -s ours Experiment

Automatic merge went well; stopped before committing as requested

git read-tree -u --prefix=ExperimentDir/ Experiment
git commit -m "Merge Experiment subtree into ExperimentDir"
git checkout master
git merge -s subtree Experiment

Merge made by the 'subtree' strategy.

Note that when logging the ExperimentDir directory I need to use --follow to find the history, but when logging Dir I need to not use -follow to avoid getting the history of ExperimentDir included as well.


Recurrent Neural Network model with more than one input

Question: I know RNNs (with LSTMs or GRUs) are now one of the most promising options for modelling sequential data, when ordering of the data matters. However, sometimes there are also some categorical variables associated to each data point from the sequence that also matter to model the pattern. For example, consider some time-series with daily frequency; one could have a sequence like $343, 365, 222, 34, 675, 546$ corresponding to the days-of-year $1, 2, 3, 4, 5, 6$ and another similar sequence corresponding to the days $45, 46, 47, 48, 49, 50$. For some reason, day $7$ could have high probability of yielding a value very similar to the previous ones, e.g. $388$, while day $51$ would have high probability of yielding some value 2 orders of magnitude greater, e.g. $23467$. Therefore, my question is:
How can we model sequential data together with class variables associated to each data point?
It occurred to me I could think about this as modelling more than a sequence simultaneously - one sequence being the data points (the "data sequence") and the others being the class-labels ("class sequences"), which for some use-cases could be deterministic (as in "Tuesday always comes after Monday" or "day 7 comes after day 6", etc.). I would then feed two (or more) synchronized sequences and try to predict the next time-step value for each. E.g.,
<code>Input: [[  1,  2,  3, 4,  5,  6],  -> Output: [[  7],
        [343,365,222,34,675,546]]              [388]]

Input: [[ 45, 46, 47,48, 49, 50],  -> Output: [[   51],
        [342,368,242,43,612,549]]              [23467]]
</code>
Is this the correct approach? How should one model an RNN with several inputs? In particular, what's the input and what's the output? Should one use one-hot encoding with a high dimensional vector? No, right?
Another case where I think this could be applied would be trying to model many correlated "data sequences" simultaneously (e.g. temperature time-series for several cities in a country)
Also, how can we implement something like this using Tensorflow?
Answer: The input, as you suggested, should be the sequence you're trying to predict concatenated with the day of year. The output should just be the predicted value of the sequence at the next time-step, there's no need to predict the next day of year as well, since you already know what it will be.
One hot should be ok with just 365 categories. It only starts to get problematic with language models when you have tens of thousands of words in your vocabulary.
Another thing which may be important is to apply some sort of transformation to your sequence (maybe log) which will make predicting huge values much easier without having huge gradients and associated issues.


Subordinate matrix norm inequality in research paper where authors replace $||A||_{op}^2$ with $||A^TA||_{op}$

Question: I'm perusing this paper. In page 8, I came across this:

My question is about equation 45. Also in this question I don't care about neither $u$ nor $M$, I mentioned them just to give some context.
In short my problem is:
Let $y \in \mathbb{R}^n$, $x \in \mathbb{R}^m$, $A \in \mathbb{R}^{n \times m}$.
$\|.\|$ is the euclidean vector norm, $\|.\|_{op}$ is a matrix norm defined as:
$$\|A\|_{op} = \max_x \frac{\|Ax\|}{\|x\|}$$
So I have: $|y^TAx| \leq \|y\|\space\|A\|_{op}\|x\|$. As long as I have an inequality with the absolute value of something, it is the same as $y^TAx \leq \|y\|\space\|A\|_{op}\|x\|$
Now I will multiply both sides by 2 and multiply the left side by:
$$\frac{\sqrt{\tau}}{\sqrt{\tau}}\frac{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{4}}}{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{4}}}$$
Then I get:
$$2y^TAx \leq\ 2
\frac{\|A\|_{op} \|y\| \sqrt{\tau}}
{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{4}}}
\frac{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{4}} \|x\| }
{\sqrt{\tau}}$$
using $a^2+b^2 \geq 2ab$:
$$2y^TAx \leq\
\frac{\|A\|^2_{op} \tau}
{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{2}}} \|y\|^2 +
\frac{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{2}} }
{\tau} \|x\|^2 $$
In equation 45, the term $\|A\|^2_{op}$ doesn't appeard, it is replaced by $\|A^TA\|_{op}$. I know that $\|A\|^2_{op} \geq \|A^TA\|_{op}$, so I don't know why the authors make this replacement. In the paper, they say that $A$ is any matrix with no special properties.
The next line comes from doing:
$$(\tau\sigma \|A^TA\|_{op})^{\frac{1}{2}}z = \frac{\|A^TA\|_{op} \tau}
{(\tau\sigma \|A^TA\|_{op})^{\frac{1}{2}}}$$
$$\tau\sigma \|A^TA\|_{op} z = \|A^TA\|_{op} \tau$$
$$z = \frac{1}{\sigma}$$
Why do authors replace $\|A\|_{op}^2$ with $\|A^TA\|_{op}$ in equation 45? With this replacement the inequalities don't necessarily hold. Is there a property I am missing?
Comment: Hint: Use `\|` instead of `||` to get better-looking double bars ($\|A\|$ instead of $||A||$).
Comment: @Rahul thanks for the hint, I didn't know about it. I'll use it next time.
Answer: I'll use $\langle x,y\rangle$ to denote the usual inner product $y^*x$.
It is always true that $$\|A\|^2_{op}=\|A^*A\|_{op},$$ where $A^*$ denotes the adjoint (the conjugate transpose, so just the transpose if $A$ is real). This is very easy to prove. The reverse inequality from the one you mention is the easy one:
\begin{align}
\|A\|^2&=\max\{ \|Ax\|^2:\ \|x\|=1\}=\max\{\langle Ax,Ax\rangle:\ \|x\|=1\}\\
&=\max\{\langle A^*Ax,x\rangle:\ \|x\|=1\}\\
&\leq \|A^*A\|,
\end{align}
where the inequality is simply Cauchy-Schwarz and definition of the operator norm. Now, from the above inequality, you have
$$
\|A\|^2\leq\|A^*A\|\leq\|A^*\|\,\|A\|.
$$
So, for nonzero $A$ (the zero case is trivial) you have $\|A\|\leq\|A^*\|$. Apply the above to $A^*$, to get $\|A^*\|\leq\|A^{**}\|=\|A\|$, so $\|A^*\|=\|A\|$. If you now go back to the inequalities we had,
$$
\|A\|^2\leq\|A^*A\|\leq\|A^*\|\,\|A\|=\|A\|^2.
$$
So $\|A\|^2=\|A^*A\|$.


Source Control at home

Question: I am trying to setup a dev environment at home. My primary development environment is VS.NET 2008 (now it is VS 2010 Premium) with SQL Server 2008.
Objectives / Requirements :
- Should be simple and lightweight
- Should have a Visual Studio plugin (cost should be less than $50, if its not free)
- should work just fine with continuous integration systems like TFS and CruiseControl.NET
- VSS is not an option.
Thanks in advance,
SK
Answer: What about just using TFS?
Or you could use subversion with ankhsvn for vs integration.
Answer: I'm using Perforce's free 2-user license, and I'm pretty happy with it.
Answer: I recommend mercurial or git as distributed version control systems seem to be the latest trend
Answer: VisualSVN offers a free, easy to install SVN server for Windows. They have a VisualSVN plugin that integrates with Visual Studio (but is not free - $49 for a personal license).
AnkhSVN is an open source alternative to the commercial VisualSVN plugin for Visual Studio integration but I've never used it before so I can't comment on it.
Answer: Subversion and switch to TeamCity for continuous integration
Answer: If you're developing alone, Perforce is free for up to two people and five machines, and works very well for a single-developer model.
If you don't mind your projects being opensource, Github implements Git servers, or for $5 a month, you can have 5 projects closed-source.  That's a very different development model than perforce/cvs/subversion/vss-- take a look at hginit for a tutorial on distributed version control before going this route, it will really help.

Source Control at home


Is this method of arriving at a statistic valid?

Question: The following is an excerpt from an article about gender bias :

Overall, 34 per cent of respondents readily admitted that their family
  preferred having more boys. Given the sensitive nature of this
  question,  this number is a lower bound since some people will
  undoubtedly give  socially desirable responses.
In order to estimate the true levels of  son preference, we ran a
  "list experiment" which protects the anonymity  of the survey
  responses. The respondents were randomly divided into two  groups,
  given a list of statements, and then asked the number of  statements
  with which they agreed. Only half of respondents were given a
  statement about son preference, and therefore, we were able to
  estimate  the share of respondents who agreed to the sensitive
  statement by  comparing the means of the two respondent groups. Using
  this technique,  we found that once granted the cloak of anonymity, 48
  per cent of  respondents have a preference for sons in their family.

Is this technique valid? If so, how exactly does it work? This the way I understand it (possibly wrong):
Since the respondents were randomly divided into 2 groups, it can be expected that the distribution of answers from both the groups to be identical if both the groups are given exactly the same survey. Now, say one group is given a list of 10 statements and the other is given a list of 11 statements, the extra statements being the one about gender preference. There will be a shift in the distribution and the difference gives us the percentage of people with a preference. Is my interpretation correct? If so, how is the exact percentage arrived at?
Comment: The survey seems fatally flawed to me, due to the way in which the original question was phrased: "For your family, is having more boys than girls preferred?" This treats boys and girls asymmetrically, implicitly suggesting boys *ought* to be preferred over girls. At a minimum half of the surveys should randomly have reversed the question to ask "For your family, is having more girls than boys preferred?" (The question phrasing has other problems, too.) Because the subsequent "list experiment" is described so vaguely, any comments about it would be purely speculative.
Comment: @whuber Nice point. Can you describe what the other problems with the question phrasing are?
Answer: It sounds valid to me although you have to make some assumptions.  Suppose there are $N$ respondants, and number them $i=1,2,\ldots,N$.  Suppose there are ten fake questions and one real question (let's suppose the real question gets put last, so it is number eleven).  Number questions $k=1,2,\ldots,K,K+1$.  So, in this case $K=10$ and the real question is $k=11$.  Denote the (real, truthful) answer of person $i$ to question $k$ as:
\begin{align}
Y_i^k &= \left\{
\begin{array}{l}
1 \; \text{agree}\\
0 \; \text{disagree}
\end{array}
\right.
\end{align}
Let's call the population mean of each $Y^k$, $\mu^k$.  We are interested in the population mean of $Y^{K+1}$.  The mean of a dummy variable is the same as the
probability that it is one which is the same, here, as the proportion of people who
want a boy.
Now, assume that as long as there are at least ten questions, the respondent tells the truth.  Person $i$ randomized into ten questions will report $S_i = \sum_{k=1}^K Y^i_k$.  The person randomized into eleven questions will report $S_i^+ = Y^{K+1}_i + \sum_{k=1}^K Y^i_k$.  Let $C$ be the set of people randomized into the ten question treatment and $\#(C)$ be the number of people in set $C$.  Similarly for $T$ and the eleven treatment.
If the randomization is good and under the assumption of truthful reporting, we get:
\begin{align}
E\left(\frac{1}{\#(C)} \sum_{i \in C}S_i \right) &= E(S) = \sum_{k=1}^K \mu^k\\
E\left(\frac{1}{\#(T)} \sum_{i \in T}S^+_i \right) &= E(S^+)= \mu^{K+1}+\sum_{k=1}^K \mu^k\\
\end{align}
The first equality in each line is true by the usual arguments in RCTs.  I.e. they get the expectations correct for the underlying latent variables in the treatment and control arms.  And that is pretty much it, the thing we are interested in, $\mu^{K+1}$ is unbiasedly estimated by the difference in those two:
\begin{align}
\mu^{K+1} &= E\left( \frac{1}{\#(T)} \sum_{i \in A}S^+_i
                             -\frac{1}{\#(C)} \sum_{i \in A}S_i\right)
\end{align}
There is a lot going on in the truthful reporting assumption here, though.  There is the obvious "maybe ten is not enough to suppress lying" thing.  But there are also the usual survey issues.  Maybe the first ten questions really annoyed respondents and they were feeling disagreeable by question 11.  Maybe some respondents have trouble counting, remembering where they are in the count, and thinking about answering questions (this isn't trivial: 50% of people have below average IQ).  Maybe this gets worse with one extra question.  If the counting mistakes are biased in one direction (and why wouldn't they be?), you can get problems.  One could go on in this vein for a while.


Obter numero de elementos dependendo do numero máximo de elementos por grupos

Question: Eu tenho duas tabelas e pretendo o obter o name da tabela A dependendo do Num3. Por exemplo , na tabela A, temos o num2 e na tabela B há os valores correspondentes ao Num1.
O que pretendo é que o name apareça repetido o número de vezes correspondente ao número máximo de num3 relacionado com o num2.
Por exemplo , o num2 = 3 tem como num3 máximo o valor 4, ou seja queraque ele apareça 4 vezes
Usando estas tabelas:
-- A (name,Num2, local)
insert into A values ('Favela',1,'WE');
insert into A values ('Lamosa',2,'NA');
insert into A values ('Luz',3,'S0');

-- B (Num1,name,Num2, Num3)
insert into B values (1,'Maria',1,1);
insert into B values (2,'Jorge',2,1);
insert into B values (3,'Teresa',2,1);
insert into B values (4,'Rui',3,1);
insert into B values (1,'Fran',3,3);
insert into B values (2,'Juliett',3,3);

Answer: Com base na resposta (do Bacco) para a pergunta, montei um gerador para repetir o aquario conforme o id_especime, veja:
SELECT A.NOME 'AQUARIO' FROM
    (SELECT (D1+D2*10+d3*100)+1 AS GERADOR FROM
        (SELECT 0 AS D1 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4
         UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) T1,
         (SELECT 0 AS D2 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4
         UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) T2,
         (SELECT 0 AS D3 UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4
         UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) T3) T
    , AQUARIO A
WHERE T.GERADOR <= (SELECT MAX(E1.ID_ESPECIME) 'MAXIMO'
                    FROM ESPECIME E1
                    WHERE E1.ID_AQUARIO = A.ID_AQUARIO
                    GROUP BY E1.ID_AQUARIO)

Observações

Ao invés de começar em zero (0), modifiquei para começar em um (1);
O gerador da forma como está, gerará números de 1 até 1000 - pode não ser compatível com seu modelo caso o ID_ESPECIME seja maior que 1000 (hoje ou no futuro);
Answer: Você pode utilizar a seguinte query se não houverem "buracos" no campo id_especime:
SELECT x.nome
  FROM (SELECT max(e.id_especime) as maximo_id_especime,
               a.nome
          FROM aquario a
               INNER JOIN especime e ON e.id_aquario = a.id_aquario
         GROUP BY a.id_aquario, a.nome) x
       INNER JOIN especime e ON e.id_especime <= x.maximo_id_especime
 GROUP BY x.nome, e.id_especime

Edit
Após a nova informação fornecida na pergunta, caso queira utilizar a subquery apenas no WHERE você deve fazer um CROSS JOIN que juntará todas as informações da outra tabela e restringi-la no WHERE.
SELECT a.nome
  FROM aquario a
       CROSS JOIN especime e
 WHERE (SELECT max(e2.id_especime)
          FROM especime e2
         WHERE e2.id_aquario = a.id_aquario) >= e.id_especime
 GROUP BY a.nome, e.id_especime

Comment: É possivel fazer isso com a  sub-interrogação fora do From ?
Comment: @ForçaChape O que é uma sub-interrogação?
Comment: o que está no From fazer fora do From
Comment: @ForçaChape não entendi sua intenção. O que está dentro do `FROM` determina o número máximo de interações.
Comment: Sim mas para o que tenho de fazer não pode haver interrogações ( que é o que está no from ) no campo do From , só no where
Comment: @ForçaChape dessa forma não há como, afinal você tem que descobrir o número máximo de alguma forma. Sua pergunta parece estar incompleta, se não deve ser utilizada `subquery` você deve informar isso na pergunta
Comment: deve ser usada subquery , mas não no campo do from , só no do where e também não sei como
Comment: @ForçaChape então coloque isso na pergunta. Sua pergunta não está clara.
Comment: sim sim , eu também só percebi agora
Comment: @ForçaChape veja a segunda solução na parte de baixo da resposta.


How to test the linearity between two non normal distributed variables

Question: I have two variables $(x_i,y_i), \; i=1, \dots 300$ and I would have liked to apply a linear regression on them, but as you can see in the scatterplot below I have a very bad linear trend.

As I need to motivate it in an essay, I would like to have a measure of the amount of linearity and I used the Pearson product-moment correlation coefficient, obtaining a value of -0.1559585. But after having tested the normality of the variables with a Shapiro-Wilk test, I have obtained that that the X values are not normally distributed, therefore I cannot use the Pearson coefficient to do that. I read that I could computed Spearman's rank correlation coefficient as the X values don't follow a normal distribution, but this coefficient gives an estimation of the monotonic association between X and Y, while I would like to have a quantification of the linearity between X and Y. Do you know how I can compute a quantity that express this, please?
Thank you very much.
Edit: The qqplot of X is the following
Comment: Have you thought about using a Box-Cox transformation?
Comment: Thank you for your advice :) No, but I have just read about it and I tried a Box-Cox transformation on X and I have obtained a better qq plot. I draw an histogram of X and the data now seem gaussian. I have calculated the Pearson coefficient for Y and X transformed, and I have obtained a result of -0.1419915, slightly less than before the transformation. But I don't know how I should interpret this result. What does this tell about the relationship between the original data? Doesn't the transformation change the nature of the data?
Comment: Two points. First, it seems that there is some hard limit with many points bunched up at about Y = 28, below which no points are found. If there is such a limit, then your investigation of a linear relation is fraught with difficulty. Second, why do you want "a measure of the amount of linearity" in the first place? Wouldn't a clear description of the relation between X and Y be better, even in transformed scales, regardless of linearity in the original scales?
Comment: 1) Yes, the Y values are all of them measure of a quantity that has been recorded if and only if it has a value greater than 26, why does this affects the investigation of the linear relationship? 2) I need to do a lot of  regressions (more than 75) on identical datasets aquired at different time scales and I would like to have something  - a number - that quantifies if I can apply the regression or not for each dataset(and the only assumption that fails is the linearity), as I cannot show a scatter plot for each dataset.
Answer: Why are you even looking at the distribution of $X$?  This has no effect on whether or not the relationship between $X$ and $Y$ is linear.  But that aside, Pearson's correlation measures the strength of linear association, period.  There are no distributional assumptions needed.  Just look at a scatterplot of the points (which by the way you haven't shown, you've provided a Q-Q plot) to see if it's linear and report the correlation.
Also, goodness of fit tests will almost always result in a rejection of the null hypothesis with any reasonable sample size, so they shouldn't be relied on too much.
Comment: The person in this article doesn't seem to give any reason for why data have to be bivariate normal in order for a correlation to be meaningful.  People justifiably calculate correlations all the time without bothering to check for such an assumption.  Just as a counterexample, let $X$ have any non-normal distribution and set $Y = X$.  The correlation is one and this tells us quite a lot about the relationship without even mentioning the distributions involved.  Statistics aside you can tell just by looking at the plot that $x$ and $y$ don't have much of a relationship here linear or otherwise
Comment: Thank you for your answer.  I am looking at the distribution of X because I hava read a lot of references(here an example http://www.statstutor.ac.uk/resources/uploaded/pearsons.pdf) that say that before computing Pearson's correlation coefficient, the normality of the variables should be checked. Aren't they correct? Thank you. P.S. Sorry, in my last question I have changed the scatterplot by chance, now it is fixed.
Answer: You might use a Q-Q plot to further assess normality. The X values do not appear too skewed.  If the Q-Q plot does not suggest a drastic departure from normality I think $R^2$ (square of the Pearson) is a reasonable measure of linearity. I have never done a Shapiro-Wilk test on real data that did not turn out to be significant so I don't depend on it alone.
Comment: Thank you very much for your help. I have posted in the question the QQ plot, it seems not a good normal distribution. Just to be sure, should I fit my points with a linear model and look at the obtained R2? As it is a measure of how much a straight line adapts to the data, can I consider it as a measure of linearity? Thank you very much.
Comment: This is the QQ plot of X? It doesn't look that bad to me, maybe a bit skewed but it is not a U-shaped distribution or bi-modal. Try replacing the qqnorm command with qqline and see if the red line does not go straight through most of it. Yes, I would report $R^2$.

Even if you transformed X to be normal, you can tell from the scatter plot that there is not much of a trend here so $R^2$ is still likely to be low. Try a box-cox transformation on X and see what happens.
Comment: Thank you for your answer. I have attached to the question the qqplot with the qqline. I also tried a box-cox transformation and I reported the result in my answer above to JohnK, but I don't know how to interpret this result. Might you have any suggestion, please?
Comment: I'd say after the transformation that makes X more normal and the Pearson coef is still low means there is no linear relationship and you have your answer. Lower by .01 is essentially the same.
Comment: But doesn't the transformation change the nature of the data? Why I can rely on a coefficiente computed after a modification of the values of the data? Sorry, I have difficulties to understand this.
Comment: It does change the nature of the interpretation. After transformation the Pearson is now a measure of linearity between Y and the transformed-X. With box-cox especially that relationship is difficult to visualize.


How to modify response body before caching inside a service worker?

Question: I thought it would be rather simple but I am struggle to successfully send response then modify my cloned response that is being stored in cache inside my service worker.
Ultimately I want to appended my cached data with the time and date of the fetch.  I am using my service worker to first fetch this data from the network and if its not available fallback to the cache.
This is the code from my service worker
<code>        event.respondWith(
            fetch(event.request)
            .then(function(e) {

                console.log('response', e)
                var responseToCache = e.clone();

                responseToCache.json().then(function(data) {
                    var today = new Date();
                    var date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
                    var time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
                    data.cached = { date: date, time: time }
                    return data
                })

                // var init = {
                //     status: response.status,
                //     statusText: response.statusText,
                //     headers: { 'X-Foo': 'My Custom Header' }
                // };

                // response.headers.forEach(function(v, k) {
                //     init.headers[k] = v;
                // });

                // var responseToCache = response.text().then(function(body) {
                //     var today = new Date();
                //     var date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
                //     var time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
                //     var bodyObj = JSON.parse(body)

                //     bodyObj.cached = { date: date, time: time }
                //     body = JSON.stringify(bodyObj)

                //     return new Response(body, init);
                // });

                caches.open(CACHE_NAME)
                    .then(function(cache) {
                        cache.put(requestURL, responseToCache);
                    });

                return e
            })
            .catch(function() {
                console.log('data offline')
                return caches.match(requestURL);
            })
        );
</code>
Answer: Something like the below should work
<code>    event.respondWith(
        fetch(event.request)
        .then(async function (originalResponse) {
            console.log('originalResponse', originalResponse)
            // consume the original response, and create a new one to return to the browser
            const originalData = await originalResponse.json();

            var today = new Date();
            var date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
            var time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
            const modifiedData = {
                ...originalData,
                cached: {date: date, time: time }
            }
            const newResponse = new Response(JSON.stringify(originalData), {
                status: originalResponse.status,
                statusText: originalResponse.statusText,
                headers: originalResponse.headers
            });
            const responseToCache = new Response(JSON.stringify(modifiedData), {
                status: originalResponse.status,
                statusText: originalResponse.statusText,
                headers: originalResponse.headers
            });

            return caches.open(CACHE_NAME)
                .then(function(cache) {
                    cache.put(requestURL, responseToCache);
                    return originalResponse
                });
        })
        .catch(function() {
            console.log('data offline')
            return caches.match(requestURL);
        })
    );
</code>
What does it do?

consumes the json from the original response.
modifies the data.
creates two responses: 1 for cached data, 1 for non-cached data.
return the addingToCache promise, which adds the response with cached data to the cache.
resolve to the original response.

Basically, you want to cache a response that is different from the actual response, so, create a new one. Note that I did not test


Is there anyway to compile/memorize a webm/mp4 file into an .exe program? (C++)

Question: Might be a stupid question.
I'm making a C++ windows.h application, and I'd like for it to play a short video. It would be nice if I could somehow(?) compile the video into the program — I plan on sending the exe to others, it'd be nice if I didn't have to send the video with it as a separate file for the exe to load in at runtime.
I'm running Windows 10, writing and compiling with just Emacs and g++.
Comment: https://balau82.wordpress.com/2012/02/19/linking-a-binary-blob-with-gcc/
Answer: This previous question looks like it would be helpful for your case: C/C++ with GCC: Statically add resource files to executable/library
Comment: That is helpful, but... You've produced a link-only answer. You can salvage it by summarizing the contents of the link so that future askers can recreate the information or find a related link when the original link rots.


Show that if $a\mid b$ then $2^a-1\mid 2^b-1$

Question: Show that if  $a|b \  \  then\ \  2^a-1|2^b-1$
I've seen this assertion in the proof of another problem but the author hadn't given any reason:
[Original problem containing this assertion][1]
  [1]: Proving that $\gcd(2^m - 1, 2^n - 1) = 2^{\gcd(m,n )} - 1$    .
Seemingly simple,I have no idea how to progress in solving this problem!!
Comment: For any $x,y\in\mathbb R, n\in\mathbb Z^+$ we have $x^n-y^n=(x-y)\left(x^{n-1}+x^{n-2}y+\cdots+y^{n-1}\right)$. Let $x=2^a$, $y=1$, $n=\frac{b}{a}$.
Comment: @HamidRezaEbrahimi No problem. There are enough answers here, and for the original problem.
Comment: @DietrichBurde Unfortunately , when typing the question the related questions are not fully listed,and the Math search engine is not perfect!
Answer: Rewrite $b=ac$ with c an integer.
Then $2^b-1=2^{ac}-1=(2^a)^c-1$.
Now you use the identity $x^n-1=(x-1)(1+...+x^{n-1})$ with $x=2^a$ and $n=c$ and here you go!
Answer: If $a \mid b$, then $b = ka$. We want to find $2^b - 1 \pmod{2^a - 1}$. But this is $2^{ka} - 1 \equiv (2^a)^k - 1 \equiv 1^k - 1 \equiv 0 \pmod{2^a - 1}$. So $2^a - 1 \mid 2^b - 1$.
Note that this applies to any integer $n$, not just 2.
Answer: If $a|b$ then $b = m.a$ where $m$ is an integer.
$2^b - 1 = 2^{m.a} - 1 = (2^a)^m - 1 = (2^a - 1).(1 + 2^a + (2^a)^2 + ... + (2^a)^{m-1} )$
Thus $2^a - 1 | 2^b - 1$.
Answer: $a|b \Rightarrow b=ka, k \in \mathbb Z$
$$2^b-1=2^{ka}-1=(2^a)^k-1=(2^a-1)((2^a)^{k-1}+(2^a)^{k-2}+...+1)$$