Custom autocomplete form and autosubmit

Question: I've added an autocomplete function for my views_exposed_form (a text input) using this tutorial and it works (https://www.linkedin.com/pulse/implementing-autocomplete-drupal-8-custom-callbacks-waliullah-khan).
I'd like to add a "submit on click" behaviour, so when I click on a value of the autocomplete it changes the value of the input and then submit the form.
How should I do this?
Answer: Attaching below code:
custom.module
<?php

use Drupal\Core\Form\FormStateInterface;

function custom_form_alter(array &$form, FormStateInterface $form_state, $form_id) {
  if ($form_id == 'views_exposed_form') {
    if ($form['#id'] == "YOUR_FORM-ID") {
      $form['combine']['#autocomplete_route_name'] = 'custom.autocomplete';
      $form['#attached']['library'][] = 'custom/custom';
      $form_state->setCached(FALSE);
    }
  }
}

custom.routing.yml
custom.autocomplete:
  path: '/want/autocomplete'
  defaults:
    _controller: '\Drupal\custom\Controller\WantAutoController::autocomplete'
  requirements:
    _permission: 'access content'

custom.libraries.yml
custom:
  version: 0
  js:
    js/custom.js: {}
  dependencies:
    - core/jquery

js/custom.js
(function ($) {
    Drupal.behaviors.custom = {
        attach: function (context) {
            $("#FIELD-ID").autocomplete({
                select: function (event, ui) {
                    if (ui.item) {
                        var itemLink = $(ui.item.label).attr('href');
                        window.location.href = itemLink;
                    }
                }
            });
        }
    };
}(jQuery));

src/Controller/WantAutoController.php
<?php

namespace Drupal\custom\Controller;

use Drupal\Core\Controller\ControllerBase;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;

/**
 * Returns tree responses for nodes.
 */
class WantAutoController extends ControllerBase {

  function autocomplete(Request $request) {
    $db_conn = \Drupal::database();
    $matches = [];
    $string = $request->query->get('q');
      $alias = \Drupal::service('path.alias_manager');
      $db_conn = \Drupal::database();
      $query = $db_conn->select('node','n');
      $query->leftJoin('node_field_data', 'nfd', 'nfd.nid = n.nid');
      $query->fields('n');
      $query->fields('nfd',['title']);
      $result = $query->execute()->fetchAll();
      if ($results) {
        foreach ($results as $row) {
          $path = $alias->getAliasByPath('/node/' . $row->nid);
          $linkTonode = '' . $row->title . '';
          $matches[] = ['value' => $row->title, 'label' => $linkTonode];
        }
      }
    return new JsonResponse($matches);
  }

}

Here in the autocomplete we are building value with "a" tag. And while searching autocomplete field our autocomplete return with "a" tag also. We are also building js here to click on that value and redirect.
Comment: There are a few points to make about your JS. You should use the passed '$' variable rather than 'jQuery' as this helps to ensure your script can run on a page with multiple jQuery instances. The attach method will run after any AJAX updates to the DOM so you should use the 'context' variable in jQuery calls to restrict their scope. And it's also good practice to use the once() method in any actions on jQuery objects that only need to be done once, to ensure that is the case.

Custom autocomplete form and autosubmit


How to calculate the difference between max/min dates on non static pivot tables?

Question: I have a pivot table like this
A | 01/01/2001 | 01/02/2001
A | 01/01/2001 | 01/02/2008
B | 01/01/1994 | 01/11/2015
B | 01/01/2006 | 01/02/2014
B | 11/01/2016 | 19/07/2018
C | 01/03/2006 | 21/02/2014
C | 01/06/2016 | 01/05/2017
...

I want to create a pivot table that displays the difference in months so either this:
A | 01/01/2001 | 01/02/2001 | 1
A | 01/01/2001 | 01/02/2008 | 85
etc...

or just the months:
A | 1
A | 85
etc...

Of course I could just use =DATEDIF(date1,date2,"M") which would work but I also want to add a slicer so I can filter A, B, C, ... this means the number of rows is not fixed and so I cannot just add that formula.
Is there a nice way to do this?
Answer: You can calculate field inside the Pivot Table itself, and then you'll be able to use the slicers
Navigation:
PivotTable Tools>>Analyze>>Field Items & Sets>>Calculate Field
And then paste your datedif formula
=DATEDIF(Date1,Date2,"M")

Hope it helps!


Missing policy in Group Policy Editor

Question: I am seeking to turn on IE Enterprise Mode: https://technet.microsoft.com/itpro/internet-explorer/ie11-deploy-guide/turn-on-enterprise-mode-and-use-a-site-list

I followed the steps but there is no "Use the Enterprise Mode IE website list" policy listed. Am I missing something? I am on the DC.
Answer: I suspect you have older versions of the Internet Explorer GPO templates installed on the machine you're opening the GP Management Console from. You can download the latest here - https://www.microsoft.com/en-us/download/details.aspx?id=40905

Missing policy in Group Policy Editor


Interpretation of certain things in $Z_7$

Question: $Z_7$ is the ring of integers modulo $7$.
I am beginner in ring theory and a question which says

Find a reasonable interpretation for the expessions $1/2\ ,\ -2/3\ ,\ \sqrt{-3}\ \&\ -1/6$ in $Z_7$.

has got me stuck.
I think $\sqrt{-3}$ will be $\sqrt 4=2$ . But I am not being able to guess what to do with the division sign. Is it some analog of cosets ?
Comment: So you want $6x \equiv-1 \pmod{7}$ this says $6x \equiv 6\pmod{7}$ which says $x=1$.
Answer: A reasonable interpretation of these expressions is via equations. For instance:

$-2/3$ is a solution to $3x \equiv -2 \bmod 7$
$\sqrt{-3}$ is a solution to $x^2 \equiv -3 \bmod 7$
Comment: Just an added comment (not enough to write a full answer): the equation may have more than one solution, just like in the reals. One of these two equations has $2$ solutions.


Dropdown List deduction

Question: I have a list of 100 employees and 50 dropdowns. I want to select some of the employees in the dropdowns but I want to prevent the user from choosing the same employee more than once.
I want to deduct the chosen employees from the possible dropdown answers. How can I manage to do so?
I tried to filter out the selected employees but it shows an ugly warning sign for those who were selected
you can view my spreadsheet here:
https://docs.google.com/spreadsheets/d/1bzMPC3-SDOjsgZVXHQgmfBnkiTEsa1Pei7p_DybIebY/edit?usp=sharing
Comment: It's not possible since the Data Validation rule is comparing the cell value with Column C, where you remove that same value. Have you considered doing this with Apps Script?
Comment: best way to do this would be to use conditional formatting to highlight duplicates - https://i.stack.imgur.com/0d3r3.png
Comment: @Jescanellas tried to do it with a script, but It's way too complicated for me. How can I do it with Apps Script?
Answer: You can use the Simple Trigger onEdit(e) to automatically do the job:

The onEdit(e) trigger runs automatically when a user changes the value
  of any cell in a spreadsheet [...]

It's not exactly the same as you had before, because the selected values will still be displayed on the list. They just will be removed if they are repeated.
The only change you should do is to remove the current Data Validation from Column D and set a new one with values from Column A.
<code>function onEdit(event){

  if (event.range.getColumn() == 4){   //Run only if the edited cell is from column D

    var sheet = SpreadsheetApp.getActiveSheet();
    var selection = sheet.getRange("D2:D22").getValues();
    var editedRow = event.range.getRow(); //Gets the edited row

    for (var i = 0; i < selection.length; i++){

      //If some value from D is equal to the edited value but in a different Row (so it ignores the inputted value)
      if ((i + 2) != editedRow &&  selection[i][0] == event.value){

          sheet.getRange(event.range.getA1Notation()).setValue(""); //Change the new value to empty

      }

    }
  }
}
</code>
The function will run every time you select an item in the column D.
Result:

References:

Range
Sheet
setValue
Comment: Thank you very much. that helped me a lot! I really wanted to view a list of the unselected employees but that will do for now.

Dropdown List deduction


Torus: $S^1\times S^1$ homeomorphic to the parametric description?

Question: I have to show that $$f:S^1\times S^1\rightarrow T^2_{R,r}, f(e^{ia},e^{ib})=(R+rcos(a))cos(b),(R+rcos(a))sin(b),rsin(a))$$ is a bijection. I already showed f is injective, but then I have to show it is surjective, I think it is not very hard but I just don't know how to do it.
I do know that $f$ is surjective if for all $(R+rcos(a))cos(b),(R+rcos(a))sin(b),rsin(a))$, there exists an $(e^{ia},e^{ib})$ such that $f(e^{ia},e^{ib})=(R+rcos(a))cos(b),(R+rcos(a))sin(b),rsin(a))$, but I can't find a way to explain this is true.
After this, I have to explain this map in a picture en convince myself it is a homeomorphism..
Can somebody help me? Thanks!
Comment: I think you already proved it by writing down the formula for $f$...maybe the only issue is that $f$ is well-defined.
Comment: yes, the formula was given.. but how do I prove it is well-defined?
Comment: If you proved it was injective that should give you the idea.
Answer: First, notice that the definition of $f$ relies on the representation of points of $S^1 \times S^1$ as pairs of (complex) numbers of the form $(e^{ia}, e^{ib})$, where $a$ and $b$ are real numbers.  In fact, $S^1 \times S^1$ is a quotient of $\mathbb{R} \times \mathbb{R}$, with projection map $\pi:\mathbb{R} \times \mathbb{R} \longrightarrow S^1 \times S^1$  defined by $\pi(a,b) = (e^{ia}, e^{ib})$.  To study your map $f$, it's helpful to study the composition $F = f \circ \pi$, that is, the map $F(a,b) = f(e^{ia}, e^{ib})$.
For example, since $T^2_{R,r}$ is defined parametrically, and the domain of $F$ is all of $\mathbb{R}^2$, $F$ is clearly surjective.  This implies, in turn, that $f$ is surjective as well.
You can also use $F$ to argue that $f$ is well-defined, injective, continuous, etc.
Hope this helps!


Where can I find Twitter Bootstrap Variables for Sass?

Question: I'd like to use Twitter Bootstrap. But there is one thing I really like about the alternative Zurb Foundation. They are using plenty of Sass Variables for styling. Example:
http://foundation.zurb.com/docs/components/topbar.html#available-scss-variables
$include-html-top-bar-classes: $include-html-classes;

/* Background color for the top bar */
$topbar-bg: #111;

/* Height and margin */
$topbar-height: 45px;
$topbar-margin-bottom: rem-calc(30);

/* Control Input height for top bar */
$topbar-input-height: 2.45em;

/* Controlling the styles for the title in the top bar */
$topbar-title-weight: bold;
$topbar-title-font-size: rem-calc(17);

I can't imagine, that Twitter isn't using such variables. Does anyone know, where to find those variables? Google didn't help.
Thanks in advance.
George
Answer: Twitter Boostrap with SASS is their official SASS support Github. They do fully support it, by the way.
See Doc for details.
Path & Description
lib/ Ruby gem code (Sass configuration, Rails and Compass integrations)
tasks/ Converter scripts (turning upstream Less to Sass)
test/ Compilation tests
templates/ Compass package manifest
vendor/assets/ Sass, JavaScript, and font files
Rakefile Internal tasks, such as rake and convert
Master Variable File
Comment: [Variables for SASS](https://github.com/twbs/bootstrap-sass/blob/master/assets/stylesheets/bootstrap/_variables.scss) are there.
Comment: Sorry. I might not phrased my question well enough. So please let me repeat it:

Where can I find Twitter Bootstrap Variables for Sass?
Comment: Thx. M8. :) This was what I was searching for.


What does this `key=func` part mean in `max(a,b,c,key=func)` in Python?

Question: What does this key=func part mean in max(a,b,c,key=func) here https://docs.python.org/2/library/functions.html#min?
I know how does this function work in Python. But usually I see it used just simply as
z = max(a, b, c)
But in Python docs I've found this part and I don't understand it. Is it some additional optional feature of this function or what?
Comment: That **exact paragraph you link to** tells you what the `key` argument does.
Comment: @DanielRoseman Well, it's one sentence saying "The optional key argument specifies a one-argument ordering function like that used for list.sort().". By itself that might not be terribly helpful, but it points you in the right direction (to look at the documentation for `list.sort()` which provides a more detailed explanation of its `key` parameter).
Comment: I agree with Tagc: a lot of users don't see the point until they see an exact example.
Answer: it allows to define a criterion which replaces the < comparison between elements.
For instance:
>>>l = ["hhfhfhh","xx","123455676883"]
>>>max(l, key=len)
'123455676883'

returns the longest string in the list which is "123455676883"
Without it, it would return "xx" because it's the highest ranking string according to string comparison.
>>>l = ["hhfhfhh","xx","123455676883"]
>>>max(l)
'xx'

Comment: Thanks a lot. ) That's more understandable for me.
So, basically in the place of `func` can be used any function, which can be applied to elements of the list with dot notation (meaning that it's included in the String class or whatever class). Is that correct?
Comment: yes. That's correct. And if it needs some tweaking, you can use `lambda` to define your own functions. Ex: provide len-3 would be: `lambda s : len(s)-3`


With which IDE develop in VB6?

Question: I must have a IDE for VB6.
I can't use DotNet, because I cannot install a Framework on a computer, on which I will use my application and I know VB.Net, so I decide to use VB6 (I am open for other options).
Which IDE do you prefer for VB6?
Comment: Um, perhaps the one that comes with VB6?
Comment: The only thing VB6 and VB.NET only have in common is the syntax.  Nearly none of the libraries are the same so knowing .NET might not be as useful as you hope.
Comment: To do this http://stackoverflow.com/questions/2844683/which-language-shouod-i-take-closed?
Answer: Well, except Microsoft's Visual Basic 6.0, I don't know any.
By the way, Microsoft's support of VB6 is now over since 2008, you might consider using another language.
Comment: That depends on what you mean by support: http://msdn.microsoft.com/en-us/vbrun/ms788708.aspx  This means VB6 is supported until at least 2020 http://support.microsoft.com/lifecycle/?p1=14481
Comment: @Bob. +1 for that comment. The VB6 runtime is still fully supported.
Comment: When will it stop?  Hard to say.  The next Windows isn't close enough to release yet for an announcement about its support of VB6 programs.
Answer: The ide is the Microsoft Visual Basic 6.0 ide. I am not aware of any other ide.
See the resource center.
Answer: You can get vb 5.0 cce for free.
http://www.thevbzone.com/vbcce.htm

details:
http://support.microsoft.com/kb/165524

also check this out
Alternative IDE for VB6 and VBA
Answer: Visual Basic Tools For Visual Studio lets you open vb6 project in visual studio 2012/2013, with basic outlining, syntax coloring, and a few more things.
Its a work in progress, but if like me you have some living dead vb6 programs to maintain, it's... intriguing. Except there's no GUI editor, no debugging. I fear these points (especially debugging) are deal breakers. (Of course, I understand that not everything is possible).

With which IDE develop in VB6?


How to create a rule that checks an extra profile field created with the core profile module?

Question: I have a Drupal 7 site with the Honeypot module installed. Honeypot removes or prevents many spam registrations, but not all. I have extra fields added to the profile e.g. 'profile_field1'. I'd like to build a rule using the Rules modules that either deletes an account or prevents it from being created if the content of the 'profile_field1' doesn't match a certain regular expression.
I cannot find how to create a rule that can handle an extra profile field.
What I'd like to create:
'before saving a new user account'
if profile_field1 does not match RegExPattern
then prevent account from being created

I can find how to create a condition based on the 'regular' fields in the account (such as the name), but I cannot find how to test for the extra fields.
Update 1: the field 'profile_field1' is part of the Users profile, i.e. the Drupal Profile module is enabled. This module is part of D7 and similar in functionality to drupal.org/project/profile (for D8). It's found in [drupal]/modules/profile. Configuration settings are in admin/config/people/profile.
Update 2: I checked in the database. There is no table called 'profile_field1'. The info is in 'profile_field' which defines all fields and in profile_value, which contains all values.
So, e.g. profile_field1 would be fid=1 in table profile_field. Then user 5 (uid = 5) would have a record in profile_value with fid = 1, uid = 5, value = 'content of field'.
Update 3: With the suggestions in the answer I was able to migrate the information from the old Profile to the Profile2 fields. Write up is here: https://www.drupal.org/node/2916659
Update 4: At least I can create a Rules rule now. Just wondering if this is the best:
{ "rules_prevent_spam_registration" : {
    "LABEL" : "Prevent spam registration",
    "PLUGIN" : "reaction rule",
    "OWNER" : "rules",
    "REQUIRES" : [ "rules" ],
    "ON" : { "user_presave" : [] },
    "IF" : [
      { "entity_has_field" : {
          "entity" : [ "account-unchanged:profile-personal" ],
          "field" : "field_profile_field1"
        }
      },
      { "AND" : [
          { "text_matches" : {
              "text" : [ "account-unchanged:field-profile-field1" ],
              "match" : "<regexPattern>",
              "operation" : "regex"
            }
          }
        ]
      }
    ],
    "DO" : [ { "entity_delete" : { "data" : [ "account-unchanged" ] } } ]
  }
}

Comment: Since your question is about D7, the profile module cannot be https://www.drupal.org/project/profile ... which only exists for D8. Can you edit your question again, to include the URL of "your" profile module? PS: I do not like the "chat"-interface on drupal.SE, sorry.
Comment: chat: no problem. Question updated
Comment: Please review the edit I applied to the title of your question (to make it more accurate). If you don't like it, just perform a rollback. Also check my updated answer ... something like "as good as it gets"?
Answer: With the limited info provided in your question, the best bet is that you cannot test for that extra field because you forgot to add a Rules Condition like "entity has field" (related to your profile_field1 field). So add that Rules Condition first (as your very first Rules Condition), and then try again.
For a video tutorial that illustrates the importance of this Entity has field condition, refer to Data types and data selection, especially what is shown between about 13:30 and 17:30 in it.
Update:
As per your updates to your question, I now understand that you're using the deprecated (core) profile module. This is a quote from its .info file:

; The Profile module is deprecated, and included in Drupal 7 for legacy
; purposes only. By default, the module will be hidden from the UI unless you
; are upgrading a site that uses the Profile module to extend user profiles.

Because of that, I doubt that there is a way to use the Rules module to check "if the content of the 'profile_field1' doesn't match a certain regular expression" (as in your question). Mostly because the Rules Condition like "entity has field" (=  typical prereq to configure these kinds of Rules Conditions) will not have any effect for such profile_field1-field. Maybe not the answer you would want to hear/read, but I'm afraid it explains that what you're asking for cannot be done with Rules.
Plan-b:
Give it some thought to consider switching to the Profile 2 module. Quote from its project page:

Designed to be the successor of the core profile module, which is deprecated for Drupal 7. In contrast to the deprecated module this module provides a new, fieldable 'profile' entity - leverage the power of fields!
Features
... Thanks to the foundational entity API the module integrates well with Rules ...

If you do, pretty sure my original answer (= add a Rules Condition like "entity has field") will turn out to be the solution for what you try to achieve.
Comment: Well, I tried that but there is no way to select the field 'profile_field1'. If I use 'entity has field' and I choose either 'account' or 'account-unchanged' I can select all kinds of fields that are part of my content types, but I cannot select 'profile_field1'. And since it's a drop down list, I cannot add it manually
Comment: @hepabolu please EDIT your question to include an EXPORT of the rule you have so far. It may help to further improve my answer also.
Comment: I'd love to but so far I have nothing, other than the event
Comment: that is what I feared ... next attempt: where does that field come from, how is it configured, some module that created it?
Comment: Thanks for sticking with me. It's added in admin/config/people/profile. It's configured as a text field, private field, but visible at user registration, user must enter a value. I think it's standard Drupal.
Comment: @hepabolu D7 core does not have a path like admin/config/people/profile , so which module did you enable that created that path in your case? I think I can ***guess*** your answer, but I want you to clarify that as part of your question, so that I don't have to guess about that in my updated answer later on ...
Comment: It's the Drupal Profile module, i.e. part of Drupal. Hope this helps.
Comment: Ah, never realised it was deprecated. I must have enabled it way back when it was a Drupal 6 site. I'm going to check out the Profile 2 module. Thanks.
Comment: Sure, I was off to figure out if I could use it.


How to use Vertical Tabs on ConfigEntity Form

Question: how are you?
I've created a ConfigEntity and there a Configuration Form with Fieldsets and Fields.
Example:

But now I want to change this form to Vertical Tabs, like that:

Do you know how can I do this?
P.s. It is for the Contrib project if you want to test it is:
https://www.drupal.org/project/modal_page
Answer: Use the vertical_tabs form element. There is an example shown in the API documentation for this form element: https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Render%21Element%21VerticalTabs.php/class/VerticalTabs
In short, instead of using nested fieldsets, create a 'vertical_tabs' form element as the container then put multiple 'details' form elements inside it. Each of the 'details' elements will serve as the container for one vertical tab. See the example in the API documentation for the exact syntax.
Comment: Perfect! Really worked for me!

Thank you so much
Comment: Please accept the answer then!
Comment: How can I do this?
Comment: https://stackoverflow.com/help/someone-answers


Definition of natural numbers in ZF

Question: I'm going through Enderton's Elements of Set Theory and he's just defined a natural number as a set that belongs to any inductive set. The problem is that this statement seems to need the notion of proper class, since the class of all inductive sets is not a set.
Am I wrong here? If not, is there any set theoretic construction of the natural numbers that doesn't need proper classes?
Thanks in advance.
Comment: Abusing notation, $\in$ for proper classes isn't like $\in$ for sets: for $x$ a set and $Y=\{z\mid\phi(z)\}$ a proper class, $x\in Y$ doesn't literally make sense since $Y$ is not in the domain of discourse, instead it's notation for $\phi(x)$.
Answer: There's no use of classes here at all: the definition is simply $$Nat(a)\quad:=\quad \forall x(Ind(x)\rightarrow a\in x)$$ (where "$Ind(x)$" is an abbreviation for "$x$ is an inductive set"). The fact that the inductive sets form a class doesn't matter.
(Admittedly, the intuitive idea of "intersection of the class of inductive sets" makes reference to a proper class, but the point is that this can be subsumed by the power of the $\forall$-quantifier in first-order logic - namely its ability, indeed duty, to range over the whole universe.)

Definition of natural numbers in ZF


Why does the following formula of multiplying and adding 1 always work?

Question: I was coming back from somewhere on the bus. I was playing with my calculator and managed to realized this formula:

9 * 9 = 81
10 * 10 = 100
100 - (9 + 9 + 1) = 81

So the formula is that you multiply x by x, then multiply x + 1 by x + 1, then (x + 1) * (x + 1) - (x + x + 1) will always give the result of x * x. I wonder why.
So I use the programming language python to check whether this always works or not, it does always work!
x = 4 # assign x as 4
a = x * x # assign a as x times x
b = (x + 1) * (x + 1) # assign b as (x + 1) times (x + 1)
c = b - (x + x + 1) # assigns c as b minus (x + x + 1)
print(a == c) # checks whether a equals to c
print(c) # outputs c
print(a) # outputs a

And the above code outputs:
True
16
16

I wonder why it's like this, I tested with all numbers and it always works.
Could anyone explain to me the logic?
Comment: Uhm... $x^2=x^2+2x+1-2x-1=(x+1)^2-(x+x+1)$
Answer: This is just the development of
$$(x+1)(x+1)= x(x+1) + (x+1)= x^2 +2x+1$$
When you sum two numbers and multiply the result by a third one, you obtain the same result when you make the sum of the first one multiplied by the third one plus the second one multiplied by the third one.
Answer: You have that $x \times x=x^2$
and $(x+1)\times (x+1)-(x+x+1)=x^2+2x+1-(2x+1)=x^2$
These two are equivalent when you simplify them which is why they  are equal for all numbers/inputs.
Answer: Rewrite $(x+1)(x+1)=x^2+2x+1$
as $(x+1)(x+1)-2x-1= x^2$.


Unable to clear sendmail mqueue-client

Question: drwxr-xr-x  2 smmsp smmsp  4.0K May  6 23:31 mqueue
drwxrws---  2 smmsp smmsp  29M May  8 10:40 mqueue-client

As you can see mqueue-client is filled with 29 megabytes of unsent mail. This seems to be likely from an internal function sending mail to localhost which is failing.
I've tried numerous attempts to clear the queue but none have succeeded. The result is a process using up significant CPU resources:
16287 smmsp     20   0 50212  44m 2416 R   85  4.4 965:35.45 sendmail-msp

I've tried killing the process, stopping the sendmail service, deleting the contents of the mqueue-client and even switching to the smmsp user. But none have worked. rm hangs when trying to remove the mail.
How can I go about emptying this queue?
Once I've done this I'll move everything over to Postfix.
Comment: Logs, my friend, logs. Have you read them at all? Boldly enter `/var/log` lair and see what it holds.
Comment: thanks @janne you helped me solve the problem, see my answer for more info
Comment: Great! Have a nice weekend. :)
Comment: To remove multiple files you can use `find` command  like `find /path/to/files/ -type f -name '*.txt'  -exec rm {} \;`
Answer: As suggested by Janne in the comments, I looked over my syslog and found many mail failing to send errors which was causing them to be deferred thus pile up in the mqueue-client folder.
After doing some digging it turns out crontab is set to mail root on completion/notices/warnings/errors. By issuing crontab -e and adding MAILTO="" to the top of the config, no more mail is sent out, problem fixed!
Now I'm clearing out the 1.5 million or so bad emails from mqueue-client using this Bash script:
#!/usr/bin/env bash

cd mqueue-client

deleted=0

for i in `ls`
do
        rm -f $i
        percentage=$(bc <<< "scale=2; ($deleted / 1035435) * 100")
        ((deleted++))

        echo "Deleted $i. Files deleted $deleted. $percentage% complete."
done

Run from /var/spool using ./filename
Update
Unfortunately after running this script all night the mail queue wasn't being significantly reduced. This was because I hadn't disabled the minute running CRON that was set to clear the queue but was resulting in the backlog growing. Hopefully now this is disabled everything should clear.

Unable to clear sendmail mqueue-client


How to tackle this tricky rhythm -- nested triplets?

Question: I play in a local brass quintet, and we were sent some new music by a local composer. When we played it through, my part contained this rhythmic pattern which stopped me dead.
I realised that when I sight read, I'm relying mostly on memory and experience to know how a rhythm goes without having to mentally calculate where beats fall.
But I had never seen this pattern before. I see the nested tuplets, and in my head I can see how it's supposed to work. But I just can't switch from the four quarter notes in the first bar to the tuplet pattern in the second. I tried to practice by setting a metronome ticking quarter notes and then clapping nine against four. But that was really hard too.
So my question is - are there any tips for playing this sort of pattern and staying in time? Or exercises?
Comment: I bet it would be helpful to look at the score.  Try to understand why the composer made these choices.  Presumably this figure makes sense in relation to the other voices.
Comment: A triplet within a triplet corresponds to exactly 1/9 of a bar (0.111...). Since an eighth note is 1/8 of a bar (0.125), and the difference is only 1% of a bar, you won't go far wrong by playing the first three notes as if they were normal eighth notes and eventually speeding them up very slightly.
Comment: I think the easiest way to think of it is to subdivide 4/4 into 9 equal parts. You will have to tongue beats 1, 2, 3, 4 and 7 within the measure. I would personally set 1 metronome to 4/4 with a clap on the 1st downbeat and another one - to 9/8 with a clap on beats 1, 4 and 7. If the 1st metronome is set to 120bpm, the 2nd one would have to be set to 135bpm exactly if my arithmetics are correct.
Comment: Did you ask the composer to demonstrate?  What is the tempo?
Answer: For practicing purposes, you can think of it as this in 9/8 which then it would reduce to:

It's very easy to play, but another tricky thing is as you mentioned is playing  a normal 4/4 bar then fitting this into a measure of 4/4. Practice them separately at first and once you are comfortable with both split the measure of 4/4 into 3 sections then split the first note into 3 again. This won't be easy, but if you practice the rhythm first it should come easier.
Comment: Yes, the rhythm is easy if you in fact modulate it to 9/8, but it's written / phrased in 4/4. It would have been much clearer to notate it as Dom has done but also include a metric modulation to clarify the unit pulse. This composer's notation is fussy and could have been written more clearly.
Comment: Well OK, but the notation really isn't the problem - I know what it's meant to convey - the problem is that I don't know how to perform the tuplet bar so that I arrive at its right-hand barline at the same moment as the other players, who are playing 4/4 non-tuplet parts. I was hoping for some feature of the polyrhythm generated by the 9 against 4 that I can use to tell whereabouts I am in the bar, or some other trick. Or maybe a suggestion telling me how I can mentally construct the tempo of the nine subdivisions in the bar...
Answer: 9 against 4 is tricky.  Take a drum loop program like Hydrogen and program the pattern, using one standard loop for the 4/4 (namely having not just a bass drum on the beat but also a bit of a fill-in) and then adding your problematic rhythm with a distinctive instrument turned up suitably.
Listen to it in order to get a feel for that kind of relation/pattern.  Play along at some point of time.  At some point of time, try to get into "free flow" where your own line makes sense of its own on the backdrop of 4/4 and has a soul and internal sense of its own, yet reaches the finishing line together with 4/4.
Answer: It might help if you can work out where the middle of the bar is. Then you can count in minims (half-notes), which is going to make life easier.
You would end up re-writing the rhythm like this - it's not much clearer at first glance, but it does at least show you clearly where the middle of the bar is, and once you can see that, you can see that it's a fairly straightforward syncopation if you feel the bar in minims (half-notes) rather than crotchets (quarter-notes):

(I have drawn a dotted line on the half-bar)
Comment: This is the approach I take to passages like this.
Answer: A consideration that no one else has mentioned is that this type of figure is often used around tempo changes. If that is the case, it will be hard to coordinate everyone (depending on proficiency) and is counterproductive to spend a lot of time working on getting the rhythm exactly accurate (since it will be in flux anyway). Rehearsal time is better spent "feeling" the rhythm together.
Assuming that this is a spot where the tempo remains steady and it's not a rubato/soloistic passage:
The rhythms we encounter can often be simplified conceptually in another meter - 3/2 = dotted notes in 3/4, etc. A hemiola like this that is split into a triplet really can't be simplified; no matter what you're going to have at least two layers of cross rhythm going on, which is too much for my brain to decode.
In case it helps you, 9/4 can be "simplified" as 32nd notes in 9/8. The 9 can be thought of as 8th notes and the 4 can be thought of as groups of nine 32nds, like this:

This rhythm is quite complex and going to be hard to think of in those terms. If that doesn't work for you, try another approach.
My general guidelines are:

Find a spot for your internal metronome where you can feel the base rhythms. Start it clicking a few measures before the trouble passage, and keep it going a few measures after to make sure you hold on to the tempo.
Break the rhythm down into simpler patterns ("check patterns"), then rebuild it.

In this case, I'd choose to set my internal metronome to half notes. For practice, I'd use an external (real) metronome also set to half notes.
For the first simplified rhythm, I'd do the three half note triplet. 3/2 is not too hard, so it shouldn't take you too long to be able to play G C B in the right rhythm in context.
Next, I'd take the inner triplet and the C, so you play G A B C, and hold the C. Learn to do this in context, and alternate between the previous pattern and this pattern to ensure that the key notes sound the same relative to the metronome (still on half notes).
After that, the whole rhythm should come together pretty well. Always practicing with the metronome on half notes will keep you firmly in duple-land, and is easy to switch between 4 and 9. Depending on tempo, you might even think in 1 (metronome to whole notes), but the fact that it's notated in 4 makes me think it won't be that fast.
Answer: Mostly playing these is a matter of practice. If you think of it hierarchically, there are 9 notes played in the space of 8. The 9 happen to b notated as quarters whereas the 8 were eights. Or you could think of it as 3 half notes in the space of two, the first being a triplet.
You could draw it out on a diagram: a 4/4 measure marked in eighth notes and a 4/4 measure with 3 half note triplets, the first of which is itself a triplet. This would give you a visual indication of where each note begins.
Answer: For this piece (specific to 9-against-4)
First approach
My first attempt would be to learn to feel the preceding measures in 1. That way, my pulse (i.e., measure length) remains constant, and I'm just switching from a 4-subdivision to a 3-subdivision. Switching from sixteenth-notes to triplets is familiar enough, that I would try this approach first. It does mean making two near-simultaneous subdivisions into three and then immediately to a "smaller" three. I would have to trust my triplet feel for that.
Second approach
Another possibility would be to feel the preceding measures in 2. In that case, I think I'd practice to feel 9-against-2, which isn't so bad. (As opposed to 3-against-2 and then subdividing the 3 into a further 3.)
General approach
I like to map out these rhythms least-common-multiple style. For 9-against-4, my map would involve 36 mini-pulses, like this:

Main pulse:   1                 2                 3                 4
9-division    1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
4-division    1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
Tuplet pulse: 1       &       a       2       &       a       3       &       a

Obviously, this is only going to work at a slow-enough speed that I can actually count each subdivision. But it lets me know the relative placement is 9 against the 4.
I practice initially by tapping one hand or foot to the main 4/4 beat, another hand or foot to the 9 subdivision, and alternate speaking the 9-counts or 4-counts.
When I have an initial feel for it, I use the metronome to gradually speed up the primary 4/4 pulse, and drop the speaking when it gets too fast.
Final comment
The approach chosen will be influenced by the tempo of the piece. The first two approaches are likely to work better at a relatively faster tempo, since one's internal pulse has to slow down. The general approach works for getting an initial feel, but in performance requires a slow enough tempo that there's time to feel all the subdivisions.


Singleton em JavaScript

Question: Como implementar o padrão Singleton em JavaScript de uma forma simples e correta - que me assegura que a instância será única?
Já vi implementações um tanto complexas, mas já não seria o suficiente algo assim?
var xyz = xyz || (function (){
    [..]
})();

O que ela teria de errado?
Comment: Não entendi bem o que você está buscando (acho que é por causa do exemplo; geralmente, simulacros de Singleton em JS usam `new`). Não sei se te ajuda, mas eu costumo recomendar usar um objeto literal em vez de tentar simular singletons. Resolve alguns casos de uso, provavelmente não todos.
Comment: Poisé, eu esqueci de comentar isso na pergunta. Se com os objetos literais globais, eu tenho necessidade de singleton.
Answer: É impossível se criar um singleton em JavaScript, dada a natureza prototípica da linguagem. A grande maioria das linguagens orientadas por objeto usa o que chamamos de OO Clássica, em que classe herda de classe e objeto instancia classe:
classe A   <==   classe B (herda de A - sua "superclasse")
   ^                ^
   |                |
objeto 1         objeto 2 (não herda de objeto 2 - não diretamente, pelo menos)

Já na OO Prototípica, não existem classes, e objeto herda de objeto:
objeto 1   <==   objeto 2 (herda de objeto 1 - seu "protótipo")

JavaScript (em sua versão atual) tem semântica de protótipos, somente, mas uma sintaxe bizarra que faz parecer que ela implementa OO Clássica, quando na verdade isso não ocorre:
function A() { this.foo = 10; }
var objeto1 = new A();

function B() { this.bar = 20; }
B.prototype = new A(); // Vou chamar esse objeto de "anônimo"; poderia ser o próprio objeto1
var objeto2 = new B();

Parece que a "classe" B herdou de A, e os objetos "instanciaram a classe", certo? Mas o código acima poderia ser escrito igualmente da seguinte forma:
var objeto1 = { foo:10 };
var objeto2 = Object.create({ foo:10 }, { bar:{ value:20 } }); // anônimo poderia ser objeto1

Em ambos os casos, anônimo é o protótipo do objeto2. Isso significa que qualquer acesso de leitura tipo objeto2.foo irá primeiro ver se existe a propriedade foo em objeto2 e, se não existir, retornará anônimo.foo. Já numa escrita, ele criará essa propriedade em objeto2 se não existir (ou atualizará, se já existir). Você pode ler mais sobre como protótipos funcionam nessa pergunta relacionada.
A consequência disso, entretanto, é que se você tiver uma referência para um objeto nada impede que você crie um outro que herde [prototipicamente] dele:
var xyz = xyz || (function (){ // Essa função só será chamada uma única vez, garantidamente
    this.foo = function() { ... };
})();

var abc = Object.create(xyz, { bar:{ value:20 } }); // Mas agora abc herda de xyz
abc.foo(); // Chamou o método foo de xyz, usando abc como this!

Ainda que xyz seja imutável, ou tenha "proibido extensões" via Object.preventExtensions ou Object.seal, nada disso impede que ele seja herdado. Se existe um meio de marcar um objeto como "é proibido usá-lo como protótipo de outros objetos", eu não conheço. Só impedindo os outros de obterem uma referência pra ele...
Finalizando, caso não tenha ficado claro, criar um objeto "vazio" herdando de outro seria o mesmo que "criar outra instância de sua classe". Pois todos os métodos do primeiro estão disponíveis para serem chamados, e o novo objeto possui uma cópia de todas as suas propriedades - podendo alterá-las ou não. Se algum método do objeto antigo presume que ele sempre será chamado usando xyz como this, chamá-lo via abc vai violar essa premissa, podendo ter consequências negativas. Ou seja: não assuma que um objeto JavaScript seja garantidamente singleton.
Answer: Se o que você quer com um singleton é ter um a instância única, realmente não precisa complicar muito. Mas sem ver um exemplo inteiro, não dá pra saber se o seu código exemplo está correto. Por exemplo, se xyz for uma variável local você vai criar uma instância nova o tempo inteiro.
function getSingleton(){
    //esse código sempre cria uma instância nova
    // é como se você tivesse escrito "var xyz = undefined || (function..."
    var xyz = xyz || (function (){
        [..]
    })();
}

Já se xyz for uma variável global, fica estranho a forma que você escreveu. Por que checar a presença de um xyz declarado anteriormente ao invés de inicializar direto?
var xyz = (function (){ ... }())

Resumindo, se o objectivo for ter uma instância única inicializada preguiçosamente, acho que o normal seria o xyz ser declarado num escopo diferente de onde ele é inicializado:
var xyz = null;
function getSingleton(){
    xyz = xyz || (function(){ ... }())
    return xyz
}

Finalmente, uma coisa pra você prestar atenção: quando você estiver escrevendo código que testa se uma variável global existe use window.xyz ou typeof xyz !== "undefined" porque tentar acessar diretamente uma variável global que não existe é um ReferenceError.
Comment: Não tenho certeza, mas acredito que a motivação por trás de `var x = x || ...` é evitar que dois ou mais scripts criem a mesma variável. Eu vejo isso muito no código fonte do jQuery, por exemplo. Esse código assume, é claro, que ambos os scripts já tenham em consenso qual deve ser o valor da variável se ela estiver ausente (se um esperar uma coisa e o outro esperar outra, aí vai ter problema...)
Comment: Concordo, me referi ao fato da função de instanciação só ser executada uma única vez (o que realmente ocorre, ainda que não caracterize totalmente o padrão *singleton*). Quanto a ter um único módulo responsável, isso é desejável sim, mas difícil de se conseguir [garantidamente] na prática (se você não é cuidadoso, acaba num caso de [*Dependency Hell*](http://en.wikipedia.org/wiki/Dependency_hell)). Se sua página tem 10 scripts externos, e você não tem certeza se dois deles incluíram a mesma dependência comum, o padrão acima ajuda na "defesa em profundidade"... :P
Comment: Sim, var x = x || ... existe, mas nunca ví alguém usar isso pra fazer um singleton. Normalmente pra isso você quer um único múdulo responsável por criar a instância ao invés de um monte de módulos testando se ela já tem valor...
Comment: Exatemente, @mgibsonbr. Foi esse o ponto que eu queria chegar com o exemplo `var x = x || ...`.

Singleton em JavaScript


Parse specific format json array in Qt4, no external libraries

Question: I need extract data from a JSON array, format '[[[1,2,3]],[[1,2,3],[1,2,3]],"string"]' in Qt. logically it's '[[[x-values]],[[y1-values],[y2-values]],"comments"]'.
Edit: x,y1,y2 arrays can be up to 1000+ elements large.
I know that that's the exact format (without the single quotes) and that it's not going to change.
What I really want is QVector xval, y1val; .
How would I parse that?
(I'm new to Qt so please forgive me if I'm missing the obvious.)
Comment: I would never assume the format will never change. The JSON structure may never change, but the formatting may. At lease prepare for white spaces between tokens which are totally legal in JSON.
Comment: What I mean by that is that my entire app is based around parsing that string. That means if the other end should introduce changes, I need to adjust my code regardless. I want to say that there won't be variance between subsequent queries.
Answer: Quick and dirty solution:
QString s = "[[[1,2,3]],[[4,5,6],[7,8,9]],\"string\"]";
QStringList parts = s.remove("[").remove("]").split(',');

QVector<int> xval, yval;
if (parts.size() >= 6)
{
    xval << parts[0].toInt() << parts[1].toInt() << parts[2].toInt();
    yval << parts[3].toInt() << parts[4].toInt() << parts[5].toInt();
}

Edit: Now supporting variable length arrays:
QVector<int> ToIntList(const QString& s)
{
    QVector<int> result;
    QStringList parts = s.trimmed().split(",");
    for (int i = 0; i < parts.size(); ++i)
        result << parts[i].toInt();
    return result;
}

QString s = "[[[1,2,3,4,5,6, 7,  8]],[[9\n,10], [11,12,13]],\"string\"]";
QStringList lists = s.remove(" ").split("],[");

for (int i = 0; i < lists.size(); ++i)
    lists[i] = lists[i].remove("[").remove("]");

QVector<int> xval, yval;
if (lists.size() >= 2)
{
    xval = ToIntList(lists[0]);
    yval = ToIntList(lists[1]);
}

Comment: Sorry, I think what I wrote was a bit ambiguous. I edited my question to reflect that the arrays aren't literally three elements long but can have arbitrary length.
Comment: Does exactly what I wanted! Thank you.


Birthdays migrate to Profile2

Question: guys.
I've this issue, i did migrate a user list from csv file with Migrate module to Profile2. I'd use one only value for the import with the format dd/mm/Y.  All was fine, but at the end the birthdays fields are empty.
What is the right way to import (update) my user birthdays.
Thanks!
Answer: I might be wrong but I believe (most) dates in Drupal are simply stored as 11 digit integers. The format you refer to is only whats displayed on the page. Your migration probably just needs to be moving the integer date data by simply moving the data from your source csv column into your new profile2 Drupal Field:
$this->addFieldMapping('csv_birthday_column', 'new_profile2_birthday_field');

Then you just use a Profile2 Manage Display under the Field UI to setup how you want it to look on the page such as d/m/Y.
EDIT:
If your CSV has for example "04/15/2013" as a date for a piece of content you could use PHP strtotime() in a Migration class prepareRow() function to get an integer value to save in Drupal.

Birthdays migrate to Profile2


Fazer alert aparecer apenas quando a pagina já estiver carregada

Question: Eu tenho um formulário que envia os dados pra mesma pagina de onde o usuário os digitou pelo método POST e faço que na URL seja criada uma variável por GET para acionar um alert por JS que tenho em meu codigo, o intuito desse código quando a pagina é recarregada é que ele exiba a mensagem para o usuário dizendo que o formulário foi enviado com sucesso!

  No entanto, ele acaba exibindo o alert antes mesmo do conteudo da pagina ser carregado, e fica feio só aquele alert, sendo que o resto da pagina só vai carregar após clicar no OK, existe alguma forma de fazer o Alert exibir a mensagem só quando a pagina for recarregada?
Comment: Oi Vinicius da pra partilhar um pouco do codigo pra entendermos melhor a situacao e tentar ajudar de forma mais eficiente?
Answer: Uma das formas é colocar seu alert dentro de um setTimeout com time 0.

  setTimeout(function(){
        alert('com setTimeout');
   },0);

 texto do site


OLS Estimation, Bias and Causality

Question: I wish to ask about the bias of an OLS estimator. In what follows I assume that the regression that we are dealing with is an approximation to a linear conditional expectations function. That is we have that:
$ E[Y|X] = \beta_{0} + \beta_{1}X_{i} $
Hence,
$ Y_{i} = \beta_{0} + \beta_{1}X_{i}+\varepsilon_{i} $
In this case, as with all CEFs,  $ \varepsilon_{i} $ is defined such that $E[\varepsilon_{i}|X_{i}]=0$. This is true, by definition and can be verified using the Law of Iterated Expectations, if necessary.
However, I will also note that in the background, we have a different model which is causal. It will be defined as follows:
$ Y_{i} = \delta_{0} + \delta_{1}X_{i}+u_{i} $
Notice that if $E[u_{i}|X_{i}]=0 $, then the causal model and the CEF coincide, and we can estimate the parameters of the causal model without any problems! However, let's suppose that $E[u_{i}|X_{i}] \neq 0 $. This means that we endogeneity in the causal model. This will imply that that the CEF is NOT the same as the causal model. More explicitly. $ \beta_{1} \neq \delta_{1} $.
My question is as follows. Suppose that I took a sample of N observations of $ (Y_{i}, X_{i}) $ and decided to run my OLS estimator. What will happen?
We know that formally, $ E[\hat{\beta}] = \beta + (X'X)^{-1}E[(X'\varepsilon)] $.
Notice however that by definition $ E[(X'\varepsilon)]=0 $. There is no such thing as endogeneity in the land of CEFs. Recall, that the OLS put nicely by Angrist (2008): "inherits its legitimacy from the CEF", which means that our estimator uses definition of the error terms found in the CEF. Hence, endogeneity cannot cause OLS to be biased?!
As we have put forward above endogeneity is only present in the causal model. My question is, how does endogeneity bias actually work? When does it affect the coefficient estimates produced by OLS?
Is it that the "bias" represents the difference between the coefficient found in the CEF and the causal model?!! That is, assuming that $E[u_{i}|X_{i}] \neq 0 $, then: $ E[\hat{\beta}] = \delta + (X'X)^{-1}E[(X'u)] $? In other words, it gives the causal model parameters plus some bias on the end? Or to put it less confusingly, $ E[\hat{\beta}] = \beta = \delta + (X'X)^{-1}E[(X'u)] $?
Lastly, when faced with this problem, do instrumental variables try to transform the regression model in such a way such that the CEF results and the causal model parameters coincide upon estimation?
Any clarifications would be much appreciated!
Comment: Looks good to me except for the nitpick that there is no error in your first equation. Sure, $\beta \neq \delta$; and OLS is unbiased for $\beta$, hence the bias for $\delta$. Instrumental variables regression aims to estimate $\delta$, not $\beta$, so they do not coincide.
Answer: You are not far from to the correct interpretation. Many times presentations are ambiguous and/or contradictory about this point, but all problems come to solve if we keep clear separated CEF and causal model as you do.

In what follows I assume that the regression that we are dealing with
is an approximation to a linear conditional expectations function.
That is we have that:
$ E[Y|X] = \beta_{0} + \beta_{1}X_{i} $
Hence,
$ Y_{i} = \beta_{0} + \beta_{1}X_{i}+\varepsilon_{i} $
In this case, as with all CEFs,  $ \varepsilon_{i} $ is defined such
that $E[\varepsilon_{i}|X_{i}]=0$. This is true, by definition and can
be verified using the Law of Iterated Expectations, if necessary.

More precisely the mean independence is precisely true if the exact CEF is linear, if you use the approximation argument even this condition is true in approximation only. For simplification let me said that the exact CEF is assumed linear.

However, I will also note that in the background, we have a different
model which is causal. It will be defined as follows:
$ Y_{i} = \delta_{0} + \delta_{1}X_{i}+u_{i} $
Notice that if $E[u_{i}|X_{i}]=0 $, then the causal model and the CEF
coincide, and we can estimate the parameters of the causal model
without any problems!

This part is ok, but a precisation about the word "coincide" can help. Even if algebraically the two equation can coincide we must keep in mind that them are two strongly different concepts; them must be not mixed.

However, let's suppose that $E[u_{i}|X_{i}] \neq 0 $. This means that
we endogeneity in the causal model.

Exactly.

This will imply that that the CEF is NOT the same as the causal model.
More explicitly. $ \beta_{1} \neq \delta_{1} $.

Yes. In this situation the previous CEF do not permits to identify the causal parameters.

My question is as follows. Suppose that I took a sample of N
observations of $ (Y_{i}, X_{i}) $ and decided to run my OLS
estimator. What will happen?
We know that formally, $ E[\hat{\beta}] = \beta +
 (X'X)^{-1}E[(X'\varepsilon)] $.
Notice however that by definition $ E[(X'\varepsilon)]=0 $. There
is no such thing as endogeneity in the land of CEFs. Recall, that the
OLS put nicely by Angrist (2008): "inherits its legitimacy from the
CEF", which means that our estimator uses definition of the error
terms found in the CEF. Hence, endogeneity cannot cause OLS to be
biased?!
As we have put forward above endogeneity is only present in the causal model.

Indeed you have that $ E[\hat{\beta}] = \beta$ regardless endogeneity problems.

My question is, how does endogeneity bias actually work? When does it
affect the coefficient estimates produced by OLS?

Endogeneity imply the biasedness of  $ \hat{\beta} $ but it is biased for what? It is biased for $\delta$.
Indeed

Is it that the "bias" represents the difference between the
coefficient found in the CEF and the causal model?!! That is, assuming
that $E[u_{i}|X_{i}] \neq 0 $, then: $ E[\hat{\beta}] = \delta +
 (X'X)^{-1}E[(X'u)] $? In other words, it gives the causal model
parameters plus some bias on the end?

Exactly.

Lastly, when faced with this problem, do instrumental variables try to
transform the regression model in such a way such that the CEF results
and the causal model parameters coincide upon estimation?

IV estimator can help for consistent estimation of $\delta$. However even OLS can be used yet, what you need is good controls.
Finally this topic is quite vast, this my explanation can help: Under which assumptions a regression can be interpreted causally?

OLS Estimation, Bias and Causality


SImplenews module. Where does the form come from?

Question: I would like to modify the content of the simplenews form, but where is it being generated from?
<?php print render($form); ?>

I would like to change the "E-mail" wording, for example.
Answer: To modify a form in Drupal you dont need to get where the form is built.
You can make use of hook_form_alter to modify the form

Perform alterations before a form is rendered. One popular use of this
  hook is to add form elements to the node form.

hook_form_alter(&$form, &$form_state, $form_id);

Parameters
$form: Nested array of form elements that comprise the form.
$form_state: A keyed array containing the current state of the form.
  The arguments that drupal_get_form() was originally called with are
  available in the array $form_state['build_info']['args'].
$form_id: String representing the name of the form itself. Typically
  this is the name of the function that generated the form.

What is a Hook ?
Basically they provide a way for a module to extend the functionality of another module. They would go in a module and 'hook' would be replaced by the actual module name.
Please try
Examples for Developers
Drupal Hooks Explained
Comment: Thank you @nikhilM

I've never worked with hooks before. Any tutorials you recommend?
Comment: @kine456, see my edit.
Comment: Thank you. Just to confirm, I don't need to create a new module, is that right? I just edit the template.php file, is that correct?
Comment: @kine456, read my answer.. `hooks` would go in a module and 'hook' would be replaced by the actual module name. Not in template.php, but there are hooks can be in template.php
Comment: thank you again @nikhilM

My modules are being recognised for some reason :(


Best way to really grok Java-ME for a C# guy

Question: I've recently started developing applications for the Blackberry.  Consequently, I've had to jump to Java-ME and learn that and its associated tools.  The syntax is easy, but I keep having issues with various gotchas and the environment.
For instance, something that surprised me and wasted a lot of time is absence of real properties on a class object (something I assumed all OOP languages had).  There are many gotchas.  I've been to various places where they compare Java syntax vs C#, but there don't seem to be any sites that tell of things to look out for when moving to Java.
The environment is a whole other issue all together.  The Blackberry IDE is simply horrible.  The look reminds me Borland C++ for Windows 3.1 - it's that outdated.  Some of the other issues included spotty intellisense, weak debugging, etc... Blackberry does have a beta of the Eclipse plugin, but without debugging support, it's just an editor with fancy refactoring tools.
So, any advice on how to blend in to Java-ME?
Comment: What do you mean by "real properties on a class object", by the way?
Comment: @MetroidFan2002 - By "real properties" he means having a feature that looks like a field when you use it but has the functionality of a getter/setter under the hood.  For example, saying Employee.Employer = SomeEmployer; instead of Employee.setEmployer(SomeEmployer);
Comment: I would probably try to run as fast away as possible from JavaME. Scala has all the nice language features you will miss when moving from C# to Java, but sadly that doesn't fix the horrible ecosystem at all. (Yes, it works when you assume that 1995 is the state of the art.) :-/
Answer: This guy here had to make the inverse transition. So he listed the top 10 differences of Java and C#. I'll take his topics and show how it is made in Java:
Gotcha #10 - Give me my standard output!
To print to the standard output in Java:
System.out.println("Hello");

Gotcha #9 - Namespaces == Freedom
In Java you don't have the freedom of namespaces. The folder structure of your class must match the package name. For example, a class in the package org.test must be in the folder org/test
Gotcha #8 - What happened to super?
In Java to refer to the superclass you use the reserved word super instead of base
Gotcha #7 - Chaining constructors to a base constructor
You don't have this in Java. You have to call the constructor by yourself
Gotcha #6 - Dagnabit, how do I subclass an existing class?
To subclass a class in Java do this:
public class A extends B {
}

That means class A is a subclass of class B. In C# would be class A : B
Gotcha #5 - Why don't constants remain constant?
To define a constant in Java use the keyword final instead of const
Gotcha #4 - Where is ArrayList, Vector or Hashtable?
The most used data structures in java are HashSet, ArrayList and HashMap. They implement Set, List and Map. Of course, there is a bunch more. Read more about collections here
Gotcha #3 - Of Accessors and Mutators (Getters and Setters)
You don't have the properties facility in Java. You have to declare the gets and sets methods for yourself. Of course, most IDEs can do that automatically.
Gotcha #2 - Can't I override!?
You don't have to declare a method virtual in Java. All methods - except those declared final - can be overridden in Java.
And the #1 gotcha…
In Java the primitive types int, float, double, char and long are not Objects like in C#. All of them have a respective object representation, like Integer, Float, Double, etc.
That's it. Don't forget to see the original link, there's a more detailed discussion.
Comment: Gotcha #9 is not enforced in the Blackberry JDE, though if you plan to use Eclipse or other IDEs you probably want to abide by it.
Answer: Java is not significantly different from C#. On a purely syntactic level, here are some pointers that may get you through the day:

In Java you have two families of exceptions: java.lang.Exception and everything that derives from it, and RuntimeException. This is meaningful because in Java exceptions are checked; this means that in order to throw any non-runtime exception you also need to add a throws annotation to your method declaration. Consequently, any method using yours will have to catch that exception or declare that it also throws the same exception. A lot of exceptions you take for granted, such as NullPointerException or IllegalArgumentException, in fact derive from RuntimeException and you therefore don't need to declare them. Checked exceptions are a point of contention between two disciplines, so I'd recommend you try them out for yourself and see if it helps or annoys you. On a personal level, I think checked exceptions improve code factoring and robustness significantly.
Although Java has supported autoboxing for quite a while, there are still quite a few differences between the C# and Java implementations that you should be aware of. Whereas in C# you can interchangeably use int as both a value type and reference type, in Java they're literally not the same type: you get the primitive value type int and the library reference type java.lang.Integer. This manifests in two common ways: you can't use the value types as a generic type parameter (so you'll use ArrayList<Integer> instead of ArrayList<int>), and the utility methods (such as parse or toString) are statically implemented in the reference type (so it's not int a; a.toString(); but rather int a; Integer.toString( a );).
Java has two distinct types of nested classes, C# only has one. In Java a static class that is not declared with the static modifier is called an inner class, and has implicit access to the enclosing class's instance. This is an important point because, unlike C#, Java has no concept of delegates, and inner classes are very often use to achieve the same result with relatively little syntactic pain.
Generics in Java are implemented in a radically different manner than C#; when generics were developed for Java it was decided that the changes will be purely syntactic with no runtime support, in order to retain backwards compatibility with older VMs. With no direct generics support in the runtime, Java implements generics using a technique called type erasure. There are quite a few disadvantages to type erasure over the C# implementation of generics, but the most important point to take from this is that parameterized generic types in Java do not have different runtime types. In other words, after compilation the types ArrayList<Integer> and ArrayList<String> are equivalent. If you work heavily with generics you'll encounter these differences a lot sooner than you'd think.

There are, in my opinion, the three hardest aspects of the language for a C# developer to grok. Other than that there's the development tools and class library.

In Java, there is a direct correlation between the package (namespace), class name and file name. Under a common root directory, the classes com.example.SomeClass and org.apache.SomeOtherClass will literally be found in com/example/SomeClass.class and org/apache/SomeOtherClass.class respectively. Be wary of trying to define multiple classes in a single Java file (it's possible for private classes, but not recommended), and stick to this directory structure until you're more comfortable with the development environment.
In Java you have the concepts of class-path and class-loader which do not easily map to C# (there are rough equivalents which are not in common use by most .NET developers). Classpath tells the Java VM where libraries and classes are to be found (both yours and the system's shared libraries!), and you can think of class loaders as the context in which your types live. Class loaders are used to load types (class files) from various locations (local disk, internet, resource files, whatnot) but also constrain access to those files. For instance, an application server such as Tomcat will have a class loader for each registered application, or context; this means that a static class in application A will not be the same as a static class in application B, even if they have the same name and even if they share the same codebase. AppDomains provide somewhat similar functionality in .NET.
The Java class library is similar to the BCL; a lot of the differences are cosmetic, but it's enough to get you running for the documentation (and/or Google) over and over again. Unfortunately I don't think there's anything to do here — you'll just build familiarity with the libraries as you go.

Bottom line: the only way to grok Java is to use it. The learning curve isn't steep, but prepare to be surprised and frustrated quite often over the first two or three months of use.
Answer: The short answer is - it's going to be annoying, but not difficult.
Java and C# have all the same underlying concepts, and a lot of the libraries are very close in style, but you're going to keep bumping your head across various differences.
If you're talking about class properties, Java has those. The syntax is
public class MyClass {
    public static int MY_CLASS_PROPERTY = 12;
}

I would seriously suggest you get a better IDE.
Any of Netbeans, Eclipse, IDEA, JBuider is going to make your transition a lot more pleasant.
Comment: He means properties, not fields... it's a concept Java lacks.  Basically, getters and setters with syntactic sugar to make them blend with the other language syntax beter... like foo.Property = 5; and bar = foo.Property; (those would access actual methods)
Comment: The problem is that the IDEs you mentioned do not integrate well with the Blackberry runtime and don't have emulator support (except for Eclipse).


Abrir PDF em java em outra máquina

Question: Fiz uma função que consigo abrir um PDF, porém essa função só está funcionando em minha maquina local. Quando eu executo o .jar gerado em outra máquina, é emitido um erro de que o arquivo PDFnão existe.
O trecho que executa tal função:
 try {

   java.awt.Desktop desktop = java.awt.Desktop.getDesktop();
   desktop.open(new File("/manual.pdf"));

} catch (IOException ex) {
    Logger.getLogger(JFrameInicio.class.getName()).log(Level.SEVERE, null, ex);
    System.out.println(ex);
}

O arquivo PDF está na seguinte localidade:
\Documents\NetBeansProjects\Reitoria\src\reitoria
Para criar o JAR, eu utilizo o método package for store que em meu pensamento, cria um jar com todos os arquivos necessários.
Creio que o PDF esteja sendo incluso no jar final pois sem colocar o PDF na pasta src, o jar fica com tamanho final de 3 MB, meu pdf também tem 3MB. Quando crio o jar com o arquivo PDF na pasta src, o .jar final fica com tamanho de 6 MB (creio que seja o jar + o arquivo PDF)
Comment: Você tem que criar um arquivo temporario do PDF em disco, e fazer a leitura do arquivo para este arquivo. A ide roda seu problema descomprimido, por isso funciona, ja executando via `jar`, o pdf está dentro do pacote e não existe externo a ele. Veja isso aqui, pode ajudar. http://stackoverflow.com/a/28808187/5524514
Comment: Coloque o erro da stack por favor. Pode ser questão de diretorio.
Comment: Java.lang.illegalArgumentException: The file \manual.pdf doesn't exist

Mas se o erro fosse no diretório, nao era para não funcionar mesmo em minha maquina local? (tanto executando pela propria IDE quanto pelo jar
Comment: vou olhar

muito obrigado
Answer: Use um caminho que sempre validará em qualquer máquina como por exemplo "C:/" ou use caminhos relativos como a pasta de usuário System.getProperty("user.home") ou o caminho da classe/jar com System.getProperty("user.dir").
Certifique-se de que o PDF está no caminho indicado.
Answer: Quando criar o objeto File utilize como argumento "./reitoria/manual.pdf" visto que no NetBeans o que seria o diretório atual na linha de comandos é a pasta src.
Assim, o seu pedaço de código ficaria:
try {

   java.awt.Desktop desktop = java.awt.Desktop.getDesktop();
   desktop.open(new File("./reitoria/manual.pdf"));

} catch (IOException ex) {
    Logger.getLogger(JFrameInicio.class.getName()).log(Level.SEVERE, null, ex);
    System.out.println(ex);
}

Abrir PDF em java em outra máquina


How does determinant method of solving equations work?

Question: Say you have two equations
$ax+by+c=0$
$dx+ey+f=0$
by determinant method
$\frac{x}{bf-ce}=\frac{-y}{af-cd}=\frac{1}{ae-db}$
Clearly, the solution is valid
How exactly was this derived?
Comment: It's logical to start by rewriting the system in terms of matrices $$\begin{bmatrix} a & b \\ d & e \\ \end{bmatrix} \begin{bmatrix} x \\ y \\ \end{bmatrix} = \begin{bmatrix} -c \\ -f \\ \end{bmatrix}$$
Comment: It is customary to write your equations as $ax+by=e$ and $cx+dy=f$ so that the determinant follows as $ad-bc$ which is then the denominator of your $(x,y)$ solutions. This is how the determinant is often introduced in texrtbooks
Comment: See if the index in your linear algebra textbook mentions "Cramer's Rule".
Comment: @ChaseRyanTaylor and then inverting first matrix? Thanks! I got it. Cant mark comments as answer though
Comment: @GEdgar no mention of Cramer's rule, I'll check Wikipedia, thanks
Comment: @AnvitGarg I posted it as an answer
Comment: A linear algebra textbook with no mention of Cramer's Rule?  That is an inferior book, in my opinion.
Comment: @GEdgar this method was taught to us 2 years earlier when we didn't know what determinants were. It was directly given as a formula. Now that we did learn determinants and I saw the relatedness that peaked my curiosity. But I do agree, the textbook isn't very good
Answer: Anvit, using my set up of equations, here is a way:
Multiply $ax+by=e$ by $d$ to get $adx+bdy=ed$ and multiply $cx+dy=f$ by $b$ to get $bcx+bdy=bf$. Now eliminating $bdy$ from both equations through subtraction yields $(ad-bc)x=ed-bf$ from which $x$ follows by division $ad-bc$ which is called determinant. For $y$ you can do the same thing. Can you give this a try?
EDIT: Considering $ad-bc$ and $ed-bf$ as determinants of two matrices, it ultimately gives Cramer's Rule
Answer: Start by rewriting the system in terms of matrices.
$$\begin{bmatrix} a & b \\ d & e \\ \end{bmatrix} \begin{bmatrix} x \\ y \\ \end{bmatrix} = \begin{bmatrix} -c \\ -f \\ \end{bmatrix}$$
Then, as you said, invert the square matrix.
Answer: it is $$x=\frac{ce-bf}{bd-ae},y=\frac{cd-af}{ae-bd}$$
and $$bd-ae$$ is the determinant, see also the Cramers rule
Comment: Our solutions are exactly the same.
Comment: I was asking for derivation not solution. Sorry if question wasn't clear
Comment: your solution is wrong, thats why i posted this, see also Cramer's rule
Comment: see here https://en.wikipedia.org/wiki/Cramer%27s_rule


Is Alpine Linux a bad choice for a server?

Question: I've been thinking of migrating one of my servers from Ubuntu to Alpine to host my web app. I came to know Alpine because of Docker. Is using Alpine as a server OS safe? Has anyone ever tried this?
Answer:
Does the change have the potential to result in loss of revenue for your company due to lack of expertise and/or inability to find consultants willing to support your platform?
Does Alpine provide must-have features that Ubuntu cannot provide?
If you're part of a larger team, does the switch have traction with other techs?
Have you been able to provide your management with a convincing business case?

Add to that: Asking whether a specific operating system is "safe" is a bit pointless. You probably wouldn't want to run any operating system in a production environment without configuring it first. Once a system fulfills your configuration needs it should be "safe" or your configuration definition is bad.


Bootstrapping - do I need to remove outliers first?

Question: We've run a split test of a new product feature and want to measure if the uplift on revenue is significant. Our observations are definitely not normally distributed (most of our users don't spend, and within those that do, it is heavily skewed towards lots of small spenders and a few very big spenders).
We've decided on using bootstrapping to compare the means, to get round the issue of the data not being normally distributed (side-question: is this a legitimate use of bootstrapping?)
My question is, do I need to trim outliers from the data set (e.g. the few very big spenders) before I run the bootstrapping, or does that not matter?
Comment: @user11852 Your general argument that outliers are legitimate is helpful. But, concerning the possibility of amplification, it seems to me that the contrary is true: bootstrapping has a chance of working only if the full sample is used.  Otherwise it presents a fairy tale, telling us how things would be if outliers didn't exist--but obviously they do. The larger problem is that bootstrapping has little theoretical justification when applied to small samples: the theory is an *asymptotic* one.
Comment: This is an important question (+1). Can you add a small sample of your dataset or a simulated sample resembling it to the question? I think providing an illustration will be more fruitful in this case.
Comment: Good question: I can probably argue pro and against the removal of outliers. Why not use medians if you worried about outliers and what you are looking for is just a "central tendency"? Given that money-related variables often have highly skewed distribution (eg. Pareto) that might not unreasonable in the first place.
Comment: @user11852 Medians tell you little about the mean, which is what is relevant to revenue.  It would be interesting to see your argument in favor of removing the "outliers," especially when these are likely the major contributors to the total revenue.
Comment: Unfortunately median would always be zero, as < 10% of users spend at all
Comment: @whuber: Let me stress, it was a comment, not an answer. I definitely not an expert on bootstrapping; I would generally argue that outliers are *legitimate* data, if they are not obviously corrupted observations. Nevertheless if one bootstraps a somewhat small sample that has obvious outliers I would worry that he could end up "amplifying" their influences or overlooking sample heterogeneity.
Comment: @whuber: Cool, thank you for the insight on the matter!
Comment: Also, consider whether in practice you make more profit from the "lots of small spenders" or the "very few big spenders." If you make your profit from those "outliers" you probably don't want to remove them--maybe you want to analyze them separately.
Comment: @whuber: I agree with what you say. Regarding your median comment: I guess an issue is that I treated revenue as roughly equivalent to income. Usually a debate between using mean or median does arise there; eg. in household income cases. (In retrospect, having just <10% of the user data generating revenue definitely it is not a good assumption.) Also I didn't mean to imply that a median "is the mean" or something like that. I specifically mentioned it *as a "central tendency"* value.
Comment: @user11852 And that's the crux of the matter: central tendency is not terribly meaningful when tracking revenue; only the sum (or equivalently, the mean) is. As far as the less than 10% goes, that depends on the business. Plenty rely on just covering costs with routine transactions and making profits on a small number of very large or high-profit sales. *E.g.*, one model for how an airline could make a profit is from the outsized margins reaped from the very small number of first-class passengers.
Answer: Before addressing this, it's important to acknowledge that the statistical malpractice of "removing outliers" has been wrongly promulgated in much of the applied statistical pedagogy. Traditionally, outliers are defined as high leverage, high influence observations. One can and should identify such observations in the analysis of data, but those conditions alone do not warrant removing those observations. A "true outlier" is a high leverage/high influence observation that's inconsistent with replications of the experimental design. To deem an observation as such requires specialized knowledge of that population and the science behind the "data generating mechanism". The most important aspect is that you should be able to identify potential outliers apriori.
As for the bootstrapping aspect of things, the bootstrap is meant to simulate independent, repeated draws from the sampling population. If you prespecify exclusion criteria in your analysis plan, you should still leave excluded values in the referent bootstrap sampling distribution. This is because you will account for the loss of power due to applying exclusions after sampling your data. However, if there are no prespecified exclusion criteria and outliers are removed using post hoc adjudication, as I'm obviously rallying against, removing these values will propagate the same errors in inference that are caused by removing outliers.
Consider a study on wealth and happiness in an unstratified simple random sample of 100 people. If we took the statement, "1% of the population holds 90% of the world's wealth" literally, then we would observe, on average, one very highly influential value. Suppose further that, beyond affording a basic quality of life, there was no excess happiness attributable to larger income (nonconstant linear trend). So this individual is also high leverage.
The least squares regression coefficient fit on unadulterated data estimates a population averaged first order trend in these data. It is heavily attenuated by our 1 individual in the sample whose happiness is consistent with those near median income levels. If we remove this individual, the least squares regression slope is much larger, but the variance of the regressor is reduced, hence inference about the association is approximately the same. The difficulty with doing this is that I did not prespecify conditions in which individuals would be excluded. If another researcher replicated this study design, they would sample an average of one high income, moderately happy individual, and obtain results that were inconsistent with my "trimmed" results.
If we were apriori interested in the moderate income happiness association, then we should have prespecified that we would, e.g. "compare individuals earning less than $100,000 annual household income". So removing the outlier causes us to estimate an association we cannot describe, hence the p-values are meaningless.
On the other hand, miscalibrated medical equipment and facetious self-reported survey lies can be removed. The more accurately that exclusion criteria can be described before the actual analysis takes place, the more valid and consistent the results that such an analysis will produce.
Comment: "then it would make sense to leave those cases out of the bootstrap sample, as they are a segment of the population that we do not wish to make any inferences about." I'm saying allow the bootstrap to sample these cases, remove them from the model fit upon the bootstrap sampled population. Doing this allows the effective sample size of each BS iteration to vary. This way, the $p$-value's distribution under $\mathcal{H}_0$ depends on sample size uncertainty (i.e. not knowing how many cases in a fixed sample from an imperfect population will need to be discarded)
Comment: I'm not sure I understand why "*if you prespecify exclusion criteria in your analysis plan, you should still leave excluded values in the referent bootstrap sampling distribution.*" You mention that this is "*because you will account for the loss of power due to applying exclusions after sampling your data.*" I don't see why it is assumed that applying exclusion criteria after sampling leads to loss of power, nor how/why leaving the excluded cases in the bootstrap sample "accounts for" (?) this, nor further why this is something that clearly must be "accounted for." Maybe I'm being dense here.
Comment: Well it depends on your sampling rule. If you collect data on 100 individuals and 5 of them are ineligible & excluded, you *could* bootstrap resample 95 observations from the 95 eligible participants, but that wouldn't reflect the fact that if you resampled 100 individuals at random from the population, potentially 10 or 8 or 4 or 0 of them would be ineligible according to your study specifications. This kind of uncertainty affects the distribution and interpretation of the $p$-value under the null hypothesis. Remember, the bootstrap is meant to simulate this kind of sampling.
Comment: Hmm, my thinking was that if one did specify the exclusion criteria in advance -- so that we are explicitly not interested in certain types of cases, and presumably future study replications would use these same exclusion criteria -- then it would make sense to leave those cases out of the bootstrap sample, as they are a segment of the population that we do not wish to make any inferences about. I do see how future replications might end up excluding a different proportion of cases, but I can't quite make the connection to why this matters for the cases that we explicitly *are* interested in..
Answer: Looking at this as an outlier problem seems wrong to me.  If "< 10% of users spend at all", you need to model that aspect.  Tobit or Heckman regression would be two possibilities.
Comment: At present, this is more of a comment than an answer. Would you mind expanding it a little bit to make it more answer-ish?


Let $R$ be a commutative ring with identity $1.$ Let $A$ and $B$ be two maximal ideals of $R.$ Prove that $AB=A\cap B.$

Question: Let $R$ be a commutative ring with identity $1.$ Let $A$ and $B$ be two maximal ideals of $R.$ Prove that $AB=A\cap B.$
My solution is as follows:
Let $x\in AB$ then, it means, $\exists a\in A,b\in B$ such that $x=ab.$ But $ab\in A,B$ as $A$ and $B$ are two ideals of $R$. Thus, we have $x=ab\in A\cap B.$ Hence, $AB\subseteq A\cap B.$
Let $r\in A\cap B$. We proceed to show $A\cap B\subseteq AB.$
$A,B$ is a maximal ideal and $R$ is a commutative ring with identity $1,$ so, $R/A,R/B$ is a field.
This means $r+A\in R/A$ and $r+B\in R/B.$
Since $R/A,R/B$ is a field  $\exists x+A\in R/A$ and $y+B\in R/B$ such that $$(x+A)(r+A)=1+A=(r+A)(x+A)$$ and $$(y+B)(r+B)=1+B=(r+B)(y+B)$$ holds.
We have, $xr+A=1+A$ and $yr+B=1+B$.
This means, $xr-1\in A$ and $yr-1\in B$. But $xr\in A$ and $yr\in B$ as, $A,B$ are ideals. This means, $1\in A,B.$
Now, $r\in A,1\in B\implies r\in AB.$ So, $A\cap B\subseteq AB.$ Hence, $AB=A\cap B.$

Does the way I presented the proof above looks good? Is it readable enough? Or is there anything that I can do to make the proof look better, or perhaps improve the proof? I want to improve the readability of my arguments if it seems necessary. Any suggestions regarding this will be greatly appreciated. Lastly, if there's anything wrong with my solution, please feel free to point them out.
Answer: If your proof involves showing that a maximal ideal contains $1$, you're doing something wrong (unless it's a proof by contradiction).
Instead, try to show that $1 \in A+B$ (or equivalently $A+B=R$) and work with that. To show this, note that $A+B$ is an ideal that contains $A$ and $A$ is a maximal ideal.
Edit: Here's the concrete mistake: if $r \in A\cap B$, then it is zero mod $A$ and mod $B$ and hence doesn't have an inverse in the field $R/A$ or $R/B$.
Comment: Thanks a lot! I do get it now. +1 from me!
Comment: But I want to know where explicitly did I make the mistake? Nothing seems to be wrong in my arguments. I think I am surely missing something. Also, if $A+B$ contains $1$ tgen $A+B=R$ and the result will follow. But then again, I am unable to show  that $1\in A+B.$
Comment: @ThomasFinley see my edit
Comment: Oh! That simplies the things so much! But just one last thing: $A+B$ is an ideal and as $A,B\subseteq A+B\subseteq R$ so, $A+B=A$ or $B$ or $R$ as, $A,B$ are maximal ideals. If $A+B=A$ then $$B\subseteq A\subseteq R\implies A=B,$$ as $A\neq R$ and then $AB=AA=A=A\cap B=A\cap A=A.$ Same is the case, if, $A+B=B.$ If $A+B=R$ then, $1\in R$ so, $\exists a,b\in A$ and $B$ respectively such that $a+b=1.$ Now, if $$r\in A\cap B\implies r=ra+rb\in BA+AB.$$ **But I don't get how can I prove, $BA+AB\subseteq AB$ ?**
Comment: @ThomasFinley we're in a commutative ring: $AB=BA$. Note also that $AB$ is an ideal, so it is closed under finite sums. It's not just the set $\{ab\mid a \in A,b \in B\}$, but the ideal generated by that set.
Comment: But how is $AB$ an ideal? Say, $a_1b_1,a_2b_2\in AB$ where $a_1,a_2\in A$ and $b_1,b_2\in B$ then how can $a_1b_1+a_2b_2\in AB$ ? I mean how can we say, that $\exists a'\in A,b'\in B$ such that $a'b'=a_1b_1+a_2b_2$?
Comment: $AB$ is defined to be the $\{\sum_{k=1}^n a_kb_k \mid n\in \Bbb N,a_k \in A,b_k\in B \}$. It's defined like that to enforce it being an ideal.


Group level random effect

Question: In a lab experiment Participants took part in four different treatment. At the beginning of each session they were randomly assigned to groups of 4 and played repeated public goods game for 30 periods.
So far I have collapsed the data to obtain group averages, so that observation in a period are independent. This data was used to estimate a model with group-level random effects. Since in some groups there were participants whose incentives were different than those of other members of their group, I would like to test whether difference in incentives lead to disparities in behaviour.
Would it be OK for me to split the group average into two observations:

With the decisions made by the "special" individuals
with the average of 3 others

The same random effect would be assigned to both.
Comment: It seems like you want to use a mixed effect regression model here, without performing any preemptive collapsing of the data into averages.  Then the fixed effect would be an indicator for "special" individuals and the group-level averages would be a random effect.  If that's not clear I'm happy to flesh out more as a full-fledged answer
Comment: Thanks for your comment. "group-level averages would be a random effect" - do you mean the groups' means of all 30 rounds? Also, my concern is that without collapsing the data, how can I take into account the fact that the earnings of each individual are correlated, ie endogenous to the group.

PS
I used the following command in Stata to estimate the model using the group averages as observations:
xtmixed Profit Treatment  Period Treatment#(c.Period ) || groupintreatment: , mle residuals(independent, by(Treatment))
Comment: Ah I overlooked the note about 30 rounds - you are correct that you have to account for that.  I'm less familiar with stata, but could lay out the formula in R (using `lme4`) if that's helpful
Comment: I'm using R, but used Stata after not being able to find a proper solution in R. Could you please post the formula?
Comment: Also, if you could refer me to some reading material on the topic that would be great.

Thanks
Comment: @MichaelOberst I'm not sure whether you got a notification about my previous comment, hence I'm adding another one. Could you please post the formula for R? Thank you and happy new year.
Comment: of course - just posted an answer, hope it's helpful!  Happy New Year
Answer: Short answer & R Formula
I believe the R code you're looking for is something along the lines of the following, from the lme4 package.
myModel <- lmer(y ~ incentive + (1 | treatGroup/player), data = df)
Basically you're saying that there's a random effect for each player, who is nested inside of a treatment group with a random effect.  In addition, there is a fixed effect for incentive.
Longer answer / explanation
First of all, I personally find the terms "fixed effect" and "random effect" to be confusing and used somewhat inconsistently.  Andrew Gelman (Prof. at Columbia) has a great blog post on the topic.
Model formulation
So in lieu of using those terms, I'll try to write out the (basic) formulation based on what I understand from your question.  In particular, it sounds like we have:

16 players (indexed by $i$)
30 observations per player (indexed by $j$)
4 possible groups (indexed by $k$)
A binary "incentive" treatment $M$, where $m_{i(k)} = 1$ if the $i$-th player receives the incentive

Let $Y_{ij(k)}$ be the $j$-th observation for the $i$-th player, who is placed into the $k$-th group.
With these in mind, our regression model is as follows:
$$Y_{ij(k)} \sim N(\mu_{i(k)}, \sigma^2)$$
$$\mu_{i(k)} = \alpha_i + \alpha_k + \beta \cdot m_{i(k)}$$
Explanation of terms / quick notes
We have a few relevant terms:

$\alpha_i$: The 'baseline' for this player
$\alpha_k$: the effect of being in the $k$-th group
$\beta$: The impact of the incentive treatement

Some quick notes:

Using (1 | treatGroup / player) describes a nesting between levels (e.g., each player is in only one treatment group).
To your point in the comments, 30 observations for each player are assumed to be drawn from a common distribution - Our goal is to estimate the impact of (a) incentives and (b) treatment group on the mean of that distribution.
This answer assumes no interaction between the incentive and the treatment group.

Additional reading

I used this to refresh my memory on lmer syntax since I'm more accustomed to using rstan (link).
Skimming this tutorial (I haven't read the whole thing), it looks like a good resource, with an intuitive overview for the beginner!
Also see this CV post on lmer formula syntax

If you really want to get into this stuff, then Gelman & Hill (2006) is an excellent book.
Comment: Not sure about measuring differences between two different treatments - I'd recommend posting as a separate question.  Same to your question about pvalues - I think (?) you should be able to use the pvalues for the treatment coefficients in the regression model but might be worth fleshing out exactly what kind of hypothesis you are trying to test and asking as a distinct question - I use multilevel models a lot but not so much in an experimentation context
Comment: Thanks again for the answer. Since I have multiple treatments, can I add a "fixed effect" if I wish to compare between them? I need to tell you that  the allocation to the treatments is random, as well as allocation to a group with in the treatment.
Comment: And another question (hopefully the last one). If I need p-values (I've seen that people suggest that we should abandon them, but I till need them), would you recommend lmerTest for my case.
Comment: or maybe with mixed function from the afex package?
Comment: Thank you again, I might add another question. On top of what was asked in the my question, what would you do if the selection to the incentive is endogenous? I think this renders multi-level approach to be inappropriate as there might be correlation between some unobserved factors and the incentive. Would removal of individuals' effects solve this problem?

Group level random effect


Spherical Bessel Functions

Question: So I have been given a formula for the spherical Bessel functions in the form of $$ j_\ell(x)=(-x)^\ell \left(\frac{1}{x}\frac{d}{dx}\right)^\ell\frac{\sin(x)}{x} $$ which is Rayleigh's formula. I've been asked to show this satisfies Helmholtz's equation, however, I don't know how to diffentiate the middle part (containing the differential operator). To differentiate this, do I use the product rule as I normally would taking the $-x $, $\frac{1}{x}\frac{d}{dx}$ and $\frac{\sin(x)}{x}$ separately and pulling down the powers?
Many thanks in advance.
Answer: Quoting Wikipedia, when solving the Helmholtz equation in spherical coordinates by separation of variables, the radial equation has the form:
$$ x^2 y'' + 2x y' + \left[x^2-n(n+1)\right] y = 0. \tag{1}$$
For $n=0$, it is straightforward to check that $y_0=\operatorname{sinc}(x)$ is a solution.
Suppose now that a solution of $(1)$ has the form $y=(-x)^n f(x)$. Then $f(x)$ satisfies the ODE:
$$ x\, f + 2(1+n)\,f' +x\, f'' = 0.\tag{2} $$
Can you prove now that a solution of $(2)$ is given by $\left(\frac{1}{x}\frac{d}{dx}\right)^n y_0$?

Spherical Bessel Functions


Confidence Interval difference in means Interpretation

Question: I have a negative confidence interval ((-35.346,-8.570) for an independent samples t-test.
I understand confidence intervals for the mean. However, I am uncertain about the interpretation of the confidence interval for a difference in means.
Is this an accurate interpretation?
The confidence interval for mean difference in life satisfaction for the two groups is (-35.346,-8.570); as this interval does not contain zero, I can be confident that I have used a method that that will produce significantly different or unequal population means 95% of the time.
Or should I interpret the CI as follows:
The confidence interval for mean difference in life satisfaction for the two groups is (-35.346,-8.570); as this interval does not contain zero, I can be 95% confident that the population means are significantly different or unequal.
Is this splitting hairs?
Comment: I would interpret it as 'The population estimate for the difference of the means is statistically significant at the 5% level'
Comment: Thank you. I would interpret this similarly. However, I need to address the CI specifically.
Comment: @KatjaBuckley why do you think you're not?
Answer: The first interpretation is incorrect, but the second one is correct.
What will happen in multiple studies is dependent on the power and the true difference, not the confidence interval, so saying that something will happen 95% of the time based on this interval is incorrect.
The second paragraph is a correct interpretation, further you can conclude that the 2nd group has a higher mean than the first group (the entire interval is negative).
Answer: By this confidence interval you are confident that if you draw random samples 100 times that in 95 cases (95%) the mean of the second sample is statistically signifanct larger than the mean of the first sample. Hence, you are 95% confident that the mean of the second sample's population is larger than the mean of the first sample's population. This is, since the test adresses to draw inference from the samples to the population.
Therefore you can also interpret that the two samples do not come from the same population since they have different means.
Comment: So you think that if the true difference in the means is around -35 (a value in the interval) that you would still see 5 out of 100 new samples with mean 1 higher than (or at least not statistically significantly lower than) mean 2?  You also think that if the true difference in means is -10 (also in the interval, but much closer to 0), that you will only see 5 out of 100 new samples with mean 1 not significantly lower than mean 2?  Your interpretation is based on power (and the true mean difference), not the confidence interval.
Comment: The idea of confidence intervals is to say P(C_l <= theta <= C_u) >= 1-alpha. In other words, in 1-alpha * 100 % of the cases the confidence interval encloses the true parameter. However, since we draw random samples, there is a probability of getting confidence intervals which do not enclose the true paramter (e.g.: [5, 10] or whatsoever). We do not know how far or close these are, but the probability of getting such sample mean/differences s is there.
Comment: But, indeed, I should have written my answer in another form, I see that there's a problem there.
Comment: Einstein said something about how things should be explained as simply as possible, but no simpler.  Interpreting confidence intervals is one thing that is very hard to find that balance.  It is very easy in the pursuit of making things simple enough to understand to make the explanation too simple and therefore potentially misleading.