Euler's number inherently in $\mathbf{x^a=a^x}$ and $\mathbf{x.a=a^x}$?

Question: I was solving some equations graphically, when I came across an identity of Euler's number in two cases:
Case 1: $\mathbf{a.x = a^x}$
Ever wondered how for two numbers $\mathbf{\left\{{ a,x,y} \right \}\in \mathbb{R_{> 0}}
 :\space a.x = a^x= y}$?
For example natural numbers: $\mathbf{2.2=2^2=4}$

What is the general formula for $\mathbf{x}$, given that $\mathbf{x.a=a^x}$?

It has multiple solutions. One is always $\mathbf{x=1}$ And interestingly, if
$\mathbf{x_1=x_2: \space a=e}$. In this case, actually: $\mathbf{x_1=x_2=1}$. So as $\mathbf{a\to e\implies x_1=x_2\to1}$.
$\mathbf{e}$ "came out of nowhere". I am sure the analytical solution would introduce something like Lambert W function which would explain the presence of $\mathbf{e}$.
Case 2: $\mathbf{x^a = a^x}$
Or second case when: $\mathbf{\left\{{ a,x,y} \right \}\in \mathbb{R_{> 0}}
 :\space x^a = a^x= y}$?
For example $\mathbf{2^2=2^2=4}$ or how $\mathbf{3^{2.478...}\approx2.478...^3\approx15.216}...$

What is the general formula for $\mathbf{x}$, given that $\mathbf{x^a=a^x}$?

It has multiple solutions. And interestingly, as $\mathbf{x\to e\implies x_1\to x_2}$.
=======================================
What are the two formulas for $x$ and how to obtain them?
Solution attempt
How to solve for $x$?
I tried this:
$\mathbf{log_ax^a = log_aa^x\\a.log_ax=x\\a.x^{-1}log_ax=1\\a.log_ax^{x^{-1}}=1}\\$
Lambert W function comes into mind. But how?
Comment: Does this answer your question? [Solution of $ax=a^x$](https://math.stackexchange.com/questions/356064/solution-of-ax-ax)
Comment: Case $2$ is a very popular duplicate here. See for example [here](https://math.stackexchange.com/questions/114453/solve-xy-yx), or many others. Case 1 is also a duplicate here. Both is solved with the Lambert W function.
Answer: It is the answer that provides the solution to your second equation:
$$ax=a^x$$
$$\frac{a^x}{x}=a$$
Let, $a^x=t$ and $x\ln a=\ln t$, then we get
$$\frac{t}{\frac {\ln t}{\ln a}}=a$$
$$\frac {t}{\ln t}=\frac {a}{\ln a}$$
$$\frac {\ln t}{t}=\frac {\ln a}{a}$$
Based on the first answer, we have
$$t=e^{-W\left(-\frac{\ln a}{a}\right)}$$
$$x=\frac {\ln t}{\ln a}=\frac {-W\left(-\frac{\ln a}{a}\right)}{\ln a}.$$
Answer: This answer deals with the solution to the first equation:
$$x^a=a^x$$
$$a\ln x=x\ln a$$
$$\frac{\ln x}{x}=\frac{\ln a}{a}$$
Let, $x=t^{-1}$
$$t\ln t=-\frac{\ln a}{a}$$
$$W\left(\ln te^{\ln t}\right)=W\left(-\frac{\ln a}{a}\right)$$
$$\ln t=W\left(-\frac{\ln a}{a}\right)$$
$$\ln x=-W\left(-\frac{\ln a}{a}\right)$$
$$x=e^{-W\left(-\frac{\ln a}{a}\right)}.$$
Comment: Or, $$x=\frac {a \ln x}{\ln a}=\frac {-aW( -\ln a/a)}{\ln a}$$


How to add additional drupalSettings to all pages with an event subscriber

Question: I need to set the user timezone on all pages in the JS drupalSettings object, so that it can be accessed on any page. As drupalSettings.user is provided by core, I want to set drupalSettings.user.timezone, with the user timezone from the server.
While this can quickly and easily be done in hook_page_attachments(), I am trying to switch from using hooks, to using Event Subscribers, as Drupal hooks are the old paradigm, and event subscribers is the new paradigm.
Can someone explain how to add attached assets (a library, or drupalSettings) to all pages from an event subscriber?
Additional Context
As mentioned by 4uk4 in the comments, this event is not dispatched by Drupal core, but rather by Symfony underneath the hood. This means it is not a true 'Drupal' methodology, as core maintainers have not chosen to dispatch an event that is equivalent to hook_page_attachments(). So which is better? The answer is 'neither' at this point, as core is ambivalent as to whether it invokes a hook or dispatches an event. It does however seem that core is moving towards creating events that match existing hooks, as can be seen in this issue: https://www.drupal.org/project/drupal/issues/2551893. And here is a discussion that considers the differences between hooks and events: https://www.previousnext.com.au/blog/alter-or-dispatch-drupal-8-events-versus-alter-hooks
Comment: Wouldn't that be the decision of the core maintainers to dispatch an event instead of invoking this hook?
Comment: The hook is part of the Drupal API. You are replacing it with a Symfony API event. There are a lot of cases where you need to switch from one API to another, but avoiding hooks is not one of them.
Comment: Core both dispatches the event, and invokes the hook. Traditionally I've implemented the hook, but I wanted to switch to using an event subscriber. Note that user timezone is not part of drupalSettings by default, I'm extending it.
Comment: I'm trying to prioritize event subscribers over hooks, as they are proper OOP and can be extended or overridden as necessary. I'd be surprised if hooks weren't eventually removed altogether, as they are legacy.
Comment: I added some additional context to the original question.
Answer: This can be done by acting upon the Symfony\Component\HttpKernel\KernelEvents::RESPONSE event. See the code below for an example (Note - you'll need to register the event subscriber, which is not shown here):
namespace Drupal\[MODULE]\EventSubscriber;

use Drupal\Core\Render\AttachmentsInterface;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpKernel\KernelEvents;

/**
 * Adds the user's timezone to the drupalSettings JS object on all pages.
 */
class SetUserTimezoneToDrupalSettings implements EventSubscriberInterface {

  /**
   * {@inheritdoc}
   */
  public static function getSubscribedEvents() {
    return [
      // Respond to KernelEvents::RESPONSE.
      KernelEvents::RESPONSE => ['onResponse', 1000],
    ];
  }

  /**
   * Event handler for KernelEvents::RESPONSE.
   */
  public function onResponse(\Symfony\Component\HttpKernel\Event\ResponseEvent $event) {
    // Only act upon the master request and not sub-requests.
    if ($event->isMasterRequest()) {
      $response = $event->getResponse();
      // Only act if the response is one that is able to have attachments.
      if ($response instanceof AttachmentsInterface) {
        // Get any existing attachments.
        $attachments = $response->getAttachments();
        // Set the user's timezone to the drupalSettings array.
        $attachments['drupalSettings']['user']['timezone'] = date_default_timezone_get();
        // Set the updated array back on the object.
        $response->setAttachments($attachments);
      }
    }
  }

}

Comment: Asked and answered in the same minute. Amazing!
Comment: Thanks, it was actually a few hour process, I just figured that someone (very potentially myself) would be interested in this in the future, so after I figured out the solution, I posted the question with the answer at the same time, so people could reference it.


Can't get this flashing light in javascript to stop flashing with clearInterval

Question: <!DOCTYPE html>

    <title>stoplight</title>
    <link rel="stylesheet" type="text/css" href="stoplight.css">

        <button id="stop">Stop</button>
        <button id="slow">Slow</button>
        <button id="go">Go</button>
        <button id="caution">Caution</button>

Basically, I'm just trying to select each div of the stoplight and change the background color. First, I thought I could just reset each div to its original color. I'm trying to do the same with the setInterval function that starts a flashing yellow light, but it doesn't seem to stop it and breaks the rest of the functions.
document.getElementById('stop').onclick = goRed;
document.getElementById('go').onclick = goGreen;
document.getElementById('slow').onclick = goYellow;
document.getElementById('caution').addEventListener('click', () => {
    blink();
});

let middle = document.getElementById('middle');
let blinking;

function goRed() {
reset();
document.getElementById('top').style.backgroundColor = '#FF0000';
};

function goYellow() {
    reset();
    middle.style.backgroundColor = '#FFFF00';
};

function goGreen() {
    reset();
    document.getElementById('bottom').style.backgroundColor = '#7FFF00';
};

function blink(){
    reset();
    blinking = setInterval(() => {
    middle.classList.toggle('onYellow');
    }, 1000);
};

function reset() {
    document.getElementById('top').style.backgroundColor = '#A0522D';
    document.getElementById('middle').style.backgroundColor = '#A0522D';
    document.getElementById("bottom").style.backgroundColor = '#A0522D';
    clearInterval(blinking);
 };

Comment: Can you provide your CSS as well -- ideally using the code snippet tool so your example can be run/executed in-line within your question.
Answer: You are mixing in-style coloring, with class based coloring, and in some cases, the browser will decide which gets a priority. Unless you add !important in the class background to enforce the attribute.
I suggest removing all in-style coloring and sticking only to classes so that you will not have to use !important.
This way, it is easier to work with blinking and setInterval.
See my code snippet below.

document.getElementById('stop').onclick = goRed;
document.getElementById('go').onclick = goGreen;
document.getElementById('slow').onclick = goYellow;
document.getElementById('caution').addEventListener('click', () => {
  blink();
});

let middle = document.getElementById('middle');
let blinking;

function goRed() {
  reset();
  document.getElementById('top').className = "onRed";
};

function goYellow() {
  reset();
  middle.className = "onYellow";
};

function goGreen() {
  reset();
  document.getElementById('bottom').className = "onGreen";
};

function blink() {
  reset();
  blinking = setInterval(() => {
    middle.classList.toggle('onYellow');
    middle.classList.toggle('onOff');

  }, 1000);
};

function reset() {
  document.getElementById('top').className = "onOff";
  document.getElementById('middle').className = "onOff";
  document.getElementById("bottom").className = "onOff";
  clearInterval(blinking);
};
#mainContainer div {
  border: 1px solid grey;
  width: 40px;
  height: 40px;
  border-radius: 40px;
}

.onYellow {
   background-color: #ffff00;
}

.onOff {
   background-color: #A0522D;
}

.onGreen {
  background-color: #7FFF00;
}

.onRed {
  background-color: #FF0000;
}
<!DOCTYPE html>

  <title>stoplight</title>
  <link rel="stylesheet" type="text/css" href="stoplight.css">

    <button id="stop">Stop</button>
    <button id="slow">Slow</button>
    <button id="go">Go</button>
    <button id="caution">Caution</button>


How to align 3 div's on 1 row

Question: Trying to make 3 of my div's (cards) to fit in one line, and it keeps going for all my divs
Code:

Onetap Master Package

Master config & private javascript

CHF 10.99

Onetap Master Config

Premium Javascripts, lifetime updates

CHF 7.99

Onetap Script

Private script with Anti-Aim, doubletap, indicators & more

CHF 5.99

Aimware Master Package

Multiple private configs with with anti-aim lua

CHF 10.99

Aimware Anti-Aim Lua

Private Aimware Anti-Aim lua

CHF 5.99

Personal Config

Private config made for you until it's good.

CHF 14.99

Config Class

Learn to sell your configs and make profit

CHF 24.99

Config Swap

Switch one of my configs to another one

CHF 5.99

If it's possible I'd want the first 3 divs to the next to eachother on one row, then the next 3, etc.
Expected (pic taken from a different site)

What it's currently showing

getting It looks like your post is mostly code; please add some more details. but i think i already explained it well enough.
Answer: You may use flex to align 3 divs per row. More flex guide can refer here

.card {
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  transition: 0.3s;
  border-radius: 5px;
  /* 5px rounded corners */
  width: 40%;
  height: 35%;
}

img {
  border-radius: 5px 5px 0 0;
}

.container {
  padding: 10px;
}
/* Flex configuration to align 3 divs per row */
.flex-container {
  display: flex;
  flex-wrap: wrap;
  justify-content: center;
}

.flex-container > .card {
  flex-basis: 30%;
  margin: 1%;
}

Onetap Master Package

Master config & private javascript

CHF 10.99

Onetap Master Config

Premium Javascripts, lifetime updates

CHF 7.99

Onetap Script

Private script with Anti-Aim, doubletap, indicators & more

CHF 5.99

Aimware Master Package

Multiple private configs with with anti-aim lua

CHF 10.99

Aimware Anti-Aim Lua

Private Aimware Anti-Aim lua

CHF 5.99

Personal Config

Private config made for you until it's good.

CHF 14.99

Config Class

Learn to sell your configs and make profit

CHF 24.99

Config Swap

Switch one of my configs to another one

CHF 5.99

Or you may use grid to align 3 divs in a row.

.card {
  box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2);
  transition: 0.3s;
  border-radius: 5px;
  /* 5px rounded corners */
  /* width: 40%;
  height: 35%; */
}

img {
  border-radius: 5px 5px 0 0;
}

.container {
  padding: 10px;
}
/* Grid configuration to align 3 divs per row */
.grid-container {
  display: grid;
  grid-row-gap: 20px;
  grid-column-gap: 30px;
  grid-template-columns: repeat(3, 1fr);
}

Onetap Master Package

Master config & private javascript

CHF 10.99

Onetap Master Config

Premium Javascripts, lifetime updates

CHF 7.99

Onetap Script

Private script with Anti-Aim, doubletap, indicators & more

CHF 5.99

Aimware Master Package

Multiple private configs with with anti-aim lua

CHF 10.99

Aimware Anti-Aim Lua

Private Aimware Anti-Aim lua

CHF 5.99

Personal Config

Private config made for you until it's good.

CHF 14.99

Config Class

Learn to sell your configs and make profit

CHF 24.99

Config Swap

Switch one of my configs to another one

CHF 5.99

How to align 3 div's on 1 row


Recuperar valor do textarea no form .serialize()

Question: Sempre usei função genérica, mas estou com um problema na recuperação do valor do textarea. Utilizo um editor de texto no textarea, o CkEditor.
Aqui segue a função que estou utilizando:
function sendPost(form, action, callback) {
    ajaxRequest == null ? ajaxRequest = null : ajaxRequest.abort();
    if (validateForm($("#" + form))) {
        var msg;
        alert($("#"+form).serialize());
        //Utilizei esse alert acima, só para verificar como o POST está indo
        ajaxRequest = $.post("crud/" + action + ".php", $("#" + form).serialize(),
            function (data) {
                dataFromAjaxRequest(data, callback);
            });
    } else {
        $("#dialog_modal_mensagem").dialog("open");
    }
}

A saída do alert:
titulo=teste&data=03%2F04%2F2014&texto=

Answer: Você tem que atualizar os campos relacionados ao CKEditor, antes de usar o serialize:
for ( instance in CKEDITOR.instances )
    CKEDITOR.instances[instance].updateElement();

Depois disso, o campo do CKEDitor estará preenchido, e deve funcionar.
Referência:
How to ajax-submit a form textarea input from CKEditor?


Best Linux Partitioning for Multiple Resizeable HDD

Question: My ISP has a machine with 6 HDDs (3TB+3TB+3TB+2TB+2TB+1TB) which we want to turn into a FTP/Download server for local users. I am planning to install CentOS 6.5 on one 2TB, rest of all space shall contain big files for download. Considering that we do not have even sized or same volume HDDs right now and HDD size may be upgraded further, what should be the best partitioning scheme for the system (LVM/Soft RAID/Combination)? And how to achieve that best arrangement?
Please focus on these points:
a. Mirroring or data retrieval is not important here
b. Performance and harddisk speed is major concern
c. The 1TB HDD may be upgraded in next few months
d. Atleast 400 LAN users download contents simultaneously from such servers
e. No hardware RAID controller available
Thanks in advance for your cooperation!
Comment: Also, how much data to you actually need to/expect to store?  That may make a big difference.  If you only need to store 2TB you could do RAID10 across 6 disks.  But that won't work if you need to store 10TB.
Comment: "Mirroring or data retrieval is not important here." Can you be clearer about this expectation? RAID is not to preserve data integrity; that's what backups are for. The purpose of RAID is to preserve uptime. Will your 400 LAN users be up a creek if this thing goes down and needs time to be rebuilt?
Comment: Grant: I expect to store maximum possible data on the HDDs, maybe 14TB now and 16TB after the upgrade.

@Skyhawk: By that I mean if any HDD crashes somehow we do not need that data rather we would like to keep the machine running with other contents on the other disks. It's a free service for the users and they should accept downtime.
Comment: @bonytasnim If you don't want the machine to go down when a disk fails, why are you proposing to use a single unmirrored disk for the operating system? You need to think this through.
Answer: My vote is for LVM. LVM allows you to stripe, resize and add and remove disks fairly easily on the fly. If your system has hot swap bays these upgrades can be done with zero downtime.
Answer: LVM adds a lot of flexibility at no performance cost, using it is a no-brainer.
While you don't want to spend space on backups (which would take up half the space), with 6 disks you do have a high risk of disk failures so you need some form of redundancy (for uptime, not point in time recovery). You can get that with raid-like technology, either Linux's raid (which is integrated with LVM) or Btrfs. The preferred raid level would depend on how much read performance, write performance, and uptime you prefer. An interesting property of Btrfs is that you can use different raid levels for data and metadata. With higher redundancy for metadata, some failures can leave big holes in large files but still keep filesystem integrity and leave a proportion of small files unaffected.
Answer: For best speed and a possibility to extend this I'd use 3TB+3TB RAID0, 2TB+2TB RAID0 connected as JBOD using md (it would create an 10TB drive). I'd wait for 1TP upgrade to 3TB and then I'd add 3TB+3TB RAID0 to JBOD (it would then be 16TB). It would all be twice as fast as a single drive.
If you'd first wait for 1TB->2TB upgrade, then another setup would be faster:
RAID0(JBOD(3+2)+JBOD(3+2)+JBOD(3+2)). This would be 15TB, 3 times faster than a single drive.
But with no redundancy it would all die with no chance for recovery with a first dying drive. So, a reasonable setup would really be:
JBOD(RAID5(3TB+3TB+3TB),degradedRAID5(2TB+2TB)), which, after upgrade 1TB=>2TB would be
JBOD(RAID5(3TB+3TB+3TB),RAID5(2TB+2TB+2TB)). This would get you 10TB with redundancy, with fast reads (3*speed of single drive) and slow writes (slightly slower than single drive).
Comment: Thanks Tometzky, for your answer; its helpful and scaled to my actual requirements. Problem is I cannot wait for 1TB -> 3TB nor 1TB-> 2TB and it is obvious that more space will be added. Is there any method to do RAID on 2 upgraded HDDs later on that system?
Comment: First option will allow you to use JBOD(RAID0(3+3)+RAID0(2+2)). JBOD will allow you to easily add and remove drives at the end of array. You'll just need to expand a filesystem after adding a new drive and shrink it before removing.
Answer: Considering the requirements you listed, I would use the following setup:

Array 1 - RAID0 (3x3TB disks)
This would yield 9TB of storage.  Keep in mind that, if a single disk fails in this array, your data is toast.  But, you did point out that you don't care about your data being retrievable but performance is important, so this throws all caution to the wind and gives you the best performance with the least amount of protection.
Array 2 - RAID1 (2x2TB disks)
I would use this as your backup storage along with OS installation.  You do intend on taking backups, right?

A couple of things to note:

Use the CentOS partitioning manager to configure the RAID configurations prior to installing the OS.

Once the OS is installed, you can then use lvm to manage things like snapshots, filesystem growth, and other tasks.  Changes to your RAID configurations would be handled with mdadm

This is a risky configuration.  I would be very surprised if an ISP is not willing to release money to buy appropriate hardware required to set up a file server.  You should really have identical disks, an appropriate storage controller, and some type of DAS enclosure.
Comment: Thanks for your support. Well, this is just a free service for local users so, the ISP is not considering enterprise costs.

What about the the other disk? Add to LVM as single disk? Another big question is what if I do want to install with 2 2TB HDDs (maybe RAID1 as per your suggestion) and add the other 3TBs later? Actually those disks have data now in ntfs format which may take some time to freed up. Is there any way to use those with existing data, by the way?
Comment: I would use the 1TB disk for the OS installation.  Adding disks to RAID arrays is pretty straightforward - it can just take a long time depending on the amount of storage in use.  Installing the OS on a standard physical drive saves the headache of having to worry about the bootloader being present on new drives added to the configuration.


How many data points to fit an ex-Gaussian distribution?

Question: I want to fit an ex-Gaussian distribution (which has three parameters: $\mu$, $\sigma$, and $\nu$) to reaction time data.
How many data points are needed as a minimum to reasonably fit this distribution?
I am aware that an answer to this question depends on many factors (e.g., how much data actually deviate from the theoretical distribution) and that "more is better" in general. Anyway, are there any rules-of-thumb, or publications which could point me to the needed minimum of data points?
Answer: If I recall correctly, as sample size decreases, mu will manifest positive bias while tau will manifest negative bias. Sigma will manifest either positive or negative bias depending on true values of the other parameters. This paper compares two methods of fitting ex-Gaussians, and from the figures I'd probably avoid going below 40 observations.
Answer: Since the parameters are estimated using maximum-likelihood method you have that
$$\sqrt{n}(\hat\theta -\theta)\to N(0,I_{\theta}^{-1}),$$
where $I_{\theta}$ is the Fisher information matrix, and $\theta$ is the vector of parameters in question. Hence
$$Var(\hat\theta-\theta)\approx 1/nI_{\theta}^{-1}$$
If you have calculated $I_{\theta}^{-1}$ and you suspect the approximate values of true $\theta$, this will give you an idea of approximate variance. You can then work out what is the minimum number of points. This is not perfect solution, but as a rule of thumb it might work.
Comment: Thanks for your thoughts on it; but I have to admit that your approach exceeds my mathematical abilities - I have no clue on how to approach this ;-)
Comment: No problem. I posted this answer to complement Mike's advice. I was just trying to quantify 30 points rule, i.e. if you have central limit theorem, then $n=30$ gives you 0.1 error. This is a very bad statement, since I do not speak about what is 0.1 error, but for thumb of rule it works. The main point is that the approximation error for assymptotic results where CLT is employed (which is the case for maximum likelihood) is always $c/\sqrt{n}$, where $c$ is the constant depending on the distribution.


Is the "$\pi$ is wrong"-movement serious or a joke?

Question: Is the "$\pi$ is wrong"-movement serious or is it to be taken as a joke?
https://www.tauday.com/tau-manifesto
If serious, what are they hoping to achieve? Throw out every book with $\pi$ in it and make up new ones with $\tau$?
Comment: It's not so much that $\pi$ is "wrong"; it's that--from a basic geometrical standpoint--$\tau$ is more intuitive.
Comment: This reminds me of a physicists who got tired of using planck's constant to write the quantity $\frac{h}{2\pi}$ and just wrote a new constant $\hbar$
Comment: Here is a video from Vihart on the topic: https://www.youtube.com/watch?v=jG7vhMMXagQ
Answer: These people have a point. But they must know that their quest is doomed to failure. It's like proposing that humanity switches from base-10 to base-12 counting, because it's more logical. Or changing the sign of the electron's charge to $+$. Or using proportional representation to choose our elected leaders. Or switching to metric worldwide. Or...the list is endless.
Think of them as hobbyists rather than crusaders.
Comment: Not everyone is from the US :)
Answer: $\pi$ is actually a little bit better than $\tau$ according to The Pi Manifesto.
If people can't change the sign of electron, they won't change the quantity for $\pi$ either.
Answer: I'm pretty sure that some people seriously believe expressing things in terms of $\tau$ is a more useful convention than in terms of $2 \pi$.
Changing the convention, one must get people to adopt the new convention. If it's useful enough, then over time it will become known enough that its adherents can use it without explanation. If it's that much better, it will eventually replace the old convention.
Either outcome would be a desirable goal for those who believe such a thing.
Answer: "No, really." :-)
They would rather replace $2\pi$ with $\tau$ as the common used constant.
The french revolution was one of the last events that led to some straightening out, dropping odd anglic measure units  (at least on the continent :-), weird currency units and tried to establish a better calendar, but the Thermidor etc. somehow did not get popular.
No idea what happens if Trump gets some funny ideas in this area.
Answer: Historically $\pi$ made alot of sense, the ratio between circumference and diameter.
It just happens to be that diameter is unhandy, and radius is so much more often used.
That's why $\tau$ could be used.
It is not about burning books, it is more about publishing books with $\tau$ in the future.
From my experience there is no real need for it tho.
Answer: I get the impression it's not the most serious debate, but Numberphile has done several videos on the subject. https://m.youtube.com/watch?v=ZPv1UV0rD8U


Why are the random variables X and Y independent in this case?

Question: Let's say that we have 2 discrete random variables, $X$ and $Y$ and we care about their joint PMF. If this joint PMF is as follows,
$$p_{X, Y} (x_1, y_1) = 2/9, \hspace{0.5cm} p_{X, Y} (x_2, y_1) = 4/9$$
$$p_{X, Y} (x_1, y_2) = 1/9, \hspace{0.5cm} p_{X, Y} (x_2, y_2) = 2/9$$
how can we know intuitively that $X$ and $Y$ are independent? By taking the definition it is easily proven, but how can I understand it without the definition?
I can see that, no matter the value $Y$ takes $(y_1$ or $y_2)$, the value of the PMF for $X=x_2$ is double the value of its value for $X=x_1$. But why does this make these 2 random variables independent??
Information about $Y$ still gives you info about $X$. If $Y=y_2$, $P(X=x_1)= 1/9$ but if $Y=y_1$, then we have that $P(X=x_1)=2/9$. So the probabilities change depending on the value of $Y$.
I probably didnt express my question perfectly because I am quite confused, so sorry about that. If someone could help me clear up the misunderstanding, I'd really appreciate it.
Thanks a lot.
Comment: The mistake in your reasoning is when you say that, with $Y=y_2$, $P(X=x_1)=1/9$. In fact, with $Y=y_2$, the probabilities of $X$ should be calculated as *conditional probabilities*. $P(X=x_1\mid Y=y_2)=1/3$ and so is $P(X=x_1\mid Y=y_1)=1/3$, so the marginal distribution of $X$ does not depend on $Y$. (In other words, knowing $Y$ does *not* reveal any information about $X$ that is not already known.)
Comment: Thanks, you are correct of course. So basically with think with respect to conditional probabilities, because thats where independence comes in. Independence of two events means that new information about one does not change my beliefs about the other, and that is expressed via conditional probabilities.
Answer: "I can see that no matter the value $Y$ takes ($y_1$ or $y_2$), the value of the PMF for $X=x_2$ is double the value of its value for $X=x_1$. But why does this make these 2 random variables independent??"
Answer
Regardless of the value of $Y$, $x_2$ is twice as likely as $x_1$, so in every option the conditional probability to have $X=x_2$ is $2/3$ and $X=x_1$ is $1/3$.
Information about $Y$ does not change these probabilities, so they are independent.
Comment: Thanks for the answer. So essentialy I have to think this with regards to conditional probabilities, because thats where independence comes in. Independence means that that NEW info does not change my beliefs, and that is expressed via conditional probabilities. In addition to this, the way you make the aforementioned calculations is this? : $P(X=x_1|Y=y_1)=\frac{P(X=x_1, Y=y_1)}{P(Y=y_1)}=\frac{2/9}{2/9+4/9}=2/6=1/3$, is that correct? And that is equal to $P(X=x_1|Y=y_2)=1/3$.


Quantiles of a compound gamma/negative binomial distribution

Question: Are there any formulae for the quantiles of a compound gamma/negative binomial distribution? That is, suppose we have
$$N \sim \text{NegBin}(\alpha, \lambda)$$
and conditional on $N$,
$$Y =
\begin{cases}
 0 & \text{if } N = 0 \\
 \sum_{i=1}^N X_i & \text{otherwise}
\end{cases}$$
where each of the $X_i$ is iid $\text{Gamma}(\mu, \sigma)$. Given values of $\alpha, \lambda, \mu$ and $\sigma$, I'd like to calculate the quantiles of $Y$.
Background: I'm modelling ATM withdrawals per day. The number of withdrawals I assume has a negative binomial distribution, and the amount of each withdrawal has a gamma distribution. I can estimate all the parameters from the data, and now I want to estimate how much cash to keep in the ATM. Keeping too little cash risks not having enough to meet demand, but too much is also bad. So, I want to estimate how much cash is needed to meet demand for a given level of cash-out risk. That is, I want an upper quantile of the compound distribution, say at the 99%, 99.9% or 99.99% level.
Right now I'm using simulation to get an estimate, but this takes a long time especially for very high quantiles. Eg if I want to get the 99.99% quantile, I need at least 10,000 replications, and ideally much more than that; I also need to repeat this for each ATM. So some kind of analytical approximation would be great.
(I realise that the actual distribution is probably more complicated than a simple negbin/gamma combination. For example the observed withdrawals would be right censored since if an ATM runs out of cash then there's no more withdrawals for the day. I'm keeping things simple to start with.)
Answer: As a practical answer to the real questions you're addressing, such high quantiles will generally be quite sensitive to issues with model choice (especially such things as whether you model the right censoring and how heavy the tails are in the components).
But in any case - especially when dealing with high quantiles where ordinary simulation becomes impractical - that has great value in its own right; it's an interesting question from both theoretical and practical standpoints.

A couple of other approaches to this problem are (1) using the Fast Fourier Transform and (2) direct numerical integration.
One useful reference on this topic is Luo and Shevchenko (2009)$^{[1]}$.
In it they develop an adaptive direct numerical integration approach that's faster than simulation and competitive with FFT.
The more traditional approach in actuarial work was been (3) Panjer recursion, which can be found in numerous texts. Embrechts and Frei (2009)$^{[2]}$ discuss and compare Panjer recursion and FFT. (Note that both of these techniques involve discretization of the continuous distribution.)

On the other hand, doing a very unsophisticated version of simulation, and with no effort whatever made to be efficient, generating from a compound gamma-negative binomial isn't particularly onerous. This is timing on my kids' little laptop:
system.time(replicate(100000,sum(rgamma(MASS:::rnegbin(1,4,2),5,.1))))
   user  system elapsed
   2.82    0.00    2.84

I think 2.8 seconds to generate 100$^{\,}$000 simulations of a compound distribution on a slow little laptop really isn't bad. With some effort to be efficient (of which one might suggest many possibilities), I imagine that could be made a good deal faster.
Here's the ecdf for $10^6$ simulations (which took about 29 seconds):
$\hspace{1cm}$
We see the characteristic discrete jump at zero you expect to see with a compound distribution.
[While it should be easy to make simulation a lot faster, all three alternatives mentioned here - if carried out sensibly - should be a lot faster still.]

You should note that the actuar package supports computation with compound distributions, and offers several methods for calculation with them.
See, for example, this vignette which discusses this facility.
[Of possibly some further passing interest, note that there is an R package for the Poisson-lognormal distribution -- poilog; if you need that distribution at some point it may be useful.]

Added in edit:
A potential quick approximation where the gamma shape parameter isn't changing -
In the gamma case, because a convolution of gammas with constant shape parameter is another gamma, you could write down the distribution of $Y|N=n$, and then evaluate the cdf and the density at a large number of grid-values at each $n$, then simply accumulate the sum directly (rather as one would for a KDE). The direct calculation only yields a lower bound to the true quantile, but if the negative binomial is not heavy tailed it should be quite rapid.

References:
[1]: Luo, X. and Shevchenko, P.V. (2009),
"Computing Tails of Compound Distributions Using Direct Numerical Integration,"
Journal of Computational Finance, 13 (2), 73-111.
[arXiv preprint available here]
[2]: Embrechts, P., and Frei, M. (2009),
"Panjer recursion versus FFT for compound distributions,"
Mathematical Methods of Operations Research, 69:3 (July) pp 497-508.
[seems to be a pre-publication version here]
Comment: Thanks Glen. The problem with simulation is the fact that the parameters are not constant. The rate of withdrawals changes during the day, and so does the amount per withdrawal; and I'm also doing this for 300-odd ATMs each of which has its own distribution. This complicates how I do the draws -- ie I can't do it all in one line as you've got there. I could be doing it wrong though.
Comment: Um, as far as I know, both the negative binomial and the gamma can vary their parameters in that call I did. If you needed 10K simulations (say - or even 30 for something short), what would you like to simulate (just as an example - I'm sure you can generalize)? Certainly the gamma is easy to make vary.
Comment: Yeah, you're right. Looking at it again the problem really seems to be the volumes involved. Each ATM can have hundreds or thousands of transactions per day; I want to get quantiles for each of 365 days (the modelling includes daily cycles); and there are 300 ATMs. When you multiply all that out, it turns into runtimes of hours.
Comment: If you want to to use simulation, it may be worth looking at using tools like Rcpp. But some of the alternatives I've described may suit your purposes.
Comment: Actually, thinking about it, there will be some additional shortcuts for the gamma case, if the scale parameter in the gammas doesn't change within one compound sum. If that's of any use, I'll write some more about it.
Comment: Sure, I'd be interested in any shortcuts possible.
Comment: Added some text on it. I really haven't thought about it in depth, but it might be of some use.


Show that
$\gcd(3k+2,5k+3)=1$

Question:
Let $k$ be an integer. Show that
  $$\gcd(3k+2,5k+3)=1$$

I know that I can use the Euclidean algorithm, but the $k$ is bugging me. Should I use induction or something?
Answer: You can do it just like normal.
$$ 5k+3 = 1(3k+2) + (2k+1)$$
$$ 3k+2 = 1(2k+1) + (k+1)$$
$$ 2k+1 = 1(k+1) + (k)$$
$$ k+1 = 1(k) + (1)$$
Since we see $1$, we're done.
Comment: Thanks for you took time to answer my question Nitin!
Answer: Suppose, there is a $d \in \Bbb N$ such that $d|(3k+2)$  and $d|(5k+3)$
$\therefore d|\{5(3k+2)\}$ and $d|\{3(5k+3)\}$
$\implies d|(15k+10)$ and $d|(15k+9)$
$\implies d|\{(15k+10)-(15k+9)\}\implies d|1$.
Now, do you get it?
Answer: Another way is to use Cramer's rule (or elimination) to solve for $\,k\,$ and $\,1\,$ below
$$ \begin{eqnarray} \ 5\,k\, +\, 3\cdot 1 &=& i\\[.3em]  3\ k\, + \ 2\cdot 1 &\ =\ & j\end{eqnarray}
\quad\Rightarrow\quad \begin{array}\  k \  = \ \ \ \,  2
\,i\, -\, 3\ j \\[.3em]  \color{#c00}{\bf 1}\ =\,  {-3}\ \color{#0a0}i\, +\, 5 \ \color{#0a0}j \end{array} $$
Therefore, by the lower equation in the RHS system: $\ n\mid \color{#c00}{\bf 1}\ $ if $\,\ n\mid \color{#0a0}{i,\,j} = 5k\!+\!3,3k\!+\!2$

Remark $\ $ In the same way we can prove more generally
Theorem  $\ $ If $\rm\,(x,y)\overset{A}\mapsto (X,Y)\,$ is linear then $\: \rm\gcd(x,y)\mid \gcd(X,Y)\mid \color{#90f}\Delta \gcd(x,y),\ \ \ \color{#90f}{\Delta := {\rm det}\, A}$
e.g. $ $ in OP we have $\,\color{#90f}{\Delta =\bf  1}\,$ so the above yields  $\ \gcd(5k+2,3k+2)\mid\color{#90f}{\bf 1}\cdot\gcd(k,1) = 1$
using the map $\ (k,1)\mapsto (5k+3,3k+2)\ $ i.e.
$$ (k,\,1)\,\mapsto\, (k,\,1)\,\underbrace{\begin{bmatrix}5 & 3\\ 3 &2\end{bmatrix}}_{\large \det A\, =\, \color{#90f}{\bf 1}} =\, (5k+2,\,3k+2)$$

Show that
$\gcd(3k+2,5k+3)=1$


Gulp watch mostra muito erros eslint. Quais dependências estão faltando?

Question: Fiz o git clone de um app Ionic 1 feito com o Generator-M-Ionic
Fiz os seguintes passos para instalar as dependências:
npm install
bower install
npm install --global gulp-cli

E os prerequisitos do próprio gerador:
npm i -g yo gulp bower
npm i -g generator-m-ionic

Após isso o comando
gulp watch

O terminal executa normal
[13:58:09] Starting 'jsonlint'...
[13:58:10] Finished 'watch' after 191 ms
[Browsersync] Access URLs:
 ----------------------------------------
       Local: http://localhost:3000
    External: http://192.168.100.235:3000
 ----------------------------------------
          UI: http://localhost:3001
 UI External: http://192.168.100.235:3001
 ----------------------------------------
[Browsersync] Serving files from: app
[Browsersync] Serving files from: .tmp
[13:58:10] Finished 'jsonlint' after 488 ms

Mas váários erros como esse abaixo aparecem em seguida no terminal e meu navegador só abre uma tela vazia.
app/main/controllers/logoutCtrl.js
   2:1   error    "SocialWaveApp" is not defined          no-undef
   3:3   warning  Unexpected console statement            no-console
   4:3   error    "mixpanel" is not defined               no-undef
   8:9   error    "hideSheet" is defined but never used   no-unused-vars
  13:32  error    "index" is defined but never used       no-unused-vars
  17:9   error    "mixpanel" is not defined               no-undef
  22:9   warning  Unexpected console statement            no-console
  30:11  error    "facebookConnectPlugin" is not defined  no-undef
  34:13  warning  Unexpected console statement            no-console

Como descobrir quais são as dependências e o que está faltando?
Answer: O problema era com algumas dependências, como fserve que só roda no Mac, então a solução foi passar a usar um macbook, agora o aplicativo funciona.
Os erros eslint mostram apenas que meu projeto não está usando as boas práticas do  JavaScript.


What's the intuition of the transpose of a matrix?

Question: I know the transpose is to swap the columns and rows of a matrix. And $A^T$$A$ is a symmetric matrix which elements are the inner product of each column of $A$. But I didn't understand the intuition of transpose. Suppose $A_{m \times n}$, and A transform a vector from $\Bbb R^n$ to $\Bbb R^m$. But $A^T$ transform a vector from $\Bbb R^m$ to $\Bbb R^n$. What's the relationship between them? Could anyone please explain the relationship between $A^T$,$A$,the inner product and symmetric matrix. I think there would be a intuition explaination.
Comment: See: http://math.stackexchange.com/questions/37398/what-is-the-geometric-interpretation-of-the-transpose
Comment: See also:  http://math.stackexchange.com/questions/484844/intuition-behind-definition-of-transpose-map
Comment: This is critical to understand why the Rank and Determinant of $A^T$ are the same as $A$. Singular Value Decomposition is a good start.
Answer: Well, $A^T$ is the adjoint matrix of $A$ with respect to the ordinary inner products, i.e. $A^T$ is the only linear mapping $B$ such that
$$\langle Av,w\rangle = \langle v,Bw\rangle$$
for all $v\in\Bbb R^n$ and $w\in\Bbb R^m$. You can easily see it if you verify it on the standard bases, noting that $\langle u,e_i\rangle$ gives the $i$th coordinate of $u$.
Comment: thank you. But how to use this conclusion? Is there any geometry explaination?
Comment: Real case: $\langle Av, w \rangle = (Av)^\top w = v^\top A^\top w = \langle v, A^\top w \rangle$ Complex case: $\langle Av, w \rangle = (Av)^\top \overline{w} = v^\top A^\top \overline{w} = v^\top \overline{A^* w} = \langle v, A^* w \rangle$
Answer: One aspect of this to consider is that the transpose lets you do the same thing in different ways. The regular matrix gets multiplied by a column on the right to give your answer as a column. The transpose gets multiplied by a row on the left to give a row as the answer.
Is one better than the other? Not really, they're equivalent.
Answer: Good geometric ilustration of transpose is if we take as linear operator rotation matrix R.
In this case is easy to see that < Rv, w> = < v, $R^T$ w> as we have 2 opportunities to change an angle between vector v and w to the same value.
One opportunity is to rotate v what   Rv operation does, the second one to rotate w in reverse direction what $R^T$  does ( transpose of R is inverse of R).
Comment: That's the geometric intuition of an inverse matrix, not of the transpose ; you took the case where $R^{\top} = R^{-1}$, which is called an orthogonal matrix.
Comment: Yes, but in this case transpose is equal to inverse, I agree .. it is a special case.
Comment: I'm doing the comment because I myself do not have much intuition about the transpose... the answer didn't help me!
Comment: Maybe some additional intuition about transpose can be taken from decomposition of any matrix $A$ into symmetric and skew-symmetric part. In this case transpose gives the same symmetric part and skew-symmetric with minus sign.


What is worst than a random, in ROC then say lower-right space is worst than random

Question: Nothing can be worst than random, if you are predicting wrong all the time then why don't you just invert your results all the time and then you will be in the upper-left area of the ROC curve i.e. above the diagonal line. Can you explain that?
Comment: What is there to explain? it's as you said
Comment: Yeah, I realised it after reading full wiki article.
Comment: Can you explain what? Why the curve gets switched when you invert the predictions? Why you can invert the predictions? Please clarify the question exactly...
Answer:
Yes, a model can actually perform worse than random.
Typical situations where this happens are when the training data differs systematically from the application/production/test data. E.g.

training was done with lab-generated data, whereas application/test data comes from [industrial] production line.
training data was curated: e.g. only cases where reference labels are easily obtained were used, but in reality borderline cases are frequent. Or training dat has been oversampled "to avoid class imbalance" - but the class imbalance was a characteristic of the application.
Leave-one-out cross validation produces a situation where the tested case's class is always underrepresented in the respective surrogate training set. The result is a pessimistic bias.
Sometimes I deliberately test edge-cases because I want to know how far I can push the model until its predictive power breaks down.

Flipping prediction "because test results were worse-than-guessing" is part of model training: with the option to flip if test results are too bad you introduced a hyperparameter that says whether a flipping postprocessing of the predictions should be applied. Thus, your test set is actually a hyperparameter optimization set (often referred to as validation set), and not any more an independent test of generalization error.
So, yes, you can do that, but in order to measure generalization error, you now need another independent test set to see how well your flipped predictions are doing.
Answer: Here is an example of flipping the prediction:
Suppose we want to classify cats and dogs by weights. And the classifier is a simple rule: if an animal's weight is larger than 2kg, then it is a dog.
Suppose we are getting an ROC curve that is worse than random, then we just need to change our rule to be: if an animal's weight is larger than 2kg, then it is a cat.
Answer: If you always guess at random, then sometimes you will perform above chance and sometimes bellow chance, although at average, in the long run, you will converge to random performance. If you always flip the results when it is convenient, then it would look like you learned something about the problem, although you are still just making random predictions. It's head I win, tail you lose situation.
You can flip your predictions, but you would have to do it fairly. So you will have to decide to flip them before you know which way is better for your test set, for example, based on an additional nested CV loop.
It is very easy to get significant bellow chance predictions without learning anything about the data, thus just flipping the projections will lead to spurious results. Imagine you want to predict Y based on X, in your dataset, there is a 0 correlation between Y and X, now you split it randomly to a training set and test set. If in training set the correlation between X and Y is positive, then in the test set it will be negative and vice versa (since int he whole dataset it is exactly 0). Therefore, you will always be predicting the relationship in the opposite direction. However, you didn't learn anything about the data, and the true relationship is 0.


How to avoid dissonance when moving from Maj7 to Dom7 chord if pedal is needed to be played

Question: I am trying to play the part where Emaj7 is followed by E7-9.
The piece is Poverty "Once upon a time in America".

The right hand plays the highest G# and the left hand plays E (bass), then the left hand plays the two chords. As far as I understand I should use the pedal to make E (bass) sound during the whole measure, because neither of the hands can reach the chords without the pedal use. But this causes dissonance due to Maj7 and Min7 sounding simultaneously. If the pedal is released to avoid dissonance the lowest E will not sound as it is written.
Is it possible to play it exactly as written without dissonance?
Comment: ...and this is a score for piano with no pedaling marks?
Comment: this is for piano. No pedaling marks.
Answer: This is an opportunity for the sostenuto pedal to be used. On some pianos, there's a middle pedal that, when pressed, will hold only the notes that are being played at that time. Unlike the damper pedal, which will hold everything played at the time it's pressed.
There are pianos that have three pedals, but the middle one is a practice pedal, which quietens the whole piano down using a felt curtain in between the hammers and the strings. That won't do the same job.
Comment: Thanks. At least I see that with "sostenuto" pedal the written text makes sense. (my acoustic piano has 2 pedals)
Answer: While the sostenuto pedal could be used if present, it's unnecessary here.
The damper pedal is not a binary device: it is not simply either on or off.  Finer gradations are possible.  You can catch a bass note like this and retain it through harmony changes by partial release.  You need to experiment with the instrument to get a feel for this yourself.
Answer: If you don't have the sostenuto pedal forget the prolonged E in the bass. just hold it a dotted   quarter , or play the last 3 8ths in the lower staff with the right hand and let dropping the G#


PHP turning 2 lines into 1 line

Question: My son seems very interested.
 Do you know why my game started?

Above is a string line, I tried to form it as
My son seems very interested. Do you know why my game started?

I tried the following code
$finish_loop = "no";

while($finish_loop=="no")
{

if(!strstr($dialogue,"  "))
{$finish_loop = "yes";}

$dialogue = str_replace("  "," ",$dialogue);
}

But the $dialogue still return the $dialogue as 2 lines, I just want everything in 1 line, how do I achieve this with php
Comment: note: use a bool. using strings (`"yes"` / `"no"`) is bad practice.
Answer: Replace the new lines from your text with space, then trim it to remove the superflus spaces :
$dialogue = str_replace("\n\r"," ",$dialogue);
$dialogue = str_replace("\r"," ",$dialogue);
$dialogue = trim(str_replace("\n"," ",$dialogue));

PHP turning 2 lines into 1 line


Pax Web on Knopflerfish on Android

Question: I'm planning to start OSGi on Android with Web Services, wherein I will be able to run Java Server Pages (jsp) and Java Servlets on Android Browser.
My initial step was installing Knopflerfish on Android. I have followed this tutorial from knopflerfish: http://www.knopflerfish.org/releases/5.0.0/docs/android_dalvik_tutorial.html
Knopflerfish did well, I have seen the localhost:8080 on the Android phone's browser.
My second step was to install Pax Web on Knopflerfish that was also installed on Android. I have also followed their tutorial from here: http://www.knopflerfish.org/kf_pax_web_tutorial.html
The tutorial said that:

Uninstall the KF HTTP bundles
The first step is to stop the HTTP root bundle and uninstall the JSDK and HTTP-server bundles as PaxWeb will take over this function. Normally you will most likely also want to uninstall the HTTP-root-bundle, but for this tutorial we will leave it installed. Make sure to do a refresh packages before going to the next step.

Then:

Install PaxWeb
The second step is install and start the following bundles from PaxWeb.

pax-web-jsp-1.0.2.jar (install)
pax-web-jetty-bundle-1.0.2.jar (install & start)
pax-web-extender-war-0.8.0.jar(install & start)

The Pax Web bundles were successfully installed.

Problem
The Pax Web bundles cannot be started.
It gives me this error messages on the console when I try to start HTTP-root-IMP which is a Knopflerfish bundle.
[stderr] ## DEBUG: errors - FrameworkErrorEvent bundle #20
[stderr] ## DEBUG: errors - FrameworkErrorEvent throwable:
Couldn't start bundle: HTTP-root-IMPL (#20) (due to: org.osgi.framework.BundleException: Bundle#20, unable to resolve: Missing package(s) or can not resolve all of the them:
org.osgi.service.http;version=[1.1.0,2.0.0) -- Could not resolve exporting bundle - org.osgi.service.http;version=1.2.0 Bundle[id=16,gen=0].
javax.servlet.http;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].)
org.osgi.framework.BundleException: Bundle#20, unable to resolve: Missing package(s) or can not resolve all of the them:
org.osgi.service.http;version=[1.1.0,2.0.0) -- Could not resolve exporting bundle - org.osgi.service.http;version=1.2.0 Bundle[id=16,gen=0].
javax.servlet.http;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].
        at org.knopflerfish.framework.BundleImpl.getUpdatedState(BundleImpl.java:1259)
        at org.knopflerfish.framework.BundleImpl.finalizeActivation(BundleImpl.java:312)
        at org.knopflerfish.framework.PermissionOps.callFinalizeActivation(PermissionOps.java:265)
        at org.knopflerfish.framework.BundleImpl.start(BundleImpl.java:295)
        at org.knopflerfish.bundle.frameworkcommands.FrameworkCommandGroup.cmdStart(FrameworkCommandGroup.java:1634)
        at java.lang.reflect.Method.invokeNative(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:511)
        at org.knopflerfish.service.console.CommandGroupAdapter.execute(CommandGroupAdapter.java:256)
        at org.knopflerfish.bundle.console.Command$2.run(Command.java:239)
        at java.security.AccessController.doPrivileged(AccessController.java:45)
        at org.knopflerfish.bundle.console.Command.run(Command.java:226)
        at java.lang.Thread.run(Thread.java:856)

Another is when I will start the bundles from the Pax Web tutorial from Knopflerfish. Here I tried to start pax-web-jetty-bundle-1.0.2.jar (OPS4J Pax Web - Jetty Bundle):
[stderr] ## DEBUG: errors - FrameworkErrorEvent bundle #16
[stderr] ## DEBUG: errors - FrameworkErrorEvent throwable:
Couldn't start bundle: OPS4J Pax Web - Jetty Bundle (#16) (due to: org.osgi.framework.BundleException: Bundle#16, unable to resolve: Missing package(s) or can not resolve all of the them:
javax.security.sasl -- No providers found.
javax.management.remote -- No providers found.
javax.management.modelmbean -- No providers found.
javax.management -- No providers found.)
org.osgi.framework.BundleException: Bundle#16, unable to resolve: Missing package(s) or can not resolve all of the them:
javax.security.sasl -- No providers found.
javax.management.remote -- No providers found.
javax.management.modelmbean -- No providers found.
javax.management -- No providers found.
        at org.knopflerfish.framework.BundleImpl.getUpdatedState(BundleImpl.java:1259)
        at org.knopflerfish.framework.BundleImpl.finalizeActivation(BundleImpl.java:312)
        at org.knopflerfish.framework.PermissionOps.callFinalizeActivation(PermissionOps.java:265)
        at org.knopflerfish.framework.BundleImpl.start(BundleImpl.java:295)
        at org.knopflerfish.bundle.frameworkcommands.FrameworkCommandGroup.cmdStart(FrameworkCommandGroup.java:1634)
        at java.lang.reflect.Method.invokeNative(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:511)
        at org.knopflerfish.service.console.CommandGroupAdapter.execute(CommandGroupAdapter.java:256)
        at org.knopflerfish.bundle.console.Command$2.run(Command.java:239)
        at java.security.AccessController.doPrivileged(AccessController.java:45)
        at org.knopflerfish.bundle.console.Command.run(Command.java:226)
        at java.lang.Thread.run(Thread.java:856)

Lastly, when I try to run pax-web-extender-war-0.8.0.jar (OPS4J Pax Web - Extender - War) this error messages will also appear:
[stderr] ## DEBUG: errors - FrameworkErrorEvent bundle #17
[stderr] ## DEBUG: errors - FrameworkErrorEvent throwable:
Couldn't start bundle: OPS4J Pax Web - Extender - WAR (#17) (due to: org.osgi.framework.BundleException: Bundle#17, unable to resolve: Missing package(s) or can not resolve all of the them:
org.osgi.service.http;version=[1.0.0,2.0.0) -- Could not resolve exporting bundle - org.osgi.service.http;version=1.2.0 Bundle[id=16,gen=0].
javax.servlet.http;version=2.1.0 -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=2.1.0 -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].)
org.osgi.framework.BundleException: Bundle#17, unable to resolve: Missing package(s) or can not resolve all of the them:
org.osgi.service.http;version=[1.0.0,2.0.0) -- Could not resolve exporting bundle - org.osgi.service.http;version=1.2.0 Bundle[id=16,gen=0].
javax.servlet.http;version=2.1.0 -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=2.1.0 -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].
        at org.knopflerfish.framework.BundleImpl.getUpdatedState(BundleImpl.java:1259)
        at org.knopflerfish.framework.BundleImpl.finalizeActivation(BundleImpl.java:312)
        at org.knopflerfish.framework.PermissionOps.callFinalizeActivation(PermissionOps.java:265)
        at org.knopflerfish.framework.BundleImpl.start(BundleImpl.java:295)
        at org.knopflerfish.bundle.frameworkcommands.FrameworkCommandGroup.cmdStart(FrameworkCommandGroup.java:1634)
        at java.lang.reflect.Method.invokeNative(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:511)
        at org.knopflerfish.service.console.CommandGroupAdapter.execute(CommandGroupAdapter.java:256)
        at org.knopflerfish.bundle.console.Command$2.run(Command.java:239)
        at java.security.AccessController.doPrivileged(AccessController.java:45)
        at org.knopflerfish.bundle.console.Command.run(Command.java:226)
        at java.lang.Thread.run(Thread.java:856)

I also tried to start pax-web-jsp-1.0.2.jar (OPS4J Pax Web - Jsp Support) this error message was displayed:
[stderr] ## DEBUG: errors - FrameworkErrorEvent bundle #15
[stderr] ## DEBUG: errors - FrameworkErrorEvent throwable:
Couldn't start bundle: OPS4J Pax Web - Jsp Support (#15) (due to: org.osgi.framework.BundleException: Bundle#15, unable to resolve: Missing package(s) or can not resolve all of the them:
javax.tools -- No providers found.
javax.servlet.resources;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.resources;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet.http;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].
javax.lang.model.util -- No providers found.
javax.lang.model.type -- No providers found.
javax.lang.model.element -- No providers found.
javax.lang.model -- No providers found.
javax.annotation.processing -- No providers found.)
org.osgi.framework.BundleException: Bundle#15, unable to resolve: Missing package(s) or can not resolve all of the them:
javax.tools -- No providers found.
javax.servlet.resources;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.resources;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet.http;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=[2.3.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].
javax.lang.model.util -- No providers found.
javax.lang.model.type -- No providers found.
javax.lang.model.element -- No providers found.
javax.lang.model -- No providers found.
javax.annotation.processing -- No providers found.
        at org.knopflerfish.framework.BundleImpl.getUpdatedState(BundleImpl.java:1259)
        at org.knopflerfish.framework.BundleImpl.finalizeActivation(BundleImpl.java:312)
        at org.knopflerfish.framework.PermissionOps.callFinalizeActivation(PermissionOps.java:265)
        at org.knopflerfish.framework.BundleImpl.start(BundleImpl.java:295)
        at org.knopflerfish.bundle.frameworkcommands.FrameworkCommandGroup.cmdStart(FrameworkCommandGroup.java:1634)
        at java.lang.reflect.Method.invokeNative(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:511)
        at org.knopflerfish.service.console.CommandGroupAdapter.execute(CommandGroupAdapter.java:256)
        at org.knopflerfish.bundle.console.Command$2.run(Command.java:239)
        at java.security.AccessController.doPrivileged(AccessController.java:45)
        at org.knopflerfish.bundle.console.Command.run(Command.java:226)
        at java.lang.Thread.run(Thread.java:856)

From what I have understood from the error messages, the bundles were looking for these packages. Is there any way I can resolve this and be able to successfully run a Java Web Application on Android with OSGi framework?
org.osgi.service.http;version=[1.1.0,2.0.0) -- Could not resolve exporting bundle - org.osgi.service.http;version=1.2.0 Bundle[id=16,gen=0].
javax.servlet.http;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet.http;version=2.5.0 Bundle[id=16,gen=0].
javax.servlet;version=[2.5.0,3.0.0) -- Could not resolve exporting bundle - javax.servlet;version=2.5.0 Bundle[id=16,gen=0].

javax.security.sasl -- No providers found.
javax.management.remote -- No providers found.
javax.management.modelmbean -- No providers found.
javax.management -- No providers found.

Answer: First of all I strongly suggest to use the same version for all bundles of Pax-Web, second you're missing a servlet API bundle. Also it looks like you're also missing a couple of other Pax-Web bundles. You'll most likely need the following bundles:

pax-web-api
pax-web-jetty-bundle
pax-web-jsp
pax-web-runtime
pax-web-spi

and if you want to use wars you'll need the

pax-web-extender-war
pax-url-war
and a couple more ...

Latest version of Pax-Web is 3.0.2 so you should choose with care depending on what you're trying to do. You might find it helpful to check the ITests of Pax-Web (depending on the version you're going to use) for inspiration on which bundles are needed.
I'd suggest to start with a minimum set. This way you start with the http-service first and enhance it to the needs you have.
Take a look at the Pax-Web project at Github and especially at the samples and itests modules for more details.
Comment: I have successfully downloaded pax-web jar files here: http://repo1.maven.org/maven2/org/ops4j/pax/web/. But one of the steps to add these jar filed on the current project I am doing is to `dexify` it, but I cannot create a dex file of `pax-web-jetty-bundle-3.0.2.jar`. I need a dex file of the jar files before I include them to the current project.
Comment: what could you recommend for servlet API bundle? @Achim
Comment: You'll find a karaf features file (the list of needed bundles) at https://github.com/ops4j/org.ops4j.pax.web/blob/master/pax-web-features/src/main/resources/features.xml#L4, this will get you a bit further I hope.

Pax Web on Knopflerfish on Android


Prompt for Certificate Password When Accessing EFS Shares

Question: My boss wants to set up encrypted file shares using EFS on Windows Server 2012 R2 Boxes accessed by Windows 8.1 clients. I've been able to set the default DRA, issue certificates in test cases and so forth. So, EFS works. However, he also wants the user to be prompted for their certificate password (the password generated when you first create the certificate) when:

The user first logs in/accesses an encrypted file
After a periodic time out when they go to access the file.
If the user locks their screen.

I've configured a group policy to clear the cache every 240 minutes and when they lock their screen. I've also set the local machine registry and domain GP to require strong protection of private keys. However, it doesn't really seem to do what he's looking for. Is what he wants possible without using a second form of authentication (smart card, etc). If so, where am I messing up?
Answer: It is not possible via built-in mechanisms.
A bit of theory: private key strong protection prompts for input only when private key material is accessed.
A bit of practice: when you encrypt/decrypt files on remote share, remote certificate is used to perform these operations. That is, EFS loads user profile on a file share hosting server, loads the certificate and performs actions (encrypt and decrypt). And dialog prompt will be exposed on a remote server and will never be exposed to you and no one will be able to type password there.
Comment: I had read that as well, but wasn't sure Thanks for clarifying and confirming. His fall back plan is to have two accounts - one let's call account.enc and one called account.clear. Account.clear is the user's day to day account. Account.enc is the encryption account.  He wants to encrypt files using account.enc on the shared drive, then use the separate account.clear to mount the drive. He believes that this will provide "two layers of protection" in case account.clear gets compromised. I'm not sure that my understanding of EFS would allow this. Do you have any thoughts?
Comment: What is your boss' the final goal?
Comment: To "prevent sensitive information being accessed if someone's account is compromised" He basically wants a secondary layer of authentication.
Comment: What you would do if "secure" account is compromised? Account separation is not the answer for you. Instead, you should consider to secure authentication mechanisms combining with 3rd party encryption. I would go with 2-factor authentication (smart cards) and 3rd party encryption tool (OpenPGP, just as an example) and storing encryption keys on smart card.
Comment: I don't disagree. But the boss  says there's no funding for smart cards. Anyways, thanks for the help. Marking as answered.


Is there any free software for Oden's Ipop?

Question: I'm trying to use software that has an Oden's Ipop Statistics function for testing spatial autocorrelation adjusted for population. I prefer knowing if something is available for FREE.
I have searched through the internet, but unfortunately I did not find nothing. I have one option for this, which is for me to use R and make a function of Oden's Ipop Statistics using the formula for it. But just in case someone knows of any free software for the Oden's Ipop, that would be a helpful to me with my research study.
Comment: I had a friend ask me the same question previously as well and I was never able to find one. The only software I know of is Biomedware's ClusterSeer program (not free).
Answer: The GeoDa is free and you can do the Oden's Ipop


Ocaml - differences between compiler and manual

Question: So I was reading http://www.podval.org/~sds/ocaml-sucks.html, which includes this quote (in the context of problems with ocaml):

there are actually three mildly different syntaxes:

the official one is described in the manual
a superset thereof is accepted by the compiler ocamlc
something similar (but ever so    slightly different) is accepted by    the preprocessor Camlp4 (e.g., it    accepts List.map [1;2;3]
  ~f:fun    x -> x, which is also accepted by the    top-level, but not
  the    compiler)

What are examples of ocaml code fragments that are accepted by the ocamlc but do not match the manual?
PS - I'm mainly interested in OCaml 4.00.1, but previous versions are also interesting...
Answer: One funny example, in the interest of curiosity only:
<code>                  OCaml version 4.00.0

# type 'a weird_list =
    | ()
    | :: of 'a * 'a weird_list;;
type 'a weird_list = () | :: of 'a * 'a weird_list
# 1::2::3::();;
- : int weird_list = :: (1, :: (2, :: (3, ())))
</code>
The differences between Camlp4 and the OCaml compiler are sometimes bugs, but most often explained by the difference in parser technology: if you write an idiomatic LR parser and a LL parser for the same language with mildly annoying syntax subtleties, you should not expect both to match perfectly. I don't think this is a problem in practice (but in practice most people don't use Camlp4).
Answer: This page is quite old, not sure, it is still true for all points.
The third item is false considering ocaml 4.00 toplevel (considering too, that the order of parameters of List.map is wrong...):
<code> $ List.map  ~f:fun x -> x [1;2;3];;
 Error: Syntax error
 $ List.map ~f:(fun x -> x) [1;2;3];;
 File "", line 1, characters  12-24:
 Error: The function applied to this argument has type 'a list -> 'b list
 This argument cannot be applied with label ~f
</code>
AFAIK, the toplevel use the same code than ocaml to parse the code...


how to convert a datatable to List>> in C#

Question: I have a datatable with three columns says, ID, Name and Value.
I want to convert this datatable into List<long,List<keyvaluepair<string,string>>>.
The only trick is ID is foreign ket from another table. so it could have repeatative values.
I want to do it in C#.
Thanks in advance.

no, not dictionaly.....actualy I have two classes says, definded as....
public class RunParameters
    {
        public long RunId { get; set; }
        public List<WorkflowParameter<string>> WorkflowParameters { get; set; }

        public RunParameters()
        {
            WorkflowParameters = new List<WorkflowParameter<string>>();
        }
    }

 public struct WorkflowParameter<T>
    {
        public string ParameterName
        { get; set; }
        public T ParameterValue
        { get; set; }
    }

I want to convert the datable into List.
I cannot use dictionary since I want to serialize the this List to send it across the network....
Comment: Do you mean `Dictionary>>`?
Comment: Kindly see the updates....
Comment: A list can be of one type only - so, if you want a list it would have to be `List>>>` which is ugly even only when defining it. Is that really what you want?
Comment: By *datatable* you mean `DataTable` class? And how your classes related to initial question?
Comment: Hello Joanna, What exectly I want is...I have datatable with three columns says, ID, Name and Value. I want to convert that datatable into List. THis RunParameters contain two variable Id and workflowParameter. THis workflowParameter again contain two fields called name and value. So ultimate output I want is List>>.

for class mapping, you can refer my question.
Answer: Group items in your query by ID and project each item from group to KeyValuePair:
(from r in table.AsEnumerable()
 group r by r.Field<long>("ID") into g
 select g.Select(i => new KeyValuePair<string, string>(
                      i.Field<string>("Name"), i.Field<string>("Value"))
         .ToList()).ToList()

Or completely with method syntax:
 table.AsEnumerable()
      .GroupBy(r => r.Field<long>("ID"))
      .Select(g => g.Select(r => new KeyValuePair<string, string>(
                          r.Field<string>("Name"), r.Field<string>("Value"))
                    .ToList())
      .ToList()


Beaglebone Black emmc gpio

Question: I have the BEAGLEBONE black, A5C.
I am having problems changing the mode pin.
I'm trying to change the way different pins but the pins P8_6 P8_03 to this mode 1 and I do not get to change mode 7
anyone can help me?
Comment: Can you post the device tree overlay file you are using to change the mode?
Comment: No deja por este metodo, ya los esta usando la beaglebone y no me deja usarlos como gpio
Comment: several gpio will not let me configure
Answer: this link is you looking for. this sit can generate source dtc file in witch you can select mode of the pin.
also try this if you are working on beaglebone, by far this best for beaglebone.

Beaglebone Black emmc gpio


Prove (if true): $P(A|C) = \sum_{i} P(A|B_i)P(B_i|C)$

Question: I think the statement $P(A|C) = \sum_{i} P(A|B_i)P(B_i|C)$ is true (where the events $B_i$ form a partition of the whole space).  Is it, and why?
Thanks!
Comment: Have you tried writing $P(A \, | \, C) = P(A \cap C) / P(C)$ and so on for the other terms? Maybe something reaaally obvious will come out of it.
Comment: Try to prove this instead: $P(A|C) = \sum_{i} P(A|B_i,C)P(B_i|C)$
Comment: Yeah, but nothing good happened :/
Comment: @p.s. That works, thanks!  Maybe I can reduce my problem to that problem instead.
Comment: @AndréNicolas: Okay, I tried that and I got $P(A|C) = \sum P(A \cap B_i | C)$ = $\sum P(A \cap B_i \cap C)/P(C)$ = $\sum P(A \cap C | B_i \cap C) P(B_i \cap C)/P(C)$ = $\sum P(A \cap C | B_i \cap C) P(B_i | C)$, which is kind of close but I'm beginning to suspect the statement is false.
Comment: @okay-at-math: I didn't push the calculation through.
Answer: It's false as stated. Suppose that the space is the sample space for two tosses of a fair coin,

$B_1$ is the event that the first toss is heads,
$B_2$ is the event that the first toss is tails,
$A$ is the event that both tosses are heads, and
$C$ is the event that at least one toss is tails (i.e, the complementary event to $A$).

Clearly $\mathbb{P}(A|C)=0$, but $$\mathbb{P}(A|B_1)\mathbb{P}(B_1|C)+\mathbb{P}(A|B_2)\mathbb{P}(B_2|C)=\frac12\cdot\frac13+0\cdot\frac23=\frac16.$$
Added: It never hurts to check some simple cases of a conjecture, if there are any. Those that are somehow atypical (like making $A$ and $C$ complementary events) are often especially useful.
Comment: Thank you!  I guess I will have to solve my problem another way then.
Comment: We would be happy to take a look at your actual question if you are still bothered about it.


Metropolis-Hastings using log of the density

Question: Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from?
That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the Metropolis-Hastings work with $\log(\pi(x))$ and $\log(q(x|x^{old}))$ as well?
When constructing a Gibbs sampler, we often encounter full conditional distributions that are non-conjugate. Techniques to sample from them include ARS,1,2 ARMS,3 and Slice sampling.4 These techniques have the convenient feature that they can take the log of a density to be sampled from. They are technically Metropolis-Hastings samplers, so  there are cases where my question will be answered in the affirmative. But is this a general feature of the Metropolis-Hastings algorithm?
The reason for my question is that you often store the log of the density when designing a Gibbs sampler, especially when using an object-oriented language and one of the three samplers mentioned above. If you have $\log( \pi(x) )$ stored, you could take $e^{ \log(\pi(x)) } = \pi(x)$ and use Metropolis-Hastings, but that can create issues with numerical overflow and underflow.
I have been unable to find a reference that explains or provides a proof to this question.
EDIT:
I didn't mean to ask whether I can sample from $\log(\pi(x) )$, since that makes no sense. My question was whether the Metropolis-Hastings can work if I pass it $\log(\pi(x) ) $. That is, is there a way to construct the algorithm that only uses the "better behaved" $\log(\pi(x) ) )$, rather than $\pi(x)$? The latter is usually a product of multiple other densities, and it can get really big really quick. Sums of log-densities are easier to use in algorithms than products of densities.

References:
1Gilks, W., Wild, P.: Adaptive Rejection Sampling for Gibbs sampling, Applied Statistics 41(2), 337–348, 1992
2Gilks, W.: Derivative-free Adaptive rejection sampling for Gibbs sampling, Baysian Statistics 4, Oxford University Press, Oxford, 641–649, Eds: Bernardo, J., Berger, j., Dawid, A., Smith, A., 1992
3Gilks, W., Best, N., Tan, K.: Adaptive Rejection Metropolis Sampling within Gibbs sampling, Applied Statistics 44(4), 455–472, 1995
4Neal, Radford M: Slice sampling, Annals of Statistics 31(3), 705–741, 2003
Comment: This makes no sense, given that those logs often are *negative*: what would it mean to "sample" from a negative quantity? There aren't any real issues with under- or over-flow with well written software: you just screen the logarithms before computing the exponentials to make sure they are not too extreme.
Comment: If you look through the code you can see that the only way the log density is used is to compare it to other logarithms. Comparing logarithms is equivalent to comparing their exponentials--but it is not limited to the range of IEEE doubles and doesn't risk over or under flowing.  As far as I can see in the code, that is all that's going on.
Comment: Acceptance of MH is $ \min\left(1 ,
  \frac{\pi(X)  q(X^{(t-1)}|X) }{\pi(X^{(t-1)})  q(X|X^{(t-1)}) }
  \right)$. Is this the same as $ \min\left(1 ,
  \frac{\log(\pi(X))  \log(q(X^{(t-1)}|X)) }{\log(\pi(X^{(t-1)}))  \log(q(X|X^{(t-1)})) }
  \right)$? Or at least, will it also converge to $\pi(x)$ as stationary distribution? To me it is not obvious. I can clarify that in the question, if you think it necessary?
Comment: That pretty much answers my question then. I was wondering whether the fact that you could pass $log(x)$ to the Slice and ARMS samplers, since they never use exponentials in there, and I was also wondering how they "sample" from a negative quantity. I thought there was something obvious I was overlooking.
Comment: Here is the Slice sampler's code. Everything happens in log terms: http://www.cs.toronto.edu/~radford/ftp/slice-R-prog
The same is true of the ARS. I was wondering why it worked in log terms, and whether it is a general feature of the MH. But as you said, it makes no sense.
Comment: That makes sense, thank you. I posted a new question, since I have really been wondering about this for a long time. Thanks for taking the time to look at it. I think a similar thing is happening in the ARS then, I will look through the code.
Answer: As hinted at by @Tim, the solution was quite simple. A function implementing the Metropolis-Hastings can take $\log(\pi(x) )$ and $\log(q(x|x^{old} ) )$, but then everything will have to happen on a log scale. Let $\alpha$ be the acceptance probability of the Metropolis-Hastings update and $x'$ be the current value of the sampler. Then we propose a new $x$ with acceptance probability of:
$$
  \alpha(x | x') = \min\left(1 ,
  \frac{\pi(x)  q(x'|x) }{\pi(x')  q(x|x') }
  \right)
$$
or in log terms
$$
 \log\big( \alpha(x | x') \big) =
\min\Big( 0 ,
\log(\pi(x)) +
\log(q(x'|x)) -
\log(\pi(x')) -
\log( q(x|x') )
  \Big) \text{.}
$$
Then $\log(\alpha)$ can then be used to accept/reject the proposed $x$.
This is a useful interpretation of the Metropolis-hastings that has practical benefits.
It is used in this textbook:
http://mcmcinirt.stat.cmu.edu/archives/320
That is, an implementation of the Metropolis-Hastings that takes the log of the density to be sampled from, and the log of the proposal. Logs are convenient to use since they are not limited to the range of IEEE doubles, and they don't suffer from numerical over and underflow, as pointed out by @whuber.


Como editar códigos gigantescos?

Question: É uma dúvida que tenho desde que comecei a ler códigos. Vejo algumas classes com mil, dois mil, milhares de linhas.
Como um programador pode editar tanta coisa sem "se perder" no meio de tanta coisa?
Mesmo comentando o código, usando nomes lógicos para as variáveis, procedimentos/funções etc, ainda assim fica muito difícil mudar até mesmo o nome de uma variável no escopo, tendo assim que mudar todas as entradas e saídas no código. No momento estou desenvolvendo um código para reserva de hotéis (um trabalho da faculdade, em grupo). O código está com aproximadamente mil linhas e ninguém consegue mais administrar tanta coisa, mesmo com comentários, switchs, if aninhado etc.
No geral, quero saber se existe alguma regra ou algum macete usado por programadores profissionais para conseguirem editar códigos enormes sem se perderem no meio dos mesmos. Espero não ter sido muito vago. Procurei por respostas em fóruns mas sem sucesso.
Comment: Se o código for bem escrito, não vai sofrer _tanto_. Caso contrário, boa sorte.
Comment: Vc só precisa editar o que vc quer mudar. O que está funcionando, à princípio, vc não deve mexer. É bom apenas dar uma revisada pra ver se algo pode ser melhorado. É trabalho de "formiguinha" mesmo. Vai vendo linha por linha se existem coisas que podem ser otimizadas para deixar o código mais enxuto, e por aí vai.
Comment: Digamos que nesse trabalho em grupo uma pessoa sozinha escreveu 1000 linhas de código. Eu falei que era desnecessário e que ninguém saberia lidar com tanta informação, o mesmo não aceitou a opinião e disse que vai ser do jeito dele. Acho que pelo jeito vou ter que fazer papel de formiguinha e sair catando mesmo. Infelizmente terei que lidar novamente com pessoas inflexíveis.
Comment: Padrões de projetos são para isso: soluções comuns para problemas comuns. Ao utilizá-los você não precisa analisar linha a linha do código.
Comment: Acredite se quiser => [o Método possui 34 mil linhas](https://pt.stackoverflow.com/q/197317/91)
Answer:
Vejo algumas classes com mil, dois mil, milhares de linhas

Mil linhas é muito? "Acho" que não.

Como um programador pode editar tanta coisa sem "se perder" no meio de tanta coisa?

Fazendo o que os computadores fazem muito bem e que os humanos deveriam saber fazer o tempo todo: dividindo para conquistar.
Lide com uma parte de cada vez. Construa cada parte para que ela dependa minimamente de outras partes. Planeje antes. Crie um resumo daquilo tudo. Entenda o que está fazendo. Não coloque nada que não precise ali.
Fazer grande, bagunçado, ineficiente é fácil, fazer organizado, pequeno e eficiente é difícil, mas isso se paga sozinho. É assim que as pessoas administram grandes bases de código, fazendo certo e decupado de acordo com a responsabilidade de cada coisa, colocando na função e na estrutura de dados só o que ela faz, sem depender de nada externo sem necessidade e dar bons nomes fazem tudo ficar simples.
Quando separa e lida com uma parte de cada vez não tem nada de gigantesco.
Um bom IDE pode ajudar. Ele é só uma ferramenta de ajuda, tem que saber usá-lo corretamente.

Mesmo comentando o código, usando nomes lógicos para as variáveis, procedimentos/funções, etc., ainda assim fica muito difícil mudar até mesmo o nome de uma variável no escopo, tendo assim que mudar todas as entradas e saídas no código.

Me parece ser coisas incongruentes. Se está dando nomes bons por que precisa comentar o código? Eu acho que sei a resposta, não está dando nomes bons.
Na verdade comentário costuma fazer mais mal do que bem.
Se mudar o nome de uma variável no escopo está difícil este escopo deve estar muito errado.
Acho que as pessoas fazem funções muito longas, obviamente com muitas variáveis, e colocam a declaração da variável longe do seu uso e de forma pouco lógica. O erro começa aí.
A descrição mostra sintomas de outros problemas.

No momento estou desenvolvendo um código para reserva de hotéis (um trabalho da faculdade, em grupo). O código está com aproximadamente mil linhas e ninguém consegue mais administrar tanta coisa, mesmo com comentários, switchs, if aninhado etc.

Além de ser muito complicado pode ser que na verdade ainda não entendem o que estão fazendo. É óbvio que se aprende coisas com um monte de lacuna não tem como usar o que aprendeu de forma adequada. E a maioria das pessoas aprendem desta forma. Elas fazem algo, mas não entendem o que estão fazendo, aí quando sai da complexidade mínima de um exercício isolado e vai para a complexidade real de um problema de verdade ela não consegue mais administrar aquilo.
E tem muito caso que não é saber programação, é não saber matemática (não falo de decorar fórmulas), comunicação e expressão, método científico, filosofia (sim, ela é importante para a lógica correta), e outras coisas básicas que constroem a capacidade da pessoa lidar com problemas.
Vou descartar a hipótese de problemas cognitivos da pessoa, mas tem muito caso assim.

No geral, quero saber se existe alguma regra ou algum macete usado por programadores profissionais para conseguirem editar códigos enormes sem se perderem no meio dos mesmos.

Regra mágica não tem não. É evoluir aos poucos, não querer queimar etapas, se dedicar, procurar melhor cada aspecto individual que está tendo dificuldade, pedir ajuda com coisas específicas, treinar, pedir feedback de pessoas experientes, reconhecidas (de verdade) e confrontar com o feddback de outras pessoas que pensam diferente. E claro, organizar e simplificar (mas antes tem que aprender fazer isto).

Digamos que nesse trabalho em grupo uma pessoa sozinha escreveu 1000 linhas de código. Eu falei que era desnecessário

Se é desnecessário faça menor. Se a equipe não aceita faça o seu e apresente, afinal algumas poucas centenas de linhas dá para fazer em algumas horas, se já está definido e só precisa codificar talvez leve minutos.
Eu poderia até dizer outras coisas sobre isto, mas ficaria muito fora de escopo.
Answer: Se você tiver um sistema bem documentado, não importa o tamanho do código é fácil identificar o que cada método faz dentro de uma classe, outra coisa, uma documentação com requisitos bem definidos não há necessidade de alterar nome de variável, no máximo acrescentar ou diminuir.

Como editar códigos gigantescos?


Best practice for Handling NHibernate parent-child collections

Question: So in a typical model where you have a parent that can have many children and a child that can have only one parent, how do you manage the adding of children.  I have been using this approach;
public class Parent
{
    public Parent()
    {
        Children = new List<Child>();
    }

    public IList<Child> Children
    {
        get;
        private set;
    }
}

public class Child
{
    public Parent Parent
    {
        get;
        set;
    }
}

var child = new Child();
var parent = new Parent();
parent.Children.Add(child);
child.Parent = parent;

Problem is that everywhere i want to add a new child i've got to remember to add a reference to both the child and parent and its a bit of a pain.  I could just add an AddChild method to the parent class and make that responsible for adding children - the problem now is that there is 2 ways to add a child, through the Children property and the method. So is this a better solution?
public class Parent
{
    public Parent()
    {
        children = new List<Child>();
    }

    private IList<Child> children
    {
        get;
        private set;
    }

    public IEnumerable<Child> Children
    {
        get
        {
            return children;
        }
    }

    public void AddChild(Child child)
    {
        children.Add(child);
        child.Parent = this;
    }
}

Are there any guidences for best practices on this, and what do you do?
Answer: This is not an NHibernate problem at all.
You should implement the AddChild method. The classes are responsible for their consistency, so they shouldn't expose anything that should not be available. For instance, the (mutable) Children list should be hidden. Exposing an IEnumerable is a good idea.
Your second code is a good starting point. You probably need some more methods, like RemoveChild or CoundChildren.
Comment: Talking the second example my only concern is how this may work with Linq to NHibernate.  As the actual mapped property is now private i would assume that it would not be visable.  Any thoughts?
Comment: You can specify in your mapping that NHibernte should use the field instead of the property when setting a value.  (Take a look at the access attribute for instance).
Comment: Yes, what I was getting at was that when Linq to NHibernate is finished how could you use it if the field is private and the public access only returns a reference to the original collection. Eg.
session.Linq().Where(x => x.Children.Name.Equals("Jonnie"))
Comment: I found the answer to this in another thread (http://stackoverflow.com/questions/195528/bidi-associations-and-nhibernate-mapping) one day by accident and thought i'd update here as well.  Basically to get around this use a naming strategy - in this case the mapping would be

field.camelcase
Answer: I do it like this:
public class Parent
{
    private ISet<Child> _children = new HashedSet<Child>();

    public ReadOnlyCollection<Child> Children
    {
        get{ return new List(_children).AsReadOnly(); }
    }

    public void AddChild( Child c )
    {
       if( c != null && !_children.Contains (d) )
       {
          c.Parent = this;
          _children.Add (c);
       }
    }
}

So, in fact, that's a bit what Stefan says as well.  I just expose a readonly-copy of the Children list, so that you can easily iterate over the children of a parent, and get the number of children that the parent has.
Adding and removing children to the parent, has to be done using the AddChild & RemoveChild member methods.
Comment: A set will not add duplicates so there is no need to check if the child already exists.
Answer: I do it like that except that I don't use property for private list
private IList<Child> _children
public IEnumerable<Child> Children
{
    get
    {
        return children;
    }
}

Comment: this is a good solution but with a drawback that if the consumer cast the Children to IList he would access the list and can modify it .. other way could be using ReadOnlyCollection<>
Answer: I'm using public IEnumerable with Add|Remove methods approach.
However I don't like it much, because it's not intuitive, and polutes class definition.
I'm wondering why people don't use CustomCollection where they override Add, Remove, Replace functions?? (like it's done everywhere in MS code) ????
Answer: I support the accepted solution to this question, however the solution presented is not complete, as using private fields requires some extra configuration in the mappers.  For the benefit of others, the following is the complete solution:
public partial class Test
{
    private readonly IList<Child> children = new List<Child>();
    public virtual IEnumerable<Child> Children
    {
        get
        {
            return children;
        }
    }
}

Note that the publicly exposed collection must be virtual for NHibernate to use it.  I also like to make it a readonly field that is initialised when the class is created to ensure it exists in all scenarios.  Here is the associated mapper:
public class TestMap : ClassMap<Test>
{
    ...
    HasMany(s => s.Children).Access.CamelCaseField();
}

The Access property tells NHibernate to use the private field when mapping values into the model.  There are also other options on the Access property allowing various naming configurations to be used.
Answer: If you have a foreign key in your DB, and you are using Identity(SQL Server) to generate your primary keys, you are going to NEED the backlink from the child to the parent. Else, the insert will complain on child because nhibernate needs to do some back and forth for the parent ID, but it hasn't done it yet.
What we ended up doing to get rid of backlinks : Use the NHibernate HiLo generator.
This way, NHibernate always has the Ids it needs to insert your parent/child relations.
<3 !
Answer: I don't like all the extra AddXXX() and RemoveXXX() methods cluttering my entity interfaces.  Instead, I have a custom list that raises events when the Add() and Remove() methods are called.
The linking then happens in the event handlers:
public class Course()
{
   public Course()
   {
     this.Topics = new EntityList<Topic>();
     this.Topics.AddItem += new AddItemEventHandler<Topic>(Topic_AddItem);
     this.Topics.RemoveItem += new RemoveItemEventHandler<Topic>(Topic_RemoveItem);
   }

   public EntityList<Topic> Topics { get; private set; }

   private void Topic_AddItem(Topic item, object args)
   {
     // Replace with your linking code:
     EntityLinker.Link(this).With(item, args);
   }

   private void Topic_RemoveItem(Topic item, object args)
   {
     // Replace with your unlinking code:
     EntityLinker.Unlink(this).From(item, args);
   }
}

Comment: I don't like this approach. :)  There is to much 'magic' going on behind the scenes.
I remember that Ayende once used this approach as well (in NHibernate generics if i'm not mistaken), but now I do not see him using this approach any more.
I just find that the 'link' logic is to much hidden.
Next to that, I don't see adding Addxxxx methods to my entities as cluttering of the interface.  Moreover, I find it better since it is more explicit.


SQL - Summing number of columns which have a true boolean value for a given ID

Question: Suppose I have a table of unique transactions with transaction ID, customer ID, and a number of different boolean columns for various features of the transaction. Customers may be associated to more than one transaction.
For each distinct customer, I would like to count how many of the boolean columns have a true value.
Columns in table: transact_id, customer_id, bool_one, bool_two, bool_three...

transact_id
customer_id
bool_one
bool_two
bool_three

10001
x
True
False
False

10002
x
True
True
False

10003
y
False
False
True

Output

Customer
Number of columns

x
2

y
1
Comment: A [mcve] is a great start when asking SQL questions.
Comment: Which DBMS product are you using?
Comment: `case when boolcol is true then 1 else 0 end + case when ...`
Comment: Thank you, I've edited my question to provide more information.
Comment: Google Big Query
Answer: Consider below approach
<code>select customer_id,
  cast(logical_or(bool_one) as int64)
  + cast(logical_or(bool_two) as int64)
  + cast(logical_or(bool_three) as int64) as col_num
from your_table
group by customer_id
</code>
if applied to sample data in your question - output is