Do different units of explanatory variables affect p-values?

Question: In multiple linear regression, does including variables with different metering units and thus values very different in dimension/size affect statistical significance/ p-values?
In my model, a variable with large values (i.e. relatively large compared to the other included variables) exhibits a very small parameter estimate. Its p-value is very large, though sum of squares is large.
Now I am looking for an explanation and I guess it could be due to the different units of explanatory variables.
Comment: @Aksakal That is true.  But people have actually seriously proposed statistical tests whose results change when values are rescaled.  It could make sense in cases where there is an intrinsic unit of measurement (counts are the clearest example of that).
Comment: @Aksakal That's really a different issue: we should expect models of linear relationships among variables to change when the values are re-expressed in *nonlinear* ways. What I was referring to is that there are some procedures whose results will change even when the units of measurement of the variables are changed (linearly). Depending on [terminology](http://stats.stackexchange.com/questions/132829), these are "non-invariant" or "non-equivariant" procedures.
Comment: @Aksakal I am neither advocating nor disparaging this approach, only reporting that it exists. The application I saw concerns automated counts of very high values, such as ion counts in a mass spectrogram or counts of a Geiger counter. See http://info.ngwa.org/gwol/pdf/990564800.PDF for an account of that application and a rebuttal.  (I have publicly agreed with that rebuttal, BTW.)
Comment: It would be strange if they did, don't you think?
Comment: @whuber, good point. You could change the units of measure from Pascal to decibel, and it's like log-transform, and this will change everything.
Comment: @whuber, I don't think the count is a good example. In this case it's more like a sample size change. You don't observe the rate, you observe the counts. So changing the units of measure of count is really increase the sample size expressed in units of time. You go from per second to per minute, this really means your sample size increased 60 folds. This is not the trivial change from centimeter to meter when measuring length directly.
Answer: The units of variables affect the parameter estimates, but not the p-values.
Think of it this way: if you multiplied an entire variable in your dataset by 10, you could just divide the coefficient of that variable by 10, and get the exact same prediction for each data point. So the model will explain just as much of the variance as before.
However, it can be easier to interpret different aspects of your model if you "standardize" the units of each variable, for instance by dividing by the standard deviation of that variable. If you do this, then all of your variables will be unitless (and all of your coefficients will have the same units as your response variable), which means they can be compared on a single value.
The upshot is that a larger coefficient doesn't necessarily mean a stronger p-value. The p-value just indicates how likely you would be to get a coefficient that large by chance. And depending on the distribution of your data/response, you can actually be fairly likely to get large values by chance, so you can't in general get a relationship between coefficient size and p-values.
Now for two caveats to this:

If your variables have very different units--like by a factor of $10^{10}$ or something--it can make it look numerically like your dataset exhibits collinearity, which means you may encounter numerical problems when you fit your model. Rescaling the predictors can help with this.
If you're using some types of regularized regression (e.g. ridge regression or LASSO), then smaller coefficients will be penalized less, so multiplying a variable by 10 and dividing its coefficient by 10 will make that coefficient more likely to be selected/have a higher value. This is why it's often advisable to center and scale your data when using these sorts of techniques.
Comment: A third caveat: If you transform the data say to "logarithmic units" then the P values may be different. Of course, that is more than a unit change, but some people get linear unit transforms mixed up with nonlinear transforms so I figured this note may help someone not make that mistake.
Comment: Good catch--thank you! I'm not used to thinking of a log transform as "scaling" or "changing units," but the OP could very well have meant that.
Comment: +1--but you seem to have lost the end of the text in the first caveat.
Comment: @BenKuhn: Thank you very much! I actually had some collinearity issue which I could fix relatively easily.
Comment: Could you explain why large differences in scale can make it look like your data has colinearity?


What was the motivation behind the removal of wrestling from the Olympics?

Question: I wonder why wrestling was removed from the Olympics without any public discussion. I do not follow wrestling on a regular basis but considering that it has been one of the oldest sports in the world history and a part of the Olympic games since 1904, I was a bit shocked when I heard the news.
Can someone shed some light into the official (and perhaps less official, more speculative) motives behind this decision?
Comment: We haven't had a `wrestling` tag until now? Blasphemy!
Comment: @sancho I was referring to lack of public discussion about it. I follow sports papers on a regular basis, and one day out of the blue there it was, an article that said wrestling was not an Olympic sport anymore.
Comment: "Without any public discussion" - the IOC's organization requires little, if any public discussion, and no doubt they don't consider this a problem.
Answer: According to a statement by an IOC spokesperson in this article, this is a move to renovate the offering of core sports in the Olympics and offer "the best program for the Olympic Games in 2020."
It was noted that wrestling ranked low in popularity, TV audience, hits on the Internet, and press coverage.
An executive board reviewed sports in the Summer Olympics with the intent to remove one to add another later this year...none may be added however.

The board voted after reviewing a report by the IOC program commission
  that analyzed 39 criteria, including TV ratings, ticket sales,
  anti-doping policy and global participation and popularity. With no
  official rankings or recommendations contained in the report, the
  final decision by the 15-member board was also subject to political,
  emotional and sentimental factors.
Comment: It is a sad day when golf is considered an Olympic sport while wrestling is not... I'd rather see Starcraft as an Olympic sport than golf, but hey that's just me. Maybe I am just not rich or noble enough :)
Comment: @posdef With the current status, they can better remove soccer from the Olympic program, imo.
Comment: didn't they already add rugby and will try to add golf?
Comment: Golf is in the 2016 games
Comment: @Bernhard true, it's not like anyone really cares about football at the olympics.. :)
Comment: Guys, lets take this to chat if we want to have discussion! @posdef, I am "offended" by your comment (lol goes here)...but I understand your viewpoint. It is an expensive game to play, but I love it :D
Comment: @edmastermind29 no offense intended, if you catch me on chat i'd be happy to discuss my opinion of golf :)


How can we retain dropdown list value, when we select the option?

Question: I want to retain dropdown list value, so that when ever someone select the option from the dropdown list should retain that value after performing the action.
/*
 *implements hook_menu().
 */

function dropdown_page_menu(){
  $items = array();
  $items['page/layout/%'] = array(
    'title' => 'Select Layout',
    'page callback' => 'drupal_get_form',
    'page arguments' => array('dropdown_page_select_form'),
    'access arguments' => array('access content'),
    );

  return $items;
}

/**
 * Implements hook_block_info().
 */
function dropdown_page_block_info() {
    $blocks = array();

    $blocks['dropdown_page_select_form'] = array(
        'info' => t('Select Layout'),
    );

    return $blocks;
}

/**
 * Implements hook_block_view().
 */
function dropdown_page_block_view($delta = '') {
    $block = array();

    switch ($delta) {
        case 'dropdown_page_select_form':
            $block['subject'] = t('Select Layout');
            $block['content'] = drupal_get_form('dropdown_page_select_form');
            break;
    }

    return $block;
}

function dropdown_page_select_form($form, &$form_state){
  $options = array(
    'select' => t('Select'),
    'layout_1' => t('Layout 1'),
    'layout_2' => t('Layout 2'),
    'layout_3' => t('Layout 3'),
    );

  $form['choice'] = array(
    '#type' => 'select',
    '#options' => $options,
    '#title' => t('Select your layout'),
    '#default_value' => $form_state['choice']['type']
    );

  $form['submit'] = array(
    '#type' => 'submit',
    '#value'=> t('Go'),
    );
  return $form;
}

function dropdown_page_select_form_submit($form, &$form_state){

switch ($form_state['values']['choice']){
  case layout_1 : {

    //echo 'hi';die;
    drupal_goto("layout_1".$options);
    break;

  }
  case layout_2 : {
    //echo 'no';die;
    drupal_goto("layout_2".$options);
    break;
  }
  case layout_3 : {
    drupal_goto("layout_3".$options);
    break;
  }
  default: {
    print 'Select it!';
  }
}

  drupal_set_message(t('Your layout %input', array('%input' => $form_state['value']['choice'])));
}

Answer: You can set the default values as follows,
$form['choice'] = array(
    '#type' => 'select',
    '#options' => $options,
    '#title' => t('Select your layout'),
    '#default_value' => variable_get('choice', $form_state['choice']['type']
)
);
provided you get the submitted values in the variable $form_state['choice']['type'] post submit action.
And the variable name should match the form field name "choice"
Hope this works for you!
Comment: it's not working
Answer: Try following code. I think you need this value to be retained through all pages while navigating.
function dropdown_page_select_form($form, &$form_state) {
    $options = array(
        'select' => t('Select'),
        'layout_1' => t('Layout 1'),
        'layout_2' => t('Layout 2'),
        'layout_3' => t('Layout 3'),
    );

    $form['choice'] = array(
        '#type' => 'select',
        '#options' => $options,
        '#title' => t('Select your layout'),
        '#default_value' => isset($_SESSION['selected_layout']) ? $_SESSION['selected_layout'] : 'select'
    );
    $form['submit'] = array(
        '#type' => 'submit',
        '#value' => t('Go'),
    );
    return $form;
}

function dropdown_page_select_form_submit($form, &$form_state) {
    $_SESSION['selected_layout'] = $form_state['values']['choice'];
    drupal_set_message(t('Your layout %input', array('%input' => $form_state['value']['choice'])));
    switch ($form_state['values']['choice']) {
        case 'layout_1' : {

                //echo 'hi';die;
                drupal_goto("layout_1" . $options);
                break;
            }
        case 'layout_2' : {
                //echo 'no';die;
                drupal_goto("layout_2" . $options);
                break;
            }
        case 'layout_3' : {
                drupal_goto("layout_3" . $options);
                break;
            }
        default: {
                print 'Select it!';
            }
    }
}


How is the javascript method referenced for the window object?

Question: When I declare a basic function in the on document ready listener, it cannot be used on any inline javascript code as shown below. However, when I call window.alertMeTwo() on the inline function, it works.
How can I reference a function of Window that hasn't been created yet?
How does this linking process possible work?

    var alertMeOne = function { alert();  }
    //works inline

    window.alertMeOne = function {alert()};
    //works inline

    jQuery(document).ready(function() {

       var alertMeTwo = function(){alert();}
        // does not work inline

       window.alertMeTwo = function(){alert()};
       //works inline????

    });

Comment: You need to go through this. http://stackoverflow.com/questions/111102/how-do-javascript-closures-work?rq=1
Comment: "*How can I reference a function of Window that hasn't been created yet?*" `onclick` doesn't "validate" that `alertMeTwo()` exists when it's defined. It does that check after it's invoked, looking for `alertMeTwo` and retrieving its current value. Though, `onevent` attributes only search within the global scope.
Answer: alertMeOne() is defined in the global scope (which coincides with the scope under window).
The first alertMeTwo() is defined is the scope of the callback passed to jQuery().ready -- which is not the global scope.
The second one is defined inside a local scope but bound to the global scope (window.alertMeTwo).
If you access functions with the inline syntax you have to specify the scope or you'll fail back to the global one.
Also, in this specific case, since you're not exporting alertMeTwo to some namespace accessible from the global one, you can only call it from that scope (inside the .ready callback).
Answer: try to assign an .on() event handler in jquery
window.alertMeTwo.on("load", function(){
//code goes here
});

Answer:
How can I reference a function of Window that hasn't been created yet?
How does this linking process possible work?

JavaScript functions, including those created from onevent attributes, don't link directly to variables or validate their existence when defined.
They reference a chain of scopes or Lexical Environments, leading up to the global scope, that can each store the variables you define in Environment Records.
While the function is executing, when it reaches alertMeTwo(), it'll only then begin searching through the environments and their records for alertMeTwo. A ReferenceError is only thrown when it runs through the chain of environments without finding the variable.

In the case of onevent attributes, these functions only search within the global scope/environment and Global Environment Record. In browsers, this object is the window object within browsers.


How to mount a local folder using docker image for Jupyter notebook?

Question: I use docker image below:
https://hub.docker.com/r/gw000/keras-full
It works great; however, I want to mount a local folder so I can save and use local notebook files. I use command below:
docker run -d -p 8888:8888 -v c:/Jupyter:/srv gw000/keras-full

The command runs successfully; however, I cannot create a new notebook because I get an error message below:

Permission denied: Untitled.ipynb

I've tried the following:

I've given "FULL" access permission to Jupter folder("everyone" user)
I've tried command from the first submitted answer
I've  tried Firefox and Edge browsers using normal and admin modes

Docker image requires password "keras" as per example below:
docker run -d -p 8888:8888 -v $(pwd):/srv gw000/keras-full

I'm not able to figure out how to use mapping folder and password at the same time.
My Environment:

Windows 10 Pro
Docker Desktop

Docker logs:
[I 10:35:10.240 NotebookApp] Writing notebook server cookie secret to /home/user/.local/share/jupyter/runtime/notebook_cookie_secret
[W 10:35:10.844 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 10:35:13.007 NotebookApp] jupyter_tensorboard extension loaded.
[I 10:35:13.009 NotebookApp] Serving notebooks from local directory: /srv
[I 10:35:13.009 NotebookApp] 0 active kernels
[I 10:35:13.009 NotebookApp] The Jupyter Notebook is running at:
[I 10:35:13.009 NotebookApp] http://[all ip addresses on your system]:8888/?token=...
[I 10:35:13.009 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 10:35:18.003 NotebookApp] 302 GET / (172.17.0.1) 0.50ms
[I 10:35:18.015 NotebookApp] 302 GET /tree? (172.17.0.1) 0.97ms
[I 10:35:21.563 NotebookApp] 302 POST /login?next=%2Ftree%3F (172.17.0.1) 1.40ms
[I 10:35:26.118 NotebookApp] Creating new notebook in
[I 10:35:26.151 NotebookApp] Writing notebook-signing key to /home/user/.local/share/jupyter/notebook_secret
[W 10:35:26.166 NotebookApp] 403 POST /api/contents (172.17.0.1): Permission denied: Untitled.ipynb
[W 10:35:26.166 NotebookApp] Permission denied: Untitled.ipynb
[W 10:35:26.167 NotebookApp] 403 POST /api/contents (172.17.0.1) 49.41ms referer=http://localhost:8880/tree?
[I 10:36:38.981 NotebookApp] 302 GET /tree (172.17.0.1) 1.66ms
[I 10:36:42.373 NotebookApp] 302 POST /login?next=%2Ftree (172.17.0.1) 1.12ms
[I 10:36:45.459 NotebookApp] Creating new notebook in
[W 10:36:45.475 NotebookApp] 403 POST /api/contents (172.17.0.1): Permission denied: Untitled.ipynb
[W 10:36:45.476 NotebookApp] Permission denied: Untitled.ipynb
[W 10:36:45.476 NotebookApp] 403 POST /api/contents (172.17.0.1) 18.42ms referer=http://localhost:8880/tree
[I 10:37:19.390 NotebookApp] Creating new notebook in
[W 10:37:19.410 NotebookApp] 403 POST /api/contents (172.17.0.1): Permission denied: Untitled.ipynb
[W 10:37:19.410 NotebookApp] Permission denied: Untitled.ipynb
[W 10:37:19.411 NotebookApp] 403 POST /api/contents (172.17.0.1) 21.50ms referer=http://localhost:8880/tree

Answer: First:
docker run -d -p 8888:8888 -v $(pwd):/srv gw000/keras-full

The above command is to mount the current directory using "pwd" Linux command ("pwd" as in print current directory) to the folder "/srv" inside the container. It has nothing to do with changing the password.
Secondly:
It seems to me here you are using folder "c:/Jupyter" to mount into the container folder. You might need to run Docker as administrator or you can mount from others drive that doesn't need administrator permission like drive d:/.
So then you might try:
docker run -d -p 8888:8888 -v d:/Jupyter:/srv gw000/keras-full

After seeing the Dockerfile at https://hub.docker.com/r/gw000/keras-full/dockerfile, I think your Docker is mounting the /srv as "root" user and the service is running under user "user", so you have to exec into the container by docker exec -it <name_or_id_of_container> bash and run chown -R user:user /home/user /srv
Comment: Can you run "**docker logs **" to see what happened?
Comment: Thank you Nivhty. I'm using Windows Environment and could not run pdw. The command worked find once removed. Unfortonately, I don't have a second drive. I've tried running docker as admin without luck.
Comment: Try mounting with **c:\Jupyter**: "docker run -d -p 8888:8888 -v c:\Jupyter:/srv gw000/keras-full
"
Comment: I'm still getting Permission denied. I've tried different browsers opening normal and as admin
Comment: I've posted the details. changed port to 8880
Comment: This is the permission error inside the container. Can you mount your folder to different location than **/srv**?
Comment: I'm not sure that your service inside the container is running as root user, that might cause the problem, you need to find where the file **Untitled.ipynb** is written inside the container and change that folder's owner to the service's account.
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/193029/discussion-between-bi-dude-and-nivhty-njgjty).
Answer: One option is to map your local user to the docker user using -u e.g.
docker run -u $(id -u):$(id -u) ...


Accuracy stagnated while training notMNIST data

Question: I am a beginner in machine learning. I have built a logistic classifier in Python using TensorFlow to train on notMNIST dataset. My code is as such:
<code>weights = tf.Variable(tf.truncated_normal(shape = [784, 10]))
bias = tf.Variable(tf.zeros(shape = [10]))
logits = tf.matmul(features, weights) + bias
prediction = tf.nn.softmax(logits)
cross_entropy = -tf.reduce_sum(labels * tf.log(prediction), reduction_indices=1)
loss = tf.reduce_mean(cross_entropy)

train_feed_dict = {features: train_features, labels: train_labels}
valid_feed_dict = {features: valid_features, labels: valid_labels}
test_feed_dict = {features: test_features, labels: test_labels}

is_correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(labels, 1))
accuracy = tf.reduce_mean(tf.cast(is_correct_prediction, tf.float32))

epochs = 5
batch_size = 50
learning_rate = 0.1

optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

validation_accuracy = 0.0

with tf.Session() as session:

    session.run(tf.global_variables_initializer())
    batch_count = int(math.ceil(len(train_features)/batch_size))

    for epoch_i in range(epochs):

        for i in range(batch_count):

            session.run(optimizer, feed_dict = train_feed_dict)
            print(session.run(accuracy, feed_dict = train_feed_dict))
</code>
However, the problem is that while the training loss is decreasing continuously, the accuracy wavers initially, and then finally stagnates (at around 0.062). I am not able to understand what's wrong with the code. Any help would be appreciated. Thanks.
Answer: I think the problem is with your learning_rate.
Try changing it.Vary it by 2-3 folds for a good range of values and check if your accuracy improves.It generally might be due to non-convergence or divergence ie for low and high learning rates  respectively.


Chrome 35 triggers popstate after back button was clicked

Question: I know that there are many topics related to chrome and popstate event but after upgrading chrome to version 35 I encountered a new one.
This is the case:
I've got a listing page with categories and filters, after You choose category, page etc. content is reloaded via ajax and everything is handled by history API which works fine.
But when You go to detail page of item in listing - standard request, whole page is reloaded - then press back button on browser, You can see a listing page in position where You last clicked an item but then popstate is triggered (and I don't talk here about chrome popstate on initial page loads cause in version 34 popstate is no longer triggered on init), and because of code which I mentioned earlier to handle categories whole page is reloaded.
So the problem is that chrome 35 triggers popstate event after we press back button, and my question is:
How to detect that this event is triggered after user came back from detail page to my listing page to not reloading this one.
I try with document object referrer etc. but when it looks like solution on first time there was always the case when it doesn't work.
Answer: I've been running into this problem. My solution is a bit of a hack, but it seems to work pretty well otherwise. I've found that waiting 150ms before setting up my onpopstate handlers is enough to wait for Chrome to run its initial onpopstate handlers, which you can use to your "advantage". For instance, to detect if an initial onpopstate was run:
<code>// wait for page to load
$(function() {
  var no_initial_onpopstate = true;
  var enabled = true;

  window.onpopstate = function() {
    if(!enabled) return;
    console.log("detected initial onpopstate");
    no_initial_onpopstate = false;
  }

  setTimeout(function() {
    enabled = false;
    console.log("No initial onpopstate: ", no_initial_onpopstate);

    // no_initial_onpopstate indicates if the browser emitted an initial
    // onpopstate event. Do with that information what you will

    // 150ms delay seems to be the "magic number" to wait for Chrome to
    // run all of its initial onpopstate handlers (in my tests on localhost, at least)
  }, 150);
});
</code>
So, from this I gather that setting up your `onpopstate' callbacks after 150ms of the browser's DOM ready event will skip chrome's initial onpopstate.


Reading huge files in C#

Question: I have to read a large files 4-10gb each line by line, the problem is that a .Net process gets and OutOfMemory exception when I read ~2gb
At first I am just attempting to count the lines, however I will need to access each line individually to strip some data from it.
From what I can see, every option keeps the previous lines in memory, where I only want it to keep the currently read line(unless anyone knows a trick to keep all of it)
Here is that I tried, and several things like it:
StreamReader reader = File.OpenText(FilePath);
while ((line = reader.ReadLine()) != null)    //This is where it errors
{
   count++;
}
reader.Close();

The exception is:
Exception of type 'System.OutOfMemoryException' was thrown.
at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
at System.Text.StringBuilder.Append(Char[] value, Int32 startIndex, Int32  charCount)
at System.IO.StreamReader.ReadLine()
at CSV.Program.NumLines() in C:\Users\ted\Documents\Visual Studio 2015\Projects\vConnect\CSV\CSV\Program.cs:line 100
 at CSV.Program.Main(String[] args) in C:\Users\ted\Documents\Visual Studio 2015\Projects\vConnect\CSV\CSV\Program.cs:line 20
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

Thanks
Comment: No, if each line is a reasonable length it should be absolutely fine, assuming you're not actually keeping the data yourself. You shouldn't need to call `DiscardBufferedData`. What is your *real* code doing with the data after stripping something from it?
Comment: It is *technically* possible, happens when the file is corrupted and no longer contains text.  Nothing but binary zeros for example.  Squarely falls in the "crap happens" category, the kind you should never code for since you still won't get any usable data out of that file.
Comment: Looks like a line ending problem - unix vs windows? Try `flip`.
Comment: Please post the *full* exception including it's call stack and the actual code where it occurs. There is no need to discard the buffered data. It's *not* the buffer that causes problems but how you handle the strings you read.
Comment: I don't think that `DiscardBufferedData` does what you think it does. It's only used to resynchronize the Reader and the underlying stream.   Have you tried this without that extra method call?
Comment: Generating a new file from data striped out. The data is an CSV style export of a database, and is not always the same size
Comment: The buffer is *only* a 1KB buffer ([source here](http://referencesource.microsoft.com/#mscorlib/System/io/streamreader.cs,48)) that is used to speed up loading bytes from disk. Cleaning it just makes your code slower. Post the full exception and the code where it occurs.
Comment: Should I guess that there are no newlines in the text?
Comment: What's the value of `count` when the exception is thrown ?
Comment: Count is equal to 368587 when its thrown
Comment: From the [source of StringBuilder](http://referencesource.microsoft.com/#mscorlib/system/text/stringbuilder.cs,1969) it's apparent that the exception is thrown when the buffer has to read *more* than its maximum capacity. This capacity though is [Int32.MaxValue by default](http://referencesource.microsoft.com/#mscorlib/system/text/stringbuilder.cs,139). Most likely this is a line ending problem
Comment: Its not a line ending problem as it reads 368587 lines first
Comment: Isn't simply StringBuilder consuming the memory?
Comment: Yes Filburt, that is the issue, however... I don't care about all the "old" lines... only the current one
Comment: There must be something missing here. Because you code is fine and if you are 100% sure about your EOL this should work.
Comment: Its just that the file is to long for StringBuilder to hold
Comment: To be totally certain and just for the heck of it, I tested reading a file line by line with a 26 GiB text file, with line lengths skyrocketing up starting at 20 and ending somewhere in the 21 million character range, and memory was very stable at ~30-60 MiB for the entire duration. The `ReadLine` code is definitely not a problem.
Comment: It defiantly is... because I just use Flints answer and it works!
Answer: You can use methods from the class FileStream: FileStream.Read and FileStream.Seek should allow you to do what you need. An example can be found here: http://www.codeproject.com/Questions/543821/ReadplusBytesplusfromplusLargeplusBinaryplusfilepl
You'll have to modify that slightly but essentially you can start at 0, read until you find a newline character, process the line, start from where you got to and repeat. It won't be terribly efficient but it will get the job done.
Hope this helps.
Comment: Yeah your correct, which a few modifications I can read line by line.
Comment: All you'd be doing is trying to reinvent the wheel. `StreamReader` already does this, quite well in fact. The chance that you'll screw up your replacement code for `StreamReader` is very high. (For binary files, you shouldn't use `StreamReader` obviously)
Comment: Agreed, in an ideal world you would use a StreamReader. Unfortunately, for whatever reason, that's not working here... For something as simple as reading line by line I'm not sure the risk of "screwing up" something would be high enough to deter me from using this solution, personally, but others may have different opinions.

Reading huge files in C#


Manipulação de colunas-lista

Question: Tenho uma tibble denominada my, a qual contém a coluna-lista data
library(tidyverse)

dataset<-data.frame(matrix(rnorm(6*30,1000,100),ncol=6))
cluster<-kmeans(dataset,centers=3)
dataset$kmeans<-as.factor(cluster[['cluster']])

my<-dataset%>%
  group_by(kmeans)%>%
  nest()

rm(dataset,cluster)

Tenho as seguintes dúvidas referente a manipulação de colunas-lista:

como aplicar a função nest e, ao mesmo tempo, manter as variáves de grupo nas tibbles? Algo semelhante a função base::split.
como faço para colocar estas colunas-lista no globalenv()? Algo semelhante a função base::list2env.
como aplicar um mutate e manter os resultados na mesma tibble? Considere que cada tibble tenha 6 colunas. Com um mutate_if(is.numeric,sum), gostaria que estas tibbles tivessem 12 colunas. Já tentei algumas ações, mas elas retornam uma nova coluna-lista, o que não quero.

Todos estes procedimentos eu consigo fazer com as funções correntes do r. Mas, tenho interesse em manipular as colunas-lista para permanecer no framework tidyverse.
Comment: A primeira já foi respondida no próprio exemplo. Não? Com o `group_by() ` + `nest()`...
Answer: A manipulação de colunas-lista dentro do tidyverse ocorre da mesma que este universo se propõe a manipular listas, ou seja, com purrr. A diferença é que esta manipulação ocorre dentro de um data.frame e portanto usa as regras do tidyverse para manipulá-los - ficando por exemplo dentro de um mutate().
Dada esta consideração de ordem geral, vamos as questões.
Manter as variáveis no nest()
Vejo duas interpretações possíveis para a pergunta. Na primeira delas, em que o resultado esperado está na tibble que é resultado de nest(), a resposta já está na própria pergunta. Na segunda delas, em que se espera que as variáveis estejam condidas dentro de cada tibble aninhada, pode ser resolvido acrescentando a variável nova na tibble aninhada com map().
my %>%
  mutate(data2 = map2(data, kmeans, ~mutate(.x, var = .y)))
# A tibble: 3 x 3
  kmeans data              data2
  <fct>  <list>            <list>
1 1      <tibble [14 x 6]> <tibble [14 x 7]>
2 3      <tibble [10 x 6]> <tibble [10 x 7]>
3 2      <tibble [6 x 6]>  <tibble [6 x 7]>

Levar a coluna-lista para o .GlobalEnv
Antes de mais nada, se você realmente pretende se manter no frameword tidyverse, você não deveria fazer isso. Neste caso as informações deveriam ser mantidas na tibble. Feita esta ressalva, a operação pode ser feita da mesma forma que colocada nesta pergunta, lembrando que para tal a lista deve ser nomeada. Assim teríamos:
ls()
[1] "cluster" "dataset" "my"
# Adiciona nomes aos elementos da lista
my$data <- set_names(my$data, paste0("tabela", seq_along(my$data)))
list2env(my$data, .GlobalEnv)
<environment: R_GlobalEnv>
ls()
[1] "cluster" "dataset" "my"      "tabela1" "tabela2" "tabela3"

Aplicar mutate
Por fim, para aplicar um mutate() numa lista-coluna se dá normalmente, porém para aplicar a operação a data elemento da lista-coluna (que é o desejado neste caso) é preciso incluir um map() dentro do mutate().
my %>%
  mutate(soma = map(data, ~mutate_if(.x, is.numeric, sum)),
         final = map2(data, soma, bind_cols)) %>%
  select(kmeans, final)
# A tibble: 3 x 2
  kmeans final
  <fct>  <list>
1 1      <tibble [14 x 12]>
2 3      <tibble [10 x 12]>
3 2      <tibble [6 x 12]>

Note que bastou incluir o seu código como fórmula dentro do map() para que funcionasse. Para produzir o resultado esperado na pergunta eu uni os dois data.frames num único.
Nem toda operação com coluna-lista precisa resultar em outra coluna lista. Para tal, basta usar alguma das variações do map_*().
my %>%
  mutate(tamanho = map_dbl(data, nrow))
# A tibble: 3 x 3
  kmeans data              tamanho
  <fct>  <list>              <dbl>
1 1      <tibble [14 x 6]>      14
2 3      <tibble [10 x 6]>      10
3 2      <tibble [6 x 6]>        6

Manipulação de colunas-lista


Having Trouble Decomposing This Partial Fraction

Question: 1/(s^2)((s-3)^2)
I'm having trouble decomposing this particular fraction.
Wolfram won't help, and other forums only suggest to use a CAS program.
Answer: Hint: Write
\begin{align}
\frac{1}{s^2(s-3)^2} = \frac{A}{s}+\frac{B}{s^2} + \frac{C}{(s-3)} + \frac{D}{(s-3)^2}
\end{align}
and solve for $A, B, C$ and $D$.
Answer: Hint: $$\frac{1}{s(s-3)} = \frac{1}{3}\left(\frac{1}{s-3}-\frac{1}{s}\right)$$
Use this twice to decompose: $$\frac{1}{s^2(s-3)^2} = \left(\frac{1}{s(s-3)} \right)^2$$
Comment: Very cute. (+1)
Comment: @JackyChong Thanks. Math is one of few places where lazy shortcuts may be considered *cute* ;-)


My pager disappears when I use hook_views_pre_execute

Question: I need to modify a query of one view. So, I use the hook_views_pre_execute.
The query is modified but the pager disappears.
My question is why? Do I need to add more lines to hook to use to display the pager?
Code:
function view_format_render_views_pre_execute(&$view) {

    if ($view->name == 'view_probe' && $view->display['block_1']->display_title =='List of categories') {
        $tid = $view->args[0];
        $tids = array_keys(taxonomy_get_children($tid));
        $tids=implode(', ',$tids);

         $search = "node_data_field_formatos_colecc.field_formatos_colecc_value = %d";
            $search_new = "node_data_field_formatos_colecc.field_formatos_colecc_value IN ( ".$tids." )";

            $query = &$view->build_info['query'];
            $query = str_ireplace($search, $search_new, $query);

            $view->build_info['query'] = $query;

        if (isset($tid) && isset($arguments)) {

                $query = str_ireplace($search_new, '', $query);
                $view->build_info['query'] = $query;
        }
    }
}

Answer: if query exec is too long (for pagination views must return total count of rows at all pages and correspond it to rows per page ) pagination not returned by views.
nid of nodes returns from one table in database and this queries is very simple & fast
another fields (cck, ubercart, stock) returned from another db tables and it can be too long. for this views just returns first page to prevent timeout


Expand div vertically to available space

Question: I have a simple page with 3 div on top of each other but I cannot get the middle content div to use all the vertical space available on screen. Note that top and bottom height cannot be fixed and that it has to be compatible with IE9.
HTML

Top

Middle

Bottom

CSS
.container {
    background-color: #dddddd;
}
.content {
    height: 100%;
    min-height: 100%;
}

I also put that on Fiddle to demonstrate the issue: http://jsfiddle.net/quaxkq40/
Thanks.
Comment: are the heights of top and bottom div fixed?
Comment: Specify clearly what you want to have, maybe with an example
Comment: @JayantVarshney I updated my question. The top and bottom div size are  not known.
Comment: see http://stackoverflow.com/questions/2351674/how-to-make-a-div-expand-to-fit-available-vertical-space
Comment: Is your container height fixed?
Comment: @Ucodia - Why don't you go with the css "100vh" rather than 100% on the container? It works, but that will have the content to be slightly larger than the available space. You can then do a CSS calculation to fix it if you know the height of the top and bottom elements. eg: height: calc(100vh - (total height of top and bottom elements));
Answer: If flexible box layout is an option, it can be done simply by displaying the container as a flex box and the nested
s as flex items:

html, body {
  height: 100%;
  margin: 0;
}

.container {
  background-color: #dddddd;
  height: 100%;
  display: flex;
  flex-direction: column;
}

.content {
  flex: 1;
}

Top

Middle

Bottom

Note: Vendor prefixes omitted due to brevity.

Otherwise, you'll probably end up with CSS tables:

html, body {
  height: 100%;
  margin: 0;
}

.container {
  background-color: #dddddd;
  display: table;
  height: 100%;
}

.container > div {
  display: table-row;
}

.content {
  height: 100%;
}

Top

Middle

Bottom

Each of the above methods has its own pros/cons, choose one that meets the requirements with care.
Comment: Unfortunately it has to work in IE9 so no flexbox :/ I am gonna try with CSS tables.
Comment: @Ucodia Just make sure that the container has a `width` of `100%` as well. It is needed to make it fill the entire horizontal space — as we have changed the default display type of the element to `table`.
Answer: You could display the div's inside .container as table rows.
body, html {
    height: 100%;
}
.container {
    background-color: #dddddd;
    display: table;
    height: 100%;
    width: 100%;
}
.container>div {
    display: table-row;
}
.content {
    height: 100%;
    min-height: 100%;
    background: lightblue;
}

UPDATED FIDDLE


best fingering for this arpeggio

Question:
Right-hand.
I've been trying to play thumb, index, middle, pinky, thumb, middle, thumb.
But I find that a bit awkward. Maybe it's correct and I just have to get used to it.
What's the standard for progressions like this?
Comment: Also possible: 2-1-2-5-3-2-1.
Comment: Maybe try 3-1-2-5-3-2-1?
Comment: My hand is big enough that 1-2-3-5-3-2-1 seems plausible. (The largest interval I can comfortably play in either hand is an octave. I can play a ninth, but I hit adjacent notes at that point.)
Comment: @Dekkadeci I can play a tenth quite comfortably, an eleventh is challenging
Answer: I would use FSimardGIS' answer of 2-1-2-5-3-2-1.  It would somewhat depend on what comes before and after this little snippet, but standing alone, that's the fingering I would use.
Comment: This is how I would answer as well.
Answer: Playing a black key with thumb is often going to create more problems than it solves. So, I'd advocate using thumb on the first C, putting either index or middle on the Bb prior, and whatever is comfy going up.
What the last Bb gets played with is up to whatever follows - could be the same finger as the beginning, but if lower notes followed, at least a middle finger would be needed to move the whole hand to the left for those notes. What does come after? And, let's face it, it all depends on what precedes that Bb in the previous bar. What comes before? (Although it does appear to be the start of a piece - but that might be just the way the OP put it.)
Without those two answers, any answer here is going to be pretty useless!
As has been said many times, 'best fingering' tends to be rather subjective. Best for one player is often what he's worked out himself. There aren't that many options!
Comment: The thumb on black key rule is not always followed in keys of scales that start on black keys. It is impossible to completely avoid the thumb on chords and arpeggios in these keys. The Chopin black-key etude uses the thumb on a black key all over the place.
Answer: You are looking at the problem backward.
The goal of fingering is to minimize stretches, twists and pulls so that the fingers can function effortlessly as an extension of the arm.  Fingering is irrelevant as it pertains to the arm placing the fingers.
If you were to swat a fly, which finger kills the fly?  It doesn't matter because all the fingers operate as one, as part of the whole hand, as an extension of the arm.  If you isolate the parts you open up the tendons for tension.
More important is how the arm places the fingers. Equally important is where the hand is coming from and where it is going before and after this passage.
Without rhyme or reason since my answer is relative to MY technique, I would finger it one of two ways:  313 (leap with the arm) 3132 (or 1 depending what is next).  My second option would be 2315132.  This is MY fingering because I use my elbow for cross over notes, I try to avoid anything that will twist my wrist in ulnar or radial deviation. Also, if you don't play from the pronator and supinator muscles, your finger for these rotations is doomed to  begin with.
Again, the goal should be to minimize stretches since when you abduct and flex at the same time you are using two muscles to pull on one bone and at that point, it doesn't matter what your fingering is because the pulls will create tension.
Playing the piano should feel more like waxing your car than whatever gymnastics most pianists subject their isolated fingers to.

best fingering for this arpeggio


A way to bind differents values of a property in each DataGrid cell

Question: I've searched on many threads but I couldn't find anything to solve my problem, and I don't really know what keywords I should use. I have a DataGrid which is populated by a DataSet, with columns that I specify manually.
In my column, 'Total', the Binding is set by :
<DataGridTextColumn Header="Total"
         Binding="{Binding RelativeSource={RelativeSource Mode=FindAncestor,
                   AncestorType={x:Type UserControl}},
                   Path=DataContext.Total}"

The Total property is not in the DataSet, which is bound to my DataGrid, that's why I use this kind of Binding.
The problem is that when I set a value inside a cell of 'Total', like any user would do, this value is repeated in each cell of my column.
So, I know it is because of this property, since everytime I change my cell's value the Total property gets this new value and sends it back to my column. I could use a condition which uses the selected row's number or something, I have some ideas of what I should do, but I can't do that in my Xaml code.
I tried to create a Binding() in the code behind part but it doesn't really work and I'm not sure it could change anything.
Is there any way to use one value per row in this case ?
Thanks !
Comment: Easiest way would be to not use a DataSet but rather custom class representing your data. Then you can add a property that represents the total of the other fields and bind your column to that. This will also support using the MVVM design pattern http://msdn.microsoft.com/en-us/magazine/dd419663.aspx
Comment: @LeeO. Hey thanks for your link, I tried to read that when I started in WPF few months ago. Unfortunately, I have no choice but to use a DataSet, that's why I didn't try something else. Actually is represents the total of the other fields, but in each cell in my column, this is the problem.
Comment: Add a column to the DataTable within the DataSet that uses an expression to calculate it's value and bind your total column to this column. http://msdn.microsoft.com/en-us/library/system.data.datacolumn.expression.aspx
Comment: Ok, I'm not sure if I can modify my DataSet but I'll give it a try, thanks for your tip. I guess there's no other solution ?
Comment: You could also use a multibinding for your total column and bind it to all the same bindings the columns you want to sum up are bound to. Then create a IMultiValueConverter that will sum up those values.
Comment: I may be misinterpreting what you want to do. I don't understand why you say you have no choice but to use a DataSet. Once you have the DataSet, you could map each row into a custom class where you can add additional properties. Then use an ObservableCollection of your custom class and bind the DataGrid to this.
Comment: I really thank you for your help, Lee O., but I have to say I'm pretty new in WPF, and I never used anything like that. Would you mind to provide me a piece of code about that please ? I'm already checking for multibinding and the IMultiValueConverter but I'm afraid I won't find anything useful before I get it.
Comment: I can't because my DataSet can contain more than 1000 rows, and it takes too much time to execute the query, then to add each result in a class, etc. Plus, I use this DataSet for a lot of other things, and I can't re-code all of that, I've a deadline, and I already spent a lot of time to find out how to bind everything in my grid and some more. But, I use OC of custom classes in my grid too.
Comment: Ah...you are looking to serve up a Squirrel Burger http://benscheirman.com/2007/10/dont-make-squirrel-burgers/
Comment: Hahaha ! I didn't know that ! But actually you're wrong, the deadline I'm talking about is just the end of my internship, my boss is pretty cool with me. It's just that I started to learn WPF few months ago, and I'm slowing down because of this kind of "bugs" everytime I try to make it better than it's supposed to do in WPF. Unfortunately, I don't have a lot of time to learn WPF optimization, see ?
Answer: So, as Lee O. kindly said, I've first decided to alter my DataSet by adding a new column and some data inside. The problem was that I couldn't find any way to bind successfully on this second column in my designer code.
Anyway, I knew it wasn't the best way because I kind of prevented myself to modify something big to avoid big problems in my code and some other reasons, so I spent out a whole day to make it work better. I didn't think it could have taken so much time to modify it, but it finally worked.
The solution is to create (or alter in my case) a class which represents one row in your DataGrid, with as many properties as you need, including every columns you may want to have. Then, create an ObservableCollection (I really love this type of list, very useful and easy to use / update) in your ViewModel.
Finally, bind your column to your property in your class in your MainWindow like that :
// new ViewModel() + ObservableCollection<Class>
this.gridCameras.ItemsSource = viewModel.OCCameras;
this.Total.Binding = new Binding("Total");

If it can help someone too, I found something to bind correctly this class to a ComboBox in your DataGrid, which is a bit more complicated, here and here as I suppose that your ComboBox should be written like that (don't forget the x:Name property used in the foreach loop) :
<DataGridTemplateColumn Header="Fps" Width="80" x:Name="Fps">
    <DataGridTemplateColumn.CellTemplate>
        <DataTemplate>
            <ComboBox x:Name="cbFps"
                      Foreground="Black"/>
        </DataTemplate>
    </DataGridTemplateColumn.CellTemplate>
</DataGridTemplateColumn>

I strongly suggest you to add an Index property in your class, since I use that to identify the object in my ObservableCollection when I edit a cell / row and some other things. It's much better than to loop on your list until you find the element everytime you'd need it.
Hope it helps and feel free to ask any question you may have. I don't say I used the best way of the world, but it works for me and that's what I wanted.
Thanks again for your tips Lee O.
Comment: This is some nice progress. I think your next step should be to learn how to do the stuff you are doing in the code behind in the xaml instead. I'm glad you were able to work your way through this!
Comment: Yay thank you ! Actually because of my C# experience I prefered to start WPF by using code behind more than Xaml. Not the best way, pretty hard, but very instructive.


Expected Value Of a Random Variable X

Question: Consider an experiment that is successful with probability $0.8$. We
repeat this experiment (independently) until it is successful for the
first time. The first $5$ times we do the experiment, we have to pay
$\$10$ per experiment. After this, we have to pay $\$5$ per experiment.
Define the random variable $X$ to be the total amount of money that we
have to pay during all experiments. Determine the expected value
$(X)$.
I tried solving it using linearity of expectations and geometric distribution, but I am sure I was wrong.
Any help on how to solve this?
Comment: Okay and what do you think about the problem?
Comment: @SeyhmusGüngören what do you mean?
Comment: your own opinions about how to solve this problem. Dont you have any idea?
Comment: no. that's way I am asking for help. as I mentioned in the question I tried using linearity of expectations and geometric distribution but I don't think it is correct
Comment: Okay so the game continues until we get the first hit. It  means we have infinitely many possible cases with all having non zero probabilities right? Every event is independent so the probability of current event and the next event dont affect each other. Then what is the probability that we stop right away? what is the probability that we stop for the second time? third time?
Answer: If Pr(N)  is the probability of reaching N trials before a successful outcome.
Assuming you mean the Expected cost of any single trial.
E(cost) = 10*( Pr(1) + Pr(2) + Pr(3) + Pr(4) + Pr(5)) + 5* sum(Pr(n)) N=6 to inf
Answer: The probability of the program lasting precisely $n$ experiments is
$$ (0.2)^{n-1}\cdot (0.8) $$
Break the problem into two regions:  Find the expected number and cost of expensive experiments, and add the cost of inexpensive experiments.
The expected cost of the expensive experiments, each of which requires $n-1$ previous failures to be needed, is
$$
 10 \sum_{n=1}^5  (0.2)^{n-1}
$$
which is just a geomentric sum with value
$$
10 \cdot  \cdot \frac{1-(0.2)^5}{1-(0.2)} = \$12.496
$$
 The expected cost of the \$5  experiments, each of which requires $n-1$ previous failures to be needed, is
$$
5 \sum_{n=6}^\infty  (0.2)^{n-1} = 5 \cdot (0.2)^5 \frac{1}{1-(0.2)} =\$0.002
$$
And the sum of these is
$$
\$12.498
$$

Expected Value Of a Random Variable X


how to access private github repos using yaml

Question: I am trying to find a way to access my private github repo via my YAML pipeline which runs in azure devops. I can create PAT access tokens, but do i need to copy this token and paste it as a secret value within github -> repo -> settings -> secret? and how would i access this token to get access to my github private repo? i have the following steps though not sure if this works:
steps:
  - shell: pwsh
    env:
      SUPER_SECRET: ${{ secrets.SuperSecret }}
    run: |
      example-command "$env:SUPER_SECRET"

what are the yaml tasks i need to access my gihuh repo? i know using below i can access public repos but not private ones:
  repositories:
    - repository: myrepo
      endpoint: svc-github
      name: projecta/privaterepo
      type: github

Any ideas on this? thanks.
Answer: To use a private github repository from your YAML pipeline, you have to grant authorization for it :

Go to the settings of your Azure devOps project on Azure DevOps
Select "Service connections"
You have to create a new GitHub service connection and give access to your private repository
After that, specify the service connection created inside "endoint:"

So, you will obtain something like that :
resources:
  repositories:
  - repository: common
    type: github
    name: Contoso/CommonTools
    endpoint: [your service connection name]

Azure DevOps documentation section about it : https://learn.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=example#resources-repositories
I hope this will help you.


Exchange 2010 domainprep messing up mailbox permissions on existing Exchange 2003 server

Question: So our environment is basically we have an Exchange 2003 server, and we're attempting to move to Exchange 2010 gradually, and move to new hardware while we're at it.
So our first step was obviously to get Exchange 2010 installed on the new box.  However, after running the domainprep steps listed in http://technet.microsoft.com/en-us/library/bb125224.aspx (including PrepareLegacyExchangePermissions) our mailbox permissions get messed up.
Normally, we have an AD security group for Exchange Administrators that allows anyone in that group to view all folders inside any user's mailbox.  However, now, this functionality is gone and our Exchange Admins can't access anyone's mailboxes.  We'd like to get this functionality back if we could.
Thanks
Answer: When you look at the ACL on a mailbox is your Exchange Administrators group still there?  Or has it been removed.
Also there is normally a Deny entry in the ACL for Domain Admins, so even if you've got another group with Allow the Deny will override that.
Comment: Yeah the exchange administrators group is still there, and as mentioned, there's an explicit deny in the rules.
Answer: Standard AD permissions on Exchange mailboxes include an explicit deny to the Domain Admins and Enterprise Admins groups (and some other ones); this ACE is applied at the organization level, and inherited by everything Exchange-related (including databases and mailboxes). I don't know how ALCs were configured in your environment, but there are very good chances the preparation steps of Exchange 2010 applied some default permissions which were different than what you are used to; the standard rule for Exchange is that nobody should have access to someone else's mailbox contents unless that kind of access is explicitly granted by the owner, and this is true for everyone, including Exchange Organization Administrators.
I suggest looking into actual object ACLs using some tool such as ADSIEdit; alternatively, you can manage security on the Exchange organization object (and downwards) using the "Services" node of the Active Directory Sites and Services console.


MVC in Groovy/Grails

Question: I'm newbie to groovy.
How can we call domain methods from controller in Grails.
Controller :
def results = User.fetch_results

User Domain :
def fetch_results {
    def users = this.get(1)
}

Sorry if the above code is wrong, i need to know how to access domain methods from controller.
thanks.

My code is like this
UserController:
def results = User.addUser

User domain:
def addUser {
    def user = new User()
    user.id = 1
    user.publication_name = pub_name
    user.publication_desc =  ""
    user.edit_date = new Date()
    user.save()
    }
}

I tried with above code but getting errors . how can we call "addUser" method from controller ?
thanks.
Answer: You need to make the method a static method.
class User {

    def static addUser() {
        def user = new User()
        ...
        user.save()
    }
}

Then make sure to import the User object in your controller.
Comment: Thanks Blacktiger, absolutely i was looking for this solution
Answer: I think a
User usr = User.get(1) // or User.findBy... etc

in your controller should do what you want. Just check out the GORM documentation from the Grails Docs, they are really good. If you are new to Groovy but not Java you should check the Groovy Documentation before to get a feeling for the Groovy language itself.
Answer: Assuming you want a list of all users and your domain class is defined as User
 def userList=User.list()

Assuming you want to access a specific field on user number 2
 def theAddress=User.get(1).address

Note this assumes you've imported what ever package your domain classes are defined in.

MVC in Groovy/Grails


Split PDF's into multiple parts

Question: Is it possible to allow pdf uploads to be split automatically by drupal?
Example1: If I upload a PDF with 4 pages. Drupal will split it into pdf(page1), pdf(page2),etc...
Example2: I upload a pdf that is 10mbs, It can logically split the files into a limit. So if the limit was 5mbs. It would split the file into 2 even parts...
Comment: Drupal wouldn't be able to do this itself. You'd have to use something like GhostScript http://pages.cs.wisc.edu/~ghost/
Answer: Drupal can't out of the box. Doesn't mean YOU couldnt in PHP :) (i think)
I've written PDFs on-the-fly using Ruby and mucked with it in PHP soooo ....
Using Fields in Drupal you could supply an File Upload widget for PDFs.
Then write a module to process the file on upload,
Using PHP's built in PDFs functions like:
http://php.net/manual/en/function.pdf-get-pdi-value.php
simply get the numbers of pages, and then retrieve each page's content and make a new simple 1 page PDF out of it. Then save the file and FID to another Field of files with unlimited values for any number of pages .... It sounds doable for most PDFs (not necessarily crappy built non-standard PDFs) -- but definately some work .... :)
Answer: Re: per page, you might look at the PDF to ImageField module.  It can create image snapshots of each page.
Comment: It would work but the client requires the file to be a pdf as these are legal documents getting sent to courts.

Split PDF's into multiple parts


A singular $n-$cube and a circumference defined the border than 2-cube

Question: This is an exercise from "Calculus on Manifolds" by Michel Spivack (first edition, p.100):

If $c$ is a singular $1$-cube in $\mathbb{R}^2-\{0\}$, with $c(0)=c(1)$, show that there is an integer $n$ such that $c-c_{i,n}=\partial c^2$ for some $2-$chain $c^2$.

Definitions: A singular $n-$cube in $A\subset \mathbb{R}^n$ is a continous fuction $\alpha:[0,1]^k\to A$. A singular $n-$chain is a sum $\sum^k_{j=1} r_j\alpha_j$, where $\alpha_j$ is a singular $n-$cube and $r_j\in \mathbb{R}$.
So, in this case, $c$ is a curve in $\mathbb{R}^2$ and $c_{1,n}$ is a circumference with radio $1$ and a total of $n$ laps around the origin.
I can consider a line that intersects the origin, and if $\{U_{+}, U_{-}\}$ is an open cover of $\mathbb{R^2}-\{0\}$, then $\{c^{-1}(U_{+}), c^{-1}(U_{-})\}$ is an open cover of $[0,1]$.
I have problems to properly write the proof.
Is this a correct idea? How can I end the proof?
Thanks.
Answer: Here's a quick proof if you know that $\pi_1(S^1,x)\simeq \mathbb{Z}$.
Since $\mathbb{R}^2-\{0\}$ is homotopy equivalent to the circle, after choosing a basepoint $x$, we have $\pi_1(\mathbb{R}^2-\{0\},x)\simeq \mathbb{Z}$. We can take the curve $c_{i,1}$ as a generator for this group. By definition of the group structure on $\pi_1$, we have
$$c_{i,n}=nc_{i,1}\;.$$
Now choose any loop $c$. Then $c$ must be in some homotopy class of loops, which implies that it is homotopic to one of the $c_{i,n}$ and the homotopy between the two gives the desired 2-chain.


What is the standard error of the sample standard deviation?

Question: I read from there that the standard error of the sample variance is
$$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$
What is the standard error of the sample standard deviation?
I'd be tempted to guess and say that $SE_{s} = \sqrt{SE_{s^2}}$ but I am not sure.
Comment: @Kjetil Your thought is an amusing one. Please note, though, that the SE as defined here is not a random variable; it has no standard error. One often *estimates* the SE by using an estimate of $\sigma^4$ and frequently--by a conventional abuse of language--still calls that *estimated* SE a "standard error." As such it is indeed a random variable and will have a standard error. I'm sure you're aware of the distinction (and had it in mind when you wrote your comment), but I want to emphasize it so that people don't misunderstand the original question as a result of pondering your comment.
Comment: Asymptotically it "does not matter". In finite samples it certainly does. For the asymptotic answer see http://stats.stackexchange.com/a/105338/28746
Comment: And next you ask for the standard error of the standard error of the standard error ...
Comment: You mean the standard error of the _sample_ variance/standard deviation I guess? If yes, any particular distribution in mind?
Comment: Yes, this is what I meant. I edited my post in reaction to your comment thanks. I am surprised that you're asking what distribution I have in mind. I would have not expect it matters. No I don't have any particular distribution in mind. The population form which my sample is taken is likely not normal. It is probably slightly skewed and has very long tails.
Comment: @kjetilbhalvorsen Is this a decreasing or an increasing sequence?
Comment: The following paper gives the standard error for different statistics. https://www.researchgate.net/publication/285590690_Standard_errors_A_review_and_evaluation_of_standard_error_estimators_using_Monte_Carlo_simulations
Answer: Let  $\mu_4 = E(X-\mu)^4$. Then, the formula for the SE of $s^2$ is:
$$
se(s^2) = \sqrt{ \frac{1}{n}\left(\mu_4 -\frac{n-3}{n-1} \sigma^4\right)}
$$
This is an exact formula, valid for any sample size and distribution, and is proved on page 438, of Rao, 1973, assuming that the $\mu_4$ is finite. The formula you gave in your question applies only to Normally distributed data.
Let $\hat{\theta} = s^2$. You want to find the SE of  $ g(\hat{\theta})$, where $g(u) = \sqrt{u}$.
There is no general exact formula for this standard error, as @Alecos Papadopoulos pointed out. However, one can drive an approximate (large sample) standard error by means of the delta method. (See Wikipedia entry for "delta method").
Here's how Rao, 1973, 6.a.2.4 put it. I include the absolute value indicators, which he incorrectly omitted.
$$
se(g(\hat{\theta})) \approx |g'(\hat\theta)|\times se(\hat{\theta})
$$
where $g'$ is the first derivative.
Now for the square root function $g$
$$
g'(u) = \frac{1}{2\thinspace u^{1/2}}
$$
So:
$$
se(s)\approx  \frac{1}{2 \sigma} se(s^2)
$$
In practice I would estimate the standard error by the bootstrap or jackknife.
Reference:
CR Rao (1973) Linear Statistical Inference and its Applications 2nd Ed, John Wiley & Sons, NY
Comment: +1 It's nice to see these results so clearly laid out and explained.  Although I don't have Rao 1973 in front of me, I would expect the multiplicative factor in his formula ought to be $|g^\prime(\hat\theta)|$, for otherwise you would conclude that any order-reversing transformation would have a negative standard error.
Comment: Thanks. You are right about the absolute value. Rao had omitted it (equation 6.a.2.4 in both the 1968 and 1973 editions.) .The proof of the delta method is really for the variance, where the multiplier is  [g']^2.
Comment: @alpha_989 The [bootstrap](https://en.wikipedia.org/wiki/Resampling_(statistics)#Bootstrap) and [jackknife](https://en.wikipedia.org/wiki/Resampling_(statistics)#Jackknife) methods use resampling in order to estimate precision. They are useful because you don't need to do the error propagation by hand.
Comment: whats the bootstrap and jackknife?


How to unset $page['#attached']['html_head_link'] from head?

Question: Help me with this question please: how to unset html_head_link from pages in my Drupal 8.0.3 site.
I mean, delete this META:
<link rel="delete-form" href="/node/{id}/delete" />
<link rel="edit-form" href="/node/{id}/edit" />
<link rel="version-history" href="/node/{id}/revisions" />
<link rel="revision" href="/mypage" />

I try this in MYTHEME.theme file:
function MYTHEME_page_attachments_alter(array &$page) {
  foreach ($page['#attached']['html_head_link'] as $key => $value) {
    unset($page['#attached']['html_head_link'][$key]);
  }
}

But this do nothing...
Comment: https://www.drupal.org/node/2406533
Comment: This is no solution..
Answer: OK. I found fresh module for this trouble. Module's author use hook_entity_view_alter() for implements head render. Check out their git respository to download the module and get the full code with comments.
Answer: function program_html_head_alter(&$head_elements) {
  unset($head_elements['system_meta_generator']);
  foreach ($head_elements as $key => $element) {
    if (isset($element['#attributes']['rel']) && $element['#attributes']['rel'] == 'canonical') {
      unset($head_elements[$key]);
    }
  }
}

Comment: This code do nothing too.


How to Delete Rows CSV in python

Question: I'm trying to compare two csv files (fileA and fileB), and remove any rows from fileA that are not found in fileB.  I want to be able to do this without creating a third file.  I thought I could do this using the csv writer module but now I'm second guessing myself.
Currently, I'm using the following code to record my comparison data from file B:
removal_list = set()
with open('fileB', 'rb') as file_b:
    reader1 = csv.reader(file_b)
    next(reader1)
    for row in reader1:
        removal_list.add((row[0], row[2]))

This is where I'm stuck and do not know how to delete the rows:
with open('fileA', 'ab') as file_a:
    with open('fileB', 'rb') as file_b:
        writer = csv.writer(file_a)
            reader2 = csv.reader(file_b)
            next(reader2)
            for row in reader2:
                if (row[0], row[2]) not in removal_list:
                # If row was not present in file B, Delete it from file A.
                #stuck here:  writer.<HowDoIRemoveRow>(row)

Comment: [`sqlite`](http://docs.python.org/2/library/sqlite3.html) is a flat-file based database and the drivers for it are included in modern versions of Python. It might be a better option considering what you are trying to do.
Comment: Sorry for the silly question but this will create an exact copy of the fileB, isn't it?
Answer: This solution uses fileinput with inplace=True, which writes to a temporary file and then automatically renames it at the end to your file name. You can't remove rows from a file but you can rewrite it with only the ones you want.

if the keyword argument inplace=1 is passed to fileinput.input() or to the FileInput constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). This makes it possible to write a filter that rewrites its input file in place.

fileA
h1,h2,h3
a,b,c
d,e,f
g,h,i
j,k,l

fileB
h1,h2,h3
a,b,c
1,2,3
g,h,i
4,5,6

import fileinput, sys, csv

with open('fileB', 'rb') as file_b:
    r = csv.reader(file_b)
    next(r) #skip header
    seen = {(row[0], row[2]) for row in r}

f = fileinput.input('fileA', inplace=True) # sys.stdout is redirected to the file
print next(f), # write header as first line

w = csv.writer(sys.stdout)
for row in csv.reader(f):
   if (row[0], row[2]) in seen: # write it if it's in B
       w.writerow(row)

fileA
h1,h2,h3
a,b,c
g,h,i

Comment: @justin You tagged it as 2.7? You can just use `set((row[0], row[1]) for row in r)` instead
Comment: A subtle improvement not addressed in the explanation: this code uses a set, a far more optimal data structure for answering "is this data present?" than a list (which must be iterated over each time).
Comment: @David Op also used a set though
Comment: D'oh. S/he clearly did. Well, small bit of advice- don't call it a removal "list", or bone-headed people like me will get confused as to the variable's type. =)
Comment: what version of python?  I don't believe this syntax is 2.4 compatible
Comment: I did.  I apologize.  I didn't realize the house for my script is on a 2.4 box until now.
Answer: CSV is not a database format. It is read and written as a whole. You can't remove rows in the middle. So the only way to do this without creating a third file is to read in the file completely in memory and then write it out, without the offending rows.
But in general it's better to use a third file.
Answer: As Lennart described, you can't modify a CSV file in-place as you iterate over it.
If you're really opposed to creating a third file, you might want to look into using a string buffer with StringIO, the idea being that you build up the new desired contents of file A in memory. At the end of your script, you can write the contents of the buffer over file A.
from cStringIO import StringIO

with open('fileB', 'rb') as file_b:
    new_a_buf = StringIO()
    writer = csv.writer(new_a_buf)
    reader2 = csv.reader(file_b)
    next(reader2)
    for row in reader2:
        if (row[0], row[2]) not in removal_list:
            writer.writerow(row)

# At this point, the contents (new_a_buf) exist in memory
with open('fileA', 'wb') as file_a:
    file_a.write(new_a_buf.getvalue())

Comment: A word of caution here: you may exhaust the available memory for your system if your input files are large.
Comment: You may as well just write to a different file and rename it at the end, that is what my solution does
Comment: @jamylak, I completely agree with you. And that's exactly what I would do in this situation. I just figured this would be useful in that in technically meets what the asker is looking for.

How to Delete Rows CSV in python


How do I prove my Fejer kernel definition is equivalent?

Question: In my notes, the definition of the Fejer kernel is
$$
F_{n} = \sum_{j=-N}^{N} \left(1 - \frac{|j|}{N+1}\right) e^{ijt}.
$$
But in most of the reference material I come across online, it is immediately defined as the average of the Dirichlet kernels
$$
F_{N} = \frac{1}{N+1} \left(D_{0} + \dots + D_{N}\right).
$$
I've tried equating these two definitions by expanding $F_{n}$'s $e^{ijt}$ and using some trigonometry to get something looking like the $\sin$ representation of the Dirichlet kernel but it has not been going well.
Is there a simple way to prove that these two definitions are equivalent?
Comment: Not sure what you're asking, you want a proof for $F_N(y)=\frac{1}{N+1} \bigg( \frac{\sin (\frac{N+1}{2})y)}{\sin (\frac{y}{2})} \bigg) ^2 \ ?$
Comment: @Itay4 Sorry if I wasn't clear, I'm just trying to establish equivalence of the two definitions I outlined.
Answer: It is sufficient to compare the coefficients of $e^{ijx}$ in both expressions.
What is the coefficient of $e^{ijx}$ in the second expression? Since
$$D_n=\sum_{k=-n}^n e^{ikx}$$
the number of Dirichlet Kernels in the average that contain $e^{ijx}$ is clearly
$N-j+1$, (because for $k<j$ the Dirichlet kernel $D_k$ does not contain $e^{ijx}$), and the coefficient is $1$ in each kernel, so all in all we have
$\frac{1}{N+1}(N-j+1)=1-\frac{j}{N+1}$. Here we assumed $j>0$. Similarly, the number of times $e^{-ijx}$ appears in the second expression is also $N-j+1$, hence the expression $1-\frac{|j|}{N+1}$.

When I posted this I wasn't aware of Itay4's answer, which is essentially the same counting argument (although somewhat condensed).
Answer: $\begin{array}{rlll}F_N(y)~&=~
\frac{1}{N+1}\sum_{n=0}^ND_n(y)=\frac{1}{N+1}\sum_{n=0}^N\sum_{k=-n}^ne^{iky}
\\ &=~ \frac{1}{N+1}\sum_{m=-N}^Ne^{imy}\#\{k \in [0,N]:m \in [-k,k]\}
\\ &=~ \frac{1}{N+1}\sum_{m=-N}^Ne^{imy}(N+1-|m|)=\sum_{m=-N}^Ne^{imy}(1-\frac{|m|}{N+1})
\end{array}$
Comment: @EricHansen That's the counting of Dirichlet's kernels which is well explained in uniquesolution's answer.
Comment: Thank you! Would you mind explaining to me what your $\#$ notation is on the second line?


MySQL config recommendation script

Question: I want to be able to get an recommendation of what values I should set my my.cnf to. Especially the query_cache_size and join_buffer_size. Is there a script out there that can monitor my server and give me a recommendation? Or any other solution?
I found a few scripts, but their recommendations did not give any specific numbers...
Answer: Written by RackerHacker, MySQL Tuner should help: http://mysqltuner.pl/mysqltuner.pl - you'll be lucky to find one that suggests the actual values though as these are heavily dependant on the code you're running.
Answer: Personally, I prefer well known tuning-primer.sh
http://www.day32.com/MySQL/

MySQL config recommendation script


Construction of Wiener measure(or Brownian motion)

Question: In this definition of Wiener's measure, we define the measure of a standard Brownian motion by extending the f.d.d. distribution on set of all continuous functions. However, we also know that the set of all continuous functions is not measurable in the measurable space of all functions, with $\sigma$-field generated by all pointwise projection maps $\pi_t(\omega)=\omega(t)$. This fact, in some textbooks(like Durrett's PTE), asks us to constraint functions on rational points and then show that the sample paths are uniformly continuous with probability 1.
So my question is that whether my following understanding is correct:

if we start with all functions $\{\omega:[0,\infty)\mapsto\mathbb{R}\}$ then the measurability of $C(\mathbb{R})$ is indeed a problem, which requires a detour via rational approximations;

but if we start with $C(\mathbb{R})$ then everything is fine. But in this case what is the $\sigma$-field?

so the reason for starting from all functions to construct the Brownian motion is that we want more generality, that we don't want to assume in advance that the sample paths are continuous almost surely?
Answer: Your understanding is correct. If you wanted to start with $C=C(\mathbb R_+\to\mathbb R^n)$ (the space of all continuous functions from $\mathbb R_+$ to $\mathbb R^n$) then the $\sigma$-algebra is $C\cap {\cal B}^{\mathbb R_+}$ which is the collection of all sets of the form $C\cap B$ where $B$ is in the $\sigma$-algebra ${\cal B}^{\mathbb R_+}\,.$ This in turn is the $\sigma$-algebra on the space of all maps $\mathbb R_+\to\mathbb R^n$ (which can be denoted by $(\mathbb R^n)^{\mathbb R_+}$) such that every projection $\mathbb R_+\ni t\mapsto \omega(t)\in \mathbb R^n$ is Borel measurable.
To make a long story short: the space of all maps and the $\sigma$-algebra ${\cal B}^{\mathbb R_+}$ are not easy to avoid  when one starts to construct the Wiener measure with finite dimensional distributions (Kolmogorov theorem).
As far as I remember, the alternative Levy-Ciesielski construction builds more directly on continuous functions.


How do I set the text format used in a message (from the Message module)?

Question: I'm using the Message module on a site that has both the Filtered HTML and Full HTML text formats. By default, Message appears to use Filtered HTML, but I would like to use Full HTML instead.
However, I couldn't find an option for changing text formats in the UI. I have exported my message types to a feature using the Features module, so instead I tried changing the format from "filtered_html" to "full_html" in the feature and reverting, but this didn't work, either.
Is is possible to change the text format used to display messages, and if so, how?
(Also asked in the module's issue queue.  When I get an answer, I will update both.)
Answer: You can set the permission for user role to use filtered_html or full_html in Administer -> people -> permission.
Comment: I'm created the message types with the user 1 account, which has all permissions.  Still, this alone does not allow me to set the text format for the message.
Answer: I wasn't able to change the format, but I was able to get what I wanted by modifying the text format itself (then I had to adjust everything else that was using the text format in another way).  Not an ideal solution, but it worked for my needs.
Answer: Comment #1 in the issue you linked to contains "a" way to do what you're looking for:

... you can select the text format in the ui while writing the message text.

That same issue also contains a screenprint about this, which looks like so:


Plugin Facebook para o Wordpress

Question: Eu instalei o plugin do facebook no wordpress a alguns meses e ele funcionou corretamente. No entanto, de um mês pra cá, um problema tem atrapalhado meus posts e eu não consigo publicar mais. Ao publicar algo no wordpress e tentar automaticamente postar no facebook, ele gera o seguinte erro:
Failed posting to MY NAME Timeline. Error: {"message":"An unexpected error has occurred. Please retry your request later.","type":"OAuthException"}

Alguém tem alguma informação sobre isso?
Plugin do Facebook para Wordpress
Comment: Atualizou o plugin? Ele requer permissões especiais de publicação via Open Graph, o que você precisa liberar no aplicativo, [exemplo](http://prntscr.com/2twmwy).
Comment: Obrigado @VictorMendonça, porém o erro foi sanado sem nenhum ajuste. Acredito que o problema estava no facebook, visto que não atualizei nenhum plugin no wordpress e nem no facebook.
Answer: Eu recomendo você criar o seu plugin do facebook no próprio developer do facebook ficaria algo mais ou menos assim

<div class="fb-comments" data-href="http://seu site/?p=<?php the_ID(); ?>" data-width="692" data-numposts="5" data-colorscheme="light">

O código já se explica, width largura data-numposts numero de post etc.. O diferencial dele é o ?p=<?php the_ID(); ?> depois do link do seu site com isso ele conserta aquele bug de mostrar em todas as páginas os mesmo comentários.
Comment: Grato @lucas-de-souza, porém o erro foi sanado sem nenhum ajuste. Acredito que o problema estava com o facebook, ainda que isso me pareça distante.

Plugin Facebook para o Wordpress


How to Share on CRM2011 with XRM services?

Question: I'm new to CRM 2011. How to Share (create share record contact between team) on CRM2011 with XRM services? I can't find on SDK and I don't know to where shared record on MSCRM db.
Answer: You're looking for the GrantAccessRequest class. Here's the SDK article.


Ubuntu on EC2 - How do I ip-restrict port 22 to a specific IP address?

Question: I'm lost in Linux, can somebody help me with an iptables command to allow only the IP Address 18.104.22.168 to access port 22?
I've got that port restricted to the same IP in the EC2 security group, but I need to block it on the instance itself as well.
Comment: If you knew more about how virtual machines worked and how the network devices are "wired" together you would understand how un-necessary is. You will basically make an environment that is no more secure but with more points of failure and more difficult to troubleshoot and administer. **Your time would be better spent making sure your ssh service itself was properly secured** with keys and such.
Comment: "I've got that port restricted to the same IP in the EC2 security group, but I need to block it on the instance itself as well." Why?
Comment: I want to verify that I am protected from other instances. I don't think this is excessive. If I am doubling up, then so be it. I'm a programmer daggumit!
Comment: possible duplicate of [Iptables: How to allow only one ip through specific port?](http://serverfault.com/questions/146569/iptables-how-to-allow-only-one-ip-through-specific-port)
Comment: I think the flaw in your reasoning is that you assume I am 100% effective at configuring EC2 security groups
Answer: Aren't you being a little excessive? Your instances on EC2 are nicely isolated from other traffic by their own private firewall on the virtual network device you are "plugged into" upstream.
iptables -A INPUT -p tcp -s 22.214.171.124/32 --destination-port 22 -j ACCEPT
iptables -A INPUT -p tcp --destination-port 22 -j REJECT

Comment: I want to verify that I am protected from other instances. I don't think this is excessive
Answer: Try:
iptables -I INPUT 1 -p tcp --dport 22 -s 126.96.36.199 -j ACCEPT
iptables -I INPUT 2 -p tcp --dport 22 -j REJECT

This adds a rule which specifically allows access from that IP, then a rule which specifically denies it from everyone else.
Answer: This has also been covered in very similar posts:
Iptables: How to allow only one ip through specific port?
Allow traffic on one port from one IP address with iptables
Comment: Right on, voting to close as dupe


Normalizing data in a factorial ANOVA

Question: Suppose you have a 2 x 2 between-subjects ANOVA, so there are four groups.  You see that the data are negatively skewed in all four cells. You can't count on the Central Limit Theorem to provide immunity because your sample sizes are small. The textbooks recommend a data transform, perhaps a square root transform:
<code>           TransformedScore = SQRT(k – RawScore), where k = the largest RawScore + 1
</code>
Here's my question. Is this transform performed separately within each of the four cells, thus using different values of k in each of the four cells? That would normalize the distributions within each of the cells, but it also means that the highest raw score in each cell ends up with the same transformed value of 1, even though those highest raw scores in each cell might be very different raw score values. It doesn't seem right that different raw scores should end up having the same value following the transformation.
So, if one doesn't transform the data separately in each cell, how is it done? Does one do a single transform on the entire data set? That would normalize the entire data set, but it doesn't seem that it would be very effective in normalizing the distributions in each of the cells.
How exactly does one transform the data in a factorial ANOVA so as to normalize the distributions in each cell?
Comment: I would have thought doing it over the whole data-set would be best. Can you clarify why you are reluctant to do that?
Answer: You certainly can't normalize differently in each cell, for the reason you posit. The results would be uninterpretable. You can transform the whole data set, and that probably will substantially reduce the skew in each cell (since all are skewed in the same direction) although it might not make them normal.
Why transform at all?  Why not use some other method than ANOVA?  For instance you could use quantile regression or robust regression, neither of which assume that the errors are normally distributed.

Normalizing data in a factorial ANOVA


Android: How to catch Fatal Exception thrown from a referenced library?

Question: I am using a referenced library called 'rome-1.0.jar'.
try {
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed = input.build(new XmlReader(file));
}
catch(Exception e) {
    Log.e("", "ROME parse error: " + e.toString());
}

input.build(new XmlReader(file)); is 'rome-1.0.jar' library call.
Although I am catching all Exceptions, my app still crashes.
05-31 13:43:06.297: W/dalvikvm(11150): threadid=22: thread exiting with uncaught exception (group=0x40a4d1f8)
05-31 13:43:06.307: E/AndroidRuntime(11150): FATAL EXCEPTION: Thread-4199
05-31 13:43:06.307: E/AndroidRuntime(11150): java.lang.NoClassDefFoundError: java.beans.PropertyDescriptor[]
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.impl.BeanIntrospector.getPropertyDescriptors(BeanIntrospector.java:39)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.impl.CloneableBean.beanClone(CloneableBean.java:129)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.impl.ObjectBean.clone(ObjectBean.java:87)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.module.DCModuleImpl.clone(DCModuleImpl.java:771)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.module.impl.ModuleUtils.cloneModules(ModuleUtils.java:35)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS090.createSyndEntry(ConverterForRSS090.java:90)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS091Userland.createSyndEntry(ConverterForRSS091Userland.java:85)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS092.createSyndEntry(ConverterForRSS092.java:46)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS093.createSyndEntry(ConverterForRSS093.java:37)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS094.createSyndEntry(ConverterForRSS094.java:58)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS090.createSyndEntries(ConverterForRSS090.java:79)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS090.copyInto(ConverterForRSS090.java:64)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS091Userland.copyInto(ConverterForRSS091Userland.java:49)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.impl.ConverterForRSS094.copyInto(ConverterForRSS094.java:47)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.feed.synd.SyndFeedImpl.<init>(SyndFeedImpl.java:144)
05-31 13:43:06.307: E/AndroidRuntime(11150):    at com.sun.syndication.io.SyndFeedInput.build(SyndFeedInput.java:123)
...(my code)

Is there a way to handle this?
Answer: Thank you Kumar, you gave me an idea.
I didn't know this was an error.
I have handled OutOfMemoryError before so I know I am able to handle errors.
try {
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed = input.build(new XmlReader(file));
}
catch(Exception e) {
    Log.e("", "ROME parse error: " + e.toString());
}
catch(Error e2) {
    Log.e("", "ROME parse error2: " + e2.toString());
}

Above code handles errors. I know it's not a good idea to do so, but it serves my purpose.
Thank you.
Answer: This is not an exception, rather an error, whose nature is more severe than exceptions. You are not supposed to handle errors.
http://www.coderanch.com/t/269746/java-programmer-SCJP/certification/Error-vs-Exception
If errors happen, the system will, in most of the cases become unstable, and you really can't do anything about it.


Second Form not Loading

Question: I've searched for hours and tried bunches of stuff with no luck.
I'm using VB.net. My main form (Form1) is not loading another form (ECR_ECO_Details). It did just hours ago. Somehow ECR_ECO_Details ended up in the My Project folder. I dragged it back to where it used to be, same place as Form1. No errors, warnings or joy.
Additionally, I have a MsgBox in the Load and Shown events. The MsgBox does not present so my program is not even finding the second form.
It has to be something simple but this is my first time in Visual Studio. I'm convinced it's something in the IDE, not code.
    Private Sub NewECR_Click(sender As Button, e As EventArgs)
        Dim NewECRForm As New ECR_ECO_Details
        NewECRForm.ShowDialog()
    End Sub

Pretty much same effective code as above, same result. Used to work tho.
    Private Sub NewECR_Click(sender As Button, e As EventArgs)
        Using NewECRForm As New ECR_ECO_Details()
            NewECRForm.ShowDialog()
        End Using
    End Sub

TIA!
Answer: You've simply lost the Handles clause on the end of your method, meaning it NEVER runs when the button is clicked. *You can lose the Handles clause if you cut/paste the button to somewhere else on your form.
You can add it back in from the IDE by selecting the Button. Then in the Properties Pane, make sure the events are listed by clicking on the "Lightning Bolt" icon. Now find the "Click" entry and change the dropdown so it says "NewECR_Click".
Alternatively, you can simply add the Handles clause back onto the end of your existing method (look at the END of the first line):
Private Sub NewECR_Click(sender As Button, e As EventArgs) Handles NewECR.Click
    Dim NewECRForm As New ECR_ECO_Details
    NewECRForm.ShowDialog()
End Sub

Comment: That totally was it exactly!!! Thank you!
Answer: this code is on my Form1 click the button and it loads and shows frmSSUpdate and Close's Form1
 Private Sub btnSSDeposit_Click(sender As Object, e As EventArgs) Handles btnSSDeposit.Click
    frmSSUpdate.Show()
    Close()
End Sub

Comment: Thank you. This works for the case you state. I'm doing modal.

Second Form not Loading


Is default Apache version in Ubuntu 14.04 secure?

Question: I have installed apache on ubuntu server 14.04 using apt-get. I set up a simple site, uploaded a few pictures etc, the website is working. I've decided to take a look at security of the thing using one of the security checking services online. The security audit says I have 2 medium severity issues:
Apache Running Version Prior to 2.4.8
Apache Running Version Prior to 2.4.10

The repository does not hold any updates so I'd assume I have highest available version... The only things I knew how to check were openproxy with curl (I tried to fetch a wikipedia page and got my own homepage which seems correct)

Should I be concerned by the version of Apache I have available or the security patches are backported and the version is irrelevant here?
How can I test for vulnerabilities of the server, what should I look at in particular?

In the logs I saw plenty of ssh brutalforce attempts thwarted by fail2ban and in apache log some strange GET requests (many indicating shellshock attempts) but also the GET requests to other sites, which - as I mentioned - return my homepage.
Answer:
No, don't worry about the old version number.  The Ubuntu packagers backport security patches to older versions.  As of right now, if you run apt-cache policy apache2, you should see version 2.4.7-1ubuntu4.1 that's sourced from security.ubuntu.com as well as us.archive.ubuntu.com.
That's too big a question to answer here.  But see Tips for Securing a LAMP Server.
Comment: Almost as you said: 2.4.7-1ubuntu4 0 instead of 4.1 but it's probably minor thing
Thanks a lot. Regarding question 2 - ok, I'll try to look around the web for some resoures, Thank you.
Comment: I added a link to a FAQ here that may help, although it's about hardening, not testing.


Not able to uncheck/check chekboxes on uncheck/check of header checkbox after individual checkboxes are interacted with - Vuejs

Question: Situation 1 - I check my individual checkboxes, my header checkbox gets checked. This is method 1 and works fine.
Code for same
index.vue
<VCheckbox
 checked={this.checkSelections[idx]}
    nativeOnChange={e => {

    this.$set(this.checkSelections, idx, e.target.checked);
    let allSelected = true;
    for (let i = 0; i < this.checkSelections.length; i++) {
        allSelected = this.checkSelections[i];
        if (!allSelected) break;
    }
    this.$root.$emit("checkmarkHead", { allSelected });
    }}
/>

Head.vue
  mounted() {
    this.$nextTick(() => {
        this.$root.$on("checkmarkHead", ({ allSelected }) => {
        console.log("checkmarkHead", allSelected);
         this.allSelected = allSelected;
      });
    }

  },

Situation 2 -
I check my header checkbox and all  my checkboxes are checked. Vice-versa is true as well. So method 2 corresponding to this works fine too.
Code for same -
Head.vue
  <HeadItem>
    <VCheckbox
        checked={this.allSelected}
        nativeOnChange={e => {
        this.allSelected = e.target.checked;
        this.$root.$emit("selectAll", {
            allSelected: this.allSelected
        });
        }}
    />
</HeadItem>

index.vue
mounted() {
   this.$nextTick(() => {
        this.$root.$on("selectAll", ({ allSelected }) => {
        this.checkSelections = Array(this.sortedData.length).fill(allSelected);
     });
   });
 }

Problem - When I do situation 2 after Situation 1, the same methods don't work as expected. The view isn't updated. Similarly, executing Situation 1 after Situation 2 won't work either.
Here's the link to
Code Link - https://codesandbox.io/s/vmwy3v4203
I'm clueless now after handling all mutations caveats etc.
Comment: Can someone mention the problem with the question downvotes?
Comment: I think it's all the duplicate text you had up there :)
Comment: I noticed, someone has changed the line in mounted function in head.vue in Codesandbox. Very strange! Therefore my use cases as described in question may not have been working. I should have read more about edit privileges before sharing sandbox.
Comment: Would be great if someone could upvote it. Now that I did freeze the question.
Comment: @HusamIbrahim What duplicate part. I cross-checked for any mistakes and still can't figure? Unless you are intending pun linking to previous question.
Comment: [See this](https://stackoverflow.com/posts/52962268/revisions). And on a side note someone added keys to your original sandbox :)
Comment: That I may have done by mistake...after reading your answer. I'll update the link.
Answer: I owe you an apology. This is indeed another reactivity issue which could be solved by providing a key attribute ..

The key special attribute is primarily used as a hint for Vue's
virtual DOM algorithm to identify VNodes when diffing the new list of
nodes against the old list. Without keys, Vue uses an algorithm that
minimizes element movement and tries to patch/reuse elements of the
same type in-place as much as possible. With keys, it will reorder
elements based on the order change of keys, and elements with keys
that are no longer present will always be removed/destroyed.

You can assign a unique key value to an element, which if changed will force Vue to re-render that element. In your case you can assign keys to your VCheckbox elements equal to their checked value, forcing Vue to re-render them when they're checked/unchecked. For example ..
<HeadItem>
  <VCheckbox
    key={this.allSelected}
    checked={this.allSelected}
    nativeOnChange={e => {
      this.$emit('change', !this.allSelected)
    }}
  />
</HeadItem>

I've taken the liberty to re-write your allSelected property as a computed property and removed the event listener you set up on the root instance. I think it's much cleaner this way ..
Comment: You did use computed getter/setter for allSelected. The header checkbox renders as per that computation value passed as prop to Head.vue. Now, when head checkbox fires change event, you have used `$emit` but haven't captured that by say `this.$root.$on('change', ...)` explicitly but by`onChange` function as a prop to `head.vue` component. How does Vuejs handles that internally? I mean in React it was explicit to call `this.props.onChange()` in such a case.
Comment: Second, you have used computed.  Getter of `allSelected` will be fired for every change event of checkbox. Now, what difference `methods{}` could have made if we just had two explicit getAllSelected and setAllSeletected methods ? What benefits in this case would computed offer?
Comment: @kushalvm For some perspective you can refer to [this issue](https://github.com/vuejs/vuejs.org/issues/1802). But basically when a child component uses `$emit`, a parent (the `table` component in your case) can set up a listener on the child instance with `v-on` (`onChange`, `onInput`, `onClick` etc when you use JSX). Any additional arguments you pass when you call `$emit` will be passed to the listener's callback function.
Comment: In your implementation you were using the root instance essentially as an event bus, which isn't idiomatic in Vue. It works fine for your simple case, but the problem is that if the table component gets destroyed the event listener won't be removed unless you specifically call `this.$root.$off('change')`. Contrast this to the other case where [Vue automatically takes care of cleanup for you](https://vuejs.org/v2/guide/events.html#Why-Listeners-in-HTML).
Comment: Also your implementation would cause problems if you had [other components using a 'change' event on the root instance](https://codesandbox.io/s/1v0ynv8q2j). This makes your components unreusable in the same tree and forces you to think about how you name your events so as to not cause a conflict. Contrast this to the case where [everything is nice and localized](https://codesandbox.io/s/vmv50jqwy7).
Comment: RE: computed properties. If we're being scientific you can imagine a situation where you allow a user to change the table's style. In this situation upon re-render a computed property would just use the cached value because you didn't check/uncheck any checkbox, whereas a normal getter method would have to be called for each and every re-render. You could further imagine a situation where you allow the user to drag a mouse on a color-picker and see the table's color change live. In this case, even with throttling, the getter would be called hundreds of times.
Comment: But if we're being practical you won't get any performance improvements from a computed property since you're not doing any of these things and you only have a couple of checkboxes. The only advantage is that with a computed you'll have your property encapsulating its getter and setter so you have everything in one place. With the methods you'd have to initialize your `allSelected` property in the `data` section and have the getter/setter down in the `methods` section.
Comment: Also if you follow the [style-guide](https://vuejs.org/v2/style-guide/#Component-instance-options-order-recommended) your `computed` properties will come right after your `data`, giving you at a glance an idea about your state and what mutations you can have on it.
Comment: All the above has nothing to do with your reactivity issue, which you could solve in your implementation by providing a `key` like I mentioned previously and everything will work fine. It's just that the implementation is unidiomatic/inelegant (opinion, not fact )
Comment: For the computed properties, just wanted to make sure, I didn't miss anything from performance perspective for this case specifically, and turns out there isn't. I do understand the scientific difference :)
Comment: It was good to know the new type of usage and difference for localisation of event. Good solid reference of Github issue. Much appreciated :)
Comment: Everything being clear except for the thing you have mentioned twice that just adding a `key` would solve my issue of reactivity. But I did to my [code](https://codesandbox.io/s/0yxyxv5qx0) and no other changes and it didn't work. Am I missing something ?
Comment: [You forgot to add a key to the header checkbox](https://codesandbox.io/s/m74vjw19vp).
Comment: Alright @Husam Ibrahim  Nice interacting with you! I surely can't always run bounties :)
I'll love to keep engaged with you somehow esp on Vuejs -  which I recently started digging into - on such small issues and maybe to discuss some deep JS/React fundamentals as well.
Comment: Please let me know if I can reach you out of SO somewhere ( linkedin/facebook/twitter ) or something.