Removing failing drive from LVM volume group ... and recovering partial data from an incomplete LV (with a missing PV)

Question: I've been fighting this issue for some time now.
I have a Logical Volume with 3 disks, 1.5TB, 2TB and 3TB. The 1.5TB drive is failing. Lots of I/O errors and dead bad sectors.
I started pvmove to move the existing extents on the failing drive to the 3TB drive (there's enough space left).
I moved 99% of the extents but the last percent seems to be impossible to read. Reading fails and pvmove exits.
Here's the current state:
root@server:~# pvdisplay
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 1500301819904: Input/output error
/dev/sdd: read failed after 0 of 4096 at 1500301901824: Input/output error
/dev/sdd: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 1500300771328: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 1500300853248: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 4096: Input/output error
Couldn't find device with uuid hFhfbQ-4cuW-CSlE-qhfO-GNl8-Jvt7-4nZTWK.
--- Physical volume ---
PV Name               /dev/sda # old, working drive
VG Name               lvm_group1
PV Size               1.82 TiB / not usable 1.09 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              476932
Free PE               0
Allocated PE          476932
PV UUID               FEoDYU-Lhjf-FdI1-Ei5p-koue-PIma-TGvs9A

--- Physical volume ---
PV Name               /dev/sdd1  # old failing drive
VG Name               lvm_group1
PV Size               1.36 TiB / not usable 2.40 MiB
Allocatable           NO
PE Size               4.00 MiB
Total PE              357699
Free PE               357600
Allocated PE          99
PV UUID               hFhfbQ-4cuW-CSlE-qhfO-GNl8-Jvt7-4nZTWK

--- Physical volume ---
PV Name               /dev/sdf # new drive
VG Name               lvm_group1
PV Size               2.73 TiB / not usable 4.46 MiB
Allocatable           yes
PE Size               4.00 MiB
Total PE              715396
Free PE               357746
Allocated PE          357650
PV UUID               qs4BVK-PAPv-I1DG-x5wJ-dRNq-vhBE-wQeJL6

Here's what pvmove is saying:
root@server:~# pvmove /dev/sdd1:335950-336500 /dev/sdf --verbose
Finding volume group "lvm_group1"
Archiving volume group "lvm_group1" metadata (seqno 93).
Creating logical volume pvmove0
Moving 50 extents of logical volume lvm_group1/cryptex
Found volume group "lvm_group1"
activation/volume_list configuration setting not defined: Checking only host tags for lvm_group1/cryptex
Updating volume group metadata
Found volume group "lvm_group1"
Found volume group "lvm_group1"
Creating lvm_group1-pvmove0
Loading lvm_group1-pvmove0 table (253:2)
Loading lvm_group1-cryptex table (253:0)
Suspending lvm_group1-cryptex (253:0) with device flush
Suspending lvm_group1-pvmove0 (253:2) with device flush
Found volume group "lvm_group1"
activation/volume_list configuration setting not defined: Checking only host tags for lvm_group1/pvmove0
Resuming lvm_group1-pvmove0 (253:2)
Found volume group "lvm_group1"
Loading lvm_group1-pvmove0 table (253:2)
Suppressed lvm_group1-pvmove0 identical table reload.
Resuming lvm_group1-cryptex (253:0)
Creating volume group backup "/etc/lvm/backup/lvm_group1" (seqno 94).
Checking progress before waiting every 15 seconds
/dev/sdd1: Moved: 4.0%
/dev/sdd1: read failed after 0 of 4096 at 0: Input/output error
No physical volume label read from /dev/sdd1
Physical volume /dev/sdd1 not found
ABORTING: Can't reread PV /dev/sdd1
ABORTING: Can't reread VG for /dev/sdd1

There's only 99 extents still left on the failing drive. I'm OK with losing this data - I just want to pull this drive and throw it away without losing data on other drives.
So I tried pvremove:
root@server:~# pvremove /dev/sdd1
/dev/sdd1: read failed after 0 of 4096 at 1500300771328: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 1500300853248: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 4096: Input/output error
No physical volume label read from /dev/sdd1
Physical Volume /dev/sdd1 not found

And then vgreduce:
root@server:~# vgreduce lvm_group1  --removemissing
/dev/sdd: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd: read failed after 0 of 4096 at 1500301819904: Input/output error
/dev/sdd: read failed after 0 of 4096 at 1500301901824: Input/output error
/dev/sdd: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 1500300771328: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 1500300853248: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 0: Input/output error
/dev/sdd1: read failed after 0 of 4096 at 4096: Input/output error
Couldn't find device with uuid hFhfbQ-4cuW-CSlE-qhfO-GNl8-Jvt7-4nZTWK.
WARNING: Partial LV cryptex needs to be repaired or removed.
WARNING: Partial LV pvmove0 needs to be repaired or removed.
There are still partial LVs in VG lvm_group1.
To remove them unconditionally use: vgreduce --removemissing --force.
Proceeding to remove empty missing PVs.

pvdisplay is still showing the failing drive...
Any ideas?
Answer: In the end I solved this problem by manually editing the /etc/lvm/backup/lvm_group1.
Here are the steps in case anyone else hits this problem:

I physically removed the dead drive from the server
I executed  vgreduce  lvm_group1  --removemissing --force
I removed from the config the dead drive
I added another stripe on a "good" drive in place of the extents that were unreadable on the dead drive.
I executed vgcfgrestore -f edited_config_file.cfg lvm_group1
Reboot
Voila! Drive is visible and can be mounted.

It just took me 4 days of learning in-and-outs of LVM to solve this...
So far it looks good. No errors. Happy camping.
Comment: do you think this could work in this case: I have 2 HDs, on same LVM group; but only have logical volume on 1st HD, the second I use for snapshots; if the second fails, there will have any problems as a missing group member?
Comment: ohh... reading it again, it seems you broke the 1st LVM law that says: "if there is one missing PV, your LV is lost", COOL! :D (of course you did loose some data, but the point was restoring the most of it); also, I believe, if you use that contiguous flag somewhere, no files may split between PVs; that config file doesnt seem too difficult to edit manually, I wonder if there is some "lvm config file fixer" somewhere out there that could do the same you did? :)
Comment: I think in the title could be added something like: ... and recovering partial data from an incomplete LV (with a missing PV)
Comment: @AquariusPower I don't think there is any automated config fixer. There isn't even much information on doing this manually. I lost couple of files, but the rest of the data was preserved.
Comment: holy SHit you have no idea how happy i am to find this answer
Comment: @AquariusPower it should work in your case as well. The entire difficulty is in editing of the config file. If you can do it right then it will work for you.
Comment: I think, just for fun, I may try to do it someday, some script that make the calculations and provide partial-auto-recovery :)
Comment: if you still have the old lvm_group1 configuration, and if you dont mind it, it would be cool to paste them here (or at least a `diff`), so we have more clues when trying it :)
Comment: oh, btw, I just found something that may be interesting/related: http://bisqwit.iki.fi/source/lvm2defrag.html, from here http://unix.stackexchange.com/a/45918/30352, it makes calculations based on current lvm layout
Answer: If you are ok to stop the LVM temporarily (and to close underlying LUKS containers if used) an alternative solution it to copy as much as possible of the PVs (or the underlying LUKS containers) to the good disk with GNU ddrescue and to remove the old disk before restarting the LVM.
While I like Sniku's LVM solution, ddrescue may be able to recover more data than pvmove.
(The reason for stopping the LVM is that LVM has multipath support and would balance write operations between the pairs of PVs with identical UUIDs as soon as LVM discovers them. Furthermore, one should stop LVM and LUKS to ensure that all data that has recently been written is visible on the underlying devices. A restart of the system and not supplying the LUKS passwords is the easiest way to make sure of it.)


angularjs ng-submit not working

Question: hey i am new and already second question for today but i need help
when i try puting the ng-submit inside a form its says attribute is not allowed and doesnt work
ill be happy if you can help me.
and if you can explain to me to when do i need to declare a new controller?
andd thats it, thanks alot for those who help

(function() {
    var app = angular.module('list', []);

    app.controller('peopleListCtrl',['$scope', function($scope){
        $scope.persons = plists;

        this.addPerson = function() {
            if (this.text) {
                persons.push(this.person);
                this.text='';
            }
        };
    }]);

    var plists = [
        { name: 'alon', job: 'web dev', home:'nir tzvi' },
        { name: 'ben', job: 'katbamflighter', home:'nir tzvi' },
        { name: 'shiraz', job: 'dentist assistant', home:'hadera west' }
    ];
})();
<!DOCTYPE html>

    {{people.name}}
    {{people.job}}
    {{people.home}}

<form name="personForm" ng-submit="peopleListCtrl.addPerson()" >

  <input type="text" ng-model="person.name"/>
  Name:{{person.name}}

  <input type="text" ng-model="person.job"/>
  job:{{person.job}}

  <input type="text" ng-model="person.home"/>
  home:{{person.home}}

  <input type="submit" value="submit" />
</form>

Answer: here is the demo

(function() {
    var app = angular.module('list', []);

    app.controller('peopleListCtrl',['$scope', function($scope){
    var plists = [
        { name: 'alon', job: 'web dev', home:'nir tzvi' },
        { name: 'ben', job: 'katbamflighter', home:'nir tzvi' },
        { name: 'shiraz', job: 'dentist assistant', home:'hadera west' }
    ];
        $scope.persons = plists;
        $scope.persion = [];
        $scope.addPerson = function() {
            if ($scope.person) {
                $scope.persons.push($scope.person);
                $scope.person=null;
            }
        };
    }]);
})();
<!DOCTYPE html>

    {{people.name}}
    {{people.job}}
    {{people.home}}

<form name="personForm" ng-submit="addPerson()" >

  <input type="text" ng-model="person.name"/>
  Name:{{person.name}}

  <input type="text" ng-model="person.job"/>
  job:{{person.job}}

  <input type="text" ng-model="person.home"/>
  home:{{person.home}}

  <input type="submit" value="submit" />
</form>

Comment: didnt solve still nothing happens on submit and when mouse over ng-submit it says attribute is not allowed here
Comment: working!! =D but why i need to define if $scope.person?? i want it to submit only if there is text
Comment: because you are using `person.name` as `ng-model` without defining `$scope.person`. you need to define `$scope.person` first..

angularjs ng-submit not working


Why use casual inference over simple year over year analysis?

Question: Let's say I'm trying to measure the effect of a marketing campaign on dwell time (length of stay at a location).
Why not just use a simple year over year calculation to measure the effect? E.g. 20% increase in time from last year. Assuming other main factors such as weather/seasonality/day of the month affect dwell time, wouldn't this calculation take this into account since it roughly maintains these variables constant?
What drawbacks does this approach have? Why should I use a more complicated method such as modeling with casual inference? In essence, why should I go down the road of time-series analysis instead of just comparing original data from the same period each year?
Comment: Casual $\ne$ causal. Affect $\ne$ effect. A more serious problem is that it's far from clear precisely what your alternatives mean.
Comment: Well, you can. Moreover, no one approach in statistics is always best, try several and see which works best.
Comment: Sorry, I made an attempt to change casual to causal but after reading the question I am not sure what it's asking, I reverted my edit.
Answer: Sometimes simple A/B baselining works just fine (also called pre-post analysis), but there are very specific conditions that need to be met.  If there are other confounding variables (long-term trends, economic indicators, seasonality, etc.), you can draw bad conclusions from single-variable analysis of this sort.
As long as you have some reasonable assumptions that you can hold all other variables constant, year-over-year can be a perfectly valid comparison.  If, however, you tried to compare summer of this year to winter of previous, you have the confounding variable of weather/seasonality which may influence your results deleteriously.  Likewise, if you ALSO changed prices or had a new competitor come in to the market, or a bed bug infestation, or what have you, "all else equal" doesn't hold anymore.  The need to control for the influence of multiple variables is what causes people to turn to more complicated modeling approaches.
Most "AB Test" or "Control Focus" methodologies attempt to measure the influence of a single variable in the fashion you describe (although there are multivariate AB tests too - this just involves using multivariate regression across multiple trial variables).  You just need to satisfy yourself that you really can hold all other significant factors "reasonably" constant over the pre-post intervals (or be able to account for the expected variability due to the inability to control for all potential confounding factors).
Control-focus tests typically designate specially-selected control groups that are held constant over the variable in question specifically to allow differential analysis versus the test group with the assumption (usually supported by historical trending analysis) that non-test factors impact the sets equally (or at least within well-defined statistical tolerance).  This approach makes impact measurement more robust against external influencing factors not directly tied to the variable being tested.


using Jackson JSON library with Jersey + Jetty

Question: I implemented a Jersey service using embedded Jetty + Jackson. Though when I try it with a GET resource (that does not consume nor produce JSON) it works, it does NOT with the POST resource. The message on the console reads:
SEVERE: MessageBodyReader not found for media type=application/json, type=class com.delta.model.EnableDisableMessage, genericType=class com.delta.model.EnableDisableMessage

The POST resource:
@POST
@Path("/enable")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.TEXT_PLAIN)
public String enablePost(EnableDisableMessage enable) throws InterruptedException {
    logger.debug("within Proxy POST method..." + enable.toString());
    return "it worked!";
}

The model:
package com.delta.model;

public class EnableDisableMessage {

    private String cell;
    private String instruction;

    public EnableDisableMessage(String cell, String instruction) {
        super();
        this.cell = cell;
        this.instruction = instruction;
    }

    public String getCell() {
        return cell;
    }

    public void setCell(String cell) {
        this.cell = cell;
    }

    public String getInstruction() {
        return instruction;
    }

    public void setInstruction(String instruction) {
        this.instruction = instruction;
    }

}

I also implemented this configuration class as detailed by Jersey specs:
@ApplicationPath("/")
public class MyApplication extends ResourceConfig {

    public MyApplication() {
        packages("com.delta.model;com.delta.rest");
        register(JacksonFeature.class);
//      property(CommonProperties.MOXY_JSON_FEATURE_DISABLE, true);
    }
}

pom.xml's dependencies:
<dependencies>

        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>

        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-server</artifactId>
            <version>9.2.3.v20140905</version>
        </dependency>

        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-servlet</artifactId>
            <version>9.2.3.v20140905</version>
        </dependency>

        <dependency>
            <groupId>org.glassfish.jersey.core</groupId>
            <artifactId>jersey-server</artifactId>
            <version>2.7</version>
        </dependency>

        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-servlet-core</artifactId>
            <version>2.7</version>
        </dependency>

        <dependency>
            <groupId>org.glassfish.jersey.containers</groupId>
            <artifactId>jersey-container-jetty-http</artifactId>
            <version>2.7</version>
        </dependency>

        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
            <version>2.7</version>
        </dependency>

    </dependencies>

Comment: You need a no-arg constructor in `EnableDisableMessage`. But that is not the current issue. That will be the next issue once you fix the current issue :-). I am not sure why you would get this exception if you register the JacksonFeature. I'm curious to see how you build the server. Are you sure that the `MyApplication` is even being used? Can you share the code where you create the server/application.
Answer: Maybe including the Jackson JSON-provider in your dependencies does help?
<dependency>
    <groupId>com.fasterxml.jackson.jaxrs</groupId>
    <artifactId>jackson-jaxrs-json-provider</artifactId>
    <version>2.6.1</version>
</dependency>

Comment: `jersey-media-json-jackson` already pulls the above dependency in. It uses it under the hood.
Comment: Ah, my bad, I overlooked that, sorry
Comment: **Hi Lars**,  the dependency you mention above was actually what I needed as well as **peeskillet**'s no-arg constructor
Comment: **Hi peeskillet**, thanks for your comment on the **missing no-arg constructor** ! as you said it was the next error... Regarding your comment on **jersey-media-json-jackson** pulls the above dependency in under the hood, it seems that applies when using Jackson 2.x, that is com.fasterxml.jackson.
Comment: Hi ekremer, thank you for accepting my answer! Glad I could help!


Views: How to list nodes by 2 vocabularies with "%" on url path for contextual filters

Question: I have a content type with 2 different Term references. I wanna use Views to list "Term one" with "%" on url path, and use "Term two" as section. See below for details.
Vocabulary term one:
Book type A
Book type B
Book type C
Vocabulary term two:
Book style A
Book style B
Book style C
List result: on Book type A
Book style A
    - node 101 title, author name
    - node 102 title, author name
    - node 103 title, author name
Book style B
    - node 111 title, author name
    - node 112 title, author name
    - node 113 title, author name
Book style C
    - node 121 title, author name
    - node 122 title, author name
    - node 123 title, author name

— — updated info @ Sep. 9th 2014 @ 1:16pm
I created a content View, which has "Content: Type Book A" and set the path as "/booka/%. It came out fine except I don't know how to use the 2nd Term as section to list all those node under their style.
Comment: how about using a table with fields, include the style (exclude from display), and then 'group' the fields by the style (under table settings).  It's not terribly pretty, but some simple css can easily improve the layout.
Comment: what have you tried so far? what about it worked, and what didn't?  In general, people here aren't in the business of doing all the work for you.
Comment: @Geoff Sorry, I just updated my question. Everything goes well on my View except the 2nd Term. I dont know how to list these nodes under their style.
Comment: @geoff Yes, grouping is the way to go, and OP can always tinker with the view template files in addition to CSS.
Comment: @Geoff That work perfect. Can you Answer that on "Answer Your Question"?
Answer: try this:

create the view as a table with fields
include the style field (exclude from display)
under table settings, 'group' the results by the style field

You can tweak the layout for the headers with some simple css

Views: How to list nodes by 2 vocabularies with “%” on url path for contextual filters


can't load C# mode for emacs because of "make-local-hook"

Question: Mode : http://www.emacswiki.org/emacs/CSharpMode
log:
Loading /.emacs.d/contrib/dev/csharp-mode.el
Done loading /.emacs.d/contrib/dev/csharp-mode.el
File mode specification error: (void-function make-local-hook)
Loading vc-git...done
When done with a buffer, type C-x #
(No files need saving)
File mode specification error: (void-function make-local-hook)
When done with a buffer, type C-x #
Making completion list... [2 times]
goto-history-element: End of history; no default available [3 times]
or: Symbol's function definition is void: make-local-hook
mouse-minibuffer-check: Minibuffer window is not active
(No files need saving)
When done with a buffer, type C-x #
(No files need saving)
File mode specification error: (void-function make-local-hook)
When done with a buffer, type C-x #
Making completion list... [2 times]
or: Symbol's function definition is void: make-local-hook

Why that? And how can I fix it?
Answer: make-local-hook has been obsolete for years, and was removed entirely in Emacs 24.
You should try to locate an updated version of the library. According to the Wiki page you linked to, the latest version is here:
http://code.google.com/p/csharpmode/
Failing that, there's a pretty good chance that the code only includes those function calls to retain backwards-compatibility with Emacs 20, and that provided there is an appropriate call to add-hook present, all you would need to do is delete all instances of (make-local-hook HOOK) from the code.
Here's the relevant bits of its old docstring:

(make-local-hook HOOK)
This function is obsolete since 21.1;
     not necessary any more.
Make the hook HOOK local to the current buffer.
     The return value is HOOK.
You never need to call this function now that `add-hook' does it for you
     if its LOCAL argument is non-nil.

See also C-hf add-hook RET
Comment: I don't see any such calls in the code, so it can't be csharp-mode then. `M-x toggle-debug-on-error` should give you a stack trace when the error happens, which will tell you exactly where the issue originates. Post the trace here if you're still unsure.
Comment: strange but now Im using last csharpmode from trunc and last ccmode and Im still getting the same error


MapView in not moving always animate to the user current location

Question: I am developing an app which finds user current location and find near by places also.I have almost completed the app,but now the problem is that when user current location is find and a marker is added to user current location, but when i want to move or scroll the map to any other location in the onTouchEvent method map is not moving and and get back to the user current location marker.In emulator onTouchEvnet working fine but on device it is not working here is my code for moving map.
//here is my code for moving map
<code>@Override
public boolean onTouchEvent(MotionEvent event, MapView mapView) {
      if(event.getAction()==MotionEvent.ACTION_MOVE) {
             GeoPoint p = mapView.getProjection().fromPixels(
             (int) event.getX(),
             (int) event.getY());
             MapController mc=mapView.getController();
             mc.animateTo(p);
             return true;
       } else{
             return false;
       }
}
</code>
Answer: Can you clarify your question a bit? Are you trying to animate to a specific location when the user touches that point but it is just going back to the users current location? This is probably because MyLocationOverlay.enableMyLocation() tries to keep the blue dot visible in the MapView.


Where is the Migrate UI page?

Question: In Drupal 8.2.x there is the new migration UI tool. I've enabled it, but I can't find any link going to any UI page. Back in Drupal 7 when you enabled a module, it would have a configure link next to the module name in the modules page, but Drupal 8 doesn't. How would one find these configure links to the module pages?
Comment: Last I heard, this was being worked on in contrib. Only the core drupal to drupal one has a UI page, but it isn't like the one you are used to with the migrate framework from d7. The route in the routing file provided by migrate_drupal_ui is at '/upgrade'
Comment: jp recently made a blog post about this, which was an interesting read http://www.jpstacey.info/blog/2016-11-15/quick-rundown-what-using-drupal-6-8s-migration-ui-feels
Answer: modules -> migrate UI -> configure.
Press on the gear and you will enter the migration process.
Answer: The Migrate Tools module provides tools for running and managing Drupal 8 migrations.
https://www.drupal.org/project/migrate_tools
Answer: The path for this module's config page is example.com/upgrade.
Comment: That is the path to update Drupal core and/or modules, not the migrate UI.
Comment: Not sure why this answer was downvoted, I clicked the gear icon ("Configure") next to Migrate UI in the modules list and it navigated to /upgrade.
Comment: Yup no idea why it was downvoted...

Where is the Migrate UI page?


Other documents features than tf-idf for clustering?

Question: What are other feature representations for documents that are used for clustering textual documents? The only representation I'm aware of is tf-idf. Are there other ones?
Answer: Technically, TF/IDF is not a feature extraction scheme.
The feature representation is "the words in the document". The TF/IDF part is the weighting of the individual features for similarity computation.
There are other representations, including bag of words, n-grams (really look this one up) and in a wider sense: minhash.
Answer: For starters, tfidf is a family of term weighting functions. There are quite a few variants of it (e.g. pivoted tdidf), each of which has different properties. You could also use BM25 weights. BM25 has the advantage (?) that BM25 can/must be tuned using two parameters.


Changing to date field based on from date using javascript

Question: I have a form written using Drupal's FAPI (Drupal v7.15). The form includes two date fields - a from date and a to date. I've installed the Date module and both fields are both set-up to use the 'date_popup' widget.
Now I want to make the to date automatically change when the from date is updated by the user. Eg if the user sets the from date to 1st Nov 2012, I want the to date to automatically change to match the from date. Ideally I would also like the date picker to disable all options earlier than this date. This is the same behaviour you see on many travel sites, such as expedia.com.
I can't find a good example of how to add the Javascript to do this. I can only see AJAX examples, and it seems unnecessary to make a round trip to the server to do this - the browser has all the info it needs to do so client side.
Answer: If I understand you right it should be as simple as dropping something like this into a JS file included in the page:
(function($) {
  Drupal.behaviors.myModule = {
    attach: function(context, settings) {
      $('#edit-from-date', context).change(function() {
        $('#edit-to-date', context).val($(this).val());
      });
    }
  }
})(jQuery);

Disabling certain dates is a bit more complicated but there's an example here that should point you in the right direction.
Comment: Yes, you understand me correctly and that works perfectly... thanks :-)  I'll check out the disabling example now.
Answer: Here's the complete solution. The 'to date' is updated to the 'from date' plus one week and dates in its datepicker are disabled for dates equal to or earlier to the 'from date'. The form fields are created with Drupal's Forms API.
First the PHP code
$default_from = date('Y-m-d 00:00:00', strtotime(date('Y-m-d')." +1 month"));
$form['dates']['from_date'] = array(
    '#type' => 'date',
    '#title' => t('Ingreso'),
    '#type' => 'date_popup',
    '#date_year_range' => '-0:+2',
    '#date_format' => 'd/m/Y',
    '#default_value' => $default_from,
    '#datepicker_options' => array('minDate' => 'today'),
    '#required' => TRUE,
);

$form['dates']['to_date'] = array(
    '#type' => 'date',
    '#title' => t('Salida'),
    '#type' => 'date_popup',
    '#date_year_range' => '-0:+2',
    '#date_format' => 'd/m/Y',
    '#default_value' => date('Y-m-d 00:00:00', strtotime($default_from." +1 week")),
    '#datepicker_options' => array('minDate' => date('d/m/Y', strtotime($default_from." +1 day"))),
    '#required' => TRUE,
);

Then the Javascript
(function($) {
Drupal.behaviors.viajes = {
    attach: function(context, settings) {
        $('#edit-from-date-datepicker-popup-0', context).change(function() {

            //change min to-date to from-date plus one day
            if ($("#edit-to-date-datepicker-popup-0" ).hasClass('hasDatepicker')) {
                $("#edit-to-date-datepicker-popup-0" ).datepicker( "option", "minDate", addDays($(this).val(),1) ); //this method if to datepicker already initialised
            } else {
                Drupal.settings.datePopup['edit-to-date-datepicker-popup-0']['settings']['minDate'] = addDays($(this).val(),1); //this method if to datepicker has never been used
            }

            //change to-date to from-date plus one week
            $('#edit-to-date-datepicker-popup-0', context).val( addDays($(this).val(),7) );

        });

        //takes date string in format dd/mm/yyyy and returns date plus 'days' in the same format
        function addDays(dateStr, days) {
            //convert to date obejct
            var dateArray = dateStr.split('/');
            var date = new Date(dateArray[2], dateArray[1]-1, dateArray[0]);

            //add days
            date.setDate(date.getDate() + days);

            //convert back to correct string format and return
            return addZero(date.getDate()) + '/' + addZero(date.getMonth()+1) + '/' + date.getFullYear();
        }

        //add leading zero if number is 9 or less
        function addZero(number) {
            if (number <= 9) {
                return '0'+number;
            } else {
                return number;
            }
        }
    }
}
})(jQuery);

Answer: This is a modified version of adamf321's script. The differences are:

Format independent, uses the format set in Drupal instead of mm/dd/yy
Preserves the time difference to 'to date' when changing the 'from date'

You can change the fromDateId and toDateId to match your input field id's.
(function($) {
  Drupal.behaviors.myModule = {
    attach: function(context, settings) {
      var fromDateId = 'edit-from-date-datepicker-popup-0';
      var toDateId = 'edit-to-date-datepicker-popup-0';
      var $fromDate = $('#'+fromDateId, context);
      var $toDate = $('#'+toDateId, context);
      var currentfromDate = $fromDate.val();
      $fromDate.change(function() {
        var dateFormat = $fromDate.datepicker('option','dateFormat');

        //change min to-date to from-date plus one day
        if ($toDate.hasClass('hasDatepicker')) {
          $toDate.datepicker( "option", "minDate", addDays(dateFormat,$(this).val(),1)); //this method if to datepicker already initialised
        } else {
          Drupal.settings.datePopup[toDateId]['settings']['minDate'] = addDays(dateFormat,$(this).val(),1); //this method if to datepicker has never been used
        }

        //move toDate the same amount as fromDate has moved
        var daysShifted = dateStrDiff(dateFormat, currentfromDate, $(this).val());
        $toDate.val(addDays(dateFormat,$toDate.val(), daysShifted));
        currentfromDate = $fromDate.val();
      });

      //takes date string in given format (e.g. 'dd-mm-yy') and returns date plus 'days' in the same format
      function addDays(format, dateStr, days) {
        //convert to date object
        var date = getDate(format, dateStr);

        //add days
        date.setDate(date.getDate() + days);

        //convert back to correct string format and return
        return jQuery.datepicker.formatDate(format, date);
      }

      function getDate(format, dateStr) {
        return $.datepicker.parseDate(format, dateStr);
      }

      function dateStrDiff(format, dateStr1, dateStr2) {
        var date1 = getDate(format, dateStr1);
        var date2 = getDate(format, dateStr2);
        return dateDiffInDays(date1, date2);
      }

      // See http://stackoverflow.com/questions/3224834/get-difference-between-2-dates-in-javascript
      // a and b are javascript Date objects
      function dateDiffInDays(a, b) {
        var _MS_PER_DAY = 1000 * 60 * 60 * 24;
        // Discard the time and time-zone information.
        var utc1 = Date.UTC(a.getFullYear(), a.getMonth(), a.getDate());
        var utc2 = Date.UTC(b.getFullYear(), b.getMonth(), b.getDate());

        return Math.floor((utc2 - utc1) / _MS_PER_DAY);
      }

    }
  }
})(jQuery);


Alternative proof or verification of given proof of convergence in probability

Question: I am asked to show that if $X_n \rightarrow c$ in probability and if $g$ is a continuous function, then $g(X_n) \rightarrow g(c)$ in probability for a statistics homework problem in a section titled "Limit Theorems". I am not quite sure about the "in probability" part of the question affecting my proof, but here is my solution:
Let $\delta > 0$. As $X_n \rightarrow c$, there exists $N$ such that for $n > N$, $|X_n - c| < \delta$. Let $\varepsilon > 0$. As there is a $\delta > 0$ such that $|X_n - c| < \delta$ and $g$ is continuous, then by definition of continuity $|g(X_n) - g(c)| < \varepsilon$, and so $g(X_n) \rightarrow g(c)$.
My primary concerns for this solution is that I am not considering the "in probability" notion, and on top we have not learned about convergence of sequences or continuity of function in this course, nor is real analysis a prerequisite to the course so I don't believe this is the type of solution expected. I am more or less looking for the following input:

Verification of correctness or notification of flaws in the solution
Input on how to incorporate the fact that the sequence converging in probability results in the function converging with the same probability
Alternative method of proving without the $\varepsilon - \delta$ definition of continuity or hint at such a solution (because without having taken real analysis before this statistics class I would not have known the definition for a sequence to converge or for the continuity of a function)

EXTRA INFO
For the alternative solution, the following definition and theorem are provided in the text are the only things I should know for continuity in this course as of this moment:
Definition

Let $X_1,X_2,\dots$ be a sequence of random variables with cumulative distribution functions $F_1,F_2,\dots$, and let $X$ be  random variable with distribution function $F$. We say that $X_n$ converges in distribution to $X$ if \begin{equation}\lim_{n\rightarrow\infty}F_n(x) = F(x)\end{equation} at every point at which $F$ is continuous.

Theorem A - Continuity Theorem

Let $F_n$ be a sequence of cumulative distribution functions with the corresponding moment-generating function $M_n$. Let $F$ be a cumulative distribution function with the moment-generating function $M$. If $M_n(t) \rightarrow M(t)$ for all $t$ in an open interval containing zero, then $F_n(x) \rightarrow F(x)$ are all continuity points of $F$.
Answer:
Let $\delta > 0$. As $X_n \rightarrow c$, there exists $N$ such that for $n > N$, $|X_n - c| < \delta$.

It's not what convergence in probability says (even with almost sure convergence, the $N$ would depend on the considered $\omega$).
Here is a way to get a proof.

Assume first that $f$ is uniformly continuous on the real line. Let $\varepsilon>0$ and $\delta$ such that $|f(x)-f(y)|<\varepsilon$ whenever $|x-y|<\delta$. Fix $\delta>a>0$. Then $$\mathbb P\{|X_n-c|>a\}\geqslant \mathbb P\{|X_n-c|>\delta\}\geqslant \mathbb P\{|f(X_n)-f(c)|>\varepsilon\}.$$
To be reduced to the case "$f$ uniformly continuous", take $A$ such that $\mathbb P\{|X_n|>a\}$ is small, and notice that $f$ is uniformly continuous on $[-A,A]$.

An alternative way is the following: a sequence $\{Y_n\}$ of random variables converges in probability to $Y$ if and only if for each subsequence $\{Y_{n_k}\}$, we can extract a further subsequence which converges almost everywhere to $Y$.
Comment: I think uniform continuity a stronger assumption that just continuity. Continuity ($\varepsilon$-$\delta$ version) is just
\begin{equation}\forall\varepsilon>0,\exists\delta>0 \text{ s.t } |x - x_0|\implies|f(x)-f(x_0)|<\varepsilon\end{equation} where $\varepsilon$ can be a function of $x_0$ and $\delta$, but uniform continuity is\begin{equation}\forall\varepsilon>0,\exists\delta>0 \text{ s.t } |x - y|\implies|f(x)-f(y)|<\varepsilon, \forall x,y\in dom(f)\end{equation} where $\varepsilon$ is restricted to only being a function of $\delta$.
Comment: You are right, it's a stronger assumption. But we can reduce the proof to this case.
Comment: Can you explain a bit more why can we make $P\{|X_n|>a\}$ small?


If $x+y+z=6$ and $xyz=2$, then find $\cfrac{1}{xy} +\cfrac{1}{yz}+\cfrac{1}{zx}$

Question:
If $x+y+z=6$ and $xyz=2$, then find the value of $$\cfrac{1}{xy}
 +\cfrac{1}{yz}+\cfrac{1}{zx}$$

I've started by simply looking for a form which involves the given known quantities ,so:
$$\cfrac{1}{xy} +\cfrac{1}{yz} +\cfrac{1}{zx}=\cfrac{yz\cdot zx +xy \cdot zx +xy \cdot yz}{(xyz)^2}$$
Now this might look nice since I know the value of the denominator but if I continue to work on the numerator I get looped :
$$\cfrac{yz\cdot zx +xy \cdot zx +xy \cdot yz}{(xyz)^2}=\cfrac{4\left(\cfrac{1}{xy}+\cfrac{1}{zy}+\cfrac{1}{zy}\right)}{(xyz)^2}=\cfrac{4\left(\cfrac{(\cdots)}{(xyz)^2}\right)}{(xyz)^2}$$
How do I deal with such continuous fraction ?
Comment: $\frac{1}{xy} +\frac{1}{yz}+\frac{1}{zx}=\frac{x+y+z}{xyz}$.
Comment: Don't automatically think you should delete a question/answer if it is flawed. People can still learn from it. That's the beauty of making mistakes.
Comment: Is $(xyz)^2$ the lowest common denominator?
Comment: lol I've messed up so badly.Shame on me....
Comment: Do I delete the question or leave it (In case someone will do the same mistake (I know that's impossible)) ?
Comment: The least common denominator, with these symbols, is $xyz$. So your expression is $$\tfrac z{xyz}+\tfrac x{xyz} + \tfrac y{xyz}$$It should now be clear how to proceed...
Comment: @Mr.Y Too late. Someone has upvoted an answer, so you can't delete your question anymore.
Comment: @ user236182 It's fine.
Comment: Thank you guys.It's nice to get some help even when one does the dumbest of the mistakes.
Answer: $$\cfrac{1}{xy} +\cfrac{1}{yz} +\cfrac{1}{zx}=\cfrac{yz\cdot zx +xy \cdot zx +xy \cdot yz}{(xyz)^2}=\frac{x+y+z}{xyz}$$
Comment: don't worry sometime it happens.
Comment: I don't know if that's a joke.
Answer: Notice, the method is straight forward simply take L.C.M. & substitute known values as follows  $$\frac{1}{xy}+\frac{1}{yz}+\frac{1}{zx}=\frac{x+y+z}{xyz}=\frac{6}{2}=\mathbb{\color{red}{3}}$$
Answer: $$x+y+z=6$$
Divide both sides with $xyz$
$$\frac{x+y+z}{xyz}=\frac{6}{xyz}$$
$$\frac{x}{xyz}+\frac{y}{xyz}+\frac{z}{xyz}=\frac{6}{2}$$
$$\frac{1}{yz}+\frac{1}{xz}+\frac{1}{xy}=3$$
Comment: Is your third equation a typo?


What to do about an 'illegal Choice' error in views?

Question: I have a view with an exposed filter and it isnt working correctly.  I am trying to change the options on the exposed filter but all that happens is I get is 'An illegal choice has been detected. Please contact the site administrator. '  I cant find anything logging anywhere.  This error is incredibly vague and tells me nothing about which choice is illegal or what legal options may be.
Is there somewhere this may be logged that I am not looking?
Comment: Re-do the same view from scratch and see if you still end up with the same error. You can use [simplytest.me](https://simplytest.me/project/drupal) just to make sure it is not just your Drupal.
Answer:
'An illegal choice has been detected. Please contact the site
  administrator. '

This type of issue generally arises when, for example, you have 2-3 select list, select list 'second' is dependent on 'first' select list. Now if you get empty result (say) in your first select list, second select list will not get populated correctly and throw this type of exceptions.
This is just an example so that you might get an idea, the sequence of selecting values is not right.
Try the whole procedure again, and this message will get disappeared.


Adding JavaScript just to one node

Question: I have added scripts[] = js/add_property.js to my .info file which works perfectly.
That loads the JavaScript for every page though, so how do you do it so that it only loads for the .tpl that it is used in?
Answer: If you want to add the Javascript to a single node, you might also implement hook_node_view:
<?php
function my_module_node_view($node, $viewmode, $langcode)
{
  if ($node->nid == MY_NODE_ID) {
    $node->content['#attached']['js'][] = array
    (
      'type' => 'file',
      'data' => drupal_get_path('module', 'my_module') . 'js/add_property.js',
    );
  }
}
?>

(Solution adapted from https://www.drupal.org/node/1993228).
Answer: Within your theme's template.php, you could use theme_preprocess_node() to distinguish between the types of nodes you like to add your JS to. For the desired type, you can use drupal_add_js():
/**
 * Implements theme_preprocess_node().
 */
function MYTHEME_preprocess_node(&$variables) {
  $node = $variables['node'];
  if ($node->bundle == 'mytype') {
    drupal_add_js(drupal_get_path('theme', 'MYTHEME') . '/js/my-js-file.js');
  }
}

Answer: drupal_add_js

Adds a JavaScript file, setting, or inline code to the page.
Comment: Thanks I will try node_view in a minute. I had done a google before I asked and it came up with so many ways I ended up confused as to which one to try
Comment: It should have been  'data' => drupal_get_path('module', 'my_module') . '/js/add_property.js',
    ); oops there was a missing / but that worked - thanks
Answer: If you want to add js to a specific node JS injector works well
https://www.drupal.org/project/js_injector
Now I'm not sure how to set it to work on different templates....

Adding JavaScript just to one node


How do I style a node page for a specific content type?

Question: I have a content-type "foo". As you know, you can look at such a node by using it's nid and going to /node/nid. I'd like to create 100% custom markup and CSS for this page.
I want code and files and only code and files. So, what I'd expect is a MVC architecture, meaning, that there's a template file to which all the node related data is passed and I can easily change the markup. But since Drupal isn't MVC at all, it somehow gets very complex and I absolutely don't get the architecture here. I'm not sure  can use one template only; if so what's the naming pattern?
Can someone tell me the cleanest and easiest way to create the HTML for the node page?
Answer: The easiest way is to create a new template file called node--[node-id].tpl.php, for example node--10.tpl.php
Then you just need to copy/paste your code from the node.tpl.php you have on your actual theme and change it like you want. For the CSS, the cleanest way I think is to add a custom class in your template high enough to be able in your CSS to target it.
You can have a look at this answer that explains the difference between node--[node-id].tpl.php and page--node--[node-id].tpl.php


Display html text as html

Question: There's a moment in my page where I connect to a database and retrieve some data. One field in special is html code. I want to display it in my page as html inside some control that understands it.
What are my options?
Comment: Define "understands it".
Comment: I want to: 1-retrieve html code stored in the database; 2-Put this code inside a control that parses the html and shows it.
Comment: Hey man, I'm not this kind of dev. I wanna try something, but I don't know what to use to do what I wanna do. There is probably some control that does this parsing to me, but I can't find it. I don't want you to give me the code.
Answer: Use a literal.
.aspx:
<asp:Literal runat="server" ID="myLiteral" />

codebehind:
myLiteral.Text = "
 this is a h2 html tag
";

this will print out
 This is a h2 html tag
Comment: Very simple and exactly what I needed. Thank you.

Display html text as html


RNNs for Non-sequential Data

Question: From what I know, Recurrent NNs perform very well in case of sequential data. However, I have also read at many places that it can be used for non-sequential data as well. For instance in the article 'The unreasonable effectiveness of Recurrent Neural Networks' by Andrej Karpathy:

Sequential processing in absence of sequences
  You might be thinking that having sequences as inputs or outputs could be relatively rare, but an important point to realize is that even if your inputs/outputs are fixed vectors, it is still possible to use this powerful formalism to process them in a sequential manner.

From what I can make out, we model our data in a sequential form. So, if the input is a data point, the output would be the data point after t time steps, and we train our model on that data.
I came across a Kaggle problem and was wondering if RNNs can be applied to it. The problem is to classify credit card frauds. The only thing which can be modelled as a sequence, I believe, is the class of the output but then wouldn't the model be just learning the sequence of zeros and ones based on the other features? Because there is nothing which identifies a previous datapoint and can be used to learn a sequence from. So, does that mean RNNs cannot be applied to this problem because it lacks that ability to be converted into sequential form, or maybe I am missing something?
Answer: 1) Talking about output, fraud detection is a subclass of anomaly detection, so classification is a correct, but not the most appropriate view. In this case we basically learn not just zeros and ones, but the whole distribution that lies under input data-points. It gets converted to a classification problem only after applying some threshold to the output probability in order to catch some rare event, so it could be used to either analyze anomaly or apply some supervised learning algorithm.
2) The representation of the output doesn't really matter so much here. Imagine classifier that takes a word in a text and outputs if it represents an animal:
hello => 0
kitty => 1
cartoon => 0
minny => 0

It could be represented as a RNN that takes the whole text as a stream:
hello kitty cartoon minny => 0 1 0 0

And in this case it could use precedence of word "hello" as a clue that "kitty" is going to be next with higher probability
3) Talking about input and RNNs, there is actually a correspondence between RNNs and CNNs (in image processing) as CNNs convolution/cross-correlation can be viewed as a particular case of RNN. This is actually a most popular way of applying (sometimes) RNNs to a non-sequential data
4) returning back to credit card case, if every input point is not actually i.i.d (identically and independently distributed) like in "hello kitty" case, then RNN could help by identifying this connection:
ok, attacker1.attack1, ok, ok, attacker1.attack2, ok, attacker2.attack

RNN could identify connection (correlation) between attacker1.attack1 and attacker1.attack2 (for instance credit card number brute-force) faster than regular network as those attacks, regardless that they could come from different IPs - would still be close in time and have something in common (let's say some digits could repeat or have some underlying generating function). Without RNN, feedforward network would have to use parameters from only one point
TLDR; RNN (especially Long-Term-Short-Term) allow you to specify a prior belief about relationships between events in "time", so you could benefit from it if those events are actually coupled (grouped) in some way even if they're not strictly sequential. Basically RNN provides a powerful way to apply linear/non-linear cross-"correlation" (much like CNN, but more general).
Comment: @smörkex I think there is a book called "Deep Learning" by Ian Goodfellow mentioning that. Also this: https://datascience.stackexchange.com/a/11621
Comment: Can you expand or give a reference for point 3? Thanks

RNNs for Non-sequential Data


Finding the vertical shift of a sinusoidal function

Question: I'm currently studying sinusoids, I've been given a graph with a few key points and have been told to find a cosine function which fits it. When it comes to finding the vertical shift of the graph the book does so by summing the highest y-value of the graph and the lowest y-value of the graph, so the maximum and minimum values of the cosine sinusoid, which in this case are $\frac{5}{2}$ and $-\frac{3}{2}$; it averages these values to get $\frac{1}{2}$. This gives the vertical shift, but why is this? Could someone explain this to me. Thanks.
Comment: If we have a cosine graph with amplitude A, then its largest value will be A and its smallest value will be -A.  If we shift this vertically by $s$ units, then the largest value will be $A+s$ and the smallest will be $-A+s$; so adding these together and dividing by 2 gives the value of $s$.
Comment: @user84413, write this up as an answer so I can reward you for helping me, thank you!
Comment: I did as you suggested -- thanks.
Answer: If we have a cosine graph with amplitude A, then its largest value will be A and its smallest value will be -A. If we shift this vertically by s units, then the largest value will be A+s and the smallest will be −A+s; so adding these together and dividing by 2 gives the value of s.


Unjustified upper bound of Taylor expansion of exponential

Question: It's simple to see that, for $x>0$, the exponential function is lower bounded by its truncated Taylor expansion:
$$e^x = \sum_{j=0}^\infty \frac{x^j}{j!} $$
If we truncate this at $k< \infty$, we are getting a lower bound, as we are just summing over positive quantities. This gives us:
$$\sum_{j=0}^k \frac{x^j}{j!} < e^x $$
However, in this paper at page 3, after equation 2.4,  I have found the following bound. Let $\epsilon \in (0,1]$, and $a \in [-1,1]$
$$e^{\epsilon a/2} \leq 1+ \frac{\epsilon a}{2} + \frac{\epsilon^2}{6}$$
This works numerically, but I don't have any intuition for the reason it works. Let's analyze the last term, which is the most troublesome one. They obviously removed the $a$ term from the "Taylor" expansion. However it's not clear how they played with the $/2$ at the denominator. The right term I expect from Taylor would be
$$\frac{(\epsilon a /2)^2}{2!} = \frac{(\epsilon a)^2}{4*2!} = \frac{(\epsilon a)^2}{8} < \frac{(\epsilon )^2}{8} < \frac{(\epsilon)^2}{6}$$
What this bound is saying is that it bounds the remaining terms of the Taylor expansion centered in 0 of $e^{\epsilon a /2}$
$$\frac{\epsilon}{6} \geq \sum_{k=0}^\infty \frac{1}{k!}(\epsilon a/2)^k $$
From here I don t know how to proceed. I wish I could apply the convergence criteria of the gometric series, but there is $1/k!$ at the denominator that prevents me from doing it, and I don t have other ideas.
Numerically, I also checked if this works also removing the $/2$ at the denominator of the exponent. To my surprise, also the bound they propose don't  work anymore.
<code>epsilon = 1
ra=np.linspace(-1, 1, 3000)

one, two, three, four = [], [], [], []
for a in ra:
    one.append(    np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon)**2/3 )  # original
    two.append(    np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/3 )  # taylor but with 6
    three.append(  np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/2 )  # taylor right
    four.append(  np.exp(epsilon*a) <= 1 + epsilon*a + (epsilon*a)**2/2 )  # taylor right

print("orig \t\t\t", np.all(one))
print("taylor but with 6 \t", np.all(two))
print("taylor right \t\t", np.all(three))
print("taylor but with 4 \t\t", np.all(four))
</code>
Can you help me please understand the idea behind this step? Thank you.
Answer: It seems that looking at the remainder of the first-degree Taylor polynomial is not good enough to get your inequality. Looking at the second-degree polynomial seems to be barely enough to get your bound.
The Lagrange form of the remainder term $R(x)$ in $e^x = 1 + x + \frac{x^2}{2} + R(x)$ is $R(x) = \frac{1}{6} e^\xi x^3$ where $\xi$ is between $0$ and $x$.
Plugging in $x=\epsilon a / 2$ yields
$e^{\epsilon a / 2} = 1 + \frac{\epsilon a}{2} + \frac{\epsilon^2 a^2}{8} + \frac{1}{48} e^{\xi} \epsilon^3 a^3$ for some $\xi$ between $0$ and $\epsilon a/2$.
It now suffices to show
$$\frac{\epsilon^2 a^2}{8} + \frac{1}{48} e^{\xi} \epsilon^3 a^3 \le \frac{\epsilon^2}{6}.$$
Or equivalently, it suffices to show
$$\frac{a^2}{8} + \frac{1}{48} e^{\xi} \epsilon a^3 \le \frac{1}{6}.$$
This holds because $\xi \le \epsilon |a|/2 \le 1/2$ and so the left-hand side can be bounded as
$$\frac{a^2}{8} + \frac{1}{48} e^{\xi} \epsilon a^3
\le \frac{1}{8} + \frac{1}{48} e^{1/2}$$
which is just barely smaller than $1/6$.
Answer: The goal is to find $c$ such that if $|x| \leq 1$ then $\left | e^{x/2} - 1 - (x/2) \right | \leq cx^2$. (This whole mess with $\epsilon$ and $a$ is really just obfuscating that.)
From the Lagrange remainder, you have
$$\left | e^{x/2} - 1 - {x/2} \right | = \left | \sum_{k=2}^n \frac{1}{k!} \frac{x^k}{2^k} + \frac{1}{(n+1)!} e^{\xi/2} \frac{x^{n+1}}{2^{n+1}} \right |$$
where $\xi$ is between $0$ and $x$ and $n \geq 1$. (With $n=1$ the first term on the RHS is just $0$ since it is an empty sum.) So you can write
$$\left | e^{x/2} - 1 - {x/2} \right | \leq \sum_{k=2}^n \frac{1}{k!} \frac{|x|^k}{2^k} + \frac{1}{(n+1)!} e^{|x|/2} \frac{|x|^{n+1}}{2^{n+1}}.$$
You can try to increase $n$ until this works out as you want it to. With $n=1$ you have $\frac{1}{8} e^{|x|/2} x^2$ but $\frac{e^{1/2}}{8}>1/6$ so this is not good enough for your goal.
With $n=2$ you end up looking at $\frac{1}{8} x^2 + \frac{e^{1/2}}{48} |x|^3$, so it suffices to show $\frac{e^{1/2}}{48} \leq \frac{1}{6}-\frac{1}{8}=\frac{1}{24}$ i.e. $e^{1/2} \leq 2$. This follows from the same kind of argument, just rearrange $e^{1/2} \leq 1 + \frac{1}{2} e^{1/2}$.
As an aside, if $x \geq 0$ is small enough that $x^{n+1} \leq (n+1)!$, then:
$$\sum_{k=0}^n \frac{x^k}{k!} \leq e^x \leq \frac{\sum_{k=0}^n \frac{x^k}{k!}}{1-\frac{x^{n+1}}{(n+1)!}}.$$
One convenient sufficient condition for this is $x \leq \frac{n+1}{e}$.
Comment: @angryavian Fixed up. Seems to be the same conclusion you came to, that it suffices to consolidate the actual quadratic term and its remainder as error terms to get the desired result.
Comment: Sorry, where is the $16$ coming from? With $f(x)=e^{x/2}$ we have $f''(\xi)=\frac{1}{4} e^{\xi/2}$ so the Lagrange remainder is $\frac{1}{2!} \frac{e^{\xi/2}}{4} x^2 = \frac{e^{\xi/2}}{8} x^2$. Maybe I've made a mistake.
Comment: @angryavian There is the $1/2$ from Taylor, another $1/2$ from the derivative of $e^{x/2}$, and the $1/4$ from $(x/2)^2=x^2/4$. Though I actually am missing a fifth factor of 2 since the derivative in question is the second derivative. So thanks.
Comment: If you are using the Taylor series of $e^{x/2}$ then shouldn't you be plugging in $x^2$? You would plug in $(x/2)^2$ if you are using the Taylor series of $e^x$. Either way the remainder will be $\frac{e^{\xi/2}}{8} x^2$.
Comment: @angryavian Good catch, I mixed some stuff up. That seems to create a problem for this argument, because $e^{1/2}/8>1/6$. Working on a fix.


how to change the size of an exposed filter autocomplete form?

Question: the source shows:
<input id="edit-field-location-taxonomize-terms-tid" class="form-text form-autocomplete ctools-auto-submit-processed" type="text" maxlength="128" size="60" value="Germany" name="field_location_taxonomize_terms_tid" autocomplete="OFF" aria-autocomplete="list">

tried hook_form_alter but didn't find size or maxlength using drupal_set_message('
' . print_r($form, TRUE) . '
');
also
function modulename_element_info() {
  $types['textfield'] = array(
    '#input' => TRUE,
    '#size' => 20,
    '#maxlength' => 128,
    '#autocomplete_path' => FALSE,
    '#process' => array('ajax_process_form'),
    '#theme' => 'textfield',
    '#theme_wrappers' => array('form_element'),
  );
}

didn't work and don't know if/how i can change the size with css.
any help is appreciated. thanks a lot!
Comment: According to https://api.drupal.org/api/drupal/developer!topics!forms_api_reference.html/7#size, the default `#size` and `#maxlength` for a textfield are `60` and `128` and thus wouldn't necessarily be found in the form, but you can certainly set them there in that `form_alter()`....
Answer: There is currently an issue in the Claro theme to reduce the exposed filter size but until then you can use this. Replace MYMODULE, MYVIEWID, and MYBLOCK. Haven't test this but it should work. I know its a lot of code for a small change. You could also do this with CSS. Oh you also have the change uid in the form variable.
/**
 * Implements hook_form_BASE_FORM_ID_alter().
 */
function MYMODULE_form_views_exposed_form_alter(&$form, FormStateInterface $form_state, $form_id) {
   if ($form_state->getStorage()) {
     $storage = $form_state->getStorage();
     if ($storage['view'] {
       if ($storage['view']->id() == 'MYVIEWID' && $storage['display']['id'] == 'MYBLOCK') {
         // Reduce the size of the input.
         $form['uid']['#size'] = 20;
       }
     }
   }
 }

With CSS. Change #EXPOSEDFILTERID. I chose data-drupal-selector because after submitting the form, the ID changes. You should also change uid.
#EXPOSEDFILTERID input[data-drupal-selector="edit-uid"]{
  width:150px;
}


How to import external sql scripts from a sql script in PostgreSQL?

Question: I was wondering if it is possible/how to include one .sql file in another .sql file, as is done in other programming languages like C or Java?
I am asking because I wanted to organize my sql scripts into support library files and application scripts, and so on. I searched around, and solutions seem to be about loading .sql files from the psql client (see e.g. postgreSQL - psql \i : how to execute script in a given path). What I am interested in is to load a library .sql script from inside a sql script.
P.S. I am using PL/pgSQL (PostgreSQL 9.3).
Comment: Make them stored procedures.
Comment: Provide an [SSCCE](http://sscce.org/) to clarify your objective. Show the code you are trying the nest, even if it's not working.
Answer: Based on the answer It is possible to reference another SQL file from SQL script, on PostgreSQL, you can include another SQL's files just using the \i syntax. I just tested and is working good on PostgreSQL 9.6:
\i other_script.sql
SELECT * FROM table_1;
SELECT * FROM table_2;


Handling missing columns in MySQL

Question: Working on parsing a bunch of databases put together in an older, more freewheeling time into a new schema. Basically it's one database per year, with database names like foo98, foo99, foo2000, etc.
So for the most recent foo data, I can do something like
SELECT foo_person.mdname AS middle_name,
...
FROM foo_person, foo_place, foo_thing

As you get back into earlier versions of the foo database, middle name isn't stored. I've tried to build a kind of universal query, something like:
SELECT IFNULL(foo_person.mdname, "") AS middle_name,
...
FROM foo_person, foo_place, foo_thing

but MySQL complains about unknown column foo_person.mdname, which is entirely reasonable as it doesn't exist.
Is there some way to handle non-existent columns with just MySQL syntax, or will I have to write database-specific import queries?
Answer: There isn't any way of handling a non-existent column in sql (as opposed to an empty column).
You can tell whether the column is there or not using the information_schema tables, like so:
select * from information_schema.columns
    where table_name='mytable' and table_schema='mydatabase';

Answer: Yes there is a way.
Let's consider these databases

DB2009 has Person with Fname, MInitial and LName
DB2008 has Person with Fname and LName
DB2007 has Person with PersonName

You can do something similar the following (I wrote this for MS SQL Server)
/*all three columns exist*/
SELECT FName, MInitial, LName
From DB2009.Person

UNION

/*one column is a forced null */
SELECT FName, NULL as MInitial, LName
From DB2008.Person

UNION

/*two columns are derived and one column is a forced null */
SELECT SubString (1, CharIndex (PersonName, ' '), PersonName) as FirstName,
       NULL as MInitial,
       SubString (CharIndex (PersonName, ' '), len (PersonName), PersonName),
From DB2007.Person

Answer: Could you rename the tables and create views in their place with the missing columns?
Not sure if this is what you're looking for, but thought I would suggest it.
-- Here is your original table
create table t (fname varchar(30), lname varchar(30));

-- Rename it to something else
alter table t rename to tOrig;

-- Create a view with the columns its missing that you need
create view t as select fname, lname, '' as mname from tOrig;

Comment: Yes, actually, this works quite well. They're old databases and the one thing I can be sure of is that nothing is pointing at them anymore. Thanks for the lateral thinking.
Answer: If you're working with heterogenous databases, I would use database-specific import queries anyways. You might need to join some columns and remove some, and truncate some, etc.
Answer: Instead of making more complex sql queries, perhaps it would be better to
alter the foo98 and foo99 tables to add in the missing columns.
For example, to add a column of type varchar(30) called "mdname" to foo98:
ALTER TABLE foo98 ADD mdname varchar(30) AFTER first_name;

Then you can relax and use the same simple SQL query no matter which tabel is being accessed.

Handling missing columns in MySQL


R: r tweets: Error: can only select one search type. Try type = 'recent'

Question: I am using 'rtweet' package to collect tweets by location.
  rt <- search_tweets(
      "lang:en",  lookup_coords("san francisco, CA", "country:US"), n = 10000

I get the following error when I mention city, state and country.
Error: can only select one search type. Try type = 'recent'

Answer: The documentation for rtweet::search_tweets tells that you need to specify which type of tweet you are looking for. There are three options to choose from: "recent", "popular", "mixed". You need to specify them in your call.
Here is a call to search tweets that will not throw an error (the coordinates I specified are in San Francisco, and I set a radius of 40 kilometers. I also picked a term that I am certain will return matches)
first_batch <- rtweet::search_tweets(q= "Kanye",
                                     geocode = "37.773972,-122.431297,40km", n = 100, type = "recent",
                                     retryonratelimit = TRUE, parse = TRUE)

I changed your call slightly, so it doesn't throw an error, but the search was retrieving 0 tweets (I think because the lookup_coords was returning empty)
second_batch <- rtweet::search_tweets(q= "Kanye",
                             geocode = rtweet::lookup_coords("San Francisco, CA", "country:US"), n = 100, type = "recent",
                             retryonratelimit = TRUE, parse = TRUE)

I have searched for tweets based on location before, and I advise you to use the first method. Grab latitude and longitude from Google, and set up a radius. Then pass them into the function with a search term and specifying the type parameter.
Comment: Can we extract tweets for multiple locations and terms with a single query?
Comment: I have never particularly tried that, but I don't think that it is possible, given how the twitter api works. You could wrap a list of locations in a vector and then loop or apply over it.
Comment: from https://rtweet.info/, It says that multiple term search is possible. But I'm not sure about the coordinates
Comment: Also, with live streaming package in R, `rt <- stream_tweets(q = q, timeout = streamtime, geocode = "37.773972,-122.431297,40km",file_name = filename)` does not work, it returns random locations. The solution here is for getting a set of records for that particular term


Relação um para muitos entre entidades

Question: Tenho o seguinte esquema de base de dados:

Neste esquema pretende-se o seguinte:
1 user pode ter várias teams e uma team pode ter até dois utilizadores (assumi que é uma relação de muitos para muitos e depois valido se uma equipa tem ou não só dois utilizadores)
1 team pode ter vários jogos e um jogo apenas tem um home_team (team) e uma away_team (team) aqui assumi que terá então duas relações 1 para muitos (1 team tem vários jogos e um jogo apenas tem uma away_team e uma home_team).
A minha dúvida é como devo fazer as relações deles nos modelos, para, por exemplo, conseguir através de um utilizador obter todos os jogos dele, tanto away_team como home_team.
Answer: Você pode definir seus modelos assim:
<?php

// Model User
class User extends Authenticatable
{
    //  ....

    public function teams()
    {
        return $this->belongsToMany(Team::class, 'teams_users', 'user_uuid', 'team_uuid');
    }
}

// Model team
// php artisan make:model Team
class Team extends Model
{
    public function users()
    {
        return $this->belongsToMany(User::class, 'teams_users', 'team_uuid', 'user_uuid');
    }

    public function homeTeamGames()
    {
        return $this->hasMany(Game::class, 'games', 'home_team_uuid');
    }

    public function awayTeamGames()
    {
        return $this->hasMany(Game::class, 'games', 'away_team_uuid');
    }
}

// Model game
// php artisan make:model Game
class Game extends Model
{
    public function homeTeam()
    {
        return $this->belongsTo(Team::class, 'teams', 'uuid');
    }

    public function awayTeam()
    {
        return $this->belongsTo(Team::class, 'teams', 'uuid');
    }
}

Exemplo query capturando usuarios + times
$users = User::with(['teams'])->get();

Relação um para muitos entre entidades


Windows script to move files to folders based on names

Question: I did a bit of research and I didn't find anything to suit/I can't alter any script.
I would like a script/link, to move a file/folder to a folder where the first characters match.
Like I want to move 'apple.txt' to the folder 'A'
Same as I want to move the folder 'John' to the folder 'J'
Thanks for the help
Comment: Do you need a script? It would be easier to do in the Windows7 file explorer.
Comment: I have about 54 folders to do, so it would be easier (granted I can get my hands on one). Each folder branches out to sub-folders so I assume it would be quicker to go through and just run the script.
Answer: I found your post while searching on another topic, but the following Powershell script does what you want. Place it in the parent directory containing your files. It will create the folder structure based on the first letter of the file name. It's case sensitive, so an 'a' is distinct from an 'A' in the file name.

<code>$OrigFolder = ".\"
$NewFolder = ".\_Sorted to Move"

# Orphans folder, where files that return null in the regex match will be moved
# Example: file "- title.pdf"
# will be moved to ".\_Orphans" folder

$Orphans = '_Orphans' # Use the underscore to sort the folder to the top of the window

# First count the number of files in the $OrigFolder directory
$numFiles = (Get-ChildItem -Path $OrigFolder).Count
$i=0

# Tell the user what will happen
clear-host;
Write-Host 'This script will copy ' $numFiles ' files from ' $OrigFolder ' to _Sorted to Move'

# Ask user to confirm the copy operation
Read-host -prompt 'Press enter to start copying the files'

# Regex to match filenames
$Regex = [regex]"(?:([A-Za-z0-9]?)[A-Za-z0-9]?)"

# Loop through the $OrigFolder directory, skipping folders
Get-ChildItem -LiteralPath $OrigFolder | Where-Object {!$_.PsIsContainer} |
    ForEach-Object {
        if($_.BaseName -match $Regex){
        $ChildPath = $_.BaseName -replace $Regex

#Caluclate copy operation progress as a percentage
[int]$percent = $i / $numFiles * 100

# If first part of the file name is empty, move it to the '_Orphans' folder
if(!$Matches[1]){
    $ChildPath = $Orphans}
else {
    $ChildPath = $Matches[1]
    }

# Generate new folder name
$FolderName = Join-Path -Path $NewFolder -ChildPath ($ChildPath)

# Create folder if it doesn't exist
    if(!(Test-Path -LiteralPath $FolderName -PathType Container)){
    $null = New-Item -Path $FolderName -ItemType Directory}

# Log progress to the screen
Write-Host "$($_.FullName) -> $FolderName"

# Move the file to the folder
Move-Item -LiteralPath $_.FullName -Destination $FolderName

# Tell the user how much has been moved
Write-Progress -Activity "Copying ... ($percent %)" -status $_  -PercentComplete $percent -verbose
$i++
    }
}

Write-Host 'Total number of files in '$OrigFolder ' is ' $numFiles
Write-Host 'Total number of files copied to '$NewFolder ' is ' $i
Read-host -prompt "Press enter to complete..."
clear-host;
</code>

Change the regular expression in line 22 to change the matching behavior.
Hope it works for you.
Also check out the program Directory Opus. It's a very mature, well-piolished product. It handles these kinds of tasks with ease. It's a little pricey for casual use but if you manage a lot of files on a daily basis it's a godsend. I don't know how I'd do without it. Lot's of support on their forums.
--Scott


How to override/enhance file_save_upload() function of file.inc?

Question: For a specific requirement, wherein whenever an image is uploaded to a particular field of a node, a watermark should be added to it, I made changes in file_save_upload() function of file.inc
if (!drupal_move_uploaded_file($_FILES['files']['tmp_name'][$source], $file->uri)) {
    form_set_error($source, t('File upload error. Could not move uploaded file.'));
    watchdog('file', 'Upload error. Could not move uploaded file %file to destination    %destination.', array('%file' => $file->filename, '%destination' => $file->uri));
   return FALSE;
}
else{
    process_image(drupal_realpath($file->uri));
}

But these changes will be lost whenever Drupal core will be upgraded. So what is the best way to override/enhance file_save_upload() function of file.inc or any function in the core drupal?
Answer: Drupal runs on a Never hack core policy.
The Drupal way to add a watermark to image is to use hook_form_alter to modify the form submit and do your things with $form_state['values']['YOUR_FIELD']
Eg:
function mymodule_form_alter(&$form, $form_state, $form_id)  {
  switch ($form_id)  {
   case 'YOURFORMID':
     $form['#submit'][] = 'yourModule_yourform_submit';
     break;
   }
 }

function yourModule_yourform_submit($form, &$form_state) {
    // do what you want to do
    // use $form_state['values']
}

Or Try ImageCache Actions module.

Watermarking - a much requested addition that enables you to place any
  image anywhere over a source picture, with alpha transparency
Comment: There already is a module, Imagecache Actions: https://drupal.org/project/imagecache_actions
Comment: @Molot Yes, Imagecache Actions are applied via image styles, not upon upload.  However, using styles for all images and disallowing access to the original files is faster than writing a new module if that fits the use case.
Comment: So, Do I need to create a module for this particular requirement?
Comment: @PatrickKenny - Image Actions, as far as I know, are leaving original file intact and create their "after actions" copy separately? Or do I recall it wrong? Long time since I was playing with images, maybe I'm confused with 6.x imagecache, so could you clarify?
Comment: I just thought that in that case it's worth to mention that unaltered files will still be there so one needs to be careful ih he does not want them to get downloaded.


Effective data visualizations for large numbers of classes

Question: Experimenting with a few approaches to visualize error modes/rates for models with a large number (100-500) of classes. For smaller numbers of classes have been using ROC graphs (with each class represented as a separate line) confusion matrices, and some simple bar charts just displaying accuracy.
When it comes to hundreds of classes though, these visualizations are generally ineffective at conveying information. Have been looking at confusion matrix variants such as CMO (confusion matrix ordering), and some other confusion-matrix-centric options. Things like displaying groups of classes with high mutual confusion rates, and pulling out strong vertical and horizontal lines.
However, outside of expecting people to scroll through individual error rates for hundreds of classes, or the confusion-matrix variants above, I haven't found any techniques that are effective for hundreds of classes.
Have people found certain visualizations that are successful in these cases? I'm specifically dealing with classification in this case, though I'm dealing both with single-label and multi-label scenarios (where individual class labels either are or are not mutually exclusive). However, suggestions for other types of problems that may be adapted to classification would also be very helpful.
Comment: Here: https://stats.stackexchange.com/search?q=visualize+many+categories is a list of some similar posts, a few of them with answers.
Comment: @kjetilbhalvorsen unfortunately only two that are the same, neither with answers :(
Answer: If you can look at only one number per category, there are some bar chart extensions to handle 100s of categories: wrapped bars, piled bars, packed bars, and combined "other" bars. Additionally treemaps and 1-D labeled scatter plots can work. All of those work best when the categories are ordered by a ranking, but I think wrapping could work with less space efficiency with any ordering if you need, say, alphabetical lookup. Labeling is always a challenge -- some of these sacrifice labeling of smaller items (relying on hover only).
For something more than one number per category, there is the small multiples technique. Here is a grid of 400 ROC curves, with background color representing some metric such as area.

Labeling is still a challenge. For a static view, one could label each row and column and provide a legend for the category of each row-column combination.


Build confusion matrix for cross validated results?

Question: I am using python , and I want to know how to build a confusion matrix after I have cross validated my dataset.
If build a confusion matrix at each fold then I have too many confusion matrices. I want one final CM with the right number of cases(not additive) as a final output.
I am currently using cross_val_score from scikit-learn to do my cross validation.
Answer: You could compute all the confusion matrices, and then compute the mean and standard deviation for each entry. You could then report a summarized confusion matrix of means $\pm$ standard deviations.
As long as you explain how you have computed the matrix you report, so that your readers can understand what it means, it should be ok!


How can I record audio and play other tracks as backtrack at the same time?

Question: I'm using a Nord Stage and a Windows 10 laptop, and I want to record an audio (not MIDI) track on top of existing tracks. So I'd like to hear the keyboard and the backtrack, and record just the keyboard.
I tried to just connect the keyboard to the line in. But then I get a terrible latency. Then I found ASIO4ALL drivers, and using that in combination with Ableton Live, the latency problem is solved. But ASIO4ALL uses the output exclusively so then I can hear either just my keyboard input or just the backtracks, not both at the same time.
I've also got a simple Behringer XENYX 502. Perhaps I could use the tape in and out ports to mix keyboard in and e.g. phone in on which I play some backtrack. But that's very inconvenient because each time I want to update the backtrack with a newly recorded track I've got to transfer it to my phone first.
Is an external audio interface going to help me out here? There's a huge price range and a lot to choose from, what would be a simple, cheap solution to my challenge?
Comment: When you have the track you are recording to record armed in Ableton live, you can't hear your keyboard?
Comment: @ToddWilcox I can, but I cannot hear the backtrack at the same time when using ASIO4ALL drivers.
Answer: if you want to hear the audio of the backtrack, be sure to put it in the same output. as your question explains, you play the backtrack outside the DAW. This happens due to an exclusive override of ASIO4ALL to the soundcard driver.
two possible cheap solution:
-to import your backtrack into the DAW.
-to buy usb soundcard and make it as the windows output. the cheap one is: http://www.ebay.com/itm/Nw-Blue-External-USB-2-0-to-3D-Virtual-Audio-Sound-Card-Adapter-Converter-7-1-CH-/300715727779
It is a lot better to use an audio interface for recording audio.and onboard soundcard is highly not recommended for professional recording.
focusrite, m-audio and presonus soundcards are available at a low price. you should check it out.
Comment: I ended up using my m-audio midi keyboard and investing some time learning vst's. Much lighter setup and works great. Thanks anyway.
Comment: you're welcome. and good luck for you.


Matplotlib interactive plots crash if mouse scroll used on plot

Question: This issue seems to happen any time I plot data using plt.show() with Matplotlib on my Macbook Pro (OSX 10.13.6).  If I create a Pandas dataframe and plot the data with Matplotlib, then show the result on the screen in the interactive window (using plt.show), the window (and matplotlib) will often crash.  It will always happen if the mouse gesture for scroll-up or down is done on the screen.  Other times it seems to happen at random.
When the plot crashes i get this as a traceback:
<code>Traceback (most recent call last):
File "./plot_lc_vs_gnss.py", line 117, in <module>
  main()
File "./plot_lc_vs_gnss.py", line 28, in main
  plt.show()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/pyplot.py", line 253, in show
  return _show(*args, **kw)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 208, in show
  cls.mainloop()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/matplotlib/backends/_backend_tk.py", line 1073, in mainloop
  Tk.mainloop()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tkinter/__init__.py", line 557, in mainloop
  _default_root.tk.mainloop(n)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
</code>
The data I am reading in is a simple CSV that was previously generated by pandas as well.  I am generating the plot with a simple script like this:
<code>#!/usr/local/bin/python3
import matplotlib
matplotlib.use('TkAgg')
import json
import os
import numpy as np
import pandas as pd
import pymap3d as pm
import matplotlib.pyplot as plt

def load_csvs():
gnss = pd.read_csv('di3_d4_1017_gnss.csv')
ins = pd.read_csv('di3_d4_1017_ins.csv')

return gnss, ins

def plot_cdf(gnss, ins):

data = [gnss, ins]

    plt.figure(figsize=[12,9])
    ax = plt.subplot(1,1,1)

    for ds in data:
        if ds[err].any != np.nan:
            dsorted = np.sort(ds['horizontal_error'])
            yvals = np.arange(len(dsorted)) / float(len(dsorted) - 1) * 100
            ax.plot(dsorted, yvals)
    ax.grid()
    ax.set_xlabel('Horizontal Error (m)')
    ax.set_ylabel('Percent of Epochs')
    title = 'Drive Test data \n DI-3 (roof) d4_1017 \n CDF Horizontal Error'
    plt.title(title)
    plt.legend(['gnss', 'ins'], loc='lower right')

def main():
gnss, ins = load_csvs()

plot_cdf(gnss, ins)
plt.show()

if __name__ == "__main__":
main()
</code>
Any advice on how to fix this issue would be greatly appreciated.
Answer: I guess by now it does not matter, but this seems to be an unresolved problem under discussion in #9637
Comment: thanks for this, although I changed to using [Bokeh](https://bokeh.pydata.org/en/latest/) for my plotting months ago and haven't looked back.  a bit more overhead at first but I think the plots look better and they're output in HTML so can be opened anywhere.


Not detecting Cookie on Vue Page

Question: Whenever the user make an API request I check if the session is valid
userCookie, err := r.Cookie("session-id")
    if err != nil {
        fmt.Println(err)
        if strings.HasPrefix(r.URL.Path, "/login") {
            Login(w, r, db) //login and create session
        }
   } else {
     //Authenticate Session Then if Valid proceed to my APIs
   }

So when I consume an API directly on the browser's search bar it works because it detects the session cookie, but when I consume it on Vue/axios it doesn't detect the cookie and it gives me this error:
http: named cookie not present
exit status 1

When I create a session I set the cookie Path: "/" so the session cookie is also generated to my Vue page even though my server and front end have different ports. So how can I make the server detect my session cookie on my Vue page?
Answer: You need to set withCredentials to send cookies on CORS requests (and since you have a different port, it is treated as if it were a different server).
axios.interceptors.request.use(
  function(config) {
    // Do something before request is sent
    config.withCredentials = true;
    return config;
  },
  function(error) {
    // Do something with request error
    return Promise.reject(error);
  }
);

or
axiox.get('/', { withCredentials: true })

Not detecting Cookie on Vue Page


How to make operations more less divided?

Question: I have a datatable (matricule, salary, number day and premium) with checkboxes and button validate, I want to calculate the sum value when I click on button validate but there is an error it gives me NaN for this value.

    <input type="checkbox" id="check_all">
    matricule
    salary
    number day
    premium

    <input type="checkbox" class="checkbox">
    1
    5000
    2
    <input type="text" name="prime" class="form-control" value="0">

    <input type="checkbox" class="checkbox">
    2
    6000
    2
    <input type="text" name="prime" class="form-control" value="0">

    <input type="checkbox" class="checkbox">
    1
    7000
    1
    <input type="text" name="prime" class="form-control" value="0">

  <button class="btn btn-success add-all" type="submit" id="hide">Pointage men</button>

jQuery code for check all and get some values and calculate sum:
$(document).ready(function() {
  $('#check_all').on('click', function(e) {
    if ($(this).is(':checked', true)) {
      $(".checkbox").prop('checked', true);
    } else {
      $(".checkbox").prop('checked', false);
    }
  });
  $('.checkbox').on('click', function() {
    if ($('.checkbox:checked').length == $('.checkbox').length) {
      $('#check_all').prop('checked', true);
    } else {
      $('#check_all').prop('checked', false);
    }
  });

  // jquery code for display array :
  $("#hide").click(function() {
    var items = [];
    $("tr").each(function(i, r) {
      if (i > 0 && $(r).find("input").first().prop("checked")) {
        //sum = (((salary/24)*nbre)+premium)
        let sum = ((($(r.cells[2]).innerText / 24) * $(r.cells[3]).innerText) + $(r.cells[4]).find('input').val());

        items.push({
          "matricule": r.cells[1].innerText,
          "salary": r.cells[2].innerText,
          "nbre": r.cells[3].innerText,
          "premium": $(r.cells[4]).find('input').val(),
          "sum": sum
        })
      }
    });
    console.log(items);
  });
})

Answer: Replace innerText with text() and use parseInt for calculation.

$(document).ready(function() {
  $('#check_all').on('click', function(e) {
    if ($(this).is(':checked', true)) {
      $(".checkbox").prop('checked', true);
    } else {
      $(".checkbox").prop('checked', false);
    }
  });
  $('.checkbox').on('click', function() {
    if ($('.checkbox:checked').length == $('.checkbox').length) {
      $('#check_all').prop('checked', true);
    } else {
      $('#check_all').prop('checked', false);
    }
  });

  // jquery code for display array :
  $("#hide").click(function() {
    var items = [];
    $("tr").each(function(i, r) {
      if (i > 0 && $(r).find("input").first().prop("checked")) {
        //sum = (((salary/24)*nbre)+premium)
        let sum = (((parseInt($(r.cells[2]).text()) / 24) * parseInt($(r.cells[3]).text())) + parseInt($(r.cells[4]).find('input').val()));

        items.push({
          "matricule": r.cells[1].innerText,
          "salary": r.cells[2].innerText,
          "nbre": r.cells[3].innerText,
          "premium": $(r.cells[4]).find('input').val(),
          "sum": sum
        })
      }
    });
    console.log(items);
  });
})

    <input type="checkbox" id="check_all">
    matricule
    salary
    number day
    premium

    <input type="checkbox" class="checkbox">
    1
    5000
    2
    <input type="text" name="prime" class="form-control" value="0">

    <input type="checkbox" class="checkbox">
    2
    6000
    2
    <input type="text" name="prime" class="form-control" value="0">

    <input type="checkbox" class="checkbox">
    1
    7000
    1
    <input type="text" name="prime" class="form-control" value="0">

  <button class="btn btn-success add-all" type="submit" id="hide">Pointage men</button>

Note: You can check input value is number or not using $.isNumeric.


Responsividade HTML/CSS

Question: Estou ajustando um código, e não consigo de forma alguma, fazer com que as setinhas de prev e next no carrosel, apareçam no mobile! Já ajustei pelas media queries, mas nada resolve!
O codigo é extenso pra colocar aqui, então, quem puder dar uma olhadinha, ele está publicado aqui:
www.extudando.esy.es/hotsite
e pode baixar os arquivos aqui:
www.extudando.esy.es/hotsite/hotsite1.zip
Comment: Primeiro de tudo. Não dá pra saber se é problemas com Media Query. Pode ser uma propriedade que se aplica na função JS que cria o Carrossel. Qual biblioteca você está usando para criar o carrossel?
Comment: Pelo que vi ele está escondendo as setas com CSS Inline. Então provavelmente quem está fazendo isso é o próprio script da biblioteca. Por isso deve ter uma propriedade que dê para escolher isso. Se você colocar uma classe `.slick-slider .slick-arrow{
display: block!important;
}` funcionaria, mas não é uma boa prática.
Comment: Infelizmente ainda assim não funcionou =/
Comment: Isso provavelmente ocorre pq a interação em mobile é arrastando com o dedo e não clicando. No plugin que vc usou provavelmente em uma opção para habilitar as setas em telas pequena. Da uma olhada na documentação do plugin e veja o Script de ativação que lá deve ter a opção de ativar as setas.
Answer: O responsável por isso é a propriedade arrows, deve ser true quando for de determinado tamanho. Você especifica.
$('.responsive').slick({
    dots: true,
    infinite: false,
    speed: 300,
    slidesToShow: 4,
    slidesToScroll: 4,
    responsive: [{
        breakpoint: 320,
        settings: {
            arrows: true
        }
    }]
});

Responsividade HTML/CSS


Discrete Mathematics book for kenneth rosen manual solution

Question: I'm looking for the manual solution for this book
Discrete Mathematics and its applications for Kenneth H. Rosen 7E Global edition

Where can I find it?
Comment: I suspect that the solutions manual is legally available only to instructors who have adopted the text. You can ask here about particular exercises that puzzle you. Each time you do, show us what you tried and where you are stuck,
Answer: You may have some chance in this link http://www.academia.edu/7558736/Solution_Manual_of_Discrete_Mathematics_and_its_Application_by_Kenneth_H_RosenOf course it requires registering but finally I could download a file containing some solution. Check whether it is useful.
Comment: the file which you downloaded has the solutions for the even numbers questions because the book has the solutions for the odd numbers questions.

just wanted to inform you in case you was looking for the odd questions answers.

thank you for the link.
Comment: Thanks for the hint!!


How do you eyeball the eigenvectors?

Question: I've been playing around--for longer than I care to admit--on this amazing site that allows you to visualize matrix transformations, determinants, and eigenvectors in order to develop an intuition of how they work.  I've gotten a great intuition of matrix transformations and determinants, but I'm having a really hard time with eigenvectors.  I understand the computation just fine.  I also understand the "words" explaining the direction concept just fine.  I even see that when I draw a vector that aligns with an eigenvector, that vector's direction doesn't change after the transformation.  However, I'm just not getting the visceral understanding of the positioning of the eigenvectors and how they relate to the transformed basis.  My gut is telling me that the transformed basis vectors define some "maximal direction of shear" and this can be used to find the direction of the eigenvectors.
QUESTION:  Is there a geometrical construction or (even better) a way to "eyeball" the eigenvectors when you know the transformed basis vectors?  An intuitive explanation (along with a drawing) for the non-mathematician is appreciated.  ELI5 plz

ASIDE:   One thing that I have noticed is that when I keep $j$ fixed and increase the magnitude of $i$, one of the eigenvectors seems to be "pulled" towards the $i$.  The other eigenvector always points in the direction of $j$.  Thus, there seems to be some interplay between the relative magnitudes of the transformed basis vectors and the angle between the original basis and the transformed basis.

12/29/2022
I left $j$ unchanged.  I systematically altered $i$ to look for patterns in how eigenvectors change w.r.t changes in $i$.  One eigenvector always remained parallel to $j$, so I only examined the effect on the other.
I first rotated $i$ from $0$ to $90$ degrees and found that the eigenvector sweeps an arc of a circle from $-90$ to $-45$ degrees.
Next, I scaled $i$ by varying factors from $0.5$ to $1000$ and rotated it as before after each scaling.  I found some interesting patterns that I'm still pondering on.
Perhaps this will stimulate some thoughts.
Plot of systematically rotating i from 0 to 90 degrees.
Comment: Try this:  https://www.youtube.com/watch?v=PFDu9oVAE-g
Comment: Having seen your graphics, you might be interested by this question of mine about families of eigenvalues of a pencil of matrices : https://math.stackexchange.com/q/2611435/305862
Comment: Thanks, Jean Marie!  This is very useful.
Answer: Here is a way to "eyeball" eigenvectors and eigenvalues as well in the particular case of symmetric matrices.
$$\pmatrix{a&b\\b&c}\pmatrix{x\\y}=\pmatrix{x'\\y'} \tag{1}$$
It sticks to  your "maximal direction of shear" intuition.
Let us consider the image of the unit circle under the transformation which is an ellipse $\frak{E}$.
The main result (see explanation below) is that the major and minor axes of ellipse $\frak{E}$ are directed by the eigenvectors of the matrix.
Moreover, one can also visualize the eigenvalues as being the values of the semi-axes $a=\lambda_1, b=\lambda_2$ of ellipse $\frak{E}$ which means that with respect to the orthonormal axes $X-Y$ defined by the eigenvectors, the equation of the ellipse is :
$$\frac{X^2}{\lambda_1^2}+\frac{Y^2}{\lambda_2^2}=1$$

Remark: Here, we have taken matrix :
$$\pmatrix{2&2\\2&5} \ \ \text{with eigenvalues} \ \ \lambda_1=6, \  \lambda_2=1$$
$$\text{with associated eigenvectors} \ \ \pmatrix{1\\2}, \pmatrix{-2\\1}$$
Explanation: Using the inverse relationship of (1) :
$$\pmatrix{d&e\\e&f}\pmatrix{x'\\y'}=\pmatrix{x\\y} $$
constraint $x^2+y^2=1$ gives:
$$(dx'+ey')^2+(ex'+fy')^2=1$$
which is the equation of a conic curve and it is easy to prove that it is the equation of an ellipse.
Comment: Isn't this rather a method of eyeballing singular vectors?
Comment: @John Madden You pinpoint an interesting point but the "rather" isn't exact. Plainly, singular vectors of $B$ enter into this frame because they are eigenvectors of *symmetric* matrix $B^TB$...
Comment: Thank you for this thoughtful response +1.  I will wait 2 days for an answer that generalizes to non-symmetric matrices before accepting.  You clearly show how the "maximum direction of shear" points in direction of the $(1, 2)$ eigenvector; and for the symmetric matrix, the other eigenvector is orthogonal to it.  To close the loop, are you saying that we can "eyeball" the eigenvectors by first drawing an ellipse based on 3 facts:  center of the ellipse is the origin, the basis vectors are two known points $(2, 2)$ and $(2, 5)$ on the ellipse, and one of the axes will always be unit length?
Comment: Not these vectors but the following unit vectors constituting the orthonormal basis $\tfrac{1}{\sqrt{5}}(1,2)$ and $\tfrac{1}{\sqrt{5}}(-2,1)$.
Answer: The answer to your question is: not really.
The visualizer linked to in the question is not the most general; it only allows for real values in the linear transformation. Furthermore, it is misleading in a sense: the initial basis vectors are always being displayed as an orthonormal basis. Finally, another point to keep in mind is that eigenvectors are not unique up to a scalar.

Is there a geometrical construction or (even better) a way to "eyeball" the eigenvectors when you know the transformed basis vectors?

Is really a question about geometry, but in order to talk about geometry we need to introduce some additional structure, like an inner product. Because you ask for a geometrical intepretation of the eigenvectors, but eigenvectors are not unique up to a scalar, any geometric answer you come up with will really have to do with the eigenrays.
Consider the linear transformation $L$ which sends the vector $\hat{e}_1$ to $2\hat{e}_1$ and sends $\hat{e}_2$ to $3\hat{e}_1.$ In the usual basis,
$$L=\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}.$$
Now let $a$ and $b$ be arbitrary vectors, and consider the linear transformation $R$ which sends $a$ to $2a,$ and $b$ to $3b.$ In the basis of these eigenvectors
$$R=\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}.$$
So is there really anything fundamentally different between these two transformation? No. They are different transformations, of course, but a priori there is no reason to prefer one set of basis vectors to another.
Of course, if we introduce some additional structure, then we can ask extra questions. Are the vectors $\hat{e_1}$ and $\hat{e_2}$ orthogonal? Do they have length $1?$ What about $a$ and $b?$
Jean Marie's answer to this question takes advantage of the fact that we live in Euclidean space, so symmetric matrices are intimately connected to rotation matrices. Thus it seems like there is a natural way to understand the eigenvalues/vectors of these matrices. Which there is, of course, but it has less to do with the eigenvalues/vectors and more to do with the rotational geometry implicit in the display we have assumed.
Comment: Thanks for your insights.  If linear transformations represent rotations, stretching, and shearing of space then intuitively we should be able to guess where vectors in that space are only being stretched.  Let's forget the rigor of math for a moment and liken linear transformations to daily things.  For example, can we consider a "rectangle of Play-Doh", subjecting it to some linear transformation (other than rotation), and then asking where are the lines only getting stretched?

How do you eyeball the eigenvectors?


Why is drop shadow offset to the left?

Question: I am trying to create a drop shadow on a div that is compatible with IE 7 & 8. I only want the drop shadow to appear below the div (not on the top, left, or right borders). Using DXImageTransform.Microsoft.Shadow() I am setting the X offset to 0 but the drop shadow still appears on the left border of my div in IE 7 & 8 only. Any ideas why?
#Container {
background-color: white;
position: relative;
z-index: 1;
width: auto;
zoom:1;
filter: progid:DXImageTransform.Microsoft.Shadow(color='grey', OffX=0, OffY=5, Strength=5);
-webkit-box-shadow: 0px 5px 5px grey;
-moz-box-shadow: 0px 5px 5px grey;
box-shadow: 0px 5px 5px grey;

}
Answer: When you blur an image, it gets larger. Even if you offset 0px on the horizontal axis (as you are doing), the larger blur will appear past the left of your content.
Comment: Thank you. Does that mean that I cannot somehow only show the drop shadow on the bottom border? If I can only move the shadow horizontally and vertically I'm not clear how that can be accomplished.
Comment: I think I am using the wrong filter altogether. I should be using DXImageTransform.Microsoft.dropshadow(). Yes? However this will not create the blurred effect.

Why is drop shadow offset to the left?