How to find exact row where SQL query fail

Question: I'm using SQL Server 2008 R2. I have 2 tables old and new and about 500k rows.
I need to convert data from old to new. Some columns were changed. For example in old table many columns are of type varchar and in new table int.
I'm executing query like this:
INSERT INTO new (xxx)
    SELECT FROM old (yyy)

And get following error:

Msg 245, Level 16, State 1, Line 4
  Conversion failed when converting the nvarchar value 'Tammi ' to data type int.

This error shows, that in old table are some rows with wrong data in columns. (Human factor).
But how can I find these wrong rows? Is it possible?
How can I find in what column wrong data is present?
Comment: check this answer - http://stackoverflow.com/q/10517777/1080354
Comment: Your query shows only a *single* column used, `yyy`.
Answer: This is a pain.  But, to find values that cannot be converted to ints, try this:
select yyyy
from old
where yyyy like '%[^0-9]%';

In SQL Server 2012+, you can use try_convert():
select yyyy
from old
where try_convert(int, yyyy) is null;

Comment: @Lari13 so what? :) do that 50 times.
Comment: I have more than 50 columns in `old` table.:)
Comment: @Lari13 if you have 50 columns with *text* data you shouldn't be trying to force them into integer columns. You should analyze your data first and decide whether the data is in error or you simply shouldn't be using integer columns. SSIS has tasks that can analyze sources without any effort from your part
Comment: That was the question, how to find `wrong rows` fast. I've found answer here https://www.mssqltips.com/sqlservertip/1522/searching-and-finding-a-string-value-in-all-columns-in-a-sql-server-table/
Comment: @Lari13 actually, that's the same as Gordon's first query. Your original question only mentioned a *single* column
Answer: Could you execute the code that this T-SQL statement generates (just change the table name):
DECLARE @TableName SYSNAME = 'DataSource'

SELECT 'SELECT * FROM ' + @TableName + ' WHERE ' +
STUFF
(
    (
        SELECT 'OR ISNUMERIC([' + name + '] + ''.e0'') = 0 '
        FROM sys.columns
        WHERE object_id = OBJECT_ID(@TableName)
        FOR XML PATH(''), TYPE
    ).value('.', 'VARCHAR(MAX)')
    ,1
    ,3
    ,''
);

For, example, if we have the following table:
IF OBJECT_ID('DataSource') IS NOT NULL
BEGIN
    DROP TABLE DataSource;
END;
GO

CREATE TABLE DataSource
(
    A VARCHAR(12)
   ,B VARCHAR(12)
   ,C VARCHAR(12)
);

GO

INSERT DataSource ([A], [B], [C])
VALUES ('1', '2', '3')
      ,('0.5', '4', '2')
      ,('1', '2', 'A');

GO

The script will generate this statement:
SELECT * FROM DataSource WHERE ISNUMERIC([A] + '.e0') = 0 OR ISNUMERIC([B] + '.e0') = 0 OR ISNUMERIC([C] + '.e0') = 0

returning two of the rows (because A and 0.5 cannot be converted to int):


Mounting an SSD

Question: I have recently got a 'dedicated GAME server' from OVH and I've been promised 2x480GB SOFT SSDs.
However, df -h shows this output:
root@dedi:/home/shavit# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        20G  4.2G   14G  24% /
devtmpfs         63G  4.0K   63G   1% /dev
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none             13G  868K   13G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none             63G  1.1M   63G   1% /run/shm
none            100M     0  100M   0% /run/user
/dev/md2        421G  392G  7.5G  99% /home

I suppose that /dev/md2 is one of the 480GB SOFT SSDs I've been promised to have.
We're at 99% disk usage at this moment and are very scared of possible data loss, I'm afraid that we don't have the other disk mounted.
root@dedi:/home/shavit# fdisk -l

Disk /dev/sdb: 480.1 GB, 480103981056 bytes
255 heads, 63 sectors/track, 58369 cylinders, total 937703088 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1768c0f0

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        4096    40962047    20478976   fd  Linux RAID autodetect
/dev/sdb2        40962048   936648703   447843328   fd  Linux RAID autodetect
/dev/sdb3       936648704   937695231      523264   82  Linux swap / Solaris

Disk /dev/sda: 480.1 GB, 480103981056 bytes
255 heads, 63 sectors/track, 58369 cylinders, total 937703088 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3767be62

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        4096    40962047    20478976   fd  Linux RAID autodetect
/dev/sda2        40962048   936648703   447843328   fd  Linux RAID autodetect
/dev/sda3       936648704   937695231      523264   82  Linux swap / Solaris

Disk /dev/md2: 458.6 GB, 458591502336 bytes
2 heads, 4 sectors/track, 111960816 cylinders, total 895686528 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md2 doesn't contain a valid partition table

Disk /dev/md1: 21.0 GB, 20970405888 bytes
2 heads, 4 sectors/track, 5119728 cylinders, total 40957824 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

I'm a complete noob with everything related to system operating and as I said I'm afraid of losing data so I'd prefer to not mess with anything by myself as I don't have enough physical disk space on my PC to even backup data on.
Could anyone please help me mount the unmounted SSD without losing data? I feel lost.
Thanks!
Edit: cat /proc/mdstat
root@dedi:/home/shavit# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] [faulty]
md1 : active raid1 sda1[0] sdb1[1]
      20478912 blocks [2/2] [UU]

md2 : active raid1 sda2[0] sdb2[1]
      447843264 blocks [2/2] [UU]
      bitmap: 3/4 pages [12KB], 65536KB chunk

unused devices: <none>

cat /proc/swaps
root@dedi:/home/shavit# cat /proc/swaps
Filename                                Type            Size    Used    Priority
/dev/sdb3                               partition       523260  15476   -1
/dev/sda3                               partition       523260  0       -2

Answer: It's really hard to read your /proc/mdstat in a comment, but it shows that md1 is a raid1 array of sda1 and sdb1, and that md2 is a raid1 array of sda2 and sdb2. And the last two partitions are used for swap.
So you are using both discs.
I guess OVH set up like that because raid1 often gives more speed, and on game servers you typically want speed. - And considering how big a complany OVH is, it is probably documented on their site, that this is what you get.
Comment: But `fdisk -l` shows two 480GB SSDs, why do I see just one drive of over 400GB in `df -h`?
Comment: Do you even know what raid1 means?
Answer: Michael really got here first, and Henrik has already posted a perfectly good answer, but you still seem to have questions, so:
You have two 480GB SSDs.  Each has been partitioned into three, /dev/sd[ab][123].  The two 1 partitions (20GB each) are mirrored as /dev/md1, which as the root file system on it.  The two 2 partitions (around 450GB each) have been mirrored as /dev/md2, which has the /home file system; after conversion (binary vs denary), file system overhead, and reserved space, this translates to about 400GB usable.  The two 3 partitions (of about half a gig each) have been made available as a striped swap device of total capacity about a gig.
This means that

You don't have any unused disc space.  You can probably make a few more free gig on /home by tuning down the free space (tune2fs -m 2 /dev/md2), but apart from that, you need to tidy up /home to avoid filling it up.
If either SSD fails, the machine will almost instantly reboot (as half of swap will go away), but provided the boot blocks have all been set up correctly, it will reboot without any persistent data loss.  This is a fairly normal way to set up a machine.
You could break the mirrors, and free up /dev/sdb for actual FS usage.  This is fragile, I wouldn't choose to do it, and you're going to want to get an expert in to help you, but it's an option if you must have more disc space right now, and you have no other options.
Comment: This is what comes out of OVH's installer if you choose to RAID your drives. Personally I can't stand their system and always do my own installs.
Comment: @MichaelHampton that would explain why the OP wasn't aware (s)he'd made the choice.  As I said, it's a fairly normal way to set up drives; I'd do it somewhat differently, and it's interesting that you would, too, but it's not a crazy default, I think.

Mounting an SSD


Output specific rows from csv

Question: I have a very large (500mb) csv file and another textfile containing one word per line.
When I find a word in the textfile in the csv in a specific collumn I want to output the line (will redirect output to a file). I had a grep solution using grep -iE "(word1|word2|word3|...|wordn)" but it crashed after using 16gb memory with a segfault and only extracted very few entries.
I tried csvtool but it refuses to produce anything valueable.
The solution must work under linux and optimally not cap on memory.
Answer: Use awk. Process the file with one word per line first, save its data in an array and for the second file check the specific column using the expression if (column in array):
awk '
    BEGIN {
        FS = OFS = ",";
    }
    FNR == NR {
        data[ $1 ] = 1;
        next;
    }
    FNR < NR {
        if ( $4 in data ) {
            print $0;
        }
    }
' textfile large_csv_file

Comment: sorry I have never used awk. Where do I specify the column I want to check? my guess is if($4 in data){ . is the $4 the column number? if yes, is it zero-indexed starting at column 0?
Comment: @Birei Worked and was lightning fast. One little error: Instead of FNR < NR I had to use NF > 1. Thank you.

Output specific rows from csv


Pegar tipo de conexão do usuário em javascript

Question: como posso pegar o tipo de conexão do usuário em javascript? Quero saber se ele está acessando meu site via wireless, 4G, internet cabeada...
Tudo isso em javascript
Answer: navegator.connection()

Aqui está um breve resumo de utilização
if(navigator.connection) {
    if((navigator.connection.downlinkMax && navigator.connection.downlinkMax>1)
      || navigator.connection.type=='wifi') {
      document.body.classList.add('hifi');
    }
    else {
      document.body.classList.add('lofi');
    }
}

Agora um retorno da função
console.log(navegator.connection)

é similar a isso
NetworkInformationdownlink: 2.85
effectiveType: "4g"
onchange: null
rtt: 150
__proto__: NetworkInformation

Answer: Você pode usar a "Network Information API" disponível em navigator.connection.
O suporte desta API ainda esta baixo e navegadores desktop (ou mobile) como Firefox, Safary e Edge ainda não a suportam outros suportam parcialmente alguns de seus recursos.
Entre algumas de suas opões você pode ter acesso a:

downloadlink: representa a capacidade da taxa de download da conexão
rtt: tempo médio de ida e volta de uma requisição (latência)
effectiveType: 2g, 3g, 4, slow-2g (definido pela média rtt e downloadlink)
ConnectionType: bluetooth, cellular, ethernet, mixed, none, other, unknow, wifi, wimax (tipo de conexão)
onchange: função para monitorar mudanças na conexão

Você pode monitorar mudanças na conexão através de um "handler" ou mesmo observando o evento change:
function changeHandler(e) {
    //
}
navigator.connection.onchange = changeHandler;

navigator.connection.addEventListener('change', changeHandler);

TESTE:

if ( navigator.connection ) {
    console.log(navigator.connection)
} else {
    console.log('Seu navegador não suporta a "Network Information API"')
}

Suporte caniuse.com

Link para a Especificação


Is this Proof of (P→Q)→((Q→R)→(P→R)) based on Lukasiewicz Axiom System for CPL Correct?

Question: Given Lukasiewicz axiom system for Classical Propositional Logic (CPL):
(L1)    α→(β→α)
(L2)    (α→(β→γ))→(α→β)→(α→γ)
(L3)    (¬α→¬β)→(β→α)
and the usual Modus Ponens, does my proof of (P→Q)→((Q→R)→(P→R)) below correct? Can someone point out where I made mistake?
(1) (P→R)→((Q→R)→(P→R))     (L1)
(2) (P→(Q→R))→((P→Q)→(P→R)) (L2)
(3) (Q→R)→(P→(Q→R))         (L1)
(4) (P→Q)→(P→R)             (2,3 MP)
(5) (Q→R)→(P→R)             (1,4 MP)
(6) (P→Q)→((Q→R)→(P→R))     (4,5 MP)
Comment: How does your step (4) work?  The only deduction I can see from (2), (3) is the derived rule HS giving $(Q\to R)\to((P\to Q)\to(P\to R))$.
Comment: You can find the proof in the answer to this [post](https://math.stackexchange.com/questions/1069497/prove-lnot-c-implies-lnot-b-implies-b-implies-c-without-the-deductio) : T2.
Answer: No. Steps 4,5, and 6 are all not correct applications of MP


Lotus Notes JavaScript Get user info

Question: User visit web site on it's own PC and javaScript creates NotesSession.
var ns = new ActiveXObject("Lotus.NotesSession");
ns.Initialize(pass); // user password

I would like to get user info such as name and corporate phone number after successful session initialize. For message like "Hello %username%, your phone is %number%" I know way to get info about specific user from Domino server if I know name or something else, but in this case I'm stuck a bit.
If I try to use GetDatabase InternetExplorer hangs.
var db = ns.GetDatabase("", "names.nsf");

Get internet address will be good too
Comment: You may check out Session class http://www.ibm.com/support/knowledgecenter/en/SSVRGU_9.0.0/com.ibm.designer.domino.main.doc/H_10_NOTES_CLASSES_ATOZ_JAVA.html
Comment: I can get username now by ns.CommonUserName, but still have no idea how to get phone, use loop to find phone by username from server database is not an option because of it take much time.
Answer: I'm wondering why you're doing this in JavaScript on the browser side instead of doing it on the server side, because this will only work for users who have a Notes client installed and configured correctly. However, if this is really the way you want to go...
The only information that you get automatically with the session is the user's name (in a few different formats). If you want anything else, you have to look up the user information on the server.
You can use
var nd = ns.getDirectory()
var userinfo = NotesDirectory.LookupNames("$Users",ns.UserName, fieldsArray)

to get more info.
Note that there are several phone number fields in the Domino Directory and depending on your organization's policy and procedures they may not all be filled in. You'll need to look up the item names (e.g., "OfficePhoneNumber", "PhoneNumber", "CellPhoneNumber" .. there are others) and put the ones that you want to retrieve into the fieldsArray that you pass to LookupNames. You'll get the result back as a NotesDirectoryNavigator object, and you can use that object's methods and properties to get the value.
Comment: I'm doing it with JS because server belong to other department of organization and located even in other region, so because of it I cannot modify anything on server side, and this is the only way how it can be done. I needed field OfficePhoneNumber, your answer solved my problem. And authorize with lotus client made to restrict access to the users account from any place except users workplace. .
Comment: Office number is a key to get info about users phone calls from my automatic telephone system server, so web page will be small members area with info about calls
Comment: It's not necessarily the only way it can be done. For example, if your web server authenticates users and runs Java, you could write code that runs on the web server to connect to the Domino server and do the lookup using NCSO.jar. Or from just about any langauge on your web server you could use the Domino REST API or perhaps even use LDAP. Yes, your Domino server would have to be set up to support those methods, but it might already be set up for one of them. In any case, doing it from the browser does work, but its likely to be a little slower.

Lotus Notes JavaScript Get user info


Make View and Sub View Programmatically

Question: I have the following code to create my window, my view and my sub view programmatically.  The problem is my subview "filterView2" when it is added [filterView addSubview:filterView2]; crashes on that line.  Is there something I forgot to include or did wrong? thanks!
    NSRect mainFrame = [[NSScreen mainScreen] frame];
    NSRect helpFrame = NSZeroRect;
    float width = 600;
    float height = 400;
    helpFrame.origin.x = (mainFrame.size.width - width) / 2.0;
    helpFrame.origin.y = 260.0;
    helpFrame.size.width = width;
    helpFrame.size.height = height;

    helpWindow2 = [[BrightnessView windowWithFrame:helpFrame] retain];

    // Configure window.
    [helpWindow2 setReleasedWhenClosed:YES];
    [helpWindow2 setHidesOnDeactivate:NO];
    [helpWindow2 setCanHide:NO];
    [helpWindow2 setCollectionBehavior:NSWindowCollectionBehaviorCanJoinAllSpaces];
    [helpWindow2 setIgnoresMouseEvents:YES];
    [helpWindow2 setBackgroundColor:[NSColor clearColor]];
    [helpWindow2 setOpaque:NO];

    // Configure contentView.
    NSView *filterView = [helpWindow2 contentView];
    [filterView setWantsLayer:YES];
    //add subview
    NSView *filterView2 = [helpWindow2 contentView];
    [filterView addSubview:filterView2];

    //CALayer for filterView
    CALayer *theLayer = [CALayer layer];
    theLayer.opacity = 0;
    [filterView setLayer:theLayer];
    CGColorRef bgColor = CGColorCreateGenericRGB(0, 200, 255, 1);
    theLayer.backgroundColor = bgColor;
    CGColorRelease(bgColor);
    theLayer.borderColor = CGColorGetConstantColor(kCGColorWhite);
    theLayer.cornerRadius = 8.0;

    float helpOpacity = (([NSApp isActive] ? 1 : 0));
    [[[helpWindow2 contentView] layer] setOpacity:helpOpacity];

    [window addChildWindow:helpWindow2 ordered:NSWindowAbove];

Comment: usally when u get bad_access it's cus you are trying to access something that's not retained(retain count < 1) due to that you released the memory (for example). in this case you actually have to look at your design. should a view be it's own subview? doesn't really sound right to me, so try to create a new instance of the contentView that you put into your object, you can even do a copy of it so that you dont loose anything.
Comment: the helpWindow2 function "contentView", does that allocate a new version of a view or atleast retain it? and what kind of crash are we talking about here? EXC_BAD_aLLOC? Provide more info about the crash and how the "contentView" function/property/variable looks. need to know what it is
Comment: EXC_BAD_ACCESS is the error message and the variable is just the (id)contentView
Answer: I think filterView and filterView2 are the same object, which causes an exception. You cannot add a view as a subview of itself.
Comment: so then how can I turn filterView2 into strictly a subview?
Comment: The most straight-forward way to create an `NSView` is to call `[[NSView alloc] initWithFrame: ...]`. However, if `filterView2` is a _subclass_ of `NSView`, it may have a different designated initializer. You should know what kind of view you want `filterView2` to be.

Make View and Sub View Programmatically


Confused by disparity between KVM host and guest CPU usage with nanosleep

Question: I've been plagued for a while by a major disparity between the CPU usage reported by a KVM host and guest. The guest will be reporting as basically idle, but the host will report one virtual core at 100% CPU.
I've finally tracked it down to a misbehaving application that ends up calling a very short nanosleep in a tight loop in the guest. The following C code will replicate the issue reliably:
#include <time.h>

void main() {
    while (1) {
        struct timespec ts = {0, 1024*100};
        nanosleep(&ts, NULL);
    }
}

If I run this program in the host, the CPU is basically idle. If I run it in the guest, the guest believes he is idle, but the host reports a core fully used.
I do intend to make the application better behaved, but I would like to understand the root cause of the mismatch. Is there potentially something misconfigured in my KVM setup that could lead to this, or is this an expected outcome somehow?
In my test setup, the host is Ubuntu 18.10, and the guest is CentOS 6. I've been able to replicate this with other guest OSes as well.
Comment: Did you replicate this with _newer_ guest OSes?
Comment: @MichaelHampton Yes, I got identical results with Ubuntu 18.04 LTS as a guest.
Answer: nanosleep probably forces the guest OS to very frequently read the CPU TSC which, in turn, can cause considerable load on the host system. This is, however, very platform and setting specific.
I would suggest to first read libvirt timer and stimer documentation here, then give at look a the following resources:

https://superuser.com/questions/1317948/qemu-enable-kvm-slower-than-pure-emulation-for-x86-64
https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html
https://wiki.qemu.org/Features/CPUModels
Comment: I wasn't able to get any significant improvement in CPU usage from adjusting these settings, but it does seem like the most likely culprit, so I'm accepting this answer.


Create Woodstox's XMLValidationSchema from multiple resource files

Question: I have a main file.xsd that has an import with a schema location that points to file1.xsd like this:
<xs:import namespace="urn:file1" schemaLocation="file1.xsd"/>

I run this on a Docker container and it complains about not finding file1.xsd in container home folder:
InputStream resourceStream = this.getClass().getClassLoader()
    .getResourceAsStream("file.xsd");

XMLValidationSchemaFactory schemaFactory = XMLValidationSchemaFactory
    .newInstance(XMLValidationSchema.SCHEMA_ID_W3C_SCHEMA);

XMLValidationSchema validationSchema = schemaFactory.createSchema(resourceStream);

How can I link these two xsd resources in same XMLValidationSchema which is the mandatory type I have to obtain? (modifying the docker-compose/explicitly copying the file1.xsd in container is not an option although it works that way)
Comment: Usually with the Java APIs you can set some resource resolver e.g. https://docs.oracle.com/javase/8/docs/api/javax/xml/validation/SchemaFactory.html#setResourceResolver-org.w3c.dom.ls.LSResourceResolver-, in the one you have you need to make sure the resources are loaded the same way as your initial schema, i.e. by doing `getClass().getClassLoader()..getResourceAsStream(systemId)`.
Answer: Woodstox does not have resolver for automatically reading schema from document content so you need to use the approach you have (but create InputStream in a way that works).
The only resources that are automatically resolved are external DTD subsets as specified by XML specification.
This is not to say that a handler could not be implemented and added to Woodstox (I am sure some users would find it useful), just that there isn't one currently.


How to detect Panelizer use in template_preprocess_html()?

Question: In Drupal 7, you could look for $node->panelizer or use page_manager_get_current_page() to check if the current page uses a panelizer layout. How do you do that in Drupal 8?
I'm trying to add a body class if the current page contains a specific pane.
Answer: I'm not sure if this is quite what you're after, but these hooks will fire if your page is using panelizer.
/**
 * Implements hook_preprocess_panels_pane().
 */
function YOURTHEME_preprocess_panels_pane(&$$variables) {
  // If this triggers then you are in a panels pane.
}

OR
/**
 * Implements hook_panels_pane_content_alter().
 */
 function YOUR_MODULE_panels_pane_content_alter($content, $pane, $args, $contexts) {
  // If this triggers then you are altering panels content.
}

If you want to check for panelizer in the node you could try something like this:
$node = \Drupal::routeMatch()->getParameter('node');
if ($node->hasField('panelizer')) {
  // Do stuff
}

Answer: Turns out you do have access to $node->panelizer. So here's what I did:
function MYMODULE_preprocess_html(&$vars) {
  $node = \Drupal::routeMatch()->getParameter('node');
  if ($node instanceof \Drupal\node\NodeInterface) {
    if (isset($node->panelizer)) {
      $panelizer = $node->panelizer->getValue();
      // do whatever checks are needed
      // and use $vars['attributes']->addClass('MYCLASS') when ready
    }
  }
}


Bootstrap Partial view modal form not submit

Question: i'm trying to create a form for add. i'm using bootstrap modal window to get data
here is my partial view

            <button type="button" class="close" data-dismiss="modal" aria-hidden="true">
                ×
            </button>

Register Datasource

            @using (Html.BeginForm("Add", "DataSource", FormMethod.Post, new { id = "order-form", @class = "smart-form", novalidate = "novalidate" }))
            {

            }

            <button type="button" class="btn btn-default" data-dismiss="modal">
                Cancel
            </button>
            <button type="button" class="btn btn-primary">
                Add
            </button>

on index page i'm firing modal show with button click
$('#btn-addDataSource').on('click', function() {
                var url = '@Url.Action("AddProvider","DataSource")';
                $.get(url, function (data) {
                    $('#register-modal').html(data).modal('show');
                });
            });

but when i submit the form it doesn't submit i used
$('#register-modal form').submit(function () {
    $.ajax({
        url: this.action,
        type: this.method,
        data: $(this).serialize(),
        success: function (result) {
            if (result.success) {
                $('#register-modal').modal('hide');

            }
        }
    });
    return false;
});

what was i missed when i submitting please ??
Comment: Instead of button type="button" to change into  type="submit"
Comment: same nothing happens
Comment: the script is added at the end of the page, right ?
Comment: yeah that script added to end of page (not partialview but index)
Comment: In you code form  id = "order-form" but yours submit function id is $('#register-modal form') . can you change your id and try
Comment: i changed but not working
Answer: Instead of button type="button" to change into type="submit"

$('#order-form').submit(function () {
    $.ajax({
        url: this.action,
        type: this.method,
        data: $(this).serialize(),
        success: function (result) {
            if (result.success) {
                $('#register-modal').modal('hide');

            }
        }
    });
    return false;
});

       @using (Html.BeginForm("Add", "DataSource", FormMethod.Post, new { id = "order-form", @class = "smart-form", novalidate = "novalidate" }))
            {

            <button type="button" class="close" data-dismiss="modal" aria-hidden="true">
                ×
            </button>

Register Datasource

            <button type="button" class="btn btn-default" data-dismiss="modal">
                Cancel
            </button>
            <button type="submit" class="btn btn-primary">
                Add
            </button>

            }

Comment: Where you write the submit function? I mean that submit code inside the partial view or main view
Comment: submit function also write inside the partial view
Answer: Its a hack but try submitting it by j-query. ie :
$(document).on("click", "#btnEdit", function(){
    $("#formEditCustomer").submit();
});


Expand dataframe following sets of rules

Question: I have a quite complex problem I am not able to tackle.
I have a dataframe I read in dplyr:
trans_id date       type
9373    2019-09-29  6-months
9945    2019-08-15  3-months
9945    2019-11-13  3-months
9615    2019-12-28  3-months
11465   2019-07-13  3-months
11465   2019-10-11  3-months

reproducible example:
library(tidyverse)

df <- data.frame(stringsAsFactors=FALSE,
          id = c(9373, 9945, 9945, 9615, 11465, 11465),
        date = c("2019-09-29", "2019-08-15", "2019-11-13", "2019-12-28",
                 "2019-07-13", "2019-10-11"),
        type = c("6-months", "3-months", "3-months", "3-months", "3-months",
                 "3-months")) %>%
  mutate(date = as.Date(date))

Each id is a transaction, happened on a given date; each transaction can be either repeated every 3 months or 6 months - as specified in type.
I want to expand these transactions in their monthly counterparts up to the current date; this means that the first transaction 9373 has to be repeated 6 times with a 30 days cycle (type == 6-months) starting from 2019-09-29 up to current day (today is 2020-01-07), aka is going to be just 4 single monthly transactions since the last two have to happen yet.
Same for the 3-months transactions, always considering the starting date and the current date.
Example of the final result:
id      date        type
9373    2019-09-29  6-months # first 6-months cycle transaction
9373    2019-10-29  6-months
9373    2019-11-28  6-months
9373    2019-12-28  6-months
9945    2019-08-15  3-months #
9945    2019-09-14  3-months
9945    2019-10-14  3-months
9945    2019-11-13  3-months #
9945    2019-12-13  3-months
9615    2019-12-28  3-months #

Any help is highly appreciated!
Answer: Here is one way using dplyr and tidyr functions.
library(dplyr)
library(tidyr)

df %>%
  #Extract the number from type column
  mutate(num = readr::parse_number(type)) %>%
  #For each transcation
  group_by(row = row_number()) %>%
  #Create a sequence from date till number of months with a break of 30 days
  complete(id, type, date = seq(date, by = "30 days", length.out = num)) %>%
  #Remove rows which have date value greater than today
  filter(date <= Sys.Date()) %>%
  ungroup() %>%
  select(-num, -row)

# A tibble: 16 x 3
#      id type     date
#   <dbl> <chr>    <date>
# 1  9373 6-months 2019-09-29
# 2  9373 6-months 2019-10-29
# 3  9373 6-months 2019-11-28
# 4  9373 6-months 2019-12-28
# 5  9945 3-months 2019-08-15
# 6  9945 3-months 2019-09-14
# 7  9945 3-months 2019-10-14
# 8  9945 3-months 2019-11-13
# 9  9945 3-months 2019-12-13
#10  9615 3-months 2019-12-28
#11 11465 3-months 2019-07-13
#12 11465 3-months 2019-08-12
#13 11465 3-months 2019-09-11
#14 11465 3-months 2019-10-11
#15 11465 3-months 2019-11-10
#16 11465 3-months 2019-12-10

Comment: Hi! the "expanding" concept works just fine, but it doesn't take into account of the "type" column... it should be expanding "at most" 6 times for the rows labeled as "6-months" and 3 times for the rows labeled as "3-months"...
Comment: @xxxvincxxx I think my code is doing the same. Which part of my output doesn't follow that rule ?
Answer: You can use rowwise and do like so:
df %>%
  rowwise() %>%
  do({
    p <- as.numeric(gsub('\\D+','',.$type))-1
    tibble(
      id=.$id,
      date=seq(.$date,pmin(Sys.Date(),.$date+p*30),30),
      type=.$type
    )
  }) %>%
  ungroup()

# A tibble: 16 x 3
# id date       type
# * <dbl> <date>     <chr>
#   1  9373 2019-09-29 6-months
# 2  9373 2019-10-29 6-months
# 3  9373 2019-11-28 6-months
# 4  9373 2019-12-28 6-months
# 5  9945 2019-08-15 3-months
# 6  9945 2019-09-14 3-months
# 7  9945 2019-10-14 3-months
# 8  9945 2019-11-13 3-months
# 9  9945 2019-12-13 3-months
# 10  9615 2019-12-28 3-months
# 11 11465 2019-07-13 3-months
# 12 11465 2019-08-12 3-months
# 13 11465 2019-09-11 3-months
# 14 11465 2019-10-11 3-months
# 15 11465 2019-11-10 3-months
# 16 11465 2019-12-10 3-months


Generic $Q_8$-polynomial in characteristic $2$

Question: Context. Apologies for the long post. As the title suggests, I am trying to compute a generic $Q_8$-polynomial in characteristic two.
Let $F$ be a field, and let $G$ be a finite group. A polynomial $f\in F(T_1,\ldots,T_n)[X]$ is called $G$-generic if the following conditions hold:

$f(T_1,\ldots,T_n,X)$ is irreducible, with Galois group $G$.

for any field extension $F'/F$, with $F'$ infinite, and every Galois extension $L'/F'$ with Galois group $G$, there exist $a'_1,\ldots,a'_n\in F'$ such that the specialized polynomial $f(a'_1,\ldots,a'_n,X)$ is irreducible in $F'[X]$ with splitting field $L'$.

Note that a $G$-generic polynomial does not necessarily exist. However, it does when Noether's problem has a positive answer. Indeed, if $V$ is a faithful representation of $G$ for which $F(V)^G/F$ is rational, then one may show that the minimal polynomial of a primitive element of $F(V)/F(V)^G$ is $G$-generic.
Claim. A $G$-generic polynomial always exists when $G$ is a $p$-group and $F$ has characteristic $p$.
Proof of the claim. A theorem of Miyata says that if $G\to \mathrm{GL}(V)$ is a triangularizable representation, then Noether's problem has a positive answer. Moreover, it is known that $p$-groups may be embedded into some $U_n(\mathbb{F}_p)$ (the $p$-Sylow subgroup of $\mathrm{GL}_n(\mathbb{F}_p)$ consisting of unipotent upper triangular matrices).
Hence, my strategy is:

to find injective morphism $Q_8\hookrightarrow U_n(F)$ for some $n\geq 1$, where $F$ is a given field of characteristic two

to compute algebraically independent elements $T_1,\ldots, T_n$ such that  $F(T_1,\ldots,T_n)^{Q_8}$

to find a primitive element  for $F(X_1,\ldots,X_n)/F(T_1,\ldots,T_n)$ and write up its minimal polynomial.

My results so far .

After tedious computations, I found that $Q_8\simeq\langle I,J\rangle\subset U_4(F)$, where
$I=\pmatrix{1 & 0 & 1 & 1\cr 0 & 1 & 0 & 1\cr 0 & 0 & 1 & 1 \cr 0 & 0 & 0 & 1}$
and
$J=\pmatrix{1 & 1 & 0 & 1\cr 0 & 1 & 0 & 1\cr 0 & 0 & 1 & 0\cr 0 & 0 & 0 & 1}$.

The corresponding action (associated to the dual representation) on $F[X_1,\ldots,X_4]$ is then given by:
$$I\cdot P=P(X_1,X_2,X_3+X_1,X_4+X_2+X_3), \ J\cdot P=P(X_1,X_2+X_1,X_3,X_4+X_2).$$

I found $F[X_1,\ldots,X_4]^{Q_8}=F[T_1,T_2,T_3,T_4]$, with
$$T_1=X_1, \ T_2=X_2(X_2+X_1), \ T_3=X_3(X_3+X_1), \ T_4=X_4^4+(X_2^2+X_2X_3+X_3^2)X_4^2+(X_2^2X_3+X_2X_3^2)X_4.$$

Note that $T_1,\ldots,T_4$ are algebraically independent over $F$.

Replacing $X_i$ by $X_i/X_1$, we can safely specialize to $T_1=1$ without changing the generic property of the corresponding extension (this reflects the fact that the triangular representation of $Q_8$ factors to a faithful projective representation)

We then get a $Q_8$-extension $F(X_2,X_3,X_4)/F_0$, where $F_0=F(T_2,T_3,T_4)$.
Set $K=F_0(X_2,X_3)$, and let
$g= X^4+(X_2^2+X_2X_3+X_3^2)X^2+(X_2^2X_3+X_2X_3^2)X+T_4\in K[X]$.
Then $X_4$ is a root of $g$, so $\mu_{X_4,K}\mid g$.
The field extension $K/F_0$ is a biquadratic extension of degree $4$. However, the polynomial $g$ cannot be irreducible over $K$, since $F(X_2,X_3,X_4)/F_0$ has degree $8$ by Artin's Lemma. But $g$ is indeed irreducible: specializing $X_2$ and $X_3$ to $T_4$ yields the polynomial $X^4+T_4^3X^2+T_4$, which is irreducible by Eisenstein's criterion.
Finally, I come to my questions:
Q1. Are my results 1) and 2) correct ?
Q2. Assuming that the answer to Q1 is YES, where did I go wrong in 3) ?
Remark. I strongly suspect that 2) is incorrect, but I did the computation several times, and ended up again and again with the same result. As far as  I know (I've tried!) SAGE does not want to compute the ring of invariants if the characteristic of the base field divides the order of the group, so I cannot check my computations.
Side question. I found $I,J$ by trial and error, and this makes me extremely unhappy. Is there a clever way to find an embedding $Q_8\subset U_4(\mathbb{F}_2)$ ?
Comment: maybe a stupid question, but what is $Q_8$?
Comment: 1) is correct, just checked on Magma.
Comment: @student91: the quaternion group of order $8$.
Comment: @DavidA.Craven: thanks!
Comment: I don't see how $T_4$ is fixed under $I$ and $J$. e.g. under $I$, there should be a term $X_1X_2X_4^2$ that only appears from the term $X_2X_3X_4^2\to X_2(X_3+X_1)(X_2+X_3+X_4)^2=X_2(X_3+X_1)(X_2^2+X_3^2+X_4^2)=X_1X_2X_4^2+\dots$. All other monomials do not seem to give this specific monomial, which should be cancelled out as it does not appear in $T_4$ itself. (Could be that I missed something...)
Comment: @student91: well, I checked on a CAS, and $T_4$ is indeed invariant by $I$ and $J$ (recall that $2=0$...)
Comment: But is the statement "$I\cdot X_2X_3X_4^2 = X_2(X_3+X_1)(X_2^2+X_3^2+X_4^2)$" (i.e. the interpretation of the $I$-action) correct? Also, I found the alternative $$T_4=(X_4+X_3+X_2+X_1)(X_4+X_3+X_2)(X_4+X_2+X_1)(X_4+X_2)(X_4+X_1)X_4$$ of which I am quite certain it is invariant under $I$ and $J$. Maybe you have more luck with that.
Comment: Yes, you interpreted correctly the action of $I$. However, I ma not sure you have the right expression for $T_4$, so let me reproduce it in this comment.

$T_4=X_4^4+(X_2^2+X_2X_3+X_3^2)X_4^2+(X_2^2X_3+X_2X_3^2)X_4.$ I triple checked with a CAS that this $T_4$ was invariant by $I$ and $J$. However, maybe there is a polynomial of degree $2$ in $X_4$ which is also invariant , but I could not find it. Sorry if I misunderstood what you said in your comments...
Comment: Oops, the "alternative" I found in my previous comment does not seem to work, I forgot the factors $K\cdot X_4$ and $K^3\cdot X_4$. I've added these in an edit to my reply below. I am as of yet unable to find invariant polynomials of degree $<8$.
Comment: OK, that's embarrassing. I asked 3 times Maple tese past few days to check $T_4$ if it was invariant, and Maple said it was.   I asked it a fourth time and I now agree with what you say. I don't understand what I did wrong the previous times. Apologies for not understanding right away your comments. I'lll go back to my computations and see what happens.
Answer: Note: This is not a full answer, but too long for a comment.

I strongly suspect that 2) is incorrect

My computations seem to indicate so too.

so I cannot check my computations.

You could check by hand, like so:
$$I\cdot T_4 = I\cdot(X_4^4+X_2^2X_4^2+X_2X_3X_4^2+X_3^2X_4^2+X_2^2X_3X_4+X_2X_3^2X_4)$$
Using action of $I$ gives $$I\cdot T_4 = (X_4+X_3+X_2)^4+X_2^2(X_4+X_3+X_2)^2+X_2(X_3+X_1)(X_4+X_3+X_2)^2+(X_3+X_1)^2(X_4+X_3+X_2)^2+X_2^2(X_3+X_1)(X_4+X_3+X_2)+X_2(X_3+X_1)^2(X_4+X_3+X_2)$$
Using $(a+b)^2=a^2+b^2$ gives $$I\cdot T_4=X_4^4+X_3^4+X_2^4+X_2^2X_4^2+X_2^2X_3^2+X_2^2X_2^2+X_2X_3X_4^2+X_2X_3X_3^2+X_2X_3X_2^2+X_2X_1X_4^2+X_2X_1X_3^2+X_2X_1X_2^2+X_3^2X_4^2+X_3^2X_3^2+X_3^2X_2^2+X_1^2X_4^2+X_1^2X_3^2+X_1^2X_2^2+X_2^2X_3X_4+X_2^2X_3X_3+X_2^2X_3X_2+X_2^2X_1X_4+X_2^2X_1X_3+X_2^2X_1X_2+X_2X_3^2X_4+X_2X_3^2X_3+X_2X_3^2X_2+X_2X_1^2X_4+X_2X_1^2X_3+X_2X_1^2X_2$$
Simplifying monomials gives $$I\cdot T_4=X_4^4+X_3^4+X_2^4+X_2^2X_4^2+X_2^2X_3^2+X_2^4+X_2X_3X_4^2+X_2X_3^3+X_2^3X_3+X_1X_2X_4^2+X_1X_2X_3^2+X_1X_2^3+X_3^2X_4^2+X_3^4+X_2^2X_3^2+X_1^2X_4^2+X_1^2X_3^2+X_1^2X_2^2+X_2^2X_3X_4+X_2^2X_3^2+X_2^3X_3+X_1X_2^2X_4+X_1X_2^2X_3+X_1X_2^3+X_2X_3^2X_4+X_2X_3^3+X_2^2X_3^2+X_1^2X_2X_4+X_1^2X_2X_3+X_1^2X_2^2$$
Collecting the same monomials gives
$$I\cdot T_4=
X_1X_2X_3^2
+X_1X_2X_4^2
+X_1X_2^2X_3
+X_1X_2^2X_4
+2X_1X_2^3
+X_1^2X_2X_3
+X_1^2X_2X_4
+2X_1^2X_2^2
+X_1^2X_3^2
+X_1^2X_4^2
+X_2X_3X_4^2
+X_2X_3^2X_4
+2X_2X_3^3
+X_2^2X_3X_4
+4X_2^2X_3^2
+X_2^2X_4^2
+2X_2^3X_3
+2X_2^4
+X_3^2X_4^2
+2X_3^4
+X_4^4
$$
Using that $2=0$ gives
$$I\cdot T_4=
X_1X_2X_3^2
+X_1X_2X_4^2
+X_1X_2^2X_3
+X_1X_2^2X_4
+X_1^2X_2X_3
+X_1^2X_2X_4
+X_1^2X_3^2
+X_1^2X_4^2
+X_2X_3X_4^2
+X_2X_3^2X_4
+X_2^2X_3X_4
+X_2^2X_4^2
+X_3^2X_4^2
+X_4^4
$$
Which is not the same as $$T_4=X_4^4+X_2^2X_4^2+X_2X_3X_4^2+X_3^2X_4^2+X_2^2X_3X_4+X_2X_3^2X_4$$
Because the terms $$I\cdot T_4-T_4=X_1X_2X_3^2
+X_1X_2X_4^2
+X_1X_2^2X_3
+X_1X_2^2X_4
+X_1^2X_2X_3
+X_1^2X_2X_4
+X_1^2X_3^2
+X_1^2X_4^2
$$ do not appear in $T_4$ but these do appear in $I\cdot T_4$.
EDIT:
You could find an alternative for $T_4$ that is certainly invariant under $Q_8$ by considering the $Q_8$-orbit of $X_4$:

$I\cdot X_4 = X_4+X_3+X_2$
$J\cdot X_4 = X_4+X_2$
$IJ\cdot X_4 = X_4+X_3$
$I^2\cdot X_4 = X_4+X_1$
$I^3\cdot X_4 = X_4+X_3+X_2+X_1$
$J^3\cdot X_4 = X_4+X_2+X_1$
$IJ^3\cdot X_4 = X_4+X_3+X_1$

And then taking the product of all these images:
$T_4=X_4(I\cdot X_4)(J\cdot X_4)(IJ\cdot X_4)(I^2\cdot X_4)(I^3\cdot X_4)(J^3\cdot X_4)(IJ^3\cdot X_4)=(X_4+X_3+X_2+X_1)(X_4+X_3+X_2)(X_4+X_3+X_1)(X_4+X_3)(X_4+X_2+X_1)(X_4+X_2)(X_4+X_1)X_4$ $$=X_4^8 +
X_4^4\left(X_3^4+X_2^4+X_1^4+X_3^2X_2^2+X_3^2X_1^2+X_2^2X_1^2+X_3^2X_2X_1+X_3X_2^2X_1+X_3X_2X_1^2\right) +
X_4^2\left(X_3^2X_2^2X_1^2+X_3^4X_2X_1+X_3X_2^4X_1+X_3X_2X_1^4+X_3^4X_2^2+X_3^4X_1^2+X_3^2X_2^4+X_3^2X_1^4+X_2^4X_1^2+X_2^2X_1^4\right) +
X_4\left(X_3^4X_2^2X_1+X_3^4X_2X_1^2+X_3^2X_2^4X_1+X_3^2X_2X_1^4+X_3X_2^4X_1^2+X_3X_2^2X_1^4\right)
$$
Is certainly invariant (because group-multiplication permutes the group). Note that $T_1,T_2,T_3$ are also of this form. But in this case it seems to be quite a stupid degree-$8$ polynomial.
Maybe we can do something smarter with something of the form $T_4=(q_1\cdot X_4)(q_2\cdot X_4)(q_3\cdot X_4)(q_4\cdot X_4)+(q_5\cdot X_4)(q_6\cdot X_4)(q_7\cdot X_4)(q_8\cdot X_4)$. The problem is that in characteristic $2$, the $X_4^4$ term will cancel out in this setting. Maybe we can take $q_1\dots q_8$ such that some lower-degree term containing $X_4$ does not cancel out.
EDIT 2:

Maybe we can do something smarter with something of the form $T_4=(q_1\cdot X_4)(q_2\cdot X_4)(q_3\cdot X_4)(q_4\cdot X_4)+(q_5\cdot X_4)(q_6\cdot X_4)(q_7\cdot X_4)(q_8\cdot X_4)$

Unfortunately we cannot:
If we assume WLOG that $q_1=e$ and $q_2\in\{I,J,K\}$, then multiplying by $q_2$ gives that the term with $q_2$ must also contain $q_2^2=\overline{e}$. But then it contains $e$ and $\overline{e}$, so it must also contain $\overline{e}q_2=\overline{q_2}$. Therefore, we have something of the form $$(e\cdot X_4)(I\cdot X_4)(I^2\cdot X_4)(I^3\cdot X_4)+(J\cdot X_4)(J^3\cdot X_4)(K\cdot X_4)(K^3\cdot X_4)$$ or with $I,J,K$ permuted. Checking these expressions shows that all non-constant terms (i.e. with a factor $X_4$) cancel out.


google maps api v3 multiple jumping location marks with info window

Question: My goal is create a map with multiple locations with a map that jumps from marker to marker when a html tag was click elsewhere on the page
i made the jumps work but, i can't seem to get the mouseover info boxes to work...i have search but can't find this specific problem
www.humphrey-ray.com/qp/locations-new27a.html

Comment: do you want to open infowindow on a specific marker on mouse move?
Comment: yes, one info per marker
Answer: working example..
<!DOCTYPE html>

                            Click Location to show on map

                                    Old City

                                    South Street/Society Hill

                                    Northern Liberties

                                    Old City


Why is the ROC curve two-dimensional instead of three-dimensional?

Question: When I understand it correctly, a ROC curve composes of a lot of ratios of TPR and FPR. Each such ratio is gained, e.g., by scanning/investigating a parameter where the ratio is the result of the used classifier.
In other words: I'm interested in how a classifier separates data points according to a frequency interval of 0-100 Hz. Hence I scan this frequency, record all the 101 ratios, put them into the plot and the ROC curve represents these datapoints, finally.

Wouldn't it be much more interesting when the frequency would be used as a third axis? Or is it implicit in the axis?
But anyway, I think it would be easier to look at with a third axis?
Comment: My question (5 days ago) was closed, because someone else asked the same (2 days ago)? :D
Comment: Your threshold strictly determines FPR and TPR so it wouldn't add much new information and it'd make the chart more harder to interpret
Answer: The ROC curve is two-dimensional because it is defined as the plot of TPR and FPR. Yes, you might be interested in the threshold used to generate the TPR and FPR, and you might want to plot that on a third axis, but then it is not a ROC curve but something else (arguably more informative, arguably harder to draw, arguably so hard to draw that the additional information is not worth the trouble).
Comment: Especially considering it's most likely going to be displayed on a 2-D screen or printed on 2-D paper...


Escrever TXT formatado através do VB6

Question: Estou tentanto copiar a estrutura de um GRID/Tabela dentro de um arquivo texto, copiando inclusive a estrutura das celulas(Tabela).
Porém, eu nao consigo alinhar as colunas no arquivo texto, apesar de possuir um tamanho padrão para cada coluna.
O processo funciona da seguinte maneira, eu meço o maior item contido em cada linha de cada columa e entao uso seu tamanho para definir o espaço total da coluna em questão. Quando o texto é menor que o tamanho total da coluna, eu preencho os caracteres restantes com espaço em branco.
O que notei, que é as letras saem de tamanhos variados e minha medida é dada pelo número de caracteres. No entanto, o espaço que a letra "A" ocupa, é diferente do espaço ocupado pela letra "T" por exemplo.
Eu tentei escolher alguma fonte true-tipe no bloco de notas(Tahoma,Arial,Impact) mas mesmo assim as letras parecem ter tamanho diferente no bloco de notas.
Sabem me dizer como poderia construir uma tabela, de tamanho padronizado, escrita no bloco de notas!? Considerando por exemplo, que a unidade de medida é o tamanho do caractere.
Answer: O problema não está no seu programa que gera o arquivo texto (VB6) e sim na visualização do arquivo, pois se o programa usado (ex: bloco de notas) estiver usando um fonte de espaçamento variável (ex: arial) o conteúdo ficará embaralhado.
Você não tem que usar fontes true-type e sim fontes de espaçamento fixo, como por exemplo: courier new.
Comment: Ao utilizar courier new, todo o alinhamento ficou perfeito.

Escrever TXT formatado através do VB6


How do reimplement this Python XML-parsing function in Haskell?

Question: I recently wrote the following Python function which will take a Google Picasa contacts.xml file and output a dictionary with ID and Name.
def read_contacts_file(fn):
    import xml.etree.ElementTree
    x = xml.etree.ElementTree.ElementTree(file=fn)
    q = [(u.attrib["id"], u.attrib["name"]) for u in x.iter("contact")]
    return dict(q)

What this function does is return a dictionary (hashtable, map) with the ID being the key and the Name being the value.
The file itself has the form:
<contacts>
 <contact id="f5fdaaee2e80fa01" name="Person A" display="A"/>
 <contact id="8d5256298fd43877" name="Person B" display="B"/>
</contacts>

What's the simplest way I can implement this in Haskell?

Just wanted to let you know that with everyone's help, I've managed to come up with the following which makes sense to me. Thanks.
parseContactsData = M.fromList . runLA (xread >>> f)
    where f =
        getChildren
        >>> hasName "contact"
        >>> getAttrValue "id" &&& getAttrValue "name"

Comment: It would help to have a description of what you want this function to do; not everyone who can help with the Haskell solution will want to figure out what this Python code is doing.
Comment: It would help other people with a similar problem (or lack of knowledge) if you showed which modules you imported, in your new code.
Answer: Here's a minimal example doing the same thing with tagsoup:
import Text.HTML.TagSoup

assocLookup k dict = [v | (k', v) <- dict, k == k']
readContactsFile fn = fmap parse (readFile fn)

parse contents = do
    TagOpen "contact" attrs <- parseTags contents
    id <- assocLookup "id" attrs
    name <- assocLookup "name" attrs
    return (id, name)

Answer: Here's an example doing it with HXT
import Text.XML.HXT.Core
import Data.Map

idAssocs = hasName "contact" >>> getAttrValue "id" &&& getAttrValue "name"
dict = fromList `fmap` runX (readDocument [] "contacts.xml" >>> deep idAssocs)

Comment: I understand this makes use of arrows. But this syntax just looks really, really complicated compared to my Python version.
Comment: @Snoqual The `>>>` syntax is forward pipelining, while `&&&` applies the two arrows to the same input and produces a pair of their results.  The primary mental contortion is to think of the pipeline structure of parsing/transformation.
Answer: One alternative would be to use the HXT library. Here are some useful information which will get you started: http://www.haskell.org/haskellwiki/HXT/Conversion_of_Haskell_data_from/to_XML
Particularly useful for you would probably be looking at 3.5 A simple application


(f o g) o f composition function

Question: If $f$ represents the function $f(x)$ and $g$ represents the functions $g(x)$, is $(f \circ g)\circ f$ essentially $f(g(f(x)))$?
I know that:

$(f \circ g) = f(g(x))$

however I'm not sure if the brackets in my equations make a difference to this new function.
Comment: short answer: yes!
Comment: Function composition [is associative](https://math.stackexchange.com/questions/523906/show-that-function-compositions-are-associative), so $\,(f \circ g) \circ f = f \circ (g \circ f) = f \circ g \circ f\,$.
Answer: The brackets make no difference: the composition of functions is completely associative, meaning that $(f\circ g)\circ h=f\circ(g\circ h)$.
Answer: Yes it is. Furthermore, $(f \circ g) \circ f  = f \circ g \circ f = f \circ (g \circ f)$ by associativity.

(f o g) o f composition function


Continuous Matrix-Valued Functions as Bounded Operators

Question: In Theorem 1.3, the following $\mathrm{C}^*$-algebra is considered:
$$A:=\left\{f\in C([0,1],M_2(\mathbb{C})):f(0),\,f(1)\text{ diagonal}\right\}.$$
I want to identify a Hilbert space that these these act on. I want to say that via left multiplication, $g\mapsto fg$, the elements of $A$ are bounded operators on:
$$H:=\left\{f\in L^2([0,1],M_2(\mathbb{C})):f(0),\,f(1)\text{ diagonal}\right\}.$$
Define matrix elements:
$$f(x)=\left(\begin{array}{cc}f_{11}(x) & f_{12}(x)\\ f_{21}(x) & f_{22}(x)\end{array}\right),$$
I want to suggest that each of the $f_{ij}$ are in $L^2([0,1])$ where the norm is got from, e.g.
$$\langle f_{ij},g_{k\ell}\rangle_{L^2([0,1])}=\int_0^1 \overline{f_{ij}(x)}g_{k\ell}(x),$$
and define the inner product on $H$ by:
$$\langle f,g\rangle =\sum_{i,j=1}^2 \langle f_{ij},g_{ij}\rangle_{L^2([0,1])}.$$
Does this actually check out? As in:

Is the representation of $A$ (via left-multiplication) as bounded operators on $H$ faithful?

Or are there some subtleties and confusions that I am missing out on?

Relevant: $A$ is generated by projections
$$p(x)=\left(\begin{array}{cc}1 & 0 \\ 0 & 0\end{array}\right)$$
and
$$q(x)=\left(\begin{array}{cc}x & \sqrt{x(1-x)}\\ \sqrt{x(1-x)} & 1-x\end{array}\right)$$
Comment: If you just want to represent it on a Hilbert space, $L^2([0,1]) \otimes \mathbb{C}^2 \simeq L^2([0,1],\mathbb{C}^2)$ works since $A$ can be identified as a subalgebra of $C([0,1]) \otimes M_2 \simeq C([0,1],M_2)$.
Comment: Are you sure that your definition of $H$ actually "works" as a Hilbert space? The elements of $L^2([0,1],M_2)$ are equivalence classes of functions, and as $\{0,1\}$ is a set of measure 0, it doesn't really matter what happens at the end points (in particular, enforcing that the endpoints are diagonal matrices).
Comment: The Hilbert space $L^2([0,1],\Bbb C^2)=L^2([0,1])\oplus L^2([0,1])$ is the most natural space onto which you can represent $C([0,1], M_2(\Bbb C))$. But you can also represent faithfully into the bounded operators of $L^2([0,1])\otimes M_2(\Bbb C)$ with the scalar product you describe (forget the condition of diagonal matrices).
Comment: @JPMcCarthy ah you're right, good example! My bad.
Comment: You are only asking why it is faithful, so I suppose you have checked all the other details (ie that it is a $*$-representation). Consider the function:
$$f_1: [0,1]\mapsto M_2(\Bbb C), \quad x\mapsto \begin{pmatrix}1&0\\0&1\end{pmatrix}$$
then $(a\cdot f_1)_{ij} = a_{ij}$. If the representation were not faithful there must be some non-zero $a$ so that $a\cdot f_1=0$ in $L^2$, but thats impossible.
Comment: As per my above comment, I think if we wanted a representation of $L^2$ functions from $[0,1]$ to $M_2$ (rather than $\mathbb{C}^2)$, $H$ would just be $L^2([0,1],M_2) \simeq L^2([0,1]) \otimes M_2 \simeq L^2([0,1]) \otimes \mathbb{C}^4$, where the representation boils down to $f \otimes A(g \otimes B) = fg \otimes AB$ (on $L^2([0,1]) \otimes M_2$).
Comment: @PStheman I am not wedded to the Hilbert space above I just want them represented on some Hilbert space. I can see the issue with 0,1 alright. Do you think the Hilbert space with C^2 valued functions works?
Comment: @s.harp thanks but I haven't actually checked anything. PStheman has identified an issue with the space above and has proposed an alternative.
Comment: @s.harp is that last space the same as $L^2([0,1],M_2(\mathbb{C}))$? So that the functions can be operators by left multiplication?
Comment: @JPMcCarthy yes
Comment: Do you want to write that as an answer? I will upvote and accept. Is it true that in this representation the ranges of $p$ and $q$ intersect trivially ie at zero? This is what I find.
Comment: I muat check in the C^2 representation do they have no trivial intersection. I must check that intersection myself... maybe that was with the incorrect diagonal condition.
Comment: @JPMcCarthy I don't think the authors are requiring that $p$ and $q$ be orthogonal projections in their paper. So I think your concern about the ranges of $p$ and $q$ having trivial intersection (i.e., they are orthogonal projections) is unnecessary (unless there is another reason for it).
Comment: @PStheman projections can have trivial range intersection without being orthogonal. Take for example the the projection onto the $x$ axis and the projection onto $y=x$.
Comment: @PStheman I am interested in any and all $B(H)$ representations, the ranges of $p$ and $q$ zero or not.
Comment: @PStheman I had the same confusion only a few days ago.
Answer: I went with $\pi(A)\subset B(L^2([0,1],\mathbb{C}^2))$ in the end.


Low PPV high Sensitivity

Question: The results from my analysis show a low PPV (84.73%) compared to Sensitivity (88.8%). Not sure how to interpret these statistics, need some help
<code>                 Manual Process
             ----------------------
                  No           Yes
     New    No   3583          75
Automated   Yes  100           550
            -----------------------
</code>
Answer: The interpretation depends on you data and your objective but generally, if your PPV is lower than your sensitivity, you have more false-positives than false-negatives in your data.
Maybe you can provide more context, for example the specificity or false-positive-rate?
Sorry for not writing this as a comment but unfortunately I don't have enough reputation for that yet.
Comment: your suggestions were helpful :) , I have developed an algorithm that can automate the current existing manual review process which is very tedious, I have updated my question with a confusion matrix.

Low PPV high Sensitivity


What is the fastest/simplest way to add/manipulate PHP on a page?

Question: As a PHP developer familiar with MVC frameworks, I am having some difficulty figuring out Drupal.
I like the block setup. I am looking for some simple step-by-step guide to:

Add a block with my own PHP code.
Alter the html/php of other blocks.

Failing that, I would like to see some code which does these things.
Here is what I've been trying to do, more specifically:

There is a button. When it is clicked a DB query is executed.
Hide the "create new account" link on the login dialog. No account for you!
Comment: This question is really general. The broad answer to question 1 is to use hook_block. The broad answer to question 2 is to use hook_form_alter to remove the markup.
Comment: I agree with jhedstrom; then, the asked questions seem unrelated to each other.
Answer: You are in fact asking four questions here.

As jhedstrom said, you need to use the hook_block_info() / hook_block_view() hooks. A quick google search pointed to to this example: http://www.summasolutions.net/blogposts/how-create-basic-block-code-new-drupal-7
You can use the hook_block_view_alter() or hook_block_view_MODULE_DELTA_alter() hooks.
You can find detailed information about how to create a form in drupal in the quickstart guide. Note that this is for Drupal 6 but works the same way in Drupal 7 for the most part.
This does not require any code at all. In fact, doing this with code is doing it the wrong way. Instead, you can disable account registrations by visitors at Configuration > Account settings, set "Who can register accounts?" to Administrators only.
Comment: Since you're learning Drupal APIs don't forget the Examples project, http://drupal.org/project/examples, which has working examples (for D7 and D6) of all of the above.
Comment: This is also an excellent answer. I had tried `hook_block_view_MODULE_DELTA_alter()` but I didn't know about the cache, hence my frustration.


In neural network, what is the good way to represent ordinal/ratio data such as age, hour, day of week?

Question: I know that age is often represented as a numerical feature. For example in linear regression, it is common to use one single independent variable (IV) to represent it. However, one single IV cannot capture non-linear relationship. In some other models, such as factorization machines, it is advised to use dummy variable in order to capture feature interaction.
So what is the good way to represent ordinal/ratio data in neural network, especially deep learning models? Numerical feature? Dummy variable?
Comment: It's not correct that a single independent variable (IV is usually used to refer to instrumental variables) cannot capture non-linear relationship in neural networks. That is true in linear models, because there is only one coefficient to express the relationship. A neural network with multiple nodes (and multiple layers for complex relationships) captures non-linearities.
Answer: Variables cannot "capture non-linear relationship", because the relationship that you are talking about is a function that we are trying to learn, not-linearity is about the function, not the inputs to this function. Linear regression is able to learn a function that is linear in parameters, while more complicated machine learning algorithms can be able to learn more complicated functions.
Moreover, you can think of regression as of a very simplified special case of neural network, so if something works for regression, it should work for neural networks as well (i.e. treat them as numerical feature). Creating features based on those variables would be feature engineering and we usually use neural networks because they don't need feature engineering and learn the features by themselves.
Finally, using dummy variables for things like age is a bad idea. With using dummy variables, your algorithm needs to find individual rules for each of the levels of the dummy variable, so it finds separate rules for people who are 30 years old, and for the ones who are 31 years old. This is not the case with age, where you don't assume that people of any possible age could behave completely different then others.
Comment: "_Finally, using dummy variables for things like age is a bad idea._" – One thing you can do is use dummy variables for "age is at least X", so that someone who is 30 would have the first 30 variables set to 1 and the remaining 70 (or however old you go) set to 0. This could ideally make the representation a little easier, though it's not without its own weirdness. Ideally in a deep network this wouldn't be necessary, but I've seen it be useful there as well.
Answer: I know for time, a common way to represent the variable is using a sinusoidal function. This allows, for example, 2PM on Monday to be the same value as 2PM on Friday.


Salvar imagem na área de trabalho com html2canvas

Question: Como faço pra salvar a imagem na área de trabalho ou em alguma outra pasta da máquina do usuário ?
Peguei o exemplo muito bom de como gerar a imagem e salvar em uma pasta do servidor aqui: Tirar print Screen e salvar imagem automaticamente em C#
Fiz uma adaptação como o exemplo abaixo:
$("#btnImprimir").click(function () {

        var target = $('#box-graphs');
        html2canvas(target, {
                onrendered: function (canvas) {
                    var dataUrl = canvas.toDataURL();
                    $.post('/Exame/Imagem', { 'data': dataUrl }, function (data) {
                        if (data == "1") {
                            alert('Imagem enviada com sucesso');
                        }
                    });
                }
            });

    });

Answer: Não é possível controlar a máquina do visitante através do navegador, isto seria uma falha de segurança, pergunte a si mesmo, você gostaria que um site pudesse controlar as pastas do seu computador?
Todavia é possível permitir o download automático, claro que cada navegador terá uma limitação, como eu mostrei aqui alguns meios de tentar contornar o problema no IE: https://pt.stackoverflow.com/a/115223/3635
$("#btnImprimir").click(function () {
    var NOME_IMAGEM = 'download.png';

    var aLink = document.createElement('a');
    var evt = document.createEvent("HTMLEvents");
    evt.initEvent("click");

    var target = $('#box-graphs');

    html2canvas(target, {
        onrendered: function (canvas) {
            var dataUrl = canvas.toDataURL();

            //Download no IE11
            if (canvas.msToBlob) {
                var blob = canvas.msToBlob();
                window.navigator.msSaveBlob(blob, NOME_IMAGEM);

            //Download para navegadores que suportam o atributo download=""
            } else if (typeof a.download !== "undefined") {
                aLink.download = NOME_IMAGEM;
                aLink.href = dataUrl;
                aLink.dispatchEvent(evt);

            //Download para outros navegadores (provavel que não funcione em mobiles)
            } else {
                window.location.href = dataUrl.replace(/^data:image\/[a-z]+[;]/g, "data:application/octet-stream;");
            }

        }
    });

});

Answer: Creio que você não conseguirá realizar este procedimento. Uma aplicação web nunca terá acesso a nenhuma informação da máquina local do usuário por segurança, como por exemplo a estrutura de pastas. Nem mesmo o navegador que está instalado na maquina do usuário não consegue este tipo de informação. O que eu indicaria para você fazer era retornar a url do documento que foi gerado no servidor e assim incluir um novo elemento na tela do usuário com o link para download. Neste caso o próprio usuário escolheria onde salvar esta imagem.
Espero ter ajudado.
Abraço!
Answer: Você terá quer usar a biblioteca FileSaver.js junto com o html2canvas.js para conseguir salvar o arquivo na máquina do usuário, mas mesmo assim o navegador dará a opção de o usuário escolher o local ao deseja salvar o arquivo.
Der uma olhada na documentação do FileSaver.js que irá resolver seu problema


VAR model and Granger causality issues

Question: Is it necessary to create a stable vector autoregressive (VAR) model satisfying all the necessary checks to create it and then and only can we say that Granger causality holds true?
Answer: Yes. In the specific case of VAR models, imagine we want to test the restriction that $x$ does not Granger-cause $y$. We have the regression (in $y$):
$$y_t=\alpha + \sum_{l=1}^p \delta_ly_{t-l}+\sum_{l=1}^q \gamma_lx_{t-l}+\epsilon_t$$
The corresponding null to our Granger causality test is $H_0:\gamma_l=0,\,\,l=1,\dots,p$. This is a Wald test, and under the asymptotic normality and consistency of your estimator, it is $\chi^2$ distributed. If your process is not covariance-stationary, the variance of $\epsilon_t$ might change over time so you're estimate will no longer be asymptotically i.i.d. normal and the test statistic may no longer be $\chi^2$ distributed.
In general, Granger causality tests are somewhat model-dependent (they are subject to, for instance, omitted variables bias) so you should make sure you have the "right" model before you run them.

VAR model and Granger causality issues


Golang : Trying to figure out why "\n" don't count

Question: As a project for my school, we need to do an ascii art text version. By this, i mean to use this command :
go run . "Hello World!" standard --output="banner.txt"
Where's standard is the figlet use for the ascii art, and output to put my result in a file call banner.txt.
For now, (and my code is very ugly) i got this :
package main

import (
    "bufio"
    "fmt"
    "os"
    "strings"
)

func main() {

    if len(os.Args) == 4 {
        args := os.Args
        arg3 := os.Args[3]
        arg4 := strings.Split(arg3, "=")

        if arg4[0] == "--output" {
            file, err := os.Create(arg4[1])

            if err != nil {
                fmt.Println(err)
                return
            }
            defer file.Close()

            police := ""
            if args != nil && len(args) > 1 {
                arg2 := os.Args[2]
                if arg2 == "standard" {
                    police = "standard.txt"
                } else if arg2 == "shadow" {
                    police = "shadow.txt"
                } else if arg2 == "thinkertoy" {
                    police = "thinkertoy.txt"
                } else if arg2 == "graffiti" {
                    police = "graffiti.txt"
                } else {
                    fmt.Println("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
                    fmt.Println("   Figlet name invalid. Available : ")
                    fmt.Println("       - graffiti")
                    fmt.Println("       - standard")
                    fmt.Println("       - shadow")
                    fmt.Println("       - thinkertoy")
                }
            }
            fichier, err := os.Open(police)

            if err != nil {
                fmt.Println("~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
                return
            }

            scanner := bufio.NewScanner(fichier) // Scan le fichier

            scanner.Split(bufio.ScanLines) // split le scan en lignes

            var lignes []string // englobe toutes les lignes
            for scanner.Scan() {
                lignes = append(lignes, scanner.Text())
            }

            asciiChrs := make(map[int][]string) // mise en place de la map (vide pour l'instant)

            dec := 31 // correspond a la case zero de la map

            for _, ligne := range lignes { //ligne = une/chaques ligne dans l'intervalle lignes du fichier txt
                if ligne == "" {
                    dec++
                } else {
                    asciiChrs[dec] = append(asciiChrs[dec], ligne)
                }
            }
            userInput := os.Args[1]
            invCharCount := 0

            for _, invalidChar := range userInput {

                if invalidChar < 31 || invalidChar > 126 {
                    invCharCount++
                    fmt.Println("Caractère non supporté: ", string(invalidChar))

                }
            }

            if invCharCount > 0 {
                fmt.Println("Try Again")
                os.Exit(0)
            } else if userInput == "\\n" {
                fmt.Print("\n")
                return
            } else if userInput == "" {
                return
            } else {
                Newline(userInput, asciiChrs)

            }
        } else {
            fmt.Println("Usage: go run . [STRING] [BANNER]\n\nEX: go run . something standard")

        }

    }

}

func Newline(n string, y map[int][]string) {
    replaceNewline := strings.Replace(n, "\\n", "\n", -1)

    wordsSlice := strings.Split(replaceNewline, "\n")
    slice := []string{}
    slice2 := []string{}
    slice3 := []string{}
    slice4 := []string{}
    slice5 := []string{}
    slice6 := []string{}
    slice7 := []string{}
    slice8 := []string{}

    for _, mot := range wordsSlice {
        //for j := 0; j < 8; /*lignes*/ j++ {
        for _, lettre := range mot {
            //fmt.Print(y[int(lettre)][j])
            slice = append(slice, (y[int(lettre)][0]))
            slice2 = append(slice2, (y[int(lettre)][1]))
            slice3 = append(slice3, (y[int(lettre)][2]))
            slice4 = append(slice4, (y[int(lettre)][3]))
            slice5 = append(slice5, (y[int(lettre)][4]))
            slice6 = append(slice6, (y[int(lettre)][5]))
            slice7 = append(slice7, (y[int(lettre)][6]))
            slice8 = append(slice8, (y[int(lettre)][7]))
        }
        fmt.Println()
        //  }
    }
    f, err := os.OpenFile("banner.txt", os.O_APPEND|os.O_WRONLY, 0600)
    if err != nil {
        panic(err)
    }
    defer f.Close()

    outpout := strings.Join(slice, "")
    outpout2 := strings.Join(slice2, "")
    outpout3 := strings.Join(slice3, "")
    outpout4 := strings.Join(slice4, "")
    outpout5 := strings.Join(slice5, "")
    outpout6 := strings.Join(slice6, "")
    outpout7 := strings.Join(slice7, "")
    outpout8 := strings.Join(slice8, "")

    if _, err = f.WriteString(outpout); err != nil {
        panic(err)
    }

    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout2); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout3); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout4); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout5); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout6); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout7); err != nil {
        panic(err)
    }
    if _, err = f.WriteString("\n"); err != nil {
        panic(err)
    }
    if _, err = f.WriteString(outpout8); err != nil {
        panic(err)
    }

}

So, my biggest problem here is whenever i type something like this :
go run . "Hello\nThere" standard --output="banner.txt"
The \n don't count as a \n and do nothing.
I hope someone can help me for this ! And if you got any question just tell me :)
Comment: Does this answer your question? [bash - How to pass a line feed to a script?](https://stackoverflow.com/questions/38642312/bash-how-to-pass-a-line-feed-to-a-script)
Comment: Shells don't interpret escape sequence in double quoted strings. Try `go run . $'Hello\nthere'` (This is for bash. If you use some other shell the syntax may be different).
Comment: FYI, Go provies the `flags` package to handle command line flags. The way you're handling them manually right now is very brittle and won't work the way people expect flags to work (ie, the position of `--output` shouldn't matter).
Answer: This is a bit tricky, because the shell does not interpret escape sequences (as clarified by Peter in his comment).
A way to solve that is passing the parameter as \\n so, for example:
go run . "Hello\\nThere" standard --output="banner.txt"

and then, in your code, you can use the Replace function from the strings package as follows on each parameter whose newline characters need to be considered:
output := strings.Replace(os.Args[1], "\\n", "\n", -1)

It is not a super clean solution but it does the job.
Comment: This has nothing to do with Go (and it certainly doesn't "escape command line parameters"). The shell doesn't interpret escape sequences, that's all, so the argument contains the two bytes '\' and 'n' instead of the single byte 0x0a.
Comment: Thank you for the clarification, I'm going to update my answer!


How to compare CNN models with non-reproducible results?

Question: I try to compare different CNN models. I use Keras and for training, I use a GPU, Google Colab with Tensorflow backend.
Unfortunately I'm not able to create the same initial conditions for the CNNs (or in other words: I always get different results).
Although by putting the following lines at the top of the code, I get always different results after every run.
from numpy.random import seed
seed(1)
from tensorflow import set_random_seed
set_random_seed(2)

Can it be that it is simply not possible  to get reproducible results?
Would it be the best way to simply repeat the training several times and then either calculate a mean (if possible) or simply practice ensemble learning?
All without using seeds or random_states or shuffle=False etc.
What would be the best way to compare these models?
Comment: Not an answer, but one issue is that [`tf.set_random_seed`](https://www.tensorflow.org/versions/r1.14/api_docs/python/tf/random/set_random_seed) ([`tf.random.set_seed`](https://www.tensorflow.org/api_docs/python/tf/random/set_seed) in TensorFlow 2) sets the seed for the default graph, but not for individual operations that might have their own random number generators
Comment: Oh, that sounds like looking for the needle in a haystack. Actually, I wanted to make this comparison only to determine which model best classifies a given data set
Comment: You can specify a seed for each operation that involves randomness. This includes obvious things like dropout (both [`tf.nn.dropout`](https://www.tensorflow.org/api_docs/python/tf/nn/dropout) and [`tf.keras.layers.Dropout`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout) accept a `seed` parameter) as well as more subtle things like the random initialization of weights for dense or convolutional layers (e.g., you can pass a `seed` parameter to the initializers in the [`tf.keras.initializers`](https://www.tensorflow.org/api_docs/python/tf/keras/initializers) module).
Comment: ok, even if I succeed in producing reproducible models, as a beginner in deep learning the question that arises for me is, whether I have really created the best conditions for comparing the performance of different models? Maybe I influence indirectly the so-called randomness in such a way that the result is forced indirectly in a certain direction and the randomness no longer really exists. Wouldn't it be the more reliable way to repeatedly run the models as mentioned above (repeating for example 5 times for each model)?
Answer: From your question it's not clear what you mean with different CNN models. Are you are comparing two models with different architecture, or two instances of the same model with different weights? Since I understand that you are a beginner in deep learning, I'll try to answer for both of the scenarios.
Comparing two different model architectures
In this case the answer is straightforward: compare the average performances of the two models. Run the training several times, introducing diversity in train/validation/test data performing the experiments, for example, in cross validation. You should use the same splits of your dataset for both of the models.
Reproduce results of the same model
This is more tricky. If you are using tensorflow, you should be aware that
a "random" operation is in fact ruled by two different seeds: a global seed, set by tf.set_random_seed, and an operation seed, provided as an argument to the operation itself. Once you have set the global seed, the system deterministically picks an operation seed in conjunction with the graph-level seed so that it gets a unique random sequence. However, it's important to set it in the right moment. You probably want to use the same graph and same operations to get the same random numbers in different sessions, so you should set the seed just before running the tf.session.
You can read more about this topic in this discussion
Comment: I'm comparing more than 5 models with different architectures and I use the Fashion MNIST data set. How many times do I have to repeat the run for each model? Since the test data set consists of 10000 images, I could do without an additional split in train/validation/test data, right? Would that mean that in this case I don't have to set individual seeds (for reproducibility) for each run?   An additional problem is that I have to optimize hyperparameters (via GridSearch), e.g. learning rate, number of epochs, before training. Do I not need seeds at least for tuning?
Comment: If the models' performances have high variance, you should repeat the runs more; otherwise, you can stick to lower number of iterations. With the experiment set that you have, I'd proceed like this: you can keep fixed the test, but produce N random splits of train/val. You will run GridSearch N times, training on train set and validating on val set. The best model coming from the GridSearch will be tested on the fixed test set. At the end of the day, you will have N test scores to average, for each of your CNN models. The only seed you need to set is the one ruling the train/val split.
Comment: You can also think to fix the tf.seed to have similar conditions for the neural networks, but if the models have different architecture I wouldn't care too much about this - differences in architectures are more important than differences in initializations. One thing that for sure could be kept the same for all the models is the parameters grid for the GridSearch.
Comment: OK, to sum up your suggestion: run GridSearchCV N times for each model (each of these N runs of GridSearchCV with a different random_state seed for StratifiedKFold). Then testing each of the N best models with the fixed test set and average the results with calculation of the standard deviation, right?  And then it is important to note that the different random_state seeds of the N runs of GridSearchCV are the same for each model.
Comment: Exactly. This is the procedure I'd follow. If you are satisfied with the answer don't forget to vote my answer ;)


ANOVA: Confidence Interval interpretation

Question: I'm reading through an example of 1-way ANOVA:

In an experiment to determine the effect of nutrition on the attention spans of ele- mentary school students, a group of 15 students were randomly assigned to each of three meal plans: no breakfast, light breakfast, and full breakfast. Their attention spans (in minutes) were recorded during a morning reading period and are shown in Table 11.1:
  None  |  Light  |  Full
---------------------------
    8   |   14    |  10
    7   |   16    |  12
    9   |   12    |  16
   13   |   17    |  15
   10   |   11    |  12
T1 = 47 | T2 = 70 | T3 = 65

The question is:

The researcher in Example 11.4 believes that students who have no breakfast will have significantly shorter attention spans but that there may be no difference between those who eat a light or a full breakfast. Find a 95% confidence interval for the average at- tention span for students who eat no breakfast, as well as a 95% confidence interval for the difference in the average attention spans for light versus full breakfast eaters.

And the given solution is:

I totally understand the solution, but I don't understand how they came up with this conclusion (emphasis mine):

You can see that the second confidence interval does not indicate a difference in the average attention spans for students who ate light versus full breakfasts, as the researcher suspected. If the researcher, because of prior beliefs, wishes to test the other two possible pairs of means—none versus light breakfast, and none versus full breakfast—the methods given in Section 11.6 should be used for testing all three pairs.

I may be completely missing something simple, but I just don't understand how they came about the conclusion in bold. What property of the 1 ± 3.36 result suggests there's no difference in the average attention spans?
Answer: A rule of thumb is that if the Confidence Interval (CI) does not include a 0, the difference between the two groups is statistically significant. In your example, the CI range is (-2.36 and 4.36) which does include a 0, therefore the two groups are not significantly different from one another. Or in other words, a difference in the average attention span does not exist.


Are link only answers allowed on Expatriates?

Question: I came across this answer in the late answer review queue which was a link only answer.
The link that the user gave did not belong to any official or government website. It is more of a forum and support website. I understand, links are generally helpful to prove the credibility of the content. However there is nothing else besides the link in that answer.
There is a mod comment, asking the OP for explanation but there are no updates after that.
I flagged it as VLQ, however it was declined by a moderator.
So my question is, are link only answers allowed on Expatriates?
I don't have a problem with this. I just want to know the right action in this case.
Comment: In this case I am more worried that the answer is SPAM and that the poster might be related to the promoted company.
Comment: It's borderline for sure. But it does provide some info related to the question. I'm inclined to provide the benefit of the doubt. But in general yeah definitely a borderline.
Answer: I was the moderator checking that answer. While it's definitely not great, it is not a link-only answer, as it does include at least some information (that you can use a B1). I added some notes that most of the information from the link should be embedded into the answer itself, not just the headline (which is an answer in itself - albeit not a great one).
Whether the information is true or not, and whether the source is reputable or not is not up to the moderators to decide. If you believe it is not a good answer or it is misleading please downvote it, and potentially add a comment on why you believe the answer is bad, and potentially include details on how it can be improved. Alternatively, edit the question and include the more details. You should have enough reputation to do that already.
Comment: Thanks for the clarification. I just wasn't sure that information like `you can use a B1` can be considered answer or not. I will keep this in mind for future reviews.
Comment: @DipenShah we don't really a have a policy on what we consider a proper answer. It might be worth starting a new meta topic on it. Some SE sites are for example quite relaxed on it (e.g. the linked answer would be considered an answer), while others are very strict. I don't remember whether we have any policy on this or not


Background image 100%

Question: Sempre tive essa curiosidade, de como deixar a imagem de background 100% de acordo com o monitor ( Todos nós sabemos que existem diversas medidas). Então me disseram que seria um script em JS ou jQuery, não me lembro muito bem, e ele funcionava da seguinte forma:
Você cria diversas imagens com as seguintes medidas:
1024x768-
1280x768-
1360x768-
1366x768-

São resoluções do meu notebook, pode ser que tenham outras medidas.
Quando você acessa o site, esse script faz a leitura da medida e carrega tal imagem.
é dessa forma mesmo que funciona? Teria a possibilidade de fazer isso no CSS?
Exemplo em um site : http://www.adobe.com/br/products/dreamweaver.html
Comment: background image 100% tu não precisa de js, é só por width e height 100%. Mas sobre o que falou de adaptação de acordo com a resolução, pode usar media queries...
Answer: Existem diversas técnicas para obter uma imagem a ocupar toda a largura e altura do ecrã. Cada técnica é considerada adequada para determinado cenário pois podemos lidar com o problema via JavaScript, CSS ou no lado do servidor.
Nesta resposta as soluções fazem uso de JavaScript com recurso a jQuery.
jQuery
Imagem consoante largura do ecrã
Este Plugin funciona da forma que tu indicas na pergunta, ou seja, ele faz uso de diversas imagens carregando a mais adequada consoante a largura do ecrã:
(function() {

var win = $(window);

win.resize(function() {

    var win_w = win.width(),
        win_h = win.height(),
        $bg    = $("#bg");

    // Carregar imagem de fundo mais estreita com base na
    // largura do ecrã, mas nunca carregar nada mais estreito
    // do que o que já está carregado, se alguma coisa estiver.
    var available = [
      1024, 1280, 1366,
      1400, 1680, 1920,
      2560, 3840, 4860
    ];

    var current = $bg.attr('src').match(/([0-9]+)/) ? RegExp.$1 : null;

    if (!current || ((current < win_w) && (current < available[available.length - 1]))) {

      var chosen = available[available.length - 1];

      for (var i=0; i<available.length; i++) {
        if (available[i] >= win_w) {
          chosen = available[i];
          break;
        }
      }

      // Definir a nova imagem
      $bg.attr('src', '/img/bg/' + chosen + '.jpg');

      // para testar...
      // console.log('Chosen background: ' + chosen);

    }

    // Determinar se a largura ou a altura deve ser de 100%
    if ((win_w / win_h) < ($bg.width() / $bg.height())) {
      $bg.css({height: '100%', width: 'auto'});
    } else {
      $bg.css({width: '100%', height: 'auto'});
    }

  }).resize();

})(jQuery);

Código retirado do web site CSS-Tricks neste artigo:
Perfect Full Page Background Image | CSS-Tricks

Vegas Background jQuery Plugin
Muito fácil de utilizar, permite-nos transformar uma simples  numa imagem a ocupar 100% X 100% do ecrã, bem como indicar a imagem no próprio PlugIn:

Incluir o Script após a inclusão do jQuery:

Incluir o CSS:
<link rel="stylesheet" type="text/css" href="/vegas/jquery.vegas.css" />

Iniciar o PlugIn:
$(function() {
  $.vegas({
    src:'/images/background.jpg'
  });
});

Vegas - Demonstração
Answer: Podes fazer isso com CSS da seguinte maneira: http://jsfiddle.net/ez7eq1Lm/
#banner{
    background:url('https://dq197.infusionsoft.com/Download?Id=2300') no-repeat;
    background-size:contain;
    width:100%;
    height:260px;
}

O valor contain (Inglês) na propriedade background-size permite:

Esta palavra-chave especifica que a imagem de fundo deve ser dimensionada para ser tão grande quanto possível, assegurando ao mesmo tempo que as suas dimensões são menores ou iguais às dimensões correspondentes da zona de posicionamento do fundo.
Answer: Dá pra fazer pelo CSS assim:
body {
    background: url(caminho_para_a_imagem);
    background-size: cover;
}

cover: Redimenciona a imagem para preencher completamente a area do background, mantendo a proporção.
Caso não se importe com a proporção, você pode usar:
body {
    background: url(caminho_para_a_imagem);
    background-size: 100% 100%;
}

Existem outros parâmetros, escolha o que atende melhor as suas necessidades.

Background image 100%


Inserir uma opção no select em um determinado indice

Question: Possuo o seguinte select:
<select id="selectChannelCam" class="custom-select">
    <option value="" disabled selected>Selecione</option>
</select>

Como podem ver, possui apenas uma opção que não é nem selecionável. Eu gostaria de adicionar novas opções, mas de forma ordenada.
Ordenar é facil, mas eu preciso inserir em um índice especifico, pois é possivel que seja necessário inserir por exemplo, o canal 2 antes do 1.

Já tentei:
$('#selectChannelCam :nth-child(' + index + '))').after("<option value='" + keys[0] +"'>" + keys[0] + "</option>");

e
$("#selectChannelCam :nth-child(" + index + ")").append(new Option(keys[0], keys[0]));

Comment: Pode ordenar esse vetor antes e quando for popular o select será apenas inserir.
Comment: As opções vêm de onde?
Não tem já um Array de Opções e depois apenas as insere por ordem?
Comment: Elas dependem de uma outro cadastro. Quando o usuario cadastrar um canal, esse canal deve ser adicionado a esse select. Pego tudo de um arquivo
Comment: Então tem como objetivo adicionar opções dinamicamente usando um arquivo para guardas as opções cadastradas pelo usuario, correto?
Comment: Exatamente. Já consegui colocar essas opções em um vetor, falta adicionar ao select (pois no arquivo há outras informações tambem, então achei melhor já colocar no vetor)
Answer: Você pode usar o eq(n) para selecionar o número do índice, e em seguida before() ou after() para inserir o novo elemento antes ou depois respectivamente.
Veja esse exemplo:

$("#btn").click(function() {
    $("#selectChannelCam option").eq(2).before($("<option>Uma nova opção</option>").text("Nova Opção"));
});

<select id="selectChannelCam" class="custom-select">
    <option value="" disabled selected>Selecione</option>
    <option value="1">1</option>
    <option value="2">2</option>
</select>

   <button id="btn">Inserir valor</button>

O eq(2) seleciona o elemento com índice 2, e insere um novo option antes (before).
Comment: como faço para que as opções sejam variaveis em vez de um texto chumbado?
Comment: é só fazer como no seu exemplo que tem `+ index +` por exemplo
Answer: Como inserir uma opção num elemento Select:
// referência para elemento select
var sel = document.getElementById('selectChannelCam');

// criar um elemento opção
var opt = document.createElement('option');

// Aqui atribui o valor de texto que tem guardado no seu vetor
opt.appendChild( document.createTextNode('Texto que aparece na sua Opção') );

// Aqui atribui o valor da opção que tem guardado no seu vetor
opt.value = 'option value';

// Adicionar a opção ao final do elemento select
sel.appendChild(opt);


How to Correctly Multi-Thread a DLL Called at Run-Time in C#

Question: All,
I wish to write a plugin .dll to be used/called by a .NET application called at runtime. My .dll is a WinForm and displays the (computationally expensive) operations being undertaken. The .dll is called from the main application are invoked via .NET System.Reflection. I have to provide to the calling application the NameSpace, Class and the Method I want to invoke.
I would like to multi-thread my .dll so that it is more UI friendly and I am only really familiar with BackgroundWorkers.
EDIT: Extension to question.
So, I call the .dll as follows:
if (classType != null)
{
    if (bDllIsWinForm)
    {
        classInst = Activator.CreateInstance(classType);
        Form dllWinForm = (Form)classInst;
        dllWinForm.Show();

        // Invoke required method.
        MethodInfo methodInfo = classType.GetMethod(strMethodName);
        if (methodInfo != null)
        {
            object result = null;
            // The method being called in this example is 'XmlExport'.
            result = methodInfo.Invoke(classInst, new object[] { dllParams });
            return result.ToString();
        }
    }
    else
    {
        // Else not a WinForm do simalar.
    }
}

So then in the WinForm .dll I want to multi-thread the time consuming work so that I can display what is going on. So in the .dll, using BackgroundWorker I have:
BackgroundWorker bgWorker; // Global.

public string XmlExport(object[] connectionString)
{
    try
    {
        bgWorker = new BackgroundWorker { WorkerReportsProgress = true, WorkerSupportsCancellation = true };
        bgWorker.DoWork +=  new DoWorkEventHandler(bgWorker_DoWork);
        bgWorker.ProgressChanged += new ProgressChangedEventHandler(bgWorker_ProgressChanged);
        bgWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(bgWorker_RunWorkerCompleted);

        // Wait until workerThread is done.
        threadDoneEvent = new AutoResetEvent(false);
        bgWorker.RunWorkerAsync();
        threadDoneEvent.WaitOne();
        return strResult; // Global String strResult
    }
    catch (Exception)
    {
        throw;
    }
}

Then I have the DoWork event handler:
void bgWorker_DoWork(object sender, DoWorkEventArgs e)
{
    BackgroundWorker thisWorker = sender as BackgroundWorker;
    strResult = (string)this.XmlExportBgw(ref thisWorker); // Or should I use bgWorker?
    e.Result = strResult;
}

public string XmlExportThreaded(ref BackgroundWorker thisWorker)
{
    try
    {
        // Some expesive work...

        // UI.
        InfoBall infoBall = new InfoBall(); // Class containing processing information.
        // Set infoBall parameters here...
        (thisWorker as BackgroundWorker).ReportProgress(infoBall.progressBarValue, infoBall);

        // Some more expensive work...

        // UI.
        (thisWorker as BackgroundWorker).ReportProgress(infoBall.progressBarValue, infoBall);
    }
    //...
}

The `ProgressChanged' event is
void bgWorker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
    // UI.
    InfoBall someBall = (InfoBall)e.UserState;

   // Update process information.
   if (someBall.showRichTextBox)
   {
       this.richTextBox.AppendText(someBall.richTextBoxText);
       this.richTextBox.ScrollToCaret();
   }
   return;
}

In addition to the above code I have the usual RunWorkerCompleted etc.
The calling application has been written to allow the user to call thereown .NET .dlls at runtime. The .dll I am trying to put together is a time intensive one and one that will only be supplied to particular users. I have run the above code and the problem is that it will not update the user interface correctly. That is that you cannot manipulate (resize, click etc.) the Form and it does not print and progress information until the very end of the processing when it only prints the final message out multiple times. However, it is correctly producing the .xml files I require. What am I doing wrong? Should I be invoking the .dll method from a separate thread?
Any help would be greatly appreciated. Thanks very much for your time.
Comment: Remove the threadDoneEvent.WaitOne() call.  Starting a thread and then waiting on it completely defeats the reason for using a thread.  And prevents the RunWorkerCompleted event from running, the one you want to use to process the thread result.
Comment: The ProgessChanged event won't fire either.  You cannot wait.
Comment: I don't want to pass back to the calling application until the .dll has done its work though. This way, the thread does its work and the WinForm that displays what the .dll is doing remains open. The main issue is that the UI for my .dll is not updating properly? Thanks very much for your time.
Comment: One more thing though. How then do I stop the calling application proceeding before the .dll is finished? Do I create another thread then invoke the .dll from that, waiting on this 'master' thread? What would be best here? Thanks again...
Comment: You don't stop it from proceeding, you *continue* proceeding when you got a result.  That's why the RunWorkerCompleted event is there.  Whatever you want to do after the wait call, move it to an event handler for that event.  The more formal definition of it is a state machine.  The *async* and *await* keywords in the next version of C# make it easier.  Asynchronous programming is a bit of an art without that.
Answer: I am not sure what kind of methods you are tryng to execute in that dll. I am assuming they are not asynchronous meaning your main thread (application) will stop until they are done executing. An easy example could be demonstrated with the Messagebox.show("myMessage"); method. For example how could you execute 3 message boxes at once? it will not be possible without using multiple threads. Hope this methods helps:
    public void SomeMethod()
    {

        Thread thread = new Thread(new ThreadStart(() =>
        {
            // this code is going to be executed in a separate thread
            MessageBox.Show("hello");

            // place additional code that you need to be executed on a separate thread in here

        }));

        thread.Start();
    }

then if I call that method 3 times as:

I will now be running 4 threads on my program. The main thread plus the 3 calls that I created.
Comment: Thank you that is great, it has reminded me about using threading this way! But I cannot launch the .dll this way as I don't have access to the calling application source code - I can only pass a namespace & class & method. However, say my WinForm class in the .dll is one of your `MessageBoxes` I want to update that message box with progress information as well, which means I want to multi-thread my method that does all the work but provide information to the user via the form. I was thinking about using the *The Event-Based Asynchronous Pattern*? Thank you very much for your answer.