Hibernate: MappingException on relationship with two @JoinColumn's with additional PK read-only mapping for one of its columns

Question: Beware of the following DB design:

NOTE: Never mind what the creator of this "design" had in mind, but I cannot change it.
It's about the relationship between MobileTans and Authenticators...
The respective mappings of the MobileTans table:
@Entity
@Table(name="MobileTans")
public class MobileTan implements Serializable
{
    private static final long serialVersionUID = 1L;

    @Id
    @Column(name="ID", insertable=false, updatable=false)
    private Integer id;    // dupe read-only field, ID already in relationship below (writable)

    @Basic(optional=false)
    @Column(name="MOBILE_PHONE_NBR")
    private String mobilePhoneNbr;

    @Basic
    @Column(name="ACTIVATION_CODE")
    private String activationCode;

    @Basic
    @Column(name="ACTIVATION_ERROR")
    private Boolean activationError = Boolean.FALSE;

    @Basic
    @Column(name="STATUS")
    private Integer status;

    @OneToOne(optional=false, fetch=FetchType.EAGER)
    @JoinColumn(name="ID", referencedColumnName="ID")
    @JoinColumn(name="AUTH_NAME", referencedColumnName="AUTHENTICATOR")
    private Authenticator authenticator;

    ...
}

The clue here is the duplicate read-only mapping (insertable=false, updatable=false) for the @Id column.
The ID column is defined to be the PK, so there's no use in setting the @Id onto the relationship. I must create an extra column mapping. I put the read-only on the extra column, because Hibernate (or JPA) won't let me specify read-only only on parts of a relationship (I think this is forbidden by the JPA spec).
EDIT: Authenticator entity
@Entity
@Table(name="Authenticators")
@IdClass(AuthenticatorPk.class)
public class Authenticator implements Serializable
{
    private static final long serialVersionUID = 1L;

    @Id
    @Column(name="ID")
    private Integer id;

    @Id
    @Column(name="AUTHENTICATOR")
    private String authenticator = "";

    @Basic(optional=false)
    @Column(name="PROCEDURE")
    private String procedure;

    @Temporal(TemporalType.DATE)
    @Basic
    @Column(name="VALID_FROM")
    private Date validFrom;

    @Temporal(TemporalType.DATE)
    @Basic
    @Column(name="VALID_TO")
    private Date validTo;

    public Authenticator()
    {
    }

    ...
}

PK class:
public class AuthenticatorPk implements Serializable
{
    private static final long serialVersionUID = 1L;

    private Integer id;

    private String authenticator = "";

    public AuthenticatorPk()
    {
    }

    ...
}

When launching the server, I get a mapping exception from Hibernate:
00:37:53,979 INFO  [org.hibernate.Version] (ServerService Thread Pool -- 72) HHH000412: Hibernate Core {5.3.6.Final}
00:37:53,980 INFO  [org.hibernate.cfg.Environment] (ServerService Thread Pool -- 72) HHH000206: hibernate.properties not found
00:37:54,088 INFO  [org.hibernate.annotations.common.Version] (ServerService Thread Pool -- 72) HCANN000001: Hibernate Commons Annotations {5.0.4.Final}
00:37:54,382 INFO  [org.hibernate.validator.internal.util.Version] (MSC service thread 1-2) HV000001: Hibernate Validator 6.0.13.Final
00:37:54,621 INFO  [org.jboss.weld.Version] (MSC service thread 1-3) WELD-000900: 3.0.5 (Final)
00:37:54,674 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 72) WFLYCLINF0002: Started client-mappings cache from ejb container
00:37:54,777 INFO  [org.jboss.as.jpa] (ServerService Thread Pool -- 72) WFLYJPA0010: Starting Persistence Unit (phase 2 of 2) Service 'mappingbug.war#BBStatsPU'
00:37:54,806 INFO  [org.hibernate.dialect.Dialect] (ServerService Thread Pool -- 72) HHH000400: Using dialect: org.hibernate.dialect.MySQL57Dialect
00:37:54,859 INFO  [org.hibernate.envers.boot.internal.EnversServiceImpl] (ServerService Thread Pool -- 72) Envers integration enabled? : true
00:37:55,007 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 72) MSC000001: Failed to start service jboss.persistenceunit."mappingbug.war#BBStatsPU": org.jboss.msc.service.StartException in service jboss.persistenceunit."mappingbug.war#BBStatsPU": javax.persistence.PersistenceException: [PersistenceUnit: BBStatsPU] Unable to build Hibernate SessionFactory
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:195)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:125)
    at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:650)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:209)
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
    at java.lang.Thread.run(Thread.java:748)
    at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: BBStatsPU] Unable to build Hibernate SessionFactory
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.persistenceException(EntityManagerFactoryBuilderImpl.java:1016)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:942)
    at org.jboss.as.jpa.hibernate5.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:167)
    ... 9 more
Caused by: org.hibernate.MappingException: Repeated column in mapping for entity: net.bbstats.entity.MobileTan column: ID (should be mapped with insert="false" update="false")
    at org.hibernate.mapping.PersistentClass.checkColumnDuplication(PersistentClass.java:862)
    at org.hibernate.mapping.PersistentClass.checkPropertyColumnDuplication(PersistentClass.java:880)
    at org.hibernate.mapping.PersistentClass.checkColumnDuplication(PersistentClass.java:902)
    at org.hibernate.mapping.PersistentClass.validate(PersistentClass.java:634)
    at org.hibernate.mapping.RootClass.validate(RootClass.java:267)
    at org.hibernate.boot.internal.MetadataImpl.validate(MetadataImpl.java:347)
    at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:466)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:939)
    ... 11 more

00:37:55,011 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "mappingbug.war")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.persistenceunit.\"mappingbug.war#BBStatsPU\"" => "javax.persistence.PersistenceException: [PersistenceUnit: BBStatsPU] Unable to build Hibernate SessionFactory
    Caused by: javax.persistence.PersistenceException: [PersistenceUnit: BBStatsPU] Unable to build Hibernate SessionFactory
    Caused by: org.hibernate.MappingException: Repeated column in mapping for entity: net.bbstats.entity.MobileTan column: ID (should be mapped with insert=\"false\" update=\"false\")"}}
00:37:55,018 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 42) WFLYSRV0010: Deployed "mappingbug.war" (runtime-name : "mappingbug.war")
00:37:55,019 INFO  [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0183: Service status report
WFLYCTL0186:   Services which failed to start:      service jboss.persistenceunit."mappingbug.war#BBStatsPU": javax.persistence.PersistenceException: [PersistenceUnit: BBStatsPU] Unable to build Hibernate SessionFactory
WFLYCTL0448: 19 additional services are down due to their dependencies being missing or failed
00:37:55,079 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
00:37:55,081 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
00:37:55,081 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
00:37:55,082 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026: WildFly Full 14.0.1.Final (WildFly Core 6.0.2.Final) started (with errors) in 14146ms - Started 449 of 645 services (23 services failed or missing dependencies, 328 services are lazy, passive or on-demand)

BTW, I am using Hibernate Core 5.3.6.Final on Wildfly 14.
QUESTION:
What's wrong here? Why does Hibernate complain about this? My impression is that this is perfectly legal.
Other chances are that this is a Hibernate bug...?
Note, that I am more interested in knowing whether this should work in Hibernate or not. I am not really interested in any non-JPA alternative mappings. I can get by this by putting the read-only stuff onto the relationship, then this works, but not the described mappings above, which should... you get it :-)
Comment: Can you post your Authenticator class so that we answerers can work on your exact situation?
Comment: Sure. See above.
Answer: I can almost get what you are looking for with this mapping here:
    @Entity
@Table(name = "mobile_tan")
public class MobileTan  {
    private static final long serialVersionUID = 1L;
    @Id
    @Column(name="ID", insertable=false, updatable=false)
    private Integer id;    // dupe read-only field, ID already in relationship below (writable)

    @Basic(optional=false)
    @Column(name="MOBILE_PHONE_NBR")
    private String mobilePhoneNbr;

    @Basic
    @Column(name="ACTIVATION_CODE")
    private String activationCode;

    @Basic
    @Column(name="ACTIVATION_ERROR")
    private Boolean activationError = Boolean.FALSE;

    @Basic
    @Column(name="STATUS")
    private Integer status;

    @OneToOne(fetch = FetchType.LAZY)
    @MapsId
    @JoinColumn(name="AUTH_NAME", referencedColumnName="AUTHENTICATOR")
    private Authenticator authenticator;
}

However, this creates a table with the following definition (HSQL)
    create table mobile_tan (
    AUTH_NAME integer not null,
    ACTIVATION_CODE varchar(255),
    ACTIVATION_ERROR boolean,
    MOBILE_PHONE_NBR varchar(255) not null,
    STATUS integer,
    primary key (AUTH_NAME)
)

When I run the following code:
    @Test
public void createMobileTan() {
    MobileTan tan = createMobileTan();
    session.flush();
    System.err.println("Tan:  " + tan.getId());
    System.err.println("Authenticator:  " + tan.getAuthenticator().getId());
    System.err.println("AUTH COLUMN:  " + session.createSQLQuery("select AUTH_NAME from mobile_tan").addScalar("AUTH_NAME", IntegerType.INSTANCE).setMaxResults(1).uniqueResult());
    System.err.println("AUTH COLUMN:  " + session.createSQLQuery("select ID from mobile_tan").addScalar("ID", IntegerType.INSTANCE).setMaxResults(1).uniqueResult());
}

public MobileTan createMobileTan() {
    Authenticator auth = createAuthenticator();

    MobileTan tan = new MobileTan();

    tan.setAuthenticator(auth);
    tan.setMobilePhoneNbr("0123456");

    session.persist(tan);

    return tan;
}

private Authenticator createAuthenticator() {
    Authenticator auth = new Authenticator();

    session.persist(auth);

    return auth;
}

I get the following output:
Tan:  1
Authenticator:  1
AUTH COLUMN:  1

java.sql.SQLSyntaxErrorException: user lacks privilege or object not found: ID in statement [select ID from mobile_tan limit ?]
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
    at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
    at org.hsqldb.jdbc.JDBCPreparedStatement.<init>(Unknown Source)
    at org.hsqldb.jdbc.JDBCConnection.prepareStatement(Unknown Source)
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:146)
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:172)

The table is created using a primary key that is also a foreign key. As you can see, it sets the object relationships correctly and I can access this using the getId() method in the MobileTan class. But, as you can also see in the table definition DDL there is only one column that is physically created in the database. Do you really need 2 physical columns? Then you can decide to use only one column and also use this single column as the foreign key. Then drop the extra column.
I created my example using the information included in this link here: set-primary-key-and-foreign-key-on-a-single-column
Comment: Thanks for your efforts. The problem is, we simply cannot change the DB design at hand. We are discussing a few things with the customer, but they are very reluctant to any DB changes. That's part of the sh...
Comment: What about a trigger? You could remove the not-null constraint on the additional column and then keep it synchronized with a trigger. Not the prettiest solution, but it would do the job.
Comment: I awarded the bounty since it gives me an alternative that I can try (but haven't yet). Thanks. But to some extent, I still get the impression that Hibernate pretends to stable here but isn't.


Prove that $x$ is algebraic over $k(y)$

Question: Let $k$ be a field and $K$ be its extension field.
Suppose $y \in K$ is algebraic over $k(x)$ for some $x \in K$ and $y$ is transcendental over $k$.
Then $x$ is algebraic over $k(y)$.
I think one way is the following,
Let $ f_0(x)/g_0(x) + f_1(x)/g_1(x)y +  f_2(x)/g_2(x)y^2 + f_n(x)/g_n(x)y^n=0$.
so if we take the numerator after adding we get,
 $f_0(x)g_1(x)...g_n(x) + f_1(x)g_0(x)...g_n(x)y + f_2(x)g_0(x)...g_n(x)y^2+...+f_n(x)g_0(x)..g_{n-1}(x)y^n=0$
Is there any other cleaner way to prove the above statement?
Answer: Not really a cleaner way, essentially your argument in fancy language: If $x$ is not algebraic over $k(y)$, then $k(Y)(X)=k(X,Y)\rightarrow k(x,y)$ would be an isomorphism of fields. But the condition $y$ algebraic over $k(x)$ shows that the kernel is not $0$.
Answer: Yes, there is a cleaner way, in some sense, using the transcendence degree.
Consider $k \subset k(y) \subset k(x,y)$ and $k \subset k(x) \subset k(x,y).$ We will have
$$tr.deg(k(x,y)/k)=tr.deg(k(x,y)/k(y))+tr.deg(k(y)/k)=tr.deg(k(x,y)/k(y))+1,$$
and
$$tr.deg(k(x,y)/k)=tr.deg(k(x,y)/k(x))+tr.deg(k(x)/k)=0+1=1.$$
Merging them, we will obtain, $tr.deg(k(x,y)/k(y))+1=1.$ Hence, $tr.deg(k(x,y)/k(y))=0,$ and $x$ is algebraic over $k(y).$
Note that, I have used the fact that an extension is algebraic $\iff$ its transcendence degree is zero.

Prove that $x$ is algebraic over $k(y)$


hostapd, rtl8192se and wpa not working: EAPOL-Key Timeout

Question: Ubuntu 12.04.2, 3.2.0-49-generic, x64
hostapd from saucy (hostapd_1.0-3ubuntu2_amd64)
Have tried from precise-updates as well to no avail.
You can see in the debug output below, hostapd sends the EAPOL-Key request to the client, but hears nothing back...
I've tried:

WPA only
WPA + WPA2
several hostapd versions from 0.7.6 onwards

Stations I've tried connecting with are a laptop (w7) and an iPhone 3S
hostapd.conf:
interface=wlan0
bridge=br0
driver=nl80211
ssid=McWilliamTest
country_code=AU
hw_mode=g
channel=11
wme_enabled=1
ieee80211n=1
ht_capab=[HT40-][SHORT-GI-40][DSSS_CCK-40]
eapol_key_index_workaround=0
eap_server=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=3
wpa_passphrase=test1234
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
macaddr_acl=0
logger_syslog=-1
logger_syslog_level=2
ctrl_interface_group=0
ctrl_interface=/var/run/hostapd

...and logging from hostapd -d /etc/hostapd/hostapd.conf:
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state INITIALIZE
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
wlan0: STA 70:f1:a1:ee:bd:53 IEEE 802.1X: unauthorizing port
mgmt::deauth cb
STA 70:f1:a1:ee:bd:53 acknowledged deauth
Removing STA 70:f1:a1:ee:bd:53 from kernel driver
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-DEAUTHENTICATE.indication(70:f1:a1:ee:bd:53, 2)
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-DELETEKEYS.request(70:f1:a1:ee:bd:53)
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
nl80211: Event message available
nl80211: Delete station 70:f1:a1:ee:bd:53
mgmt::auth
authentication: STA=70:f1:a1:ee:bd:53 auth_alg=0 auth_transaction=1 status_code=0 wep=0
wlan0: STA 70:f1:a1:ee:bd:53 IEEE 802.11: authentication OK (open system)
wlan0: STA 70:f1:a1:ee:bd:53 WPA: event 0 notification
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-AUTHENTICATE.indication(70:f1:a1:ee:bd:53, OPEN_SYSTEM)
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-DELETEKEYS.request(70:f1:a1:ee:bd:53)
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
authentication reply: STA=70:f1:a1:ee:bd:53 auth_alg=0 auth_transaction=2 resp=0 (IE len=0)
mgmt::auth cb
wlan0: STA 70:f1:a1:ee:bd:53 IEEE 802.11: authenticated
mgmt::reassoc_req
reassociation request: STA=70:f1:a1:ee:bd:53 capab_info=0x611 listen_interval=2 current_ap=00:1c:10:1b:68:22
Validating WMM IE: OUI 00:50:f2  OUI type 2  OUI sub-type 0  version 1  QoS info 0x0
  old AID 1
update_sta_no_ht STA 70:f1:a1:ee:bd:53 - no HT, num of non-HT stations 1
hostapd_ht_operation_update current operation mode=0x17
hostapd_ht_operation_update new operation mode=0x17 changes=0
wlan0: STA 70:f1:a1:ee:bd:53 IEEE 802.11: association OK (aid 1)
mgmt::reassoc_resp cb
wlan0: STA 70:f1:a1:ee:bd:53 IEEE 802.11: associated (aid 1)
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-REASSOCIATE.indication(70:f1:a1:ee:bd:53)
wlan0: STA 70:f1:a1:ee:bd:53 MLME: MLME-DELETEKEYS.request(70:f1:a1:ee:bd:53)
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
wlan0: STA 70:f1:a1:ee:bd:53 WPA: event 1 notification
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
IEEE 802.1X: Ignore STA - 802.1X not enabled or forced for WPS
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state AUTHENTICATION2
WPA: Assign ANonce - hexdump(len=32): 1e 04 83 e5 4e 02 47 68 d1 d0 1c cb a6 4d 4c 94 7d 70 09 a1 3d e5 40 0f 33 16 8c 4d 73 5d 23 a5
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state INITPSK
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state PTKSTART
wlan0: STA 70:f1:a1:ee:bd:53 WPA: sending 1/4 msg of 4-Way Handshake
WPA: Send EAPOL(version=1 secure=0 mic=0 ack=1 install=0 pairwise=8 kde_len=0 keyidx=0 encr=0)
WPA: Use EAPOL-Key timeout of 100 ms (retry counter 1)
nl80211: Event message available
nl80211: New station 70:f1:a1:ee:bd:53
wlan0: STA 70:f1:a1:ee:bd:53 WPA: EAPOL-Key timeout
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state PTKSTART
wlan0: STA 70:f1:a1:ee:bd:53 WPA: sending 1/4 msg of 4-Way Handshake
WPA: Send EAPOL(version=1 secure=0 mic=0 ack=1 install=0 pairwise=8 kde_len=0 keyidx=0 encr=0)
WPA: Use EAPOL-Key timeout of 1000 ms (retry counter 2)
wlan0: STA 70:f1:a1:ee:bd:53 WPA: EAPOL-Key timeout
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state PTKSTART
wlan0: STA 70:f1:a1:ee:bd:53 WPA: sending 1/4 msg of 4-Way Handshake
WPA: Send EAPOL(version=1 secure=0 mic=0 ack=1 install=0 pairwise=8 kde_len=0 keyidx=0 encr=0)
WPA: Use EAPOL-Key timeout of 1000 ms (retry counter 3)
wlan0: STA 70:f1:a1:ee:bd:53 WPA: EAPOL-Key timeout
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state PTKSTART
wlan0: STA 70:f1:a1:ee:bd:53 WPA: sending 1/4 msg of 4-Way Handshake
WPA: Send EAPOL(version=1 secure=0 mic=0 ack=1 install=0 pairwise=8 kde_len=0 keyidx=0 encr=0)
WPA: Use EAPOL-Key timeout of 1000 ms (retry counter 4)
wlan0: STA 70:f1:a1:ee:bd:53 WPA: EAPOL-Key timeout
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state PTKSTART
wlan0: STA 70:f1:a1:ee:bd:53 WPA: PTKSTART: Retry limit 4 reached
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state DISCONNECT
hostapd_wpa_auth_disconnect: WPA authenticator requests disconnect: STA 70:f1:a1:ee:bd:53 reason 2
wlan0: STA 70:f1:a1:ee:bd:53 WPA: event 3 notification
wpa_driver_nl80211_set_key: ifindex=5 alg=0 addr=0x22d6ca0 key_idx=0 set_tx=1 seq_len=0 key_len=0
   addr=70:f1:a1:ee:bd:53
WPA: wpa_sm_step() called recursively
WPA: 70:f1:a1:ee:bd:53 WPA_PTK entering state DISCONNECTED

Comment: The answer is https://serverfault.com/a/1043009/197563
Answer: I met the similar problem with hostapd_1.0-3+b1_armhf.deb several days ago, when i setup wpa on my ap (raspberry pi), evenif the passpharse was inputted correctly, the connection always failed by password authentication.
After trying a newer version(hostapd_2.1-0ubuntu4_armhf.deb), the problem is gone. I don't know the reason of this issue, looks like it is caused by hostapd itself, not your config.
you can try a newer version of hostapd, hope it works on your device.


NestestedScrollView with RecycleView and ViewPager is not scrolled up views

Question: I'm trying to achieve to scroll views in xml layout which contain recycleview and viewpager but somehow nestedScrollView is not scrolled up the views.

layout file.
 <android.support.v4.widget.SwipeRefreshLayout
    android:id="@+id/swipe_refresh"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:layout_behavior="@string/appbar_scrolling_view_behavior">

    <android.support.v4.widget.NestedScrollView
        android:id="@+id/nested_scroll"
        android:layout_width="wrap_content"
        android:layout_height="match_parent"
        android:fillViewport="true"
        >

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:orientation="vertical">

            <RelativeLayout
                android:layout_width="match_parent"
                android:layout_height="@dimen/dimen_200">

                <android.support.v4.view.ViewPager
                    android:id="@+id/viewpager_media"
                    android:layout_width="match_parent"
                    android:layout_height="wrap_content" />

                <RelativeLayout
                    android:id="@+id/viewPagerIndicator"
                    android:layout_width="match_parent"
                    android:layout_height="wrap_content"
                    android:layout_alignParentBottom="true"
                    android:gravity="center"
                    android:padding="@dimen/dimen_10">

                    <LinearLayout
                        android:id="@+id/viewPagerCountDots"
                        android:layout_width="wrap_content"
                        android:layout_height="wrap_content"
                        android:layout_centerVertical="true"
                        android:gravity="center"
                        android:orientation="horizontal" />

                </RelativeLayout>

            </RelativeLayout>

            <android.support.v7.widget.RecyclerView
                android:id="@+id/rc_media"
                android:layout_width="match_parent"
                android:layout_height="match_parent"
                app:layout_behavior="@string/appbar_scrolling_view_behavior"/>
        </LinearLayout>

    </android.support.v4.widget.NestedScrollView>
</android.support.v4.widget.SwipeRefreshLayout>

Java file

    mediaAdapter = new MediaDiscoverAdapter(getActivity(), mediaCallbackListener);
    discoverBinding.rcMedia.setNestedScrollingEnabled(false);
    discoverBinding.nestedScroll.setFillViewport(true);
    discoverBinding.rcMedia.setAdapter(mediaAdapter);
    layoutManager = new StickyHeaderLayoutManager();
    layoutManager.setAutoMeasureEnabled(true);
    discoverBinding.rcMedia.setLayoutManager(layoutManager);

Answer: I figure out the solution with the help of StackOverFlow. I just make Viewpager part of RecycleView as row item using getItemViewType and removed the NestedScrollView so now layout looks like..
mainLayout.xml

<data>

    <variable
        name="MediaDiscoverFragmentNew"
        type="com.hpc.fragments.media.MediaDiscoverFragmentNew" />
</data>

<android.support.v4.widget.SwipeRefreshLayout
    android:id="@+id/swipe_refresh"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:layout_behavior="@string/appbar_scrolling_view_behavior">

    <android.support.v7.widget.RecyclerView
        android:id="@+id/rc_media"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</android.support.v4.widget.SwipeRefreshLayout>

and created two separate item xmls raw_media_top.xml for top Viewpager and another raw_media.xml for rest of items.

      if(viewType == HEADER_VIEW) {
        RawMediaTopBinding mediaTopBinding =
        DataBindingUtil.inflate(mLayoutInflater, R.layout.raw_media_top, parent, false);
        mediaTopBinding.setMediaDiscoverAdapterNew(this);
        return new HeaderViewHolder(mediaTopBinding);
    } else {
        RawMediaBinding inspirationItemsBinding = DataBindingUtil.inflate(mLayoutInflater, R.layout.raw_media, parent, false);
        inspirationItemsBinding.setMediaDiscoverAdapterNew(this);
        return new ItemViewHolder(inspirationItemsBinding);
    }

Cheers!


How do I exclude return traffic from my ACL?

Question: I have this network topology. What I try to achieve is: Deny access from Network 172.28.224.0 to Network 172.28.226.208. But the other way around it should work. 172.28.226.208 should be allowed to access 172.28.224.0
I have tried it with the following ACL on the Router
access-list 101 permit ospf any any
access-list 101 permit icmp any any
access-list 101 deny ip any 172.28.226.208 0.0.0.7
access-list 101 permit ip any any

int f0/0.1
access-group 101 out

I am using ospf and my problem is, that when I return traffic is also blocked by the ACL. Does anyone have a suggestion?
Comment: This question is perhaps better asked at the Network Engineering Stack Exchange: https://networkengineering.stackexchange.com
Answer: You state you want to deny traffic from 172.28.224.0 to 172.28.226.208 (which subnet masks?) yet you deny traffic from ANY to 172.28.226.208/29.
Next you apply the ACL in the outbound direction on Fa0/0.1 but the destination (172.28.226.208/29) does not live there so you should apply the ACL inbound instead.
Apply the ACL inbound instead of outbound and fix the "ANY" statement and it should work.


Getting C# Int64 in Python

Question: I have to get an Int64 object in Python. My code looks like this:
from System import Int64
l = Int64.Parse('123')
print(l, type(l))

but it returns: 123 <class 'int'>
When I did the same with DateTime c# object, it returned just right type: <class 'System.DateTime'>
So why does Python convert an Int64 C# object back to it's own int class?
I need exactly this type, because I'm sending it to external program, which otherwise fails.
Thanks a lot.
Comment: `long` (`Int64`) is a primitive; I would *expect* python to map that to its own closest primitive... ?
Comment: I am not sure if there exists int64 in native python. Maybe you want to use numpy??
Answer: Figured out a way, but it's quite a harakiri:
Int64.__new__(Int64, value)

Getting C# Int64 in Python


How to find what processes were running at a time in the past?

Question: I am asked to investigate a high CPU usage alert which occurred this morning. I used sar -p and saw the high CPU usage showing up at that time
Next I used ps -eo pcpu,pid,user,args | sort -r -k1 | less to list the top 10 memory hogs at this hour
Now how do I find out what processes caused the bottleneck at that specific time in the morning. I am a java developer and not a Linux expert.
Is it even possible?
Answer: There are several options:

use a script which writes needed data on a regular basis to a logfile. You could use cron to write the output of ps (and other commands) every x minutes into a logfile.
Better it would be to use a specialized program, which does this for you. atop is very good at this, at it takes care of logfile retention.

atop is available via the EPEL repo for CentOS/RHEL/Fedora and via the default repos of Debian/Ubuntu.
You can use atop like a normal real-time top utility, with slightly different behaviour (check out the manpage for keystrokes).
The more interesting part is: Once installed a daemon starts logging data into /var/log/atop and you can read these files with atop again:
atop -r /var/log/atop/atop_20160128

You have then access to all 'top' like functions (sorting/looking at memory/CPU/IO usage, etc.) and you can jump 10 minutes forward in time via 't' and 10 minutes back with 'T' or jump at a specific time via 'b'.
Check out the atop manpage and google has lots of howtos about it.
There might be other solutions, but atop is easy to understand and use and a good start before doing some more bespoke setups.
Comment: option 1: set up a cronjob to log every x minutes the data you need. PS: then it should be the job of the admin to provide you the data...
Comment: So - is there anything that linux offers out of the box? I am not the linux admin and do not have control over installing atop. assuming i can't install anything new is it possible to get the list of processes from the past?
Comment: @Shiv: Unless you had something running to record them in the past, there is no way to get that information now. So the only way to get what you want in the future is to install something now, like atop, or run a custom shell script from cron. Or I suppose you could download atop, configure it to run in your user directory and run it from there, if you really cannot install system packages.


Which algorithm can learn exactly a tree structure without noise?

Question: Let's say I have a response variable $Y$ satisfying a complicated tree-structure:
$$Y=f(X_1,X_2,X_3,\ldots,X_p) + \varepsilon$$
where:

$\varepsilon = 0$
$f$ is a known deep tree-structure function (not necessarily binary). That is to say we can have as many observations as we want.

My question is: Which highly interpretable algorithm can learn perfectly $f$? I first thought of CART, but my expriment on R via rpart doesn't give the desirable result.
I tried two parameters in rpart: $cp = -1$ or $0$ -> every split decreasing the overall lack of fit is attempted.
Comment: What parameters did you use on `rpart`? What are the characteristics of the tree that generates your data?
Comment: I updated the information in the question. As the characteristics of the tree, I asked about the tree structure in general, in no particular form. But does it change something?
Comment: I'm not sure, but perhaps it's a tree so deep/complex you have to take some extreme parameters to recreate it.
Comment: @Firebug What do you expect by "extreme parameters"? I had the same thought, that's why I tried cp = 0 at first then changed to -1, the result got better but still I didn't get $f$. Other parameters doesn't make sense to me in this context.
Comment: `minsplit`/`minbucket` comes to mind, and also is depth beyond 30?
Comment: @Firebug all these parameters will only limit the complexity of the tree if I understood correctly.
Comment: Why add $\varepsilon$ if it's equal to $0$.  Might you have meant $\operatorname{E}(\varepsilon) = 0$ rather then $\varepsilon=0$? $\qquad$
Answer: General purpose decision tree procedures (rpart included) do not attempt to find the tree that best fits the data.  Instead they simply attempt to find a "good" tree by searching greedily.
Greedy means that, any time a split is under consideration, the best split is returned without considering any information from potential future splits.  It is rather easy to concieve of a situation where this will fail to find the optimal tree
x_1 <- runif(100)
x_2 <- runif(100)

y <- ifelse(x_1 < .5,
            0.99 * ifelse(x_2 < .51, 0, 1),
            1.01 * ifelse(x_2 < .49, 0, 1)
)

In this example y is a true tree on (x_1, x_2)
                  root
              /          \
        x_1 < .5         x_1 >= .5
        /      \         /        \
x_2 < .51  x_2 >= .51  x_2 < .49   x_2 >= .49

but constructed in a way that

The effect of true first split on the response is imperceptible when compared to the second split.
The difference in second true second splits based on the first split is very small.

Here's a visualization of the effect
color = ifelse(x_1 < .5, "red", "blue")
plot(x_2, y, col = color)

This should trick a greedy algotithm into choosing x_2 < .5 as its first split
D <- data.frame(x_1, x_2, y)
rpart(y ~ x_1 + x_2, data = D)

which results in
1) root 100 25.124860 0.5012
  2) x_2< 0.4971595 50  0.000000 0.0000 *
  3) x_2>=0.4971595 50  0.004712 1.0024 *

which is pretty much what I predicted would happen (maybe even a bit worse, as the tree does not even find any dependence on x_1).
To avoid this, you would have to use a non-greedy algorithm that searches the entirety of the collection of possible trees and finds the truly optimal one.  The only way I can think of to do this is an exhaustive search.  This would be feasible for small depth trees, but the computational time blows up exponentially, so it will quickly become impossible for deep trees.
Note: I suspect that fitting a tree exactly is an NP problem, but I'm still searching for a reference.
Comment: Very good explaination of why rpart fail to find the optimal tree, could you elaborate more on how to construct a "non-greedy algorithm that searches the entirety of the collection of possible trees and finds the truly optimal one"?
Answer: I think you may try the Bayesian Additive Regression Tree method and there is a bartMachine R package you may try.
Comment: How does this address the posters question?


How to use arc4random in JavaScript?

Question: How do I use arc4random in JavaScript? I tried to use var randomNumber = (arc4random() % 83) + 1; but I got UncaughtReferenceError: arc4random is not defined. I am not sure which libs to include to use arc4random.
Answer: arc4random() is not defined in JavaScaript. To generate random numbers in JavaScript you can use Math.random(). But I think you can easy emulate arc4random() as:
function arc4random(){
  return Math.random()*4294967296;
}

Answer: I do not believe it possible, given all research points to the fact that arc4random is not a js function, but one built in to objective-c, or ios.  I'm beginning to believe you did little research into the matter.  Even the tag wiki entry on SO says as much.
Comment: Agree. Even I did not find any pointers for this function in javascript.

How to use arc4random in JavaScript?


Calculating joint probability density

Question: Say I have $n$ independent random variable $X_1,...,X_n: \Omega \rightarrow \mathbb{R}$ on a probability space $\Omega$ with the same density $p:\mathbb{R}\rightarrow [0,\infty)$. Could someone explain to me in the most measure-theoretic terms (i.e. assuming I know plenty of measure theory but not much probability theory terminology) how you can explicitly write down the density of the joint probability distribution $\mathbb{R}^2\rightarrow [0,\infty)$ of $X_1$ and $t= \sum_{j=1}^nX_j$ using only $p$?  For instance I have an example where $p(x) = xe^{-x}$, and I'm supposed to get $$\frac{1}{\Gamma ( 2n-2)}x_1(t-x_1)^{2n-3}e^{-t}$$ for the density of the joint distribution.
Comment: The density can be viewed as mappping $[0,\infty)^2$ into $[0,\infty).$ Thus you could say $\text{"} \ldots \text{for } x_1\ge0,\, t\ge 0.\text{''} \qquad$
Comment: @MichaelHardy Sorry I don't understand. The density isn't defined on all of $\mathbb{R}^2$?  Or its determined by its values on nonnegative $x_1, t$?
Comment: It can be taken to be defined on all of $\mathbb R^2$ but equal to $0$ except in the first quadrant.
Comment: You have $p(x) = xe^{-x}$ for $x\ge0.$ Note that
$$(\,\underbrace{p* \cdots \cdots *p}_{(n-1)\text{-fold} \\ \text{convolution}}\,)(x) = \frac 1 {\Gamma(2(n-1))} x^{2(n-1)-1} e^{-x} \quad \text{for } x\ge 0,$$
where the convolution $f*g$ of $f$ and $g$ is
$$
(f*g)(w) = \int_0^w f(x)g(w-x) \, dx.
$$
If we have two _independent_ random variables $U,V$ for which $\Pr(U\ge0) = \Pr(V\ge 0) =1,$ with respective probability density functions $f,g,$ then their joint density is $(u,v)\mapsto f(u) g(v),$ and the probability density function of the sum $X+Y$ is then$\,\ldots\qquad$
Comment: $\ldots\,$the convolution $f*g.$ You should be able to prove by induction that the $(n-1)$-fold convolution is as asserted above. $\qquad$
Comment: But then you must also deal with the non-independence of $X_1$ and the sum. More later, maybe$\,\ldots\,\qquad$
Comment: Ok thanks. Yes, so the correlation between $X_1$ and the sum is the missing piece.
Comment: @MichaelHardy btw I'm self-studying stats partially as a result of being inspired by your excellent answer to "statistics for mathematicians" on MO :)
Comment: So one of my answers got noticed. :-) $\qquad$
Comment: One fact that may be worth knowing here is that the functions $$ f_\alpha (x) = \frac 1 {\Gamma(\alpha)} x^{\alpha-1} e^{-x} \quad \text{for } x>0$$ with $\alpha> 0$ satisfy $$f_\alpha * f_\beta = f_{\alpha+\beta}$$ (where the asterisk represents convolution as above). $\qquad$
Comment: . . . . . and therefore if $X,Y$ are independent random variables with probability densities $f_\alpha, f_\beta$ then $X+Y$ has probability density $f_{\alpha+\beta}. \qquad$
Comment: . . . and you're looking at the case $\alpha=2. \qquad$
Answer: What if you just push the measure from $\mathbb{R}^n$ to $\mathbb{R}^n$ via $$L: (x_1,...,x_{n-1}, x_n)\mapsto (x_1,...,x_{n-1},\sum_{i=1}^n x_i)$$ and then project down $ \mathbb{R}^n\rightarrow \mathbb{R}^2$ via
$$\pi :(x_1,...,x_{n-1}, t)\mapsto (x_1,t).$$
Then the density function is easy to track: Since $L$ has an inverse, the density $p$ goes to  $p\circ L^{-1}$ and then since $\pi$ is just a projection, the density can be pushed down just by integrating over the fibers of points.
You will need the following identity,
$$\frac{C^{2k+1}}{(2k+1)!} = \int_{A_k(C)}\big(C-\sum_{i=1}^k x_i\big)\cdot x_1x_2...x_k~~ dx_1dx_2...dx_k,$$
where $$A_k(C):= {\{(x_1,...,x_k)\in \mathbb{R}^k:~~~~\sum_{i=1}^k x_i\leq C,~~\text{and}~~ x_i\geq 0~~\text{for all} ~i~\}}.$$
This can be proved by induction:
$$\int_{A_k(C)}\big(C-\sum_{i=1}^k x_i\big)\cdot x_1x_2...x_k~~ dx_1dx_2...dx_k =$$  $$\int_0^C ~x_k \bigg( ~\int_{A_{k-1}(C-x_k)}\big((C-x_k)- \sum_{i=1}^{k-1} x_i\big)\cdot x_1x_2...x_{k-1}~~ dx_1dx_2...\bigg) ~dx_k=$$ $$\int_0^C ~x_k\frac{(C-x_k)^{2k-1}}{(2k-1)!}dx_k.$$
Then just integrate by parts.

Calculating joint probability density


Does Clickhouse provide continuous (automated) aggregation functions/tables?

Question: I read that InfluxDB offers continuous aggregation features. Does Clickhouse provide something along those lines?
For example, I have certain real-time measurements arrive every couple seconds, sometimes multiple per second, but I am interested in frequently polling down-sampled measures, for example, summary stats of measurements per hour. Ideally, those aggregates should be calculated continuously server-side. I want to configure an aggregation that continuously updates my queries and generates views upon the arrival of new data or at specified intervals, such as every minute.
Is that possible?
Answer: Clickhouse materialized views you are looking for

https://clickhouse.com/docs/en/guides/developer/cascading-materialized-views
https://altinity.com/webinarspage/2019/6/26/clickhouse-and-the-magic-of-materialized-views
https://altinity.com/blog/clickhouse-materialized-views-illuminated-part-1
https://altinity.com/blog/clickhouse-materialized-views-illuminated-part-2
https://den-crane.github.io/Everything_you_should_know_about_materialized_views_commented.pdf

SummingMergeTree & AggregatingMergeTree
they are able to downsample continuously the same time data is coming

https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/summingmergetree
https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/aggregatingmergetree

Postponed downsampling

https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/graphitemergetree
https://kb.altinity.com/altinity-kb-queries-and-syntax/ttl/ttl-group-by-examples/


Identificar modelo do roteador via PHP ou Javascript

Question: É possível identificar o modelo do roteador de uma pessoa através do PHP ou Javascript?
Queria que quando a pessoa acessasse uma página "X" o php ou javascript identificasse qual é o modelo do roteador dela e guardasse essa informação em txt ou BD.
Isso é possível ?
Comment: Se o PHP rodar dentro da rede da pessoa, e tiver acesso aos IPs internos, seria o caso de acessar a página do roteador e tentar identificar. Mas pra isso você teria que elaborar um método de verificação adequado para cada modelo. Pra simplificar, a melhor resposta seria "não, não tem e ainda bem que não, por uma questão de privacidade e segurança". Dentro de condições bem limitadas, e longe do ideal "dá pra identificar em alguns casos pontuais", mas em geral, seria uma porcaria o resultado.
Comment: Você teria um exemplo de como eu posso verificar nesses "casos pontuais"? Só pra eu ter uma ideia, pq nao faço nem ideia de como começar.
Comment: @Marcos acho que ele quis dizer: não comece porque não vai dar certo.
Answer: Há diversos meios mas nenhum confiável pois nem sempre retornará o que se espera. Um exemplo de gambiarra que pode tentar fazer é obter o IP do gateway do usuário. Normalmente o IP do gateway é o IP que leva a página de administração do roteador. Mas note que só conseguirá acessar caso o usuário esteja na mesma faixa de IP do gateway e nem sempre o IP do gateway leva a página admin do roteador. Os roteadores permitem ao usuário definir uma faixa de IP customizada ou um conjunto pré definido que difere da faixa do gateway. Tornando esse método inviável. E mesmo que aceda ao gateway, não há garantia alguma de obter a informação que necessita.
Mas se quiser entender a ideia é, precisa obter o IP do gateway. Pode fazer isso com JavaScript usando permissões especiais de acesso via browser ou por meio de plugins JAVA (O JAVA é mero exemplo, pode ser outra linguagem).
Com o IP do gateway em mãos, a ideia é conseguir ler a página que retorna do protocolo HTTP.
Conseguindo a página, tente extrair as metatags ou o título da página. Muitos roteadores colocam o nome e modelo na tag <title>.
É esse o alvo. Obter o conteúdo da tag <title>.
Saliento novamente que é apenas um de diversos meios de alcançar o que pretende e ainda assim requer permissões de acesso para execução de plugins e scripts JavaScript que normalmente são bloqueados por normas básicas de segurança. E ainda tem a falta de garantia de que o nome e modelo do roteador estará descrito na página HTML conforme método citado acima.
obs: o meu roteador não exibe o nome, marca ou modelo na tag title. Ao invés disso exibe "クイック設定Web".
Outros meios é usar funções de rede. Obviamente precisa de permissões para executar no usuário. Alguns roteadores retornam o modelo como nome de host ou DNS local. Normalmente o nome vem abreviado num padrão definido pelo fabricante. Mas como pode ver é também inseguro em relação a funcionalidade pois nem sempre terá essa informação e mesmo que consiga em alguns casos, ainda terá que interpretar as abreviaturas.
Um detalhe, pode descartar qualquer tentativa com usuários conectados em redes de telefonia 3G, 4G, 5G. E também usuários de dispositivos móveis como smartphones e tablets que obviamente não permitirão fazer nada disso. A menos que instalem explicitamente um aplicativo que extraia os dados da rede local.
No final das contas, terá um trabalho enorme com dezenas de gambiarras que não vale o retorno. É mais fácil um dromedário passar pelo orifício de uma agulha.


Wikipedia Referencing and Content Rules - Is this time for me?

Question: I've received my share of policing by automated rules and manual moderation in these two interactions:

Do actively managed ETFs help the economy in any way, and if so, how?
What happens if commercial banks restricted and we have just a centeral bank

Admittedly, these two answers are below what I can potentially do if I had more time, but I don't have extra time. The automated labels and cease and desist notes are quite discouraging. I noticed similar cases in other threads.
Is this site going to enforce Wikipedia rules for reputable sources and original content or is it going to be like other stack exchange sites where good answers get ahead by virtue of upvotes.
I'm not complaining, but curious which way is it going?
Comment: Are post notices (blue boxes) automated or placed by a moderator? Can I explicitly opt-out from [automated responses](https://economics.meta.stackexchange.com/questions/2088/all-generic-responses) added by moderators that 'encourage' me to improve the answer?
Comment: they are automated in a way that they have automatically generated text based on pre-defined issues. Same as if your Q get closed or question deleted you will get automatically rendered text based on reasons for closing/deletion. Those comments are mainly for new users, there is no official way of opting out - I will try to remember you already got comment (although those comments can be used by regular users as well or other mods) - however your post will in that case get the post notice instead of the comment
Answer: I am not fully familiar with referencing rules of Wikipedia but looking at this Wikipedia page  our referencing policy is far less strict.

We require referencing in order to avoid plagiarism.  Plagiarism is generally discouraged by SE in general and Economics.SE (although this does not seem to be  your case).
Next we encourage sources in cases where:

... for answers that talk about statistics, research or results that are not common knowledge in economics (i.e. something that a person with undergraduate degree in economics should know) and which are not self-contained (e.g. it should not apply to self-contained answers presenting a model), and which lack any references to credible sources supporting the answer.

If the answer does not have sources it won't get automatically deleted. It will just get post notice encouraging other users to edit and improve it. If you do not have time for adding sources then why should not other members of community step in and add those? The post notice does not automatically marks the answer for deletion and the answer is still able to receive upvotes.
Next, you talk about enforcing, but let me stress again nobody forces you to add those sources. You won't get banned for not adding sources. Your question won't get deleted, by moderators (but could be by community or Roomba if it gets too many low quality flags).
I would hardly call this 'enforcement'.
There are also no cease and desist notices. The notices and comments encourage you to edit answer, in none of the comments or notices is there any mention or even insinuation you should not attempt to answer the question.
Notices like these are common across all science stacks. Just take a peak at History.SE where they not just use the notice but also actually enforce it by closing or deleting the answers outright by moderators (here your answer will not be deleted at all except if by users).
Next, the notices are encouraging not discouraging, they encourage you to improve the answer by adding references. That is why they exist in the first place. We encourage answers that are based on facts, theory, sources and literature in general. This to maintain some minimum standards of the site, every science Stack Exchange I know of has a variant of this, and I dare to say that the way how we handle this on Economics:SE is one of the most light touch approach among science stacks.
In addition, note that we do not require that you make proper academic citation with ISBN etc like Wikipedia does. If you can put a hyperlink to paper, book or article from some credible medium or heck even just to dataset/graph that is generally enough. And this is not required for self contained answer based on some model and so on as mentioned above.


Solving $\rm\dot v = \frac q m v \times B + \frac q m E$

Question: I need to solve the following ode: $\rm\dot v = \frac q m v \times B + \frac q m E$, where $\mathrm{ v_0}=(0,0,v_0), \mathrm{B}= (0,B,0), \mathrm{E} = (0,E,0) $.
If it helps anyhow, this comes from a particle entering a magnetic/electric field with speed perpendicular to both fields.
I believe that $v_y=\frac q m E \,t $ and $v_x=v_0$. But I couldn't get $v_z$ because the equation yields $\dot v_z=\frac q m \sqrt{v_x^2+v_y^2+v_z^2}B \sin(\alpha)$ where $\alpha$ is the angle between $\rm v, B$, and I don't know how to solve that nonlinear ode...
Answer: After working out the cross product, we see that the system of ODEs is $$\dot v_x=-\frac{qB}{m} v_z,$$ $$\dot v_y=\frac{q E}{m},$$  $$\dot v_z=\frac{qB}{m}v_x.$$ The solution to $v_y$ can be found by direct integration: $$v_y=\frac{qE}{m}t.$$ Let $qB/m=w$. For the other two, take another derivative:
$$\ddot v_x=-w\dot v_z=-w^2 v_x$$ $$\ddot v_z=w\dot v_x=-w^2 v_z.$$ These are standard ODEs whose solutions are well known (simple harmonic oscillator) $$v_x=A\sin(wt)+B\cos(wt)$$ $$v_z=C\sin(wt)+D\cos(wt).$$ Since we know that $v_x(0)=0$, we know that $B=0$. Also, since $v_z(0)=v_0$, $D=-v_0$. Next, $\dot v_x(0)=-wv_z(0)=-wv_0$, so $A=-v_0$. Finally, $\dot v_z(0)=wv_x(0)=0$, so $C=0$. Thus, $$v_x=-v_0\sin(wt)$$ and $$v_z=-v_0\cos(wt).$$
Comment: All too easy ... +1 for taking time to post this solution.
Comment: Damn, I made such a newbie mistake, I bindly wrote down the norm of $\rm v \times B$ instead of actually calculating it. Thanks.


Dont want horizontal scroll bar for views

Question: I am using a view called "exhibit" with multiple fields.
I am getting a horizontal scroll bar since the view have multiple fields.
But I don't want the horizontal scroll bar.
How can I achieve this.
Answer: You can use css like this:
<code>overflow-y: scroll;
overflow-x: hidden;
</code>
Comment: Can you please tell me where to put the css file in my drupal folder
Comment: Find your current theme, it should be in sites/all/themes, then place it there or in a subfolder css. I guess you already have a css file, add those 2 rows there.
Comment: I am using bartik theme. which is in themes folder itself. Added those two rows in a test.css file, and added this css file to a view. but still getting scroll bar.
Comment: you need to add these rows inside the css rule that describe the specific views element.


Remove index from MultiIndex dataframe if child index has column value meeting criteria

Question: I had originally asked this question here, and I believe it was incorrectly marked as a duplicate. I will do my best here to clarify my question and how I believe it is unique.
Given the following example MultiIndex dataframe:
import pandas as pd
import numpy as np

first = ['A', 'B', 'C']
second = ['a', 'b', 'c', 'd']
third = ['1', '2', '3']

indices = [first, second, third]

index = pd.MultiIndex.from_product(indices, names=['first', 'second', 'third'])

df = pd.DataFrame(np.random.randint(10, size=(len(first)*len(second)*len(third), 4)), index=index, columns=['Val1','Val2',' Val3', 'Val4'])

Goal: I would like to retain a specific level=1 index (such as 'a') if the value of column 'Val2' corresponding to index value 1 in level=2 is greater than 5 for that level=1 index. Therefore, if this criteria is not met (i.e. column 'Val2' is less than or equal to 5 for index 1 in level=2), then the corresponding level=1 index would be removed from the dataframe. If all level=1 indices do not meet the criteria for a given level=0 index, then that level=0 index would also be removed. My previous post contains my expected output (I can add it here, but I wanted this post to be as succinct as possible for clarity).
Here is my current solution, the performance of which I'm sure can be improved:
grouped = df.groupby(level=0)

output = pd.concat([grouped.get_group(key).groupby(level=1).filter(lambda x: (x.loc[pd.IndexSlice[:, :, '1'], 'Val2']>5).any()) for key, group in grouped])

This does produce my desired output, but for a dataframe with 100,000's of rows, the performance is rather poor. Is there something obvious I am missing here to better utilize the under-the-hood optimization of pandas?
Answer: I got the same result as your example solution by doing the following:
df.loc[df.xs('1', level=2)['Val2'] > 5]

Comparing time performance this is ~15X faster (in my machine your example takes 36ms while this take 2ms).
Comment: Excellent answer, exactly what I was looking for. Thanks!


What is the technique about how this this violin is held called?

Question: the violinist in this video holds the violin in a really peculiar way, I'm wondering if there is a name for the way he holds the violin?
Comment: At this point, it ceases to be a violin. It's now a *fiddle*!
Comment: I highly recommend watching this film if you haven't already. It's excerpted from _The Last Waltz_, which is an excellent Scorsese-directed part-documentary of The Band's eponymous farewell concert and part-biopic of the group. You may observe some more unusual playing styles within!
Answer: I think this way is called "holding the fiddle at the chest" or against the chest or "low down on the chest" (Chris Haig, The Fiddle Handbook).
This hold is sometimes used for genres which usually refer to the violin as fiddle and playing it is called fiddling. E.g. in country or bluegrass.
While this position has obvious disadvantages e.g. vibrato (especially fast) is more difficult and playing in higher positions is much more difficult as well as changing the positions. These are techniques that are in general rarely used in fiddling.
Advantages of these positions are that it is stylistic typical to do so which adds to the mood of the presentation but also that its easier to sing while playing (I don't have an overview how many fiddlers do so, but there are some doing so.). It's easier to move on stage and keep contact with your fellows. At least they say so, but this claim seems quite plausible. Drew Beisswenger mentions (North American Fiddle Music - A Research and Information Guide, 20011, Routledge) that the positions used by fiddlers also "make it possible for them to play for six or eight hours straight".
This way of holding the violin was also applied in the early days of playing violin. For reference see the following drawing by the Dutch painter Frans Hals (about 1630s):
Comment: @Tim it's quite common for baroque violinists to hold the instrument this way, too.  Another effect is to make the downbow and upbow less even.  The meaning of "fiddle" is not so clearly defined as you seem to want it to be.  Even if we agree that the video clearly shows "fiddling," that doesn't mean that the way the instrument is held defines fiddling or that it is contradictory to say that fiddling can also be done in "standard" position.
Comment: I disagree. Fiddling is so much more than holding the fiddle in this special way and in fact many (or most?) fiddlers use standard position or something close to standard position.
Comment: Yeah, every time I imagine a stereotypical fiddler, s/he is always holding the violin in standard position.
Comment: So, *fiddling* would in fact be a pretty accurate term for doing this.
Comment: So what defines *fiddling*? There's a contradiction here.


AngularJS: ui-router secured states

Question: I have one main controller for my app - AppCtrl and use ui-router. How can I make secured states?
$rootScope.$on('$stateChangeStart',function(event, toState, toParams, fromState, fromParams){
    var authorization = toState.data.authorization;

    if(!Security.isAuthenticated() && authorization != false)
        $location.path('/login');
});

For example I want to make books and authors states secured, and login state not secured.
.state('login', {
                url: '/login',
                templateUrl: /**/,
                controller: /**/,
                data: {
                    authorization: false
                }
            })
.state('books', {
                url: '/books',
                templateUrl: /**/,
                controller: /**/,
                data: {
                    authorization: true
                }
            })
.state('authors', {
                url: '/authors',
                templateUrl: /**/,
                controller: /**/,
                data: {
                    authorization: true
                }
            })

Security.isAuthenticated() function returns boolean. When I open /books everything works perfectly, page are being redirected to /login, when after redirecting I open /authors, page loads and it's content are shown, but browser's url is /login, so page being redirected, but somehow it's content are shown.
Comment: This example might be also useful: http://stackoverflow.com/questions/22537311/angular-ui-router-login-authentication
Answer: Figured out that I have to prevent opening next route, and go to login state. Made some changes and all works perfectly.
if (!Security.isAuthenticated() && authorization != false){
        event.preventDefault();
        $state.go('login');
    }

Comment: Looks neat. May I ask where you put the if (!Security.isAuthenticated()... block ?
Comment: Sorry. Just saw now you have it with the rootScope state change handler.
Comment: I still wonder how to have it in my multiple modules application. In my case I need to have my rootScope in a module utilsModule.run( or in the app.run(. But then no way to reference the $state.current.data.securedRoute route data.
Comment: I would also be interested in alternatives to the `.run()` method.

AngularJS: ui-router secured states


Mass data rendering performance with JavaScript

Question: I am developing a special grid solution for a software product in javascript. The data are collected with a PHP script on the server side and pushed to JavaScript via a JSON array. In my script I have to parse this array and render the grid rows. And here is my problem. If I receive for example 4000 rows, javascript is rendering this very fast, but I think the bottleneck is the browser...
My question is, is it possible to render only the visible parts? I need to scroll to the other information, but the browser does not need to render it if it is not visible. is it possible to render something outside of the viewport?
I need to set width and positions and this is only possible if I added the new element to the viewport and this is very slow by a huge mass on data... How could I solve this problem?
Answer: The solution here might be to paginate your data on client side. That way, you can sort your object array with JS and simply insert a section of the data into the DOM at a time.
Client side pagination library options have be discussed here.
Comment: pagination is no option because the data should be stick together. it has to be one big grid.
Answer: DOM updates are the slowest part of the chain. Process the result in memory and insert it into the DOM in one go if you can.
Comment: that is an idea, but this is a only a little bit faster. if i calculate 5000 rows in memory and add this by once it needs 3 seconds to be inserted. too slow... i think i will implement a partial rendering like in slickgrid. only rendering rows available in a viewport.


My jquery/ajax website not working

Question: I have a script which allows to display cross domain data, example: Jsfiddle
I have put the exact same code on my website moskah.nl as you can see I get a blank page. How is this possible ?????? Do you think my website provider is somehow blocking/restricting the code?? Is it the ajax call to "//query.yahooapis.com/v1/public/yql?callback=?" , that maybe is not working in the Netherlands ?? I have talked to the helpdesk of my provider but they don't know (anything!) what is going on.
Answer: You are getting js error, which you can see in firebug console and you are getting jquery is not defined error. try loading jquery like this(added type attribute),
<code><script type="text/javascript" src="http://scripts.embed.ly/jquery.embedly.min.js"></script>
</code>
Comment: HAHAHAHAAAHAHHHHAAA (Im so embarrassed right now...) Thank you very much:) I knew I should have had that cup of coffee first..
Comment: try laoding the official jquery from google cdn...lets chcek if that works

My jquery/ajax website not working


Prediction Result Interpretation

Question: I got a prediction result and it is not seemingly good.
But, I am not sure how horrible is this result.
Brief summary of the result is as follows:

My dependent variable is a binary variable
I separated my data set in two data set (1) cross-validation set (2) test set
(In both sets, the proportion of the dependent variable's values is 1:9)
(That is, y = 1 is 10% and y = 0 is 90%)
I selected the best method and model only using the cross-validation set.
After fitting model only using the cross-validation set, I predicted the test set's dependent variable
using the test set's features
the result is pretty shocking. Every predicted result is just zero.

In this situation, I am not sure is it horrible or a just bad result because the real proportion of y = 0 is 1:9 in the test set.
How do you think that? I want some comments.
Thank you.
Answer: To formalize things a little bit, you are solving a binary classification problem where you have a modest level of class disbalance.
It would be helpful to get a more detailed description of what you mean by selecting the best method and model.
Given that you have a dedicated test dataset and, as far as I understood, you used cross-validation (by the way, which type did you use?) to fine-tune hyperparameters, it seems that there is nothing obviously wrong.
That said, since your classes are imbalanced, double-check that you are using stratified splits so as to avoid the cases when one of the splits consists entirely of the first-class objects.
It's also worth noting that using accuracy as a metric in such unbalanced scenarios is not the best approach. Getting an accuracy of $0.9$+ and most predictions as zeros makes sense and you should also pay attention to other metrics.
What also matters for such tasks is determining how you are going to use your model in the future. It's often the case that trying to concentrate too much on 0/1 outcomes from the beginning does not correlate very much with the really useful possible applications of the model. Try to check other metrics like AUC ROC that can be much more representative of your model's true prediction capability (but please also keep in mind that this is a metric that evaluates your model's ranking talent and is a composite indicator across all possible thresholds, not just one as opposed to accuracy, precision, recall, f1.
Another thought regarding the distribution of classes: as long as you are confident that it matches the data in the wild, there is no need to do downsampling or oversampling, but you could try and use the so-called "cost-sensitive learning" and use the class_weight="balanced" parameter available in most sklearn-ish libraries.

Prediction Result Interpretation


Removing Number from Textbox

Question: Trying to use the following class to limit a textbox to text only:
However, it sees that its not text and e.Handled is false but its leaving the number in the textbox. How can i remove it?
Public Class LettersOnlyTextbox
Inherits TextBox

Public Class LettersOnlyTextbox
Inherits TextBox

Protected Overrides Sub onkeydown(e As System.Windows.Forms.KeyEventArgs)
    Dim c = Convert.ToChar(e.KeyValue)
    Select Case e.KeyCode
        Case Keys.Back, Keys.Delete
            e.Handled = False
        Case Else
            e.Handled = Not Char.IsLetter(c)
    End Select
End Sub
End Class

Answer: I was able to solve it this way
Protected Overrides Sub onkeydown(e As System.Windows.Forms.KeyEventArgs)
    Dim c = Convert.ToChar(e.KeyValue)
    Select Case e.KeyCode
        Case Keys.Back, Keys.Delete
            e.Handled = False
        Case Else
            If Not Char.IsLetter(c) Then
                e.SuppressKeyPress = True
            End If
    End Select
End Sub

Answer: If you set Handled = False, it sends the event to the operating system to be handled by default.  So, you want the opposite to be true to STOP something from being entered in the textbox.
Therefore, you want...
e.Handled = Not Char.IsLetter(c)
'if the character is not a letter then handle it (i.e. stop)

You would also want to change the statement above it for the Back and Delete keys to be False
Comment: I changed it as you suggested and it still allows them to enter numbers. (See above edit of code)
Answer: Here's exactly how I handle restricting text within a text box. It's actually quite simple and I use the textbox keypress handler. I'll also include a picture to show you how to get to it.
Please note "AllowedChars" is the variable that you use to determine what characters you want to "allow" the user to be able to enter into the text box. If they try to press any other key, it simply won't go into the textbox.
With this method it also allows the use of backspace and the shift key for Capital letters.
 Private Sub Textbox1_KeyPress(sender As Object, e As System.Windows.Forms.KeyPressEventArgs) Handles Textbox1.KeyPress

    Dim AllowedChars As String = "abcdefghijklmnopqrstuvwxyz" 'Change the value of this variable to suit your needs.

    If e.KeyChar <> ControlChars.Back and ModifierKeys <> Keys.Shift Then
        If AllowedChars.IndexOf(e.KeyChar) = -1 Then
            e.Handled = True 'This is what prevents the keys from being entered into the textbox
        Else
        End If
    End If

End Sub

If you do not want the backspace or the shift key being allowed then simply remove that part of the code as such:
    If AllowedChars.IndexOf(e.KeyChar) = -1 Then
        e.Handled = True 'This is what prevents the keys from being entered into the textbox
    Else
    End If

Also note that this does not prevent copy and paste into the form, so you still want to put in error preventative methods when you include your actions/submits/buttons (however you want to state it lol).
The pictures at this point are to direct you to the keypress event handler.

Hopefully this helps you! Cheers :)
Answer: Private Sub MyTextBox_KeyPress(sender As Object, e As System.Windows.Forms.KeyPressEventArgs) Handles MyTextBox.KeyPress
If Not IsNumeric(e.KeyChar) And Not e.KeyChar = ChrW(Keys.Back) Then
    e.Handled = True
End If

End Sub

Removing Number from Textbox


Valor de um "for" retornado infinitamente

Question: Contextualizando: Tenho um exercício que sugere a criação de um programa para ler notas de alunos e mostrar as notas iguais ou maiores que a média.
Dessa forma, o seguinte código foi escrito:
#include <stdio.h>

int main(){

    float media;
    float alunos[5];
    int posicao;
    float soma=0;

    for(posicao=0;posicao<5;posicao=posicao++)
    {
        scanf("%f", &alunos[posicao]);
        soma=soma+alunos[posicao];
    }
    media=soma/5;
    for(posicao=0;posicao<5;posicao++)
    {
        if(alunos[posicao] >= media)
        printf("%.2f \n", alunos[posicao]);

    }
    return 0;
}

Primeira dúvida: Por que quando o terceiro parâmetro da estrutura de repetição é igual posicao++ ela ultrapassa o limite que deveria ser 5?
Trocando o valor que gerou a primeira dúvida por outro equivalente posicao=posicao+1, sigo na construção do código.
Segunda dúvida: Por que as notas maiores ou iguais à média não são retornadas? Se for digitado por exemplo, 1, 2, 3, 4, 5, a soma seria 15 e a média 3, portanto, os números 3, 4 e 5 deveriam ser exibidos, mas não são.
Answer: A primeira dúvida é que gera comportamento indefinido, o compilador pode fazer o que quiser com aquela atribuição em cima de algo que já é uma atribuição (no caso ela está atribuindo antes de fazer o incremento). Oi faz a atribuição "manual" ou deixa a "automática" do incremento acontecer, não faça as duas.
Só é estranho fazer certo em um lugar e errado no outro, faça igual ao que estava certo.
Não consegui reproduzir o segundo problema, eu só reorganizei o código e tirei o erro da primeira dúvida e funcionou, como era esperado.
#include <stdio.h>

int main() {

    float alunos[5];
    float soma = 0;
    for (int i = 0; i < 5; i++) {
        scanf("%f", &alunos[i]);
        soma += alunos[i];
    }
    float media = soma / 5;
    for (int i = 0; i < 5; i++) if (alunos[i] >= media) printf("%.2f\n", alunos[i]);
}

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.
Comment: Há, sim! Entendi. Obrigado pela ajuda mano!
Answer: posicao++ é um pós-incremento logo, em posicao=posicao++, posicao está recebendo ele mesmo e só depois sendo incrementado.
Se fosse um pré-incremento (posicao=++posicao), posicao seria incrementada.
Se a intenção for apenas incrementar em 1, basta usar posicao++ ou ++posicao, dispensando a atribuição.
Answer: Para código C++ sugiro que você substitua
    scanf("%f", &alunos[posicao]);

por
cin >> alunos[posicao];
e
printf("%.2f\n", alunos[i]);
por
cout << alunos[i] << endl;
Evite usar recursos C ou depreciados pra evitar conflito com compiladores mais modernos


Why does the modal system approach to jazz sometimes fail?

Question: When I first dived into studying jazz guitar, I learnt that the modal system and harmonized scale were the ultimate key to jazz improvisation and mastery. This method worked for me in most cases but today I'm beginning to realize that the modal approach is not all that there is to speaking the jazz language and it has its limitations. For instance, the modal system doesn't argur well when played against a simple straight ahead 12-bar blues but a hexatonic blues scale tends to be a perfect fit in such cases. This brings me to this question;
What are the common limitations of the modal system?
Comment: Asked, answered, accepted ... but what exactly is the "modal system" and what are its limitations? It would help a great deal in understanding this question, if you explained the modal system in detail, and how you used it on a 12-bar blues.
Answer: First off you came across a standard education method of exposing new students to the connection between modes are chords but that is just one ingredient to understanding music.  By the way this connection isn't special to Jazz, it exists in classical music too.  Not sure how you derived the statement "...were the ultimate key to jazz improvisation and mastery".  They are one of many ingredients to understanding Jazz.
The key to mastery is to understand that music is a language like English or Spanish and it has a structure that has evolved.  With that evolution "rules" of grammar have also been discovered or imposed on it to quantify what seems to be a standard use.  The modes and chords are barely the alphabet.  What one need to do is analyze the solos of great masters and see what they are doing, how they speak!  Then start inventing your own take on these words and phrases.
It never ceases to amaze me how people push this system, and gravitate to it, as a formulaic method of always playing the right notes.
I've transcribed and picked apart solos by Charlie Parker, Wes Montgomery, Miles, etc.  You name it.  I'm always amazed and amused that Wes can play every single "wrong" note, every avoid note, the 4th, the major 3rd over a Minor chord, and it sounds cool.  He had an ear that was truly gifted.  Jazz is a culture.  To master it you need to get immersed in the culture, listen to everything from Benny Goodman, Duke Ellington, up through Dizzy, Wes, Pat, Miles, into modern times if you like what people are doing with it, Jazz rock fusion , smooth jazz, or whatever.  Learn how every generation copied from the one before.  Pay more attention to phrasing rather than streams of notes that chase chords.
As for Blues, you can absolutely solo modally over it.  Changing key on every chord, using a different mixolydian with the starting notes on the I, IV and V respectively.  Guess who does this?  Wes.  Most players do not pigeonhole themselves to one approach but apply a mixture of modal, blues scale, and other approaches to building a solo.
Mastery comes from experience and trying to integrate all these approaches.  There is no formula for mastery in any form of music, especially Jazz.
I learned Jazz, and guitar in general, is the 70s and 80s.  Perhaps this "modal system" is new, or perhaps my teachers didn't buy into it.  But I've never heard it called a system.  Like I said it is one of many tools in the musician's arsenal and with that it is full of limitations when taken out of context.  But when integrated with classical harmony theory, and going through the act of transcribing and analyzing the work of the masters, that one tool gets integrated in your mind with several others.  I can honestly say I fully understand this tool, or system, yet when I solo it never enters my mind in the decision making process  You could argue that Wes new 100s of volumes of music theory and thought about all of that when soloing, but I'd bet my money on the opposite.
Answer: Aebersold and Berklee have a lot to answer for!  They formalised a system of improvisation that was teachable and testable.  But they gave a generation of players the idea that this was THE way to approach jazz.
It's harder to emulate Armstrong or Beiderbecke 'by numbers' (though their approach CAN be analysed and studied).   I recommend you do so, alongside your scale/chord work.
Comment: I am very familiar with Abersold, worked through many of his series on LP, and know people who teach at his summer camp.  I am glad my guitar and bass teachers walked me through these but also told me what I've come to repeat "it's just one of many tools".
Answer: The modal system is just a set of scales to play over chords. This gives you an option of what to play however it does not mean it will sound good. You must train your ear by listening to jazz and you will soon start to come up with lines that fall into the scales.
Answering your second question, limitations of the modal system include that playing purely diatonically (using only the notes in the scale) can send boring after a time. This is how the bebop was formed. If you study solos by Charlie Parker, Bud Powell, or Thelonious Monk, you will notice that instead of just playing the chord/scale tones, they use something called a chromatic enclosure. That is the heart of bebop.
Answer: Today's trend in jazz education is to focus a lot on scales, but this
seems to be pretty recent. I think it can be linked to the rise of the
Jamey Aebersold method. He started producing his play-a-longs in the
1970s, and each one contains a primer on "chord/scales." This series
is still being produced and is probably the most popular jazz teaching
aid in the world, and many of today's jazz artists learned using
Aebersold books.
That being said, it wasn't how older jazz musicians conceived of their
music. I have a Jimmy Dorsey saxophone method book (published in the 40s) with a small section on improvisation. It's basically a page of
triadic arpeggios and broken scales and another page of short etudes
(which are based on triadic arpeggios and broken scales). You can find
some statements from Dizzy Gillespie, and he's always talking about
chord extensions or super-imposing chords over other chords (e.g.
tritone subs). Nothing about scales or modes anywhere, so you will have a hard time sounding like them if you use a modal approach.
Answer: The example you quote (playing jazz modes over a 12 bar blues) is a peculiar example. I would say that a classical 12 bar blues harmony (I7, IV7, V7) is a genre different to jazz. It's non-diatonic (no single scale or mode will contain all the notes in that harmony), so the jazz rules don't apply. As I like to say, blues is minor and major at the same time.
I think that the modal approach to jazz works well in jazz, but as soon as you move to other genres, other rules apply.
Comment: What? I mean... _what_? So these examples clearly aren't jazz https://www.youtube.com/watch?v=EHpF3WEqZpQ   https://www.youtube.com/watch?v=HQ_TzhblPjw  https://www.youtube.com/watch?v=EYVmxKxMqE4  So did I get this right ... if it's not diatonic, it's not jazz? What? Do you have any links to these "jazz rules"? Diminished scale, blues scale, whole-tone scale, ... --> not jazz? _This changes everything!_ :)
Comment: Your statements are just not true.  In Rock, Metal, Shredders use "the modal system".  It is also used in other genres.  And it really isn't at the heart of Jazz.  Jazz is about 100 years old and this system wasn't formalized until the 70s as far as I know.
Comment: @ggcg I think I took a very "colloquial" approach in the response (plus my own biased understanding of jazz), and it contains lots of ambiguity. I will try to rewrite. Thanks for the feedback.
Comment: I'm not an expert in jazz. I studied jazz guitar for a couple of years but I was always struggling somewhat. The way I see it, jazz musicians have a wider palette of harmonic resources to choose from (maj 7 chords, diminished, etc). On the other hand, many blues musicians can create great music with just 3 dominant chords. We all agree that there are no hard and fast rules, only suggestions and tools to play with. However, saying "other rules apply" is a more succint way of saying the above. I never said that a diminished scale, a blues scale or a whole-tone scale are not jazz.
Comment: @ggcg - I've obviously failed to convey the idea I had in my mind because my answer has gotten a lot of pushback. I will try to edit it. Can I ask a question though: what part of my answer says that "the modal system" is not used in Rock, Metal or Jazz. Something that has become obvious to me is that my perception of jazz is warped by the Berklee method (and I suspect it influences the pushback that my answer has received). Thanks for your comment.
Comment: @mkorman "I think that the modal approach to jazz works well in jazz, but as soon as you move to other genres, other rules apply."  This, to me, implies it does not work anywhere.  Perhaps I've misunderstood your intent but logic would imply no other genre can use the modal system.  Unless you meant that "modal approach to jazz" is different than the modal approach to Rock.
Comment: @mkorman, also, I interpret the statement "Other rules apply" to mean that the modal system is not one of those rules where you may have meant that other rules are added.  If that's the case I am truly sorry.
Comment: Some people dont like verbosity.   I get slammed a lot for over explaining,  especially on the physics exchange.  But then again, I tend to dissect things and reconstruct them which leads to multiple views of the same concept.  Fun being human. ;-)
Comment: Something really odd has been going on in music schools, since they managed to make students think that jazz means diatonic and modal playing. And that blues and jazz are separate. What are these "jazz rules"? What is the mythical diatonic jazz? Books, web pages, recordings, anything. If you listen to actual jazz, for example the links I posted, how can it be possible to conclude that jazz means spreading a single diatonic scale over chords.
Answer: For a start, if you just stick to modes you won't be getting chromatic and you won't be getting whole tone, and they are two flavours well-known in jazz.
Comment: What, do you mean by "... you won't be getting a whole tone"?
Comment: Just as there is no mode that moves completely or substantially in semitones (chromatically), there is no mode that is constructed completely of whole tones. You won't get the flavour of chromatic movement or the flavour of sustained whole tone movement if you rely solely on modes. These flavours are common in jazz.


Create a views "exposed filter" to compare two fields

Question: First of all:
I know that there's a static filter with Global: Fields comparison that would do it. But I need an exposed filter in views to fullfill my needs.
Let's say I have a table with 3 cols: "nid" | "changed" | "another datetime".
Now I want to have an exposed filter as a select list with options "none", "old" and "new".
If "old" is selected, show rows where "another datetime" > "changed".
If "new" is selected, show rows where "another datetime" < "changed".
I've been looking for a solution for multiple hours.
Comment: This would need some custom coding, would that be an option for you? Or are you strictly looking for site-building options?
Comment: Absolutely yes!
Comment: You could also use a taxonomy field (none, old, new) and when the node is saved or updated, use Rules to compare time fields and set the taxonomy field accordingly. Then expose the field. Easy option if you're already using rules.
Answer: Solution 1
Make use of views php to do the compare with the help of https://www.drupal.org/project/views_php
In that case you have to write some code which is executed on each row, in the end it will work but it is very bad in maintaining. You do not have any integrated editor for error code checking etc etc. It is a solution but I do not recommend it.
Solution 2
Make use of the hook views_post_execute with parameter of the view name.
function YOURMODULENAME_views_post_execute(&$view){
For easy maintenance start with a switch
<code>switch ($view->name){

    case "YOURVIEWNAMEA";
    case "YOURVIEWNAMEB";
    case "YOURCASEVIEW";

    foreach ($view->result as $row => $rowinfo) {

      // put here your code to do something with each result.

       // E.G Delete a value in the result of the view making use of unset statement
        unset ($view->result[$row]);

      }
</code>


Apresentar conteúdo resumido no Laravel 5.1

Question: Tenho um controller que realiza uma consulta paginada:
$noticias = Noticias::orderBy('created_at', 'desc')->paginate(4);

Porém, uma das colunas retornada é a conteudo. A coluna em questão tem muito texto, como poço fazer no Laravel para que este retorne somente um resumo contendo os 380 primeiros caracteres ou então as 200 primeiras palavras?
Answer: No Laravel há um recurso que nos permite retornar um valor, numa propriedade do model, como se o mesmo estivesse na tabela.
Por exemplo, se você tem o campo conteudo na sua tabela, você poderá fazer com que a propriedade mágica conteudo_resumido seja retornada, a partir desse conteúdo original.
Podemos fazer isso adicionando um método no Model, que começe com get, seguido do nome com StudlyCase, e em seguida com a palavra attribute. Ou seja:
MeuModel::getConteudoResumidoAttribute()
Exemplo:
class Post extends \Eloquent
{

    protected $appends = ['conteudo_resumido'];
    public function getConteudoResumidoAttribute()
    {
        return str_limit($this->getAttribute('conteudo'), 200, '...');
    }
}

A partir disso, podemos fazer o seguinte na chamada do nosso model Post:
@foreach($posts as $post)

{{ $post->conteudo_resumido }}
@endforeach

Eu usei a propriedade appends para determinar que o método mágico será carregado junto com o resultado trazido de uma consulta comum, pois caso necessite do mesmo no json, ele já será retornado automaticamente.
Atualização
Se você estiver enfrentando problemas com tags HTML quebrada por causa do "truncate" gerado pela função str_limit, basta alterar o método descrito acima, colocando a função strip_tags antes do uso de str_limit.
str_limit(strip_tags($this->getAttribute('conteudo')), 200);

Assim, só serão contados os caracteres de texto, não os da tag, e você não terá problemas com tags HTML quebradas.
Answer: Consegui fazendo assim:
{{ preg_replace('/(<.*?>)|(&.*?;)/', '', \Illuminate\Support\Str::words($noticia->conteudo, 70, "...")) }}

Com isso consigo remover as tags do html e formatações especiais, bem como limitar a um total de 70 palavras antes de inserir os três pontos, elementos estes que representam a continuidade do conteúdo.
Comment: Se o problema eram as tags, era só falar, que eu editava as perguntas. E pra que usar preg_replace se já existe strip_tags?
Comment: Inserir esse tipo de lógica na view não é uma boa ideia. Caso eu precise desse mesmo resumo em outra view da aplicação preciso duplicar essa lógica enorme?
Comment: sim de fato, mas se tiver que fazer isso no controller terei que sobrescrever aquela posição do array, coreto?
Comment: Veja a resposta do @wallace. Ela já esta explicando muito bem como resolver esse tipo de problema
Comment: Só mais uma coisa, do banco é retornado um array com o conteúdo, no seu exemplo acho que está lendo somente uma string. Para fazer o resumo de tudo terei que usar um laço certo? Ou o laravel disponibiliza algum recurso para isso?
Comment: Não entendi muito bem. Como assim o banco retorna um `array`?


Causal Inference where the treatment assignment is randomized

Question: I have mostly worked with Observational data where the treatment assignment was not randomized. In the past, I have used PSM, IPTW to balance and then calculate ATE.
My problem is:
Now I am working on a problem where the treatment assignment is randomized meaning there won't be a confounding effect. But treatment and control groups have different sizes. There's a bucket imbalance.
Now should I just analyze the data as it is and run statistical significance and Statistical power test?
Or shall I balance the imbalance of sizes between the treatment and control using let's say covariate matching and then run significance tests?
Answer: Having different sample sizes in each group won't interfere with causal inference arguments.  The statistical power under the alternative will be highest when comparing two groups with equal sample size.  The p-value and the type I error rate under the null are not affected by imbalanced sample size when using a valid test.
Comment: Yes to both questions.  Typically with IPTW we are balancing the distribution of patient features between treatment groups.  We could create the weights so that they sum to any $n$ of choice in each group.  However, this is artificially augmenting the sample size, imagining the sample size was something different from what was actually collected.  When implementing IPTWs I recommend constructing the weights so they sum to the actual sample size in each group.  Therefore, in your setting that does not require balancing of patient features there would be no IPTW adjustment.
Comment: Statistical significance refers to the p-value threshold or type I error rate of the test under the null hypothesis for decision making.  You can simply report the p-value and confidence interval without declaring a "reject" or "not reject" decision about a particular hypothesis.  You could instead make a "reject" or "not reject" decision about a particular hypothesis based on a particular p-value threshold.
Comment: Thanks, Geoffery. Type II error and statistical power will be affected by the imbalance. Assuming the treatment group is ~15-20% of the total control group, in which case the statistical power will be lower?
Is balancing the data to 50-50% before analysis is incorrect?
Comment: Thanks a lot, Geoffery, really helpful. Can the high confidence interval level with the imbalance and low power be considered statistically significant?


The best way to backup and restore a Linux Mail Server?

Question: I have a Linux Postfix mail server, and I am doing a configuration backup via FTP. but actually thinking about it, this will give me a 3-4 day period for recover my server...
whats is the best way to recover my server in case of failure (hours instead of days)?
thanks...
Answer:
Use automated machine build tools (Kickstart, Debian Installer Preseeding) to make machines quick and easy to create
Use configuration management to maintain the configuration of the server so it's easy to reapply the configuration
Regularly test that the backups you take are useful by running an entire restore from bare metal.
Answer: in addition to womble's suggestion, tar zcvf + ssh is your friend on a linux server. Once you have a server's configuration where you want it, simply
<code>tar zcvf - directory_to_backup/ | ssh someotherserver "cat > serverbackupfile.tgz"
</code>
to save a copy of the important configurations.


Coordinate-free proof that dual Minkowski norm is indeed a Minkowski norm on the dual space

Question: Context: let's say that a Minkowski norm on a vector space $V$ is a map $F\colon V\to \Bbb R_{\geq 0}$ such that $F$ is smooth on $V\setminus \{0\}$, $F$ is positive-homogeneous of degree $1$, and for every $x\in V\setminus \{0\}$, the symmetric bilinear form $g_x$ on $V$ defined by $$g_x(v,w) = \frac{1}{2} \frac{\partial^2}{\partial t\partial s}\bigg|_{t=s=0}F^2(x+tv+sw)= \frac{1}{2} D^2(F^2)(x)(v,w)$$is positive-definite. Now define $F^*\colon V^* \to \Bbb R_{\geq 0}$ by $F^*(\xi) =\max \{\xi(x) \mid F(x)=1\}$.
I want to understand the proof that $F^*$ is a Minkowski norm, in the above sense, on $V^*$. Everything is clear except the positivity condition.
I am following the notes by Matias Dahl (clearly based on the book by Zhongmin Shen). He does a heavy coordinate computation on page 7 that, as I see it, hides the actual idea behind the proof. This is Lemma 3.1.2 on Shen's book. So, I want a proof without coordinates, but I'm struggling to translate what he did there.
We are free to use the Legendre transform $\ell\colon V\to V^*$ given by $\ell(x) = g_x(x,\cdot)$ and $\ell(0) = 0$, and its properties --- in particular that $F = F^*\circ \ell$.
From $F^2 = (F^*)^2 \circ \ell$, we have that $$D(F^2)(x)(v) = D((F^*)^2)(\ell(x)) \circ D\ell(x)v,\tag{$1$}$$and thus $$D^2(F^2)(x)(v,w) = D^2((F^*)^2)(\ell(x))(D\ell(x)v,D\ell(x)w) + D((F^*)^2)(\ell(x))((D^2\ell)(x)(v,w)),\tag{$2$}$$so the proof is concluded once we establish that $$D((F^*)^2)(\ell(x))((D^2\ell)(x)(v,w)) = 0,\tag{$3$}$$as it will follow that $g = \ell^*(g^*)$, where $g$ is the assignment $x\mapsto g_x$ on $V\setminus \{0\}$ and $g^*$ is the assignment $\xi \mapsto g^*_\xi$ on $V^*\setminus \{0\}$ induced by $F^*$. So each $g_x$ being positive-definite implies each $g^*_{\ell(x)}$ positive-definite as well, and we're done as $\ell$ is surjective.
I cannot verify that $(3)$ is true. From Dahl's coordinate computation, the homogeneity relation $g_{\lambda x} = g_x$ (for $\lambda > 0$) must enter. This implies that $\ell(\lambda x) = \lambda\ell(x)$ (for $\lambda>0$), and (1) reads $g_x(x,v) = g^*_{\ell(x)}(\ell(x), D\ell(x)v)$, while $(D\ell(x)v)w = g_x(v,w)$, but we only have that $(D^2\ell)(x)(x,\cdot) = 0$ and I don't see how this is enough.
Answer: I have cooked up a proof. Differentiating the relation $(1/2)F^2=(1/2)(F^*)^2\circ \ell$ twice at $x$ and evaluating at $v$ and $w$, we directly have that \begin{align*} g_x(x,v) &= g_{\ell(x)}^*(\ell(x),D\ell(x)v), \\  g_x(v,w) &= \ell^*(g^*)_x(v,w)+ \frac{1}{2} D((F^*)^2)(\ell(x))((D^2\ell)(x)(v,w)).  \end{align*}It remains to show that this error term vanishes. First, consider the vector $z\in V$ such that $g_x(z,\cdot) = (D^2\ell)(x)(v,w)$. Thus, $D\ell = g$ gives that $D\ell(x)z = (D^2\ell)(x)(v,w)$. With this in place, the above relations give that
\begin{align*}
  \frac{1}{2}  D((F^*)^2)(\ell(x))((D^2\ell)(x)(v,w)) &= g_{\ell(x)}^*(\ell(x),(D^2\ell)(x)(v,w) ) \\ &= g_{\ell(x)}^*(\ell(x),D\ell(x)z) \\ &= g_x(x,z) \\ &= [(D^2\ell)(x)(v,w)]x.
  \end{align*}
As for why we have $[(D^2\ell)(x)(v,w)]x=0$, we differentiate the previously established identity $(D\ell(x)w)x = \ell(x)w$ at the point $x$, in the direction of $v$, to obtain $$  [(D^2\ell)(x)(v,w)]x + (D\ell(x)w)v = (D\ell(x)v)w.  $$Since $(D\ell(x)w)v = (D\ell(x)v)w$ (as $g_x$ is symmetric), we are done.
Answer: Here's how I think of this:
$\newcommand{\R}{\mathbb{R}}$ First, if $F: V \rightarrow \R$ is a convex homogeneous function positive away from the origin, then the function $F^*: V^* \rightarrow \R$, defined to be
$$
F^*(\xi) = \sup \left\{ \frac{\langle \xi,x\rangle}{F(x)} \right\},
$$
is also convex and homogeneous, because it is the sup of linear functions.
Now assume that $F$ is $C^2$ and let $\phi = \frac{1}{2}F^2$ and $\phi^* = \frac{1}{2}(F^*)^2$. Their differentials are maps
\begin{align*}
D\phi = F,DF: V &\rightarrow V^*\\
D\phi^* = F^*\,DF^*: V^* &\rightarrow V.
\end{align*}
The Hessian $D^2\phi = F\,D^2F + DF\otimes DF$ is a symmetric $2$-tensor-valued function of $V$ that is homogeneous of degree $0$. Let $g$ be the restriction of this tensor to the unit sphere, $F = 1$. $g$ is a Riemannian metric on the sphere if and only if $D^2\phi$ is always positive definite.
If we assume that the Hessian $D^2\phi$ is always positive definite away from the origin, then $D\phi$ and $D\phi^*$ are homogeneous diffeomorphisms away from the origin. Moreover, you can show that the maps $D\phi$ and $D\phi^*$ are inverses of each other. This implies that, if $\xi = D\phi(v)$, then
$$
D^2\phi^*(\xi) = (D^2\phi(v))^{-1}
$$
In particular, $D^2\phi^*$ is always positive definite and therefore its restriction to the tangent space of the unit sphere $F^* = 1$ is positive definite.
You can find this all spelled out in a little survey paper I wrote: https://www.math.nyu.edu/~yangd/papers/affine_survey.pdf
Comment: Thank you Deane! This is a very nice and reader-friendly write-up. For whoever sees this, the part of the survey addressing the issue here is Section 5.6.


Adicionar parâmetro em todas as minhas URLs internas

Question: Tipo, meu site tem vários posts e tal, gostaria que todos os links do meu Próprio site ficasse com ?ref=app no final.. mesmo eu mudando de pagina ele ficasse com o parâmetro.
Existe um código ? uma forma de fazer isso ?
Comment: Pesquise isso relacionado a htaccess, pode ser que te ajude
Comment: Poderia ser mais objetivo na pergunta? Esclarecer alguns pontos de como está fazendo?
Comment: Tipo, meu site quero que abre dentro de um App, e quero que quando ele abra dentro de um app fique com todas as url dele com a ref=app, assim vou saber que e de pelo app e editar o tema como eu quiser so pro app
Answer: Você pode usa rum script para capturar sua URL e editar ela para a nova referencia.
<code><script type="text/javascript">
  // PROTETOR DE LINK
  (() => {
  const urlsBases = ["seusite.com", "localhost"];

  for (let urlBase of urlsBases) {

    const anchors = document.querySelectorAll("a[href*='" + urlBase + "']");

    anchors.forEach(el => {
        let urlProtegida = el.getAttribute("href") + "?ref=app"

        el.setAttribute("href", urlProtegida)

        console.log(urlProtegida)
    });
  }
  })();
  </script>
</code>
Ou seja, ele pegou seu domínio principal e adicionou a referencia: ?ref=app no final.
Comment: Muito bom!valeu cara.


extracting middle OR final part of a string

Question: I want to extract only the first fontname out of a URL-string from the Google Webfont Directory. Here are some examples of possible strings and what part should be returned:
fonts.googleapis.com/css?family=Raleway              // "Raleway"
fonts.googleapis.com/css?family=Caesar+Dressing      // "Caesar Dressing"
fonts.googleapis.com/css?family=Raleway:300,400      // "Raleway"
fonts.googleapis.com/css?family=Raleway|Fondamento   // "Raleway"
fonts.googleapis.com/css?family=Caesar+Dressing|Raleway:300,400|Fondamento  // "Caesar Dressing"

So sometimes it's just one fontname, sometimes it has a weight indicated by a colon (:) and sometimes there are more fontnames divided by a pipe (|).
I have tried /family=(\S*)[:|]/ but it only matches the strings with :or |. I could do it like this, but it's not a nice solution:
var fontUrl = "fonts.googleapis.com/css?family=Caesar+Dressing|Raleway:300,400|Fondamento";
var fontName = /family=(\S*)/.exec(fontUrl)[1].replace(/\+/, " ");
if (fontName.indexOf(':') != -1){
    fontName = fontName.split(':')[0];
}
if (fontName.indexOf('|') != -1){
    fontName = fontName.split('|')[0];
}
console.log(fontName);

Is there a nice regex solution to this?
Answer: Instead of matching the character that (might) follow the string you want, match only the string you want except those characters:
/family=([^\s:|]*)/

Alternatively, you'd use a lookahead like this:
/family=(\S*?)(?=$|[:|])/

Comment: could you please explain `[^\s:|]` to me?
Comment: Check out [character classes](http://www.regular-expressions.info/charclass.html). Since `\S` == `[^\s]`, (`\S` without `[:|]`) == `[^\s:|]`
Comment: @AvL You could say it means any character except white spaces (`\s`), colons and pipes. `[^]` basically means 'not matching' this and that character following the caret in the square brackets.
Answer: That should be better:
/family=([^:|]*)/

Of course for the + case, you'll have to replace it afterwards (or before maybe).
Comment: Thank you. I did the `+`part like this: `/family=([^:|]*)/.exec(fontName)[1].replace(/\+/, " ")`
Answer: try this:
/family\=(\S+?)[\:\|,]{0,2}\S*/ims

Comment: this is not cutting the string before `:`or `|`. It's the same as `family=(\S*)`
Answer: You can use (choose the i and m modifier in all case):
family=([a-z]+\+?[a-z]+)

or more simply
family=([a-z+]+)

or to avoid matching the + char:
family=([a-z]+)\+?([a-z]+)?

but it is an easyer way to use the second solution, and to replace the + chars with a space after.
Comment: @Jerry: You don't have read! I precised to use the i modifier. Underscores are not needed
Answer: No regex is required in this case, unless you are good with regex's or test them thoroughly then you are likely to make mistakes.
var fontUrls = [];

fontUrls.push("fonts.googleapis.com/css?family=Raleway");
fontUrls.push("fonts.googleapis.com/css?family=Caesar+Dressing");
fontUrls.push("fonts.googleapis.com/css?family=Raleway:300,400");
fontUrls.push("fonts.googleapis.com/css?family=Raleway|Fondamento");
fontUrls.push("fonts.googleapis.com/css?family=Caesar+Dressing|Raleway:300,400|Fondamento");

function getFirstFont(url) {
    return url.split("=")[1].split("|")[0].split(":")[0];
}

fontUrls.forEach(function (fontUrl) {
  console.log(getFirstFont(fontUrl));
});

on jsfiddle