Full and fractional factorial designs

Question: I have a data set consisting of six independent environmental variables (all binomial: present / absent) and one dependent variable (binomial: disease present / absent).
In order to determine the combination of factors that have the highest probability of leading to disease, I first need to conduct an Expert Opinion poll where I will have several experts rank all possible combinations of variables according to their probability of leading to the occurrence of disease. Then, I will obtain regression parameters for each variable using a conjoint analysis approach where each expert conforms a level (hierarchical design), the six environmental variables are independent variables, and the rankings are the dependent variable. There being six factor variables, there exist a total of 64 possible orthogonal combinations.
I reduced this overwhelming number of possible combinations (while retaining orthogonality) using the AlgDesign package of R. Here is the code followed only by relevant pieces of output:
levels.design = c(2,2,2,2,2,2)
full.design <- gen.factorial(levels.design)

   X1 X2 X3 X4 X5 X6
1  -1 -1 -1 -1 -1 -1
2   1 -1 -1 -1 -1 -1
3  -1  1 -1 -1 -1 -1
   .................
63 -1  1  1  1  1  1
64  1  1  1  1  1  1

set.seed(69)
fractional <- optFederov(~., data=full.design, approximate=FALSE, criterion="D")
fractional

The result is a subset of 12 combinations to be included in the conjoint analysis:
$design
    X1 X2 X3 X4 X5 X6
4   1  1 -1 -1 -1 -1
5  -1 -1  1 -1 -1 -1
    ................
57 -1 -1 -1  1  1  1

From what I understand, doing a regression analysis on all 64 combinations should lead to the same regression parameters as those obtained if I use only the reduced set (i.e. 12 combinations from the fractional factorial design).
Questions:

Do the code and the resulting output make sense?
Could anyone point me to a good and simple reference on how this fractional design works? I am afraid I might be doing things wrong by selecting a subset that produces different results from those obtained if a full factorial design was employed.
Answer: Speaking to question 2, you're doing a D-optimal design against the full model using a criterion for linear models.  You have no guarantee that main effects are not correlated with other main effects or higher order terms.  Though your design is likely a non-regular fractional factorial, you might be better served looking at the regular fraction $2^{6-3}_\mathrm{III}$ in 8 runs, or that design but augmented using the $D$-criterion to the 12 runs you want to do, or the regular fraction $2^{6-3}_\mathrm{III}$ in 16 runs.  Check out Montgomery's "Design of Experiments" book or the Box, Hunter, and Hunter "Statistics for Experimenters" book.  Montgomery's book has all the regular fractions in the back.  You can also check out NIST's handbook page on fractional factorials
The times in which the fractional factorial design is guaranteed to fit the same parameter estimates as the full factorial are when only one of any pair of terms that are correlated or confounded in the fractional design has a non-zero effect size.  It's very hard to get a $D$-optimal design to do this.  $D$-optimal designs are mainly used (in industrial settings at least) when there's some kind of restriction on the number of runs you can do (so you can't do a power of 2) or on the design space.

Full and fractional factorial designs


Reverse of the Central Limit Theorem

Question: I have a question about a statistical concept and would like to gather your thoughts on it. As we know, the Central Limit Theorem states that under certain conditions, the sum or average of independent and identically distributed random variables tends to a normal distribution as the sample size increases. However, I am pondering the reverse of this theorem:
If the sum of random variables $x_1, x_2 ... x_n$ exhibits a normal distribution, can we infer that these variables are independent? In other words, is a sum that conforms to a normal distribution sufficient to indicate the independence of the variables?
Do we have any idea or reference?
Comment: Nope. There are central limit theorems for sums of dependent variables, such as the Hypergeometric Distribution. There are also trivial counterexamples like $x_1 = Z$, $x_2 = x_3 = \dots = 0$.
Answer: The answer is NO. It is not possible to grant that if $S = \sum_{i=1}^{n} x_i$ has normal distribution, then $x_1, \ldots, x_n$ are independents.
In fact, if you check the Wiki page of the CLT, you will find that it exist several versions of the CLT and the "CLT under weak dependence" does not require the independence of the samples.

Reverse of the Central Limit Theorem


Unit test-Give a fake value to the variables initialized by constructor

Question: I have a class
Test.ts:
class Test
{
     public property1: string;
     public property2: boolean;
     constructor(property1, property2)
     {
            this.property1 = property1;//will get value while instantiation
            this.property2 = property2;//will get value while instantiation
            this.property3 = somevalue;//want to add fake value here
     }
}

As given above I want to give a fake value for property3 but property1 and and property2 I will get it from instantiation. How can I achieve This?
Comment: TIP: Use the "StackSnippet" utility and click the `<>` button to format your code.
Comment: Please can you edit your question? Otherwise in its current state it's likely to be closed.
Comment: where did that come from? I gave a tip, and asked for you to format your code, that was all.
Comment: _"please concentrate on the main objective"_ I think you need to re-read [ask]
Answer: I believe you are struggling with declaring the property3 member. Simply do it like you declared the other properties. Additional you can use property initializer if you wanted:
class Test
{
     public property1: string;
     public property2: boolean;
     public property3 = somevalue;
     constructor(property1, property2)
     {
            this.property1 = property1;//will get value while instantiation
            this.property2 = property2;//will get value while instantiation
     }
}

More
Some docs on TypeScript classes https://basarat.gitbooks.io/typescript/docs/classes.html
Comment: Thanks for reply. I cant move the property up because i want to initialize it at the instantiation time only.So i have to give a fake value there


Matrix square roots of -I

Question: Since we can see matrices as generalizations of complex numbers, I asked myself if there is a way to classify those matrices which are the "Basis" for the complex part. That is, I would like to identify the set of $n\times n$ real valued matrices $M$ whose square $M^2$ is equal to $-I$, where $I$ is the $n\times n$-identity matrix. Is this set already classified?
The matrix $J = \left( \begin{smallmatrix}0 & -1\\1 & 0 \end{smallmatrix} \right)$ satisfies $J^{2} = -I$ in the 2-dimensional case.
EDIT:
Thanks to your comments and answers I have the following observation (if its wrong, please tell me):
Suppose that $M$ satisfies that $M^t=-M$, then we can observe for two vectors $x,y\in\mathbb{R}^{2n}$ that

$y^tAx=(Ax)^{t}y=x^{t}A^ty=-x^tAy$

My interpretation is that if the angle between $x$ and $Ay$ is $\alpha$ then $\pi+\alpha$ is the angle between $Ax$ and $y$, since ${\displaystyle \cos \;x=-\cos(x+\pi )}$ (here we suppose that the length of the vectors are not relevant). If now $y=x$ and then we can see that $Ax$ is orthogonal to $x$.
Is it true that $A^2$ will be a rotation of 180 degrees, right? Because of vector length we will have something like $A^2=-CI$, where $C$ is just a constant. Is that true?
Comment: You want to classify such matrices up to conjugation because $M^2+I=0$ implies $(PMP^{-1})^2+I=0$ for every invertible matrix $P$. In other words, you want to classify the linear operators $T$ such that $T^2+id=0$.
Comment: For one thing, $n$ must be even (consider the determinant of $-I$).
Comment: The Cayley-Hamilton Theorem states that every square matrix over a commutative ring satisfies its own characteristic equation. A matrix $A$ with $A^2=-I$ would have to satisfy $p_A(t)=t^2+1$. So we have the condition $\det(tI-A)=t^2+1$.
Comment: Thank you for all your comments.Actually what I try to understand is the following: when I have a matrix, it is no more than a linear application of a vector. In order to have real value eigenvectors we need symmetric matrices. And what I try to figure out is the mathematical motivation behind symmetric matrices.What I thought is that, when a matrix is symmetric, it is because there is no "complex" part, since we can go into both directions (conjugate) and the direction is the same.Hence, what I need is to see what is the "complex part" of a matrix.Maybe I am wrong with my way of seeing  this.
Comment: Your comment suggests that what you may be looking for is the idea of a _skew symmetric matrix_. Every real matrix is the sum of a symmetric matrix and a skew symmetric matric:  https://en.wikipedia.org/wiki/Symmetric_matrix/wiki/Symmetric_matrix
Comment: You can also represent quaternions this way.
Comment: Real or complex matrices?
Comment: Actually, the main point is that, if I am able to see what is the "complex part" of a matrix, then it will be easier to see why the definition of the conjugate of the matrix is defined by their way...
Comment: I have edited my question. Skew symmetric matrices are definitiely what I was looking for. Now I try to relate it to my original question...
Answer: Here's a solution for the $n=2$ case (too long for the comments, sorry):
Using the determinant condition I gave in the comments, and letting
$$A = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix}$$
we have
$$\det(tI-A) = t^2-a_{22}t-a_{11}t+a_{11}a_{22}-a_{12}a_{21} = t^2+1$$
Equating coefficients gives $-a_{11}=a_{22}$ and $a_{11}a_{22}-a_{12}a_{21}=1$. Rearranging a bit gives the matrix
$$ A = \begin{pmatrix} a_{11} & \frac{-a_{11}^2-1}{a_{21}} \\ a_{21} & -a_{11} \end{pmatrix} $$
for $a_{21} \in \mathbb{R}, a_{21} \neq 0$ and $a_{11} \in \mathbb{R}$. So in the case of the example you gave, $a_{11}=0$ and $a_{21}=1$.
Answer: You might be interested in complex structures on real vector spaces.
The basic idea is this: suppose I have a complex vector space $V$ of dimension $n$. If I forget how to multiply by $i$, and only use scalars that are real, then $V$ is also a real vector space of dimension $2n$ (let's call this $V_{\mathbb{R}}$, to make it clear I've forgotten anything complex). An interesting question is: what exactly did I forget when I forgot how to multiply by $i$? Can I remember it for later, and get back my original complex vector space?
On the complex vector space $V$, there is a linear map $J: V \to V$ which is multiplication by $i$, defined by $Jv = iv$. In fact, this map is $\mathbb{R}$-linear, and so gives a linear map $J: V_\mathbb{R} \to V_\mathbb{R}$. We can check that $J^2v = i^2v = -v$, and so $J^2 = -\mathrm{id}_V$. This map $J$ is exactly what we forgot when we forgot how to multiply by $i$: if I have a vector $v \in V_{\mathbb{R}}$ and want to scale it by $(a + bi)$, then I use $(a \,\mathrm{id}_V + bJ)v$.
Your question is essentially going the opposite way: starting with a $2n$-dimensional real vector space $W$, what are the kind of maps $J: W \to W$ such that $(W, J)$ becomes a complex vector space with action $(a + bi)w = (a \, \mathrm{id}_W + bJ)w$.
Comment: I have edited my question. I can see that there must be a connection between skey symmetric matrices and complex structures on real vector spaces. Do you know any result related to it?
Answer: If you want to make progress, you will need to read some books...
The matrices $M\in M_2(\mathbb{R})$ s.t. $M^2=-I_2$ are similar to your $J$ (there are an infinity of such matrices).
Yet, if you consider that $\mathbb{R}^2$ is euclidean (with an inner product), then it is natural to consider the solutions $M$ that are normal ($MM^T=M^TM$). Then you find only $2$ solutions: $\pm J=Rot(\pm \pi/2)$, that are skew-symmetric. If we consider the isomorphism $f:a+ib\in \mathbb{C}\rightarrow \begin{pmatrix}a&-b\\b&a\end{pmatrix}=aI+bJ\in M_2(\mathbb{R})$, then $\pm J=\pm f(i)$ (cf. the Joppy's post). Of course, if $M$ is skew-symmetric ($a=0$), then $M^2=-b^2I$; yet, it's false when $n>2$.
Now, in dimension $2n$, the matrices s.t. $M^2=-I_{2n}$ are similar to $diag(J_1,\cdots,J_n)$ where $J_k=J$.
In the euclidean case, where $M$ is normal, then the solutions are skew-symmetric and in the form $Pdiag(J_1,\cdots,J_n)P^T$ where $P$ is an orthogonal matrix;  that is to say $\mathbb{R}^{2n}$ is an orthogonal sum of $n$ planes $\Pi_k$ s.t., over each $\Pi_k$, the linear transformation is $Rot(\pm \pi/2)$.

Matrix square roots of -I


VS2005: how to find text in the current function

Question: In the VS2005 code-editor using C#, how do you search for text in the current function only?  It allows searching over the Current Document but I cannot see how to limit the search to only the current function.
Comment: hopefully your 'functions' are short enough for this to be a small problem?
Answer: Select the function's code and then a new option will be available in the dropdown list. This option is called Current Block. This will allow you to search only within that specific function.
From the source on MSDN: Quick Find, Find and Replace Window
Answer: Most of the time, your functions should be short enough that this really isn't an issue, and I mean that.
Occasionally, you might have an almost-legitimate need for some kind of indexing function that has a lot of distinct tokens.  Even in this case, you should refactor to use a database or reflection or something, as this is the kind of thing that'll get your code featured on DailyWtf.com.
But if it's really unavoidable to have a function this long, if it's legacy code, or for some other weird reason, there are some things you can do:

The Find All References feature in visual studio lists all results at once, in order. You can size the window to show just the results in your function.
You can use partial classes to isolate your function to a specific file
Highlight the function with your mouse, and the Find/Replace dialog can search within selected text only.


Qt: right mouse click handling on QDockWidget title

Question: Clicking right mouse button on any QDockWidget title causes list of all QDockWidget in application.
To disable this beheavor I implemented special title class
class QMyTitleBar : public QLabel
{
public:
    QMyTitleBar(QWidget *parent = Q_NULLPTR, Qt::WindowFlags f = Qt::WindowFlags()) : QLabel(parent, f)
    {
    }

    QMyTitleBar(const QString &text, QWidget *parent = Q_NULLPTR, Qt::WindowFlags f = Qt::WindowFlags()) : QLabel(text, parent, f)
    {
    }

    ~QMyTitleBar()
    {
    }

protected:
    void mouseReleaseEvent(QMouseEvent * qevent) override
{
    if (qevent->button() == Qt::MouseButton::RightButton)
    {
        qevent->accept();
        return;
    }
}

void mousePressEvent(QMouseEvent * qevent) override
{
    if (qevent->button() == Qt::MouseButton::RightButton)
    {
        qevent->accept();
        return;
    }
}

and set to my QDockWidget this custom title widget
myDockWidget->setTitleBarWidget(new QMyTitleBar(QObject::tr("My Title")));

but this does not work. It handles mouse event but menu with all QDockWidgets appears.
How can I remove this menu?
Comment: try: (parent, Qt::Window | Qt::WindowStaysOnTopHint |
        Qt::X11BypassWindowManagerHint | Qt::FramelessWindowHint)
Answer: dock_widget->setContextMenuPolicy(Qt::PreventContextMenu);
From: How to disable the context menu of a QDockWidget title bar
Comment: Thanks! I hed a trouble to find the anser


MySQL - How to make more than 1 where clause on same column?

Question: Well i'm trying to make a where clause on same column many times i.e:
  product
----------------------------------------
| product_id | product_name | group_id |
----------------------------------------

  group
-------------------------
| group_id | group_name |
-------------------------

  group_val
-------------------------
| group_id | product_id |
-------------------------

select b.product_name
from group_val a
inner join product b using(product_id)
inner join group c using(group_id)
where a.group_id in(10,15,88)
group by b.product_id;

it's same as or ... or but i need to do something like: it only will return product that are on group 10 and group 15 and group 88: I mean that a product need to be in these 3 groups.
i.e:
  group_val
-------------------------
| group_id | product_id |
-------------------------
| 10       | 1          |
-------------------------
| 15       | 1          |
-------------------------
| 88       | 1          |
-------------------------
| 15       | 2          |
-------------------------
| 10       | 2          |
-------------------------

In this case it will only return product 1
How can I do it?
Comment: [What have you tried?](http://www.whathaveyoutried.com/) See [ask advice](http://stackoverflow.com/questions/ask-advice), please.
Comment: Have you tried `and ... and` in place of `or ... or`?
Comment: It's a bad idea to use SQL reserved words as table or column names (in your case *group*).
Comment: it's only a example ;p
Comment: @ultranaut yes, it doesn't work
Comment: Maybe a side issue, maybe not, but does `group_id` make sense as a field in your  `product` table? Isn't `group_val` what's associating a product with a particular group(s)?
Answer: You don't need the field group_id in table product
SELECT p.product_name
  FROM product AS p
    INNER JOIN group_val AS g1 ON p.product_id = g1.product_id
    INNER JOIN group_val AS g2 ON p.product_id = g2.product_id
    INNER JOIN group_val AS g3 ON p.product_id = g3.product_id
  WHERE g1.group_id = 10 AND g2.group_id = 15 AND g3.group_id=88;

Try it in SQL Fiddle
Answer: Just working with your example query, and assuming that you explicity know, programmatically or otherwise, which group_ids you're looking to match and how many there are:
select `product_name`, count(gv.`product_id`) as cnt
from `product` p
left join `group_val` gv
on p.`product_id` = gv.`product_id`
where gv.`group_id` in (10, 15, 88)
group by p.`product_id`
having cnt = 3;

will select all the product_ids in group_val that is in all of those groups.
Comment: it worked but i need to use distinct on count(distinct(gv.product_id)), thanks ;p
Comment: @ultranaut I think your approach is better than mine, but it can be improved, see this [fiddle](http://sqlfiddle.com/#!2/58b1d9/4)
Comment: @grahamj42 yeah, you don't really need to get the count back I guess


How might I find the values of $a$ in terms of $b$ such that ${2^a}+1=3^{b}c$?

Question: I currently have the expression $2^a+1=3^bc$, where $a$, $b$ and $c$ must be positive integers. I am looking for the possible values of $a$ in relation to $b$. I know the solution is $a=3^{b-1}(2n+1)$, where $n$ can be a positive integer, by trial and error, however I cannot seem to come up with a solid proof for this. I have tried using logarithms and different series but it has not worked so far, so I was wondering if anyone could give me a push in the right direction.
EDIT:
A rephrasing of my question is: "What values of $a\in\mathbb{N}$ ensure that $2^a+1$ is some multiple of $3^b$?" - if possible giving a solution using elementary methods.
Thanks :)
Comment: Write $2 = (-1) + 3^1 + k\cdot 3^2$. Raise to the third power (binomial expansion) to find $2^{3^1} = (-1) + 3^2 + m\cdot 3^3$. Repeat to find something about $2^{3^n}$.
Comment: I suggest rephrasing the question as: *what constraints on $a\in\mathbb{N}$ ensure that $2^a+1$ is a multiple of a fixed power of $3$ ?*
Comment: You may just try to re-prove by elementary techniques that $2^a+1$ is a multiple of $3^b$ if and only if $a$ is an odd multiple of $3^{b-1}$.
Comment: $c$ is not the solution you know?
Comment: What do you mean? I'm looking for the value of $a$ such that $b$ and $c$ are integers and $2^a+1=3^bc$
Comment: Ok thanks, but surely $k$ would just be 0? Did you mean $2=$ or something else?
Comment: Thanks, definitely more concise and I have just added it to my original question. I'm currently trying to comprehend your answer (I have not yet studied any university level Mathematics)
Comment: I believe I have managed to do this with two proofs by induction - do you reckon there is an easier/more elegant (elementary) method?
Answer: Let $\nu_3(n)=\max\{m\in\mathbb{N}: 3^m\mid n\}$. Since $2^a+1\equiv(-1)^a+1\pmod{3}$, $3$ is a divisor of $2^a+1$ iff $a$ is odd. Since $2$ is a generator for $\mathbb{Z}/(9\mathbb{Z})^*$ we have that $9\mid(2^a+1)$ iff $a$ is odd and a multiple of $3$. By Hensel's lifting lemma we have that $2$ is a generator for $\mathbb{Z}/(3^k\mathbb{Z})^*$, hence for any $k\geq 1$ we have that $3^k$ is a divisor of $2^a+1$ iff $a$ is odd and $a$ is a multiple of $3^{k-1}$:
$$ \nu_3(2^a+1) = \mathbb{1}_{\text{odd}}(a)\cdot\left(\nu_3(a)+1\right).\tag{1} $$
In particular, if $2^a+1 = 3^b\cdot c$, then $\color{red}{a\geq 3^{b-1}}$.
Comment: Lifting The Exponent Lemma is relevant.
Answer: Basically it is equivalent to find some $a\in \mathcal{N}$ such that $2^a+1$ can be divided by 3. It is easy to find that all the odd numbers is a solution to $a$, but the even numbers are not. Now $b$ and $c$ can be easily obtained once you get $a$.
Comment: Ok, but wouldn't that imply $b$ is 1? It also has to be divisible by $3^b$ not just $3$. I'm looking for a set of solutions for $a$ in terms of $b$
Comment: For $b=1$, it is easy to see $a=2k+1$. For $b=2$, $a=6k+3$. For $b=3$, $a=18k+9$
Comment: Yes, this is the result I found - however I have not found not any concrete or intuitive way of deriving it (without using maths out of my scope of understanding)


WooCommerce: filter products by attribute and hide product if variation is out of stock

Question: I use the built in WooCommerce filter by attribute widget which works for what I need, but my issue is, that if I filter on a size, I get all products with that size as an attribute, also when they are out of stock...
My products are created as a variable product with variations on size and color
What I want, is to filter a size and only display products in that size that is in stock.
Answer: It may not be possible for variable products and their variations (but only for other product types).
It can only works for variable products when stock management is handled by the product variation itself (but not by the variations).
Note that WooCommerce product query doesn't handle post type 'product_variation', but only post type 'product'.
To exclude out of stock products using widget filters (except for variable products) you can use:
add_filter( 'woocommerce_product_query_tax_query', 'filter_product_query_tax_query' );
function filter_product_query_tax_query( $tax_query ){
    if ( ! is_admin() && isset($_GET['filter_color']) && ! empty($_GET['filter_color']) ) {
        // Exclude products "out of stock"
        $tax_query[] = array(
            'taxonomy' => 'product_visibility',
            'field'    => 'name',
            'terms'    => array('outofstock'),
            'operator' => 'NOT IN'
        );
    }

    return $tax_query;
}

Code goes in functions.php file of the active child theme (or active theme). Tested and works.
Comment: You mention that it only is possible if the stock management is handled by product variation, but not by the variations, what exactly do you mean? Currently we have enabled the option for managing stock on each variant and entered the amount in stock
Answer: I've used this code for my stores to hide product sizes that are not in stock whilst filtering.
add_filter( 'woocommerce_product_is_visible', 'hide_product_with_outofstock_variation', 10, 2 );
if(!function_exists('hide_product_with_outofstock_variation')){
    function hide_product_with_outofstock_variation( $is_visible, $id ) {
        $product = wc_get_product($id);
        if ( $product->is_type('variable')) { // if variation product is out of stock
            $available = $product->get_available_variations();
            if ( $available )foreach ( $available as $instockvar ) {
                if(isset($_GET['filter_size'])){
                    $destostock = $_GET['filter_size'];
                    if($destostock ==$instockvar['attributes']['attribute_pa_size']){
                        if($instockvar['is_in_stock']){
                            $is_visible = true;
                        }else{
                            $is_visible = false;
                        }
                    }
                }
            }
        }
        return $is_visible;
    }
}

Answer: filter_size will be converted in array if many filter being selected.
As for product variations, best if its stored in an array as well and use array_intersect to search if one of the selected item available in stock.
add_filter( 'woocommerce_product_is_visible', 'hide_product_with_outofstock_variation', 10, 2 );
if(!function_exists('hide_product_with_outofstock_variation')){
    function hide_product_with_outofstock_variation( $is_visible, $id ) {
        if(isset($_GET['filter_size'])){
                $filter_size = explode(',',$_GET['filter_size']);
                $product = wc_get_product($id);
                if ( $product->is_type('variable')) { // if variation product is out of stock
                    $available = $product->get_available_variations();
                    $available_size = array();

                    if ( $available )foreach ( $available as $instockvar ) {
                        array_push($available_size, $instockvar['attributes']['attribute_pa_size']);
                    }
                    if (count(array_intersect($filter_size, $available_size)) === 0) {
                        $is_visible = false;
                    }else{
                        $is_visible = true;
                    }
                }
        }
        return $is_visible;
    }
}

This code works for me
Answer: You can hide out of stock products by checking the option of "Out of stock visibility" on the Settings page:
Woocommerce > Settings > Product > Inventory
Comment: Okay, so you are managing the stock in the variations of all the attributes correct?
Comment: I've already done that, but the products still show up if the size is sold out... I want to mention that almost all products are variable products and many of them also have brand, size and color as an attribute
Comment: Yes, that is correct. Stock is managed on each product variantion. Is there another way to manage it? Since we have a limited amount of each size


Instant app won't start directly, shows a screen first

Question: I have successfully published a (very simple) Instant App linked to my personal website.
When I click on my website's link, I was expecting the app to start directly, but instead, I first see a screen with 2 buttons: "Open app" and "Open in browser":

Why?
Other instant apps that I tried do not show this screen and open the app directly (the desired behavior).
Comment: The thing is, on the same device, I have both behaviors, depending on the app.
Comment: I also can see the same behavior on a Nexus 5X API 24 emulator, but when I use other setups (Pixel API 24 and Nexus 5X API 26) instant app starts directly, no "Open in browser" option is present. Try?
Comment: Can you check your app with these tests? https://developer.android.com/training/app-links/index.html#testing App Linking must first work before Instant Apps can.
Comment: @JuliaK It *does* work though - my only problem is that it first shows the above screen. But other than that it works.
Comment: Please try this: 1. Google > Instant Apps > turn "off" (top-right) | 2. Uninstall ⸢Google Play services for Instant Apps⸥ (and if the installed-app is installed, also uninstall) | 3. re-run your instant-app
Comment: Not sure why I would want to do that @TWL, the question has been answered.
Comment: @BoD did you get this fixed?
Answer: We call this screen the "speed bump". It's a security feature. You're right in noticing that it doesn't always trigger. But there's nothing you, as a developer, can do to control whether it shows. The intention is for it to be a black box from your perspective.
Comment: Thanks for answering. Well I wouldn't say it is problematic but since it is not documented and not happening with other apps I tried I thought there was a problem with my code. Therefore I *strongly* suggest you document this behavior in the guides/reference documentation - that will save people from being confused.  Thanks a lot.
Comment: This happened to my new instant app as well. I noticed https://www.buzzfeed.com/tasty (my go-to reference instant app) always opens up directly, while my instant app presents the choice. Good to know there's nothing I can do about it! Maybe after some period of time and because of early concerns with "instant app viruses," Google trusts the instant app enough to open immediately?
Comment: @philo I have never seen the speed bump on wish and BuzzFeed. Is there any way you (Google) could remove it from my app? If so could you please post the details.
Comment: my instant app download size is 1.2 MB and I'm still seeing this speed bump. There are no extra permissions. Is there a way to bypass it? Otherwise my purpose on going from mobile web to instant app is useless
Comment: @philo I've reached out to google play support. But they are not even understanding the problem. This is very critical for us. Is there any way we can get over this?
Comment: @ScottyB, thanks for sharing about  buzzfeed.com/tasty. I am surprised to know how they're able to bypass speed bump screen if this is not possible to bypass as developer.
Answer: This is designed in my opinion to happen at least once to give the user an opportunity to decide what to use: either browser or the instant apps feature.
Comment: You need to have this link somewhere else, not in the browser.  Copy/paste it in a messaging app for instance.
Comment: Alas, you didn't read the part where I say that it doesn't happen with other apps...
Comment: @BoD could you tell me another instant app, I am curious by myself. Wish for example was exactly like that.
Comment: Well for instance the StackOverflow app   You can click on this to trigger it: https://stackoverflow.com/questions/44858798/instant-app-wont-start-directly-shows-a-screen-first/44953956#
Comment: @Bod sorry, but it opens for me just in a normal browser. How you get this triggered with an instance app?
Comment: @BoD still opens a browser from the Gmail application for example.
Comment: It looks like you don't have Instant Apps enabled


How to remove certain statement from a function conditionally

Question: So let's say I have a function of
def question():
    print("")
    question_section = str("B" + str(quiz_nmbr + 1))
    print("Question", str(quiz_nmbr) + ": " + str(sheet[question_section].value))
    question_become = str(input("Type your question: "))
    sheet[question_section].value = question_become
    book.save('Quiz_Problems.xlsx')

Then let's say one time I wanted to call the question() function again.
However, I don't want print("Question", str(quiz_nmbr) + ": " + str(sheet[question_section].value)) to be printed.
Is there anyway to just remove that statement for certain condition? and by condition what I mean is let's say I wanted to call the function in if else statement (which condition matters to give different output)
Comment: Do you know how to pass parameters to functions? Are you familiar with conditional expressions? Once you know these things you'll have your answer
Answer: Try this:
def question(prompt):
    print("")
    question_section = str("B" + str(quiz_nmbr + 1))
    if prompt:
        print("Question", str(quiz_nmbr) + ": " + str(sheet[question_section].value))
    question_become = str(input("Type your question: "))
    sheet[question_section].value = question_become
    book.save('Quiz_Problems.xlsx')

Then, inside your if/else clause, you can call question(True) or question(False) as desired.


How rasa_nlu using lookup_tables for entity extraction?

Question: I am trying to develop a chatbot using rasa nlu and rasa core. But I am not getting the link how rasa_nlu using lookup_tables for entity extraction. I had already go through (http://blog.rasa.com/improving-entity-extraction/) link but not getting it. How should lookup_table used for to extract the entity?
Answer: Requirements:
If you want to use lookup tables, make sure:

you have the components intent_entity_featurizer_regex and ner_crf in your NLU pipeline
the entities you want to match fit have a well defined and narrow scope

entities like food names, company names, car brands are unlikely to appear in contexts you in which you don't want to match them. Hence, look up tables are a good use case for them.
entities like objects (e.g. "car", "house", "paper") appear in a variety of contexts in which you don't want to match them at all. Therefore, using look up tables might even lead to worse results.

In your training data
In order to use look up tables, you can either define them directly in the training data, e.g.:
## intent:check_balance
- what is my balance
- Could I pay in [yen](currency)?

## lookup:currency
- Yen
- USD
- Euro

Or you can write them in a text file:
Yen
USD
Euro

And then include the path to the text file in your training data:
## intent:check_balance
   ... like before

## lookup:food
    <path to your look up table text file>.txt

Taking an input like Could I pay in Euro?, Rasa NLU then sets the value of the slot currency to Euro.
How they work
The single items in a look up table are added to a regular expression (regex) which is applied to the the messages which your users send to the bot. However, look up tables don't work if your user inserts typos, e.g. a look up table entry Pesos would not match Peesos. To also match these cases you can try fuzzy matching which is described in the blog article you linked. Make sure that your look up tables don't become too large as Rasa NLU has to check every sentence whether it matches one of your look up table entries.
Maybe the Rasa NLU documentation can also help you.


Strange log message and node not found for anonymous

Question: I have a node found at the teaser of front-end for both admin and anonymous users. However, going to the node details page using anonymous user leads to 404 error while it works fine with admin.
The log message details is strange as shown in the following screen shot:

What is custom_404_page? I'm afraid of it is a hacking against the website.
I tried to clear all cache, I disabled XMLsitemap module. I could not able to determine where is the origin of custom_404_page.
Also in the log and after clearing it, and the trying to visit this node's details, this is the only log message available. i.e there is no other messages or warnings.
Answer: Check your custom 404 configuration at admin/config/system/site-information.
Also check if this custom configuration is directing users to a Panels page (check Structure > Panels or Structure > Pages if you have Panels and Page Manager installed). They may be handling different custom 404 pages for different roles. Perhaps the Panel variant that handles the 404 page for anonymous is missing or redirecting to a non-existing page.
Comment: I removed the custom 404 page settings. i.e I have left the input field blank. The idea here the content is really available and appears correctly with admin and other like contents works fine with both admin and anonymous.
Comment: I'm not sure what you are saying here. Did removing the custom setting fix the problem?
Comment: No it is not fixed.
Comment: OK, then I don't understand what you meant in the comment above.
Comment: So, what you are saying is that the content DOES exist, but Drupal is logging a "page not found" report? Is Drupal ALSO showing the "404 Page not found" page when trying to access the content?
Comment: lets we say the problem is related with `node/14148` other nodes of the same type works fine without any problem. This node is not available for anonymous. Anonymous user can only see it as teaser.


Parallel processing queries

Question: I have written a program in matlab and my algorithm depends on few parameters let's say a and b with a=1:10 and b=1:10 I want to find those values of a and b which gives me the best results.
My main code is as follows:
a= 0.1:0.1:1;
b= 1:1:10;
arr_mat = zeros(length(a),length(b));
for i=1:length(a)
    for j=1:length(b)
        disp(['loop no = ',num2str(i),' & ',num2str(j)]);
        knn = a(j);
        eta = b(i);
        arr_mat(i,j) = called_function(knn,eta);
    end
end

The program runs but is very computationally expensive. I was wondering if there is any matlab inbuilt parallel processing toolbox which will be helpful for me in this case. I was thinking along these lines :
Divide my main program into parts:
    a1= 0.1:0.1:0.5;
    b1= 1:1:5;
    arr_mat1 = zeros(length(a1),length(b1));
    for i1=1:length(a)
        for j1=1:length(b)
            disp(['loop no = ',num2str(i1),' & ',num2str(j1)]);
            knn1 = a1(j);
            eta1 = b1(i);
            arr_mat1(i,j) = called_function(knn1,eta1);
        end
    end

    a2= 0.6:0.1:1;
    b2= 6:1:10;
    arr_mat2 = zeros(length(a2),length(b2));
    for i2=1:length(a2)
        for j2=1:length(b2)
            disp(['loop no = ',num2str(i2),' & ',num2str(j2)]);
            knn2 = a2(j);
            eta2 = b2(i);
            arr_mat2(i,j) = called_function(knn2,eta2);
        end
    end

Run the codes in parallel. My system configuration is : Intel Core i7-3770 @ 3.40 GHz with 32.0 GB RAM. I have MATLAB 2013b installed.
I have consulted this question and have tried to write my own code in that format:
clc;clear all;close all;
% run ixmas for different modifications
a= 0.1:0.1:1;
b= 1:1:10;
arr_mat = zeros(length(a),length(b));

matlabpool open local 2

    parfor i=1:length(a)
        for j=1:length(b)
            disp(['loop no = ',num2str(i),' & ',num2str(j)]);
            knn = a(j);
            eta = b(i);
            recog = 0;
            for k=1:5
                recog(k) = ixmas(knn,eta);
            end
            arr_mat(i,j) = mean(recog);
        end
    end
end

matlabpool close

This code throws up the error : "Illegal use of reserved keyword "end"."
My queries :

Kindly please tell me where I am going wrong.
I tried to introduce a  parfor in the inner loop but it throws up an error?
What does the statement matlabpool open local 2 mean ?
How many threads can I use in parallel for my system configuration ? How can I even check those limits and optimally use them to the full capacity ?
Does parallel processing means utilizing different cores of my own machine or utilizing resources of other machines ? Is there any way either options can be selected and how does one go about doing so?
Comment: You have 3 loop beginnings (one `parfor` and 2 `for`'s) but 4 `end`'s. That's the problem the error is telling you about. Also read [this](http://au.mathworks.com/help/distcomp/introduction-to-parfor.html), you should try to understand what your code is doing.
Comment: Please see the highlighted link , in the given example there is also one extra end
Answer: matlabpool does not open a statement that needs a closing end. In the linked question the end is closing the spmd statement.
The meaning of the statement matlabpool open local 2 is:
matlabpool open or closes a pool of matlab parallel processing worker processes.
open tells matlab to run these workers. (this is the default)
local tells matlab to open these workers on the local machine (this is the default)
2 tells matlab to run 2 such workers (default is number of cpus)
You can just matlabpool without any parameters.
in order to configure the local profile go to the matlab home tab and choose 'manage clusters' under the parallel menu.
Also, for some strange reason matlab can't split the array between the workers when the parfor is not in the inner most loop. change the inner for to parfor and it will work.
Comment: Thanks for your reply. I removed the last "end" as suggested. Now I am getting an error : The variable arr_mat in a parfor cannot be classified. Why is that ? please suggest the fix

Parallel processing queries


It's driving me crazy, 24 errors produced when I run this simple Java program.

Question: public class Planet {
    double x;
    double y;
    double xV;
    double yV;
    double mass;
    double imgName;
    public Planet (double x; double y; double xV; double y; double mass; double imgName;) {
        this.x = x;
        this.y = y;
        this.xV = xV;
        this.yV = yV;
        this.mass = mass;
        this.imgName = imgName;
    }
    public static void main(String[] args) {
        return 0;
    }
}

Planet.java:8: ')' expected
Planet.java:8: illegal start of type
Planet.java:8: ';' expected
Planet.java:9: illegal start of type
Planet.java:9:  expected
Planet.java:9: ';' expected
Planet.java:9: illegal start of type
...
Planet.java:16: class, interface, or enum expected    public static void
  main(String[] args)
Planet.java:18: class, interface, or enum expected }

I see different types of error! Could anyone explain them one by one to me as it is so annoying.
Comment: [Basic official Java method tutorial](https://docs.oracle.com/javase/tutorial/java/javaOO/arguments.html)
Comment: What is the `;` character for?
Comment: In the class constructor signature change the **;** for **,** (parameters are separated by commas, not semicolons). And although it's not forbidden, the idea of the `main` method is to be run from command line and your POJO should not contains this method.
Comment: Thank you. How could it produce so many errors?
Answer: You are not declaring your parameters properly. Use:
public Planet (double x, double y, double xV, double y, double mass, double imgName)

Additionally, main should not return 0. If you want to return a system exit code, use System.exit(0). Simply returning from the main method (using return; or by reaching the end of it) will act analogously to return 0; from main languages like C.
The reason there are so many errors is due to the way the Java parser/compiler works. The first error is already an incorrect state, so the following errors are just artifacts of the compiler being "confused".
A practical rule of thumb for compile/syntax errors is to focus on fixing the first error given, as the parsing of the rest of the file is likely affected by it.
Although I am not 100% certain I believe that the following is occuring:

Semicolon causes parser to emit error of missing closing ), and to end the statement (in fact, ending the constructor then and there).
After the last semicolon, ) is an illegal type, and a semicolon is additionally expected.
Although nearly any human would conclude that this.x = x is part of the constructor, the previous line confused the parser so it thinks it's working at class scope (not in a method). It can't parse this.x = x because this.x isn't a valid type, and this "field" doesn't have a name. At this point, it is clear that the previous errors have messed up the parser state enough to begin panicking for valid statements that it now believes are out of place.
As it reaches the closing } it ends the class scope. The following main method and final closing } are likewise unparseable, and cause errors to be emitted (since these elements are not inside a class at this point)
Comment: Thank you and that's what I really ask for. As I rebuild the code, I see another error produced "y is already defined in Planet"...so why y? not the parameters parallel to it
Comment: @tonyabracadabra If you look in the list of parameters, you have `double y` in there twice. Looking at the rest of your code, this looks like a simple typo, since there isn't a field of `Planet` that got "left out".
Comment: Oh I apologize for that, thanks for your patience toward a newbie. : )
Comment: @tonyabracadabra Not a problem, always glad to help :) As everyone else here, I was a newbie at one point as well.


Navegando entre URLs com Javascript (alterando valor da url)

Question: Boa noite, estou tentando fazer um comando em javascript que redirecione para uma página html e adicione uns caracteres no texto final, exemplo:

index.html

após o comando (pega o valor anterior do link e adiciona +abcd no final):

indexabcd.html

E se fosse uma pagina com outro nome, por exemplo:

joao.html

Com o comando ficaria:

joaoabcd.html

Agradeço a quem ajudar
Answer: Isso aqui deve resolver.

var urlAnterior = document.referrer;
//remove o .html
urlAnterior = urlAnterior.replace('.html','');
var complementoURL = "abcd";

function comando(){
  window.location.href = urlAnterior+complementoURL+".html";
}
<button type="button" onclick="comando()">Comando</button>
Answer: Com uma pequena combinação de substrings você consegue, se todos os links terminarem com a extensão do arquivo, apenas procure pelo ultimo ponto e adicione o texto escolhido atrás

lastIndexOf: Para ver onde está o último ponto.
Substring: Para pegar tudo que está antes do último ponto.
Substr: Para pegar a extensão novamente, poderia ser o Substring de novo, mas é um parâmetro a menos.

Resultado:

function mudarLink (link, texto){
  ultimoPonto = link.lastIndexOf('.');
  return link.substring(0, ultimoPonto) + texto + link.substr(ultimoPonto)
}


A question about Quadratic residue

Question: I need help with this question :
Prove that for each prime number p there exist $a,b \in Z$ such that $-1\equiv a^{2}+b^{2}\pmod p $
When $p\equiv1\pmod4$ it is easy because -1 is a quadratic residue, I got stuck when $p\equiv3\pmod4$
Comment: This is a special case of [an earlier question](http://math.stackexchange.com/q/891861/11619). Probably the argument has presented even earlier, but I couldn't find it easily.
Answer: Hint: suppose for the sake of contradiction that $-1-a^2$ is not a quadratic residue mod $p$, for any $a \in \{0, 1, 2, \ldots, p-1\}$. Then count how many quadratic nonresidues you get as a result. Use the fact that $x^2 \equiv y^2 \mod p$ if and only if $x \equiv \pm y \mod p$.
Answer: If $p\equiv 3\pmod 4$, in the sequence $\{2p-1, 6p-1, 10p-1,\ldots,4np+2p-1,\ldots\}$ there are infinitely many primes (Dirichlet's theorem) and all of them are $\equiv 1\pmod 4$.
Let $q$ be any prime from this sequence. Since $q\equiv 1\pmod 4$, there exists $a$ and $b$ such that $q=a^2+b^2$. Therefore
$$a^2+b^2\equiv q\equiv 4np+2p-1\equiv -1\pmod p$$
Comment: I don't see how this helps. Let $q = 4np+2p-1$ be prime, then there exists an $a$ so that $-1 \equiv a^2 \mod q$, which means $a^2+1 = x(4np+2p-1)$ for some $x \in \mathbb Z$. Then $a^2+1 \equiv -x \mod p$, so... ??? Was this what you were suggesting, or did you have something else in mind?
Comment: Edited. I hope it helps now.

A question about Quadratic residue


Namespaces and Sub-Namespaces and the libraries they reference in C#

Question: Can I have
namespace somenamespace
{
//references functionality in DLL_1
      namespace somesubnamespace
      {
       //references functionality in DLL_2
      }
}

And if I do this, when I use just somenamespace, will it only include DLL_1 in the solution and if I use somenamespace.somesubnamespace will it include DLL_1 and DLL_2 in the solution?
Or are DLL_1 and DLL_2 set on the project that contains the namespace and no matter which I use, both DLLs will be copied?
Comment: It's worth noting that I have never had the need to nest namespaces in this way, nor have I ever seen libraries that do this, including the BCL. Instead, the namespaces are nested via their naming convention, as in `Namespace.SubNamespace.SubSubNamespace`. But maybe I'm missing the point.
Answer: You only need to add references to the DLLs that your code actually uses (directly or indirectly).
Therefore, if library X has two classes that reference library A and B respectively, and you only use the class that references library A, you do not need library B at all.
Namespaces are organizational concepts and have nothing to do with it.
Answer: If you add references to the DLLs, they will be copied. Since you must add the reference in order to use them, both will always be copied.
Comment: Wrong.  You don't need to add a reference to an indirect dependency if you never use it.
Comment: @Slaks - my understanding is that he is asking if using namespaces affects what DLLs are copied to the output directory.
Answer: Your code above is exactly equivalent to:
namespace.somesubnamespace
{

}

If you write it your way, or if you write it this way, accessing somesubnamespace is done exactly the same way in your code.
Namespaces are essentially just labels to organize your code.  Depending on what your code is accessing will dictate what references you need - but the answer to your question is "no", the references for namespace will not necessarily be required.  namespace.somesubnamespace is an independent namespace in that regard.


Decision tree without the "tree"

Question: I would like to construct something like a decision tree. However, instead of using "recursive partitioning" to build a tree, I would like to find an optimal set of "global" splits.
For example, in a normal decision tree, you might have something like this:
1) root
    2) A >= 1
        4) B >= 1
        5) B < 1
    3) A < 1
        6) C >= 1
        7) C < 1

Here, you only use variable "B" when the value of "A" is greater than 1 (but not when "A" is less than 1). However, I want to find a set of "global" variables that are not conditioned on each other. For example, these three splits would divide the feature space into 8 blocks:
1) A >= 1
2) B >= 1
3) C >= 1

In terms of a decision tree, I only want to consider decision trees that look like this (note: each path in the tree uses ALL of the splits defined above):
1) root
    2) A >= 1
        4) B >= 1
            8) C >= 1
            9) C <= 1
        5) B < 1
            10) C >= 1
            11) C <= 1
    3) A < 1
        6) B >= 1
            12) C >= 1
            13) C <= 1
        7) B < 1
            14) C >= 1
            15) C <= 1

I have done a lot of searching and I cannot figure out how to do this. If anyone can point me in the right direction, I would REALLY appreciate it.
I have looked into other approaches, like Naive Bayes and regression. Naive Bayes seems like it might be similar in concept, but it does not partition the feature space into "p-dimensional hyperblocks". I have looked into using lasso regression, but because this is multi-class classification, there is nothing constraining the model to use the same variables across the classes.
Comment: Do you need to determine the order of each decision (in your example: A then B then C) or the optimal threshold for each feature? Or both? Do you need your tree to perform only one test for each feature (i.e. can your tree look like: A $\rightarrow$ B $\rightarrow$ A $\rightarrow$ C $\rightarrow$ B)?
Comment: Also: are you sure you want to build trees with all variables only? Those trees are likely to show a higher variance than smaller trees.
Comment: Decision tree with a single split is called [stump](https://en.wikipedia.org/wiki/Decision_stump). An ensemble of such stumps can be used in random forest.
Comment: @RomainReboulleau I would like to constrain the tree so that it is only k levels deep. I would like to find the best order of features, and their optimal thresholds. I also need to perform only one test per feature.
Comment: @jank I want to use a small subset of the total features. I anticipate I will need to do a greedy search? For example, my total dataset contains 1000 features, but I would like to constrain it to use only 5 features (i.e. splitting the feature space into 32 hyperblocks)
Comment: Just to clarify, I am interested in doing this as a form of feature selection, rather than classification. Furthermore, I am not overly concerned about overfitting, because in this particular context, the results are easy for me to validate.
Comment: @adnbps In the 3rd list, you have overlapping categories with C >= 1 and C <= 1. You might want to change C<= 1 to C < 1.
Answer: I am not aware of this method already existing, but assuming a continuous, or at least ordered response, this sounds like tree boosting with a learning rate of 1, maximum tree depth of 1 and a maximum of 5 iterations. (If you have a response from a different GLM family, the same applies, but obtaining the pseudo-response will involve slightly more computation). The following code does such a thing:
> airq <- airquality[complete.cases(airquality), ]
> library("partykit")
> data <- airq
> splitvars <- c()
> partvars <- names(airq)[-1]
> trees <- list()
> for (i in 1:5) {
+   form <- as.formula(paste0("Ozone ~ ", paste(partvars, collapse = " + ")))
+   ## Find split:
+   tree <- ctree(form, data = data, maxdepth = 1)
+   if (length(tree) < 3) {
+     print(paste("Exiting: Tree of depth 0 grown in iteration", i))
+     break
+   }
+   splitvars <- c(splitvars, names(data)[tree[[1]]$node$split$varid])
+   partvars <- partvars[!partvars %in% splitvars]
+   trees[[i]] <- tree
+   ## Update response (i.e., boosting with learning rate of 1):
+   data$Ozone <- data$Ozone - predict(tree)
+ }
[1] "Exiting: Tree of depth 0 grown in iteration 3"
> ## Inspect results:
> splitvars
[1] "Temp" "Wind"
> trees
[[1]]

Model formula:
Ozone ~ Solar.R + Wind + Temp + Month + Day

Fitted party:
[1] root
|   [2] Temp <= 82: 26.779 (n = 77, err = 42143.2)
|   [3] Temp > 82: 76.794 (n = 34, err = 20659.6)

Number of inner nodes:    1
Number of terminal nodes: 2

[[2]]

Model formula:
Ozone ~ Solar.R + Wind + Month + Day

Fitted party:
[1] root
|   [2] Wind <= 5.7: 32.958 (n = 12, err = 22615.1)
|   [3] Wind > 5.7: -3.995 (n = 99, err = 25572.6)

Number of inner nodes:    1
Number of terminal nodes: 2

> ## Obtain predictions:
> preds <- rowSums(sapply(trees, predict, newdata = airq))
> cor(preds, airq$Ozone)
[1] 0.7848191

Answer: What you are looking for may be called regularized trees. Have a look at this paper: Feature Selection via Regularized Trees. It seems to provide a framework to do exactly what you need.

Decision tree without the "tree"


How to minimize the $2$-norm?

Question: I encountered the following question today:
Given $n$ real numbers $r_1\le ...\le r_n.$ Can we find $k_1,...,k_n \ge 0$ real with $\sum_{i=1}^n k_i=1$ such that
$$(r_1-k_1)^2+...+(r_n-k_n)^2$$
is minimal?
Is there a way to do this analytically or are there good optimization algorithms that can do this? (The problem has apparently a unique solution (compactness) and a unique-solution by convexity of the euclidean norm).
Answer: Use Lagrange multipliers:
$$F(\mathbf{k},\lambda) = \|\mathbf{r}-\mathbf{k}\|_2^2 - \lambda(\sum_{i}k_i-1).$$
Then
\begin{align}
\frac{\partial}{\partial k_i}F(\mathbf{k},\lambda) = & 2(k_i - r_i) - \lambda,\\
\frac{\partial}{\partial \lambda}F(\mathbf{k},\lambda) = & \sum_{i}k_i - 1.
\end{align}
The first equation is zero if, and only if
$$k_i = r_i + \frac{\lambda}{2}.$$
Then the other condition gives
$$1 = \sum_{i}k_i = \sum_{i}r_i + \frac{n}{2}\lambda,$$
so that $\lambda = \frac{2\left(1-\sum_{i}r_i\right)}{n}$. Therefore,
$$\mathbf{k} = \mathbf{r} + \frac{2\left(1-\sum_{i}r_i\right)}{n}\pmatrix{1\\\vdots\\1}.$$
This finds the solutions in the interior of the region where $\mathbf{k}$ is allowed to live. If you don't find any solution in the interior, then re-do all the process above on the boundary (where one or more $k_i$'s are zero).
Comment: you forgot the condition that $k_1,..,k_n \ge 0.$ Do you think you can build this into your neat approach?
Comment: @Optimization2.0 Added a paragraph explaining how to go on. I'll let you do the work, though.
Comment: I still think that there is something wrong in what you say. Imagine $\sum r_i =1$ then $k=r$ according to your algorithm. Now there is no need that all entries in $r$ are positive. Therefore, $k$ is not in the interior of the region where $k$ is allowed to live.
Comment: @Optimization2.0 Then, as I have written, there is no solution in the interior of the region, and you have to look for one constrained in the boundary.
Comment: For negative $r_i$, you simply set $k_i = 0$ and consider the problem where you leave out this term.
Answer: You are projecting the point $(r_1,\ldots,r_n)$ onto the probability simplex. This projection is needed often as a substep in convex optimization algorithms, and the solution can be found for example in this paper:
"Projection onto the probability simplex:
An efficient algorithm with a simple proof, and an application"
by Wang and Carreira-Perpinan.
This paper provides the following Matlab code which projects each row vector in the $N × D$ matrix $Y$ onto the probability simplex in $D$ dimensions.
<code>function X = SimplexProj(Y)
    [N,D] = size(Y);
    X = sort(Y,2,'descend');
    Xtmp = (cumsum(X,2)-1)*diag(sparse(1./(1:D)));
    X = max(bsxfun(@minus,Y,Xtmp(sub2ind([N,D],(1:N)',sum(X>Xtmp,2)))),0);
</code>
See also slide 8-12 of Vandenberghe's 236c notes:
http://www.seas.ucla.edu/~vandenbe/236C/lectures/proxop.pdf

How to minimize the $2$-norm?


prevent explorer.exe from launching internet explorer

Question: In a Windows Terminal Server (2003 R2) where the users are not allowed to run a remote desktop, but just the applications they want (that's Citrix work), i've published "Explorer.exe" for the users to get their files.
How can i prevent Explorer.exe to launch Internet Explorer when a user writes an url in the address bar instead of a local file path?
Answer: explorer.exe is silently calling iexplore.exe in the background for this. Don't allow these users to launch iexplore.exe through Software Restrictions (or any other way really) and it should do the trick.


why am I not able to convert string type column to date format in pyspark?

Question: I have a column which is in the "20130623" format. I am trying to convert it into dd-mm-YYYY. I have seen various post online including here. But I only got one solution as below
from datetime import datetime
df = df2.withColumn("col_name", datetime.utcfromtimestamp(int("col_name")).strftime('%d-%m-%y'))

However, it throws an error that the input should be int type, not the string type. I tried to convert with the help of int() function. But even that doesn't seem to be helping.
below is the error that I see when converting
invalid literal for int() with base 10: 'col_name'

I am not sure if it is taking the col_name as string, or its value as string.
Please suggest, how can I do this, or the best way to get the required output
Note: I cannot use pandas in my environment.
thank you.
Comment: `df.withColumn("date", to_date("date_str", format="yyyyMMdd"))` where `from pyspark.sql.functions import to_date`
Comment: `withColumn` should strictly use Spark types. You can not use python datetime there.
Comment: Please see example on https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.date_format.html
Comment: Add one more line `df = df.withColumn("date_ddMMyyyy", F.date_format("date", format="dd-MM-yyyy"))`
Comment: ok, Thank you for yur help, it is working. But as I said, the final output should be dd-mm-yyyy. "to_date()" doesn't seem to do that. I undertand that the string which I gave here is having "YYYY" first. But now, since to_date has converted to date format, is there any way to get "dd-mm-yyyy"?
Answer: withColumn() should use Spark types. You can not use python datetime there.
First parse string to date using to_date and then format date to string using date_format:
import pyspark.sql.functions as F

df = spark.createDataFrame(data=[["20130623"]], schema=["date_str"])

df = df.withColumn("date", F.to_date("date_str", format="yyyyMMdd"))
df = df.withColumn("date_ddMMyyyy", F.date_format("date", format="dd-MM-yyyy"))

df.show(truncate=False)
df.printSchema()

Output:
+--------+----------+-------------+
|date_str|date      |date_ddMMyyyy|
+--------+----------+-------------+
|20130623|2013-06-23|23-06-2013   |
+--------+----------+-------------+

root
 |-- date_str: string (nullable = true)
 |-- date: date (nullable = true)
 |-- date_ddMMyyyy: string (nullable = true)

Whatever you see as output 2013-06-23 in date column is actually default to_string() representation of date.


Two questions about explanatory variables and transformations

Question: I'm using Excel and R to do this. I'm new to R.
I have a dataset of energy savings arising from capital investments supported by government grants. I've done a multiple linear regression of energy savings against grant size, total investment cost, and several dummies about investment type. The data is far from normal and looks like it needs transformation.
I did a Box-Cox test on y (ie energy savings), resulting in lambda of -0.14, so I transformed y using a natural log. Then, thinking the explanatory variables also needed transforming, I followed this (where it speaks about transformations of predictors), and the lambdas of cost and grant are 2.25 and 1.75 respectively (both have significant p-values), suggesting a X^2 transformation. Additionally (ie as an alternative), I did log transformations on both grant and cost as I had a hunch that would normalise them.
My questions are as follows.
Question 1: Using transformed explanatory variables
So, looking solely at the adjusted R^2 and not removing any non-significant predictors, the adj-R^2 values are as follows.

model 1 (no transformation): 0.5596
model 2 (y = ln(y)): 0.6070
model 3 (y=ln(y), x=x^2 where not dummy): 0.5478
model 4 (y=ln(y), x=ln(x) where not dummy): 0.6809

I think I should use the ln transformation (ie model 4), but the box.tidwell lambdas recommended a x^2 transformation.
Am I justified using a ln transformation on the explanatory variables despite what the box.tidwell test says? Why? This page suggests I can compare models in R with anova(fit1, fit2), but I don't really know how to evaluate the output (the smaller RSS is better?).

Question 2: Addressing multicollinearity
While doing all this, I realised I had forgotten about multicollinearity. Grant size and investment cost are highly correlated. I want to use the model to estimate energy savings for various grant sizes, so I think it's okay that they are correlated. But I was wondering, can I transform the grant column to be something like 'percentage granted'? I suspect that would remove the correlation, but might do something weird because it would be a constrained value (ie always < 1). Or maybe it could make it a dummy or something?
How can I deal with the multicollinearity of grant size and project cost? Is it just a matter of trying different transformations and seeing how they affect the fit?

Thanks so much.
Comment: The main reason for transforming here should be that way you get closer to linearity. Normality of any marginal distribution is not a problem if you get it, but it is not an assumption for regression. Every good text on regression explains this point. That aside, you are asking for advice on how to model the data, but you never show the data either in listings or in graphs.
Comment: (1) Adjusted $R^2$ may be useful for comparing transformations of predictors--at least if you adjust it for the additional parameters implied by the freedom to transform the predictors--but it is meaningless for comparing transformations of the responses. (2) Don't use Box-Cox parameters like $2.25$ or $1.75$. Indeed, be very suspicious of any estimate of $\lambda$ that lies beyond the interval $[-1,1]$ unless theory suggests it should. (3) Maximum likelihood makes strong assumptions; its Box-Cox estimates can be poor. (4) For a robust approach, see https://stats.stackexchange.com/a/35717/919.
Answer: For question 1, I would suggest not transforming variables based on statistical considerations but on substantive ones. Presumably, energy savings is in dollar terms (or some other currency). It often makes sense to take the log of this sort of variable since we often think of money in multiplicative terms rather than additive ones. For instance, the difference between a \$10,000 house and a \$12,000 one is large; the difference between a \$1,000,000 house and a \$1,002,000 house is rounding error.  Whether this is the case in your situation isn't completely clear to me, but it seems possible.
If you decide to not transform, based on the above, note that OLS regression does not assume that the variables are normally distributed; it assumes that the errors (estimated by the residuals) are normally distributed.  If you find that this assumption is violated, then you can use methods that don't assume linearity of the residuals (e.g. robust regression, quantile regression).
Comment: Peter: Linearity of the residuals is just a typo here for normality of the errors, but not quite trivial errors should be corrected by the OP!


ASP.NET Core 7 Web API - how to serialize in Pascal case?

Question: In Visual Studio 2022, and .NET 7, I created a new project with the template ASP.NET Core Web API. The template creates the WeatherForeCastController, which uses the WeatherForecast model. The properties of the model are in PascalCase:
public class WeatherForecast
{
    public DateOnly Date { get; set; }
    public int TemperatureC { get; set; }
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
    public string? Summary { get; set; }
}

When I execute the API from Swagger, I get the properties in camelCase:
[
  {
    "date": "2023-11-28",
    "temperatureC": 53,
    "temperatureF": 127,
    "summary": "Hot"
  },
...
]

But I want to get them as they are defined in the .NET class. I added the following two statements in Program.cs right after
builder.Services.AddSwaggerGen();

builder.Services.ConfigureHttpJsonOptions(
    options => options.SerializerOptions.PropertyNamingPolicy = null);
builder.Services.Configure<Microsoft.AspNetCore.Http.Json.JsonOptions>(
    options => options.SerializerOptions.PropertyNamingPolicy = null);

But I still get the JSON properties in camelCase.
How can I get the JSON properties in exactly the same case as defined in the .NET class, in this case Pascal case?
Answer: Try to add it to AddControllers
builder.Services.AddControllers()
.AddJsonOptions(x =>
{
    x.JsonSerializerOptions.PropertyNamingPolicy = null;
});

Answer: You can use the JsonPropertyName attribute to specify the exact JSON property names.
public class WeatherForecast
{
    [JsonPropertyName("Date")]
    public DateOnly Date { get; set; }

    [JsonPropertyName("TemperatureC")]
    public int TemperatureC { get; set; }

    [JsonPropertyName("TemperatureF")]
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);

    [JsonPropertyName("Summary")]
    public string? Summary { get; set; }
}

Comment: Hey `Alex`, you can use `Shahar Shokrani` solution also.


Interpretation of an output from a SURVIVAL ANALYSIS - COX

Question: I am struggling to understand the output of my survival analysis. These are the following codes and outputs:
summary(coxph(Surv(A_diffdate, Arrythmia) ~AST , data = df))

plot(res.cut, "AST", palette = "npg") #  cut-off = 25.4

res.cat <- surv_categorize(res.cut)
head(res.cat)

df$AST_cat = ifelse(df$AST < 25.4,'Healthy','High_AST')
fit <- survfit(Surv(A_diffdate, Arrythmia) ~ AST_cat , data = df)
ggsurvplot(fit, data = df, risk.table = TRUE, conf.int = TRUE, pval = TRUE)

res.cox <- survfit(coxph(Surv(A_diffdate,Arrythmia) ~ 1, data = df))
ggsurvplot(res.cox, data = df, title ="AST (Arrythmia)",conf.int = FALSE,risk.table = "absolute")

res.info<- coxph(Surv(A_diffdate,Arrythmia) ~ AST + sex + age + bmi, data = df)
summary(res.info)

It would be very very helpful to get a comprenhensive interpretation of these results, that I created based on this web:
https://rpkgs.datanovia.com/survminer/reference/surv_cutpoint.html
Thank you so much in advance!
Best
Answer: There are several ways in which you could improve your modeling of these data. Much of what's below is based on what you can learn from Frank Harrell's course notes or book in terms of multiple-regression modeling in general (especially Chapter 4) and Cox survival models in particular (Chapters 20 and 21).
First, do not categorize a continuous predictor based on its association with outcome. See this thread for why categorization is seldom a good idea. Although your p-value of 0.0094 after dichotomizing AST looks very "significant" (graph of survival curves based on the cutoff of 25.4), that p-value is uninterpretable as it doesn't take into account the fact that you used the data to pick the cutoff.*
If you choose a cutoff based on an association with outcome in one particular data set, that "optimal" cutoff will probably not apply to a new data set. Thus your results will not generalize reliably to new data sets, which is typically what one wants to accomplish. You could demonstrate this for yourself: try the automated cutoff choice on multiple bootstrapped samples of your data and see how much the "optimal" cutoffs vary.
Second, a flexible spline model is often the best way to incorporate a continuous predictor into a model when there is no theoretical basis for some fixed functional form. The pspline() function in the R survival package and the rcs() function in the rms package provide different ways to do that.
Such modeling lets the data tell you the functional form of how a continuous predictor is associated with outcome. For example, none of the models using AST as a continuous predictor show it to be significantly associated with outcome, but it's treated in those models as linearly associated with log-hazard. If it has a true association with outcome that is more complicated, you could miss it unless you go beyond the linear fit. A spline fit can demonstrate the actual form of its relationship to outcome.
Third, you seem to be starting with a bottom-up approach to modeling, starting with a single predictor of interest (AST) and then adding other predictors (age, sex, BMI). Regression model building better starts with an overview of the data and all the predictors, more of a top-down approach. With survival modeling, look at the number of events as a guide to how complex a model you can build. Then use your knowledge of the subject matter and things like associations among the variables (without considering their associations with outcome in your data) to come up with a set of predictors of a scale consonant with the scale of your data. For survival modeling without overfitting, that's usually about 1 predictor (including interaction terms, extra coefficients for spline fits, etc) per 15 events.
With 425 events in your data set, you might be able to develop a much more complex model. You could use flexible continuous fits not just for AST but also for age and BMI. You could include interactions of those continuous predictors with sex, interactions among the continuous predictors, and additional predictors associated with outcome.
Including as many predictors associated with outcome as possible, without overfitting, is generally a good strategy particularly in survival modeling. In survival modeling, as with logistic regression, omitting any predictor associated with outcome runs a risk of biasing the coefficients of included predictors toward lower than their true magnitudes.
Finally, it's important to document the discrimination and calibration of your model. The concordance index is one measure of discrimination between case outcomes; it's the fraction of pairs of comparable cases in which the model-predicted and observed order of events agrees. (As noted in another answer, that's not very good for your models thus far, as 0.5 is what you get just by chance.) Calibration shows how well things like predicted and observed event probabilities agree over the range of the data. The Harrell reference and the rms package provide tools for evaluating calibration and additional measures of discrimination.

*Given the strong association of age with outcome in the final model, I'm particularly concerned that your 2 AST groups simply differ in average age, so that your AST categories are just a proxy for age.
Answer: First of all, you should know that the Cox model fits a risk function h (X, t), that is, it tries to predict the risk h based on the covariates X at time t.
To evaluate the predictive capacity of the model, the concordance index is used. Your concordance index is close to 0.5 which is pretty bad (The concordance index must be at least 0.6).
The Pseudo R squared is also used and it's not shown in the summary, which indicates that it does not make sense to show it, so I assume that it must be very bad too (For Cox models it is enough that the R ^ 2 is greater at 0.3, yours should be below 0.1, if I'm not mistaken). You can see the Pseudo R squared with summary (…) $ rsq.
More important than the above, you must test the proportional hazard hypothesis. You can do this using the cox.zph () function. The P-values of all variables must be greater than 0.5 for the model to be valid.
Sorry for the bad news.
Comment: I have substituted AST for another blood biomarker that has rsq= 0.01023002   maxrsq = 0.82335777 (which one of these is valid?). p values from the cox.zph are ALT (the new marker)= 0.13, sex =0.57, age =0.44, bmi=0.72, GLOBAL =0.54. This looks good does it? how do you interpret the first density plot as well, I would be interested. thanks
Comment: The p-values from cox.zph are good. The density plot is a usual density plot camparing two populations. The **log-rank test** compares the risk functions of two populations and since the p-value of the test is less than 0.5 (0.0094), this means that there is no evidence that tells us that the risk functions of the two populations are equal. So it's a good partition. You should use the variable ` ATS_cat `  in your model.
Comment: This is so helpful thank you! I have three more questions: (1) what does it mean the graph with the dots&cutpoint below the density? (2) what is the difference between the first plot with the two lines and the second with just one line? does this means that the first is univariate and the second is multivariate? (3) regarding the r square in my previous comment rsq= 0.01023002 maxrsq = 0.82335777 which one of these is valid? - Thank you so much!!
Comment: You are welcome. In response to your questions (1) Every point in that graph is a statistic resulting from one Stadarized Log_Rank test. We select the largest to determine the optimal cut-off point, in your case 25.4.  (2) The first plot compares the Kaplan -Meier Stadistic of two population, which includes a Log-rank test to compare the hazard functions. The second one is just a plot of the Kaplan-Meier stadistic of all population. (3) The first one is your R-square and the second one is the max R-square posible.
Comment: Thank you! understood!


Trouble with Javascript object properties - Eloquent Javascript example

Question: var thing = {"gabba gabba": "hey", "5": 10};
show(thing["5"]);
thing["5"] = 20;
show(thing[2 + 3]); // = 20
show(thing[2]); // = undefined
show(thing[3]); // = undefined

I do not understand why show(thing[2+3]) = 20 and the other two below it return undefined. What do the 2 and 3 refer to? Could someone help me out with this? Thanks.
Answer:
I do not understand why show(thing[2+3]) = 20 and the other two below it return undefined.

thing[2+3] evaluates to thing[5] (==20), but properties with the names "2" or "3" don't exist on your object - and therefore yield undefined when being accessed.

What does the 2 and 3 refer to?

They're plain number literals.
Answer: The point is that thig is not an Array, it is an Object (key : value pairs).
If you like to get "hey" youd need to do:
show(thing["gabba gabba"]); // = "hey"

And because there is an entry with key "5", 2 + 5 works:
show(thing["5"]);   // = 10
show(thing[2 + 3]); // = 10

Answer: The expression thing[2] means to find a property on the object named thing with the name "2".  Since property names are strings, the number 2 is automatically converted to the string "2".  Since that object has no such property, the result is undefined.
The expression thing[2+3] means to first evaluate 2+3 which is 5.  Then 5 is converted from a number to a string "5", and then find the property on thing with that name.  Thus the result is 20.
Comment: Thanks dsh, you answered it perfectly. I'd upvote but I don't have the reputation yet.
Answer: You have given the 5th term of "thing" the value of 20
thing["5"] = 20;

Javascript is clever enough to understand that when you type
thing[2 + 3]

What you really want is
thing[5]

What you're not doing is
thing[3] + thing [2]

However, thing[2] and thing[3] have no value, hence, they return as undefined. You'd need to give them a value, like you did before when you gave thing[5] a value of 20.
Answer: That's the way JavaScript Objects work. You can consider them as a kind of HashMap with keys and values.
In your example you've got a HasMap object called thing which has only 2 pair of key-values. "gabba gabba" is the key for the first one with hey as its value, the second pair has "5" as its key and 10 is the value. To make it more comprehensible we have a function called Object.key() in JavaScript which gives you the collection of keys you have in your object:
Object.keys(thing);

which its result is:
["5", "gabba gabba"]

as you see there is no 2 or 3 in the keys, so if you want them to be there you should add a pair with these keys and your desired values:
thing[2] = "whatever";
thing[3] = "whatever";

then the key collection would be:
["5", "gabba gabba", "2", "3"]


Does the association $V \mapsto GL(V)$ define a functor?

Question: As is stated in the title:

Question: for $k$ a field, does there exist a functor $F: k$-$\mathrm{v.s.} \to \mathrm{Grp}$ which on objects is $V \mapsto GL(V)$?

My guess is no, because it doesn't look to me like there's a sensible way to map the morphisms. However, I can't come up with a counterexample in a similar way to Arturo Magidin in Why is there no functor $\mathsf{Group}\to\mathsf{AbGroup}$ sending groups to their centers?.
Here is my attempt at imitating his answer: take $f: k \to k^2$, $f = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$, $g: k^2 \to k$, $g = \begin{pmatrix} 1 & 0 \end{pmatrix}$. Then $g \circ f = 1_k$, so $F(f):k^{\times} \to GL_2(k)$ is injective and $F(g): GL_2(k) \to k^\times$ is surjective. On the other hand, $f \circ g = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}$, but this doesn't really give me anything.
Comment: Generally taking automorphisms is only "naturally" functorial with respect to isomorphisms. It would be interesting to rule out "non-natural" functors also, I guess.
Comment: That doesn't produce another map from $W$ to $W$; you need to also precompose with the inverse of $f$ which need not exist.
Comment: Yep. The only thing I could think of for morphisms would be $f: V \to W \mapsto f_*:GL(V) \to GL(W)$, $f_*(A) = f \circ A$, but obviously $f$ can be anything and so $f \circ A$ isn't necessarily an isomorphism. Is this what you meant by your remark?
Comment: True! Even more reason to believe such a functor doesn't exist.
Comment: The "natural" functor with respect to isomorphisms that Qiaochu alludes to sends an isomorphism $f:V\to W$ to the conjugation by $f$ map $GL(V)\to GL(W)$.  Any "natural" functor with respect to all morphisms ought to extend this, i.e. be given by conjugation on isomorphisms.
Answer: No such functor exists in general.  Note that if $n\geq m$, then $k^m$ is a retract of $k^n$, so if such a functor existed, then $GL_m(k)$ would have to be a retract of $GL_n(k)$.  I suspect this is basically never true for $n>m>1$, though I don't know a proof for general $k$.  In particular, when $k=\mathbb{F}_2$, the group $GL_n(\mathbb{F}_2)$ is simple for all $n>2$, so it has no nontrivial retracts, so such a functor definitely cannot exist for $k=\mathbb{F}_2$.
Comment: This should at least work when $k$ is finite and I think it always works: $GL_n(k)$ has a Jordan-Holder decomposition in which the only nonabelian simple group is $PSL_n(k)$ (except for a small list of exceptions where this group isn't simple), and (I think) these are nonisomorphic for $n \neq m$.
Answer: (Edit 3: The first argument I found is long but there's a shorter argument below it.)
Let's complete Eric's nice argument. First I'll say the first thing he said in a little more detail, which is also what Arturo used. In any category, a retract of an object $c$ is an object $d$ together with two morphisms $f : c \to d, g : d \to c$ such that $fg = \text{id}_d$. This means $f$ is a split epimorphism and $g$ is a split monomorphism. Retracts are the "particularly nice subobjects" (which are also simultaneously quotients); e.g. in an additive category they are precisely the direct summands. The significance of retracts for our purposes is that

retracts are preserved by arbitrary functors, and
in $\text{Vect}$, $k^n$ is a retract of $k^m$ iff $n \le m$.

So if $V \mapsto GL(V)$ were a functor, this would imply in particular that $GL_n(k)$ is a retract of $GL_m(k)$ whenever $n \le m$. (Incidentally, for groups retracts correspond exactly to semidirect product decompositions.) So, suppose $f : GL_m(k) \to GL_n(k)$ is an epi and $3 \le n \le m$. It's known that when $m \ge 3$ the subgroup $SL_m(k)$ is perfect, so its image in $GL_n(k)$ must also be perfect and hence must land in $SL_n(k)$. So $f$ induces an epi $SL_m(k) \to SL_n(k)$. An epi maps centers to centers, so $f$ induces an epi $PSL_m(k) \to PSL_n(k)$.
It's also known that when $m \ge 3$, $PSL_m(k)$ is simple. So $f$ is an epi between two nontrivial simple groups, which therefore must be an isomorphism. In other words, we've proven:

Claim: if $V \mapsto GL(V)$ is a functor, then for $3 \le n \le m$, the groups $PSL_n(k)$ and $PSL_m(k)$ are isomorphic.

So it suffices to disprove this.

Edit #2: Okay, I think I can actually complete the argument now. The following weaker result suffices:

Proposition: There exist positive integers $3 \le n < m$ such that $PSL_n(k) \not\simeq PSL_m(k)$.

Proof. We'll proceed by finding $3 \le n < m$ and a nonabelian finite simple group $G$ that embeds into $PSL_m(k)$ but not $PSL_n(k)$. By simplicity, if such a group $G$ embeds into $GL_m(k)$ then any such embedding has image in $SL_m(k)$, and by simplicity again it also embeds into $PSL_m(k)$. Hence if we take $m$ to be the smallest dimension of an irreducible representation of $G$ (which is necessarily faithful) then $G$ embeds into $PSL_m(k)$.
On the other hand, $PSL_n(k)$ acts by conjugation on $\mathfrak{sl}_n(k)$ (the vector space of traceless $n \times n$ matrices), which has dimension $n^2 - 1$ over $k$, so if $G$ embeds into $PSL_n(k)$ then it embeds into $GL_{n^2 - 1}(k)$. Hence if we take $n$ to be such that $n^2 - 1$ is less than the smallest dimension of an irreducible representation of $G$ over $k$, then $G$ does not embed into $GL_{n^2-1}(k)$ and so does not embed into $PSL_n(k)$.
Now it suffices to find a nonabelian finite simple group $G$ such that the smallest dimension of an irreducible representation of $G$ over $k$ is at least $9$ (which means we can take $n = 3$ and $m$ the dimension of this irrep). At this point we'll want to say something about the extent to which this problem is independent of the choice of $k$.
Case 1: $\text{char}(k) = 0$. Any embedding $\rho : G \to GL_d(k)$ of a finite group $G$ into $GL_d(k)$ has image lying in $GL_d(R)$ where $R$ is the $\mathbb{Q}$-subalgebra of $k$ generated by the matrix entries of $\rho(g), g \in G$; since $G$ is finite $R$ is finitely generated, and then by the Nullstellensatz $R$ has a maximal ideal $M$ such that $R/M$ is a number field (a finite extension of $\mathbb{Q}$). By simplicity $G$ also embeds into $GL_d(R/M)$. In other words, the smallest dimension of an irreducible representation of $G$ over an arbitrary field $k$ of characteristic zero is bounded from below by the smallest dimension of an irreducible representation of $G$ over a number field, which is in turn bounded from below by (and is in fact equal to) the smallest dimension of an irreducible representation of $G$ over $\mathbb{C}$. So the characteristic zero case reduces to the case $k = \mathbb{C}$.
At this point we can appeal to standard results; for example it's known that for $n \ge 7$ the alternating group $A_n$ has no nontrivial representations over $\mathbb{C}$ of dimension $\le n-2$. I don't know how to prove this off the top of my head but I think it shouldn't be too hard to at least get a bound of $\le \frac{n-2}{2}$ using standard facts about the representation theory of $S_n$, which would suffice for our purposes.
Explicitly, it follows that $A_{10}$ doesn't embed into $GL_8(\mathbb{C})$ and hence doesn't embed into $PSL_3(k)$, and since it does embed into $PSL_9(k)$ it follows that $PSL_3(k) \not\simeq PSL_9(k)$.
Case 2: $\text{char}(k) = p$ is positive. Repeating the same construction as above, any embedding $\rho : G \to GL_d(k)$ has image lying in $GL_d(R)$ where $R$ is now the $\mathbb{F}_p$-subalgebra of $k$ generated by the matrix entries of $\rho(g), g \in G$. The Nullstellensatz gives that $R$ has a maximal ideal $M$ such that $R/M$ is a finite field $\mathbb{F}_q$ for some $q = p^i$, and we get an embedding $G \to GL_d(\mathbb{F}_q)$. So the characteristic $p$ case reduces to the case $k = \overline{\mathbb{F}_p}$.
In this case we can argue as follows. $GL_d(\mathbb{F}_q)$ has a Sylow $p$-subgroup given by the unipotent subgroup $U_d(\mathbb{F}_q)$ of upper triangular matrices with diagonal entries $1$. This group is nilpotent of class $d-1$, hence all of its subgroups have nilpotency class $\le d-1$. So if $G$ has a Sylow $p$-subgroup of nilpotency class $\ge d$ then it can't embed into $GL_d(\mathbb{F}_q)$. Now it suffices to take $G = PSL_{d+1}(\mathbb{F}_p)$, whose Sylow $p$-subgroup is $U_{d+1}(\mathbb{F}_p)$ (since it lies in the kernel of the determinant map and also has trivial intersection with the center of $SL_{d+1}(\mathbb{F}_p)$).
Explicitly, it follows that $PSL_9(\mathbb{F}_p)$ doesn't embed into $GL_8(\overline{\mathbb{F}_p})$ and hence doesn't embed into $PSL_3(k)$, and since it does embed into $PSL_9(k)$ it follows that $PSL_3(k) \not\simeq PSL_9(k)$.  $\Box$
(There's probably a cleaner way to do this...)

Edit 3: There is! We'll show that no functor $F : \text{Vect} \to \text{Grp}$ sending $V$ to $GL(V)$ can exist by using functoriality a bit more.
First, writing $0$ for the zero-dimensional vector space, we have $GL(0) = 1$ (the trivial group). It follows that the zero linear transformation $0 : V \to W$ between any two vector spaces, which factors through $0$, is trivial, since the induced map $F(V) \to F(0) \to F(W)$ factors through the trivial group.
Now consider a short exact sequence $0 \to k^n \to k^{n+m} \to k^m \to 0$ (for example, given by the inclusion of the first $n$ coordinates, and the projection to the last $m$ coordinates). Applying $F$ produces a sequence
$$GL_n(k) \xrightarrow{f} GL_{n+m}(k) \xrightarrow{g} GL_m(k)$$
where $f$ is a split monomorphism, hence in particular injective, and $g$ is a split epimorphism, hence in particular surjective, and $gf$ is trivial. For $n, m \ge 3$ the same conditions hold after passing to commutator subgroups, so we get a sequence
$$SL_n(k) \xrightarrow{f} SL_{n+m}(k) \xrightarrow{g} SL_m(k)$$
where, again, $f$ is injective and $g$ is surjective and $gf$ is trivial. $\text{ker}(g)$ must therefore be a normal subgroup $N$ of $SL_{n+m}(k)$ such that both $N$ and the quotient by $N$ are nonabelian. But by the simplicity of $PSL_{n+m}(k)$ (here $n, m \ge 3$ so $n+m \ge 6$), the only proper normal subgroups of $SL_{n+m}(k)$ are subgroups of its center, which are abelian; contradiction. So no such sequence exists.
Comment: Sorry for the long wait, I finally had the time to go through the answer and accept it. Thanks so much, it was a great opportunity and pleasure for me to learn some group theory while trying to understand this!
Comment: Hi, thanks for the amazing answer! One question about the $k$ infinite case - how do you define $SL_m(k)$ and $PSL_m(k)$ for $m$ infinite and are they also perfect/simple in that case?
Comment: @Matija: ah, you're right, that's a big gap. Of course we can't define the determinant anymore. It should still be possible to write down a simple group related to $GL_n$ but I don't know what it is anymore off the top of my head...
Comment: @Matija: okay, so the "obvious" conjecture is that if $V$ is a countable-dimensional vector space then the quotient $GL(V) / GL_{\text{fin}}(V)$ of $GL(V)$ by the subgroup of automorphisms fixing a subspace of finite codimension is simple. The analogous result for permutations of a countable set is known: https://mathoverflow.net/a/12307/290 But I haven't found a reference stating or proving the linear version.
Comment: Ah, sorry, we need to further quotient by the center. So $PGL(V) / PGL_{\text{fin}}(V)$ or something like that.
Comment: @Matija: okay, I fixed the argument, we don't need to consider the infinite-dimensional case anymore.
Comment: @Matija: you're very welcome, and thanks for asking the nice question!


1/4 inch TRS male plug spacing difference

Question: The TRS length varies in the female receptacles for male plugs 1/4-inch TRS (stereo) cables. I've seen the problem with Behringer and Donner headphone amps and Hosa 1/4-inch TS female to TRS male adapters.
You get only one channel unless you back the male plug out partially.
How do you get TRS male plugs that fit the receptacle yet match the TRS bands?
Comment: Good jack sockets have a stop at the back so you have a defined point they go to. Cheap ones will let you push a long plug too far. I've no idea why they can't get manufacturing tolerances sorted for something as big as a jack plug. It's not rocket science. For cheap sockets you have to rely on the indent, assuming the spring is tight enough to tell when it's centred.
Comment: Brand recommendation - **Neutrik**. Buy cheap, buy twice. Neutriks will outlast all of us. https://www.neutrik.co.uk
Comment: Are you sure it's not that the plug on the headphones is too long? I haven't run into this problem, although I don't have the exact items you mention
Comment: Purchase a good quality TRS jack, solder to cable end, job done! No brand recommendations, sorry! Never had problems with Behringer headphone amp sockets - I use two of these amps.
Answer: I've been observing that variations in tolerance is a bigger and bigger problem, with many plugs being made apparently by cheap manufacturers, even for equipments of supposedly good brands.
Just the other day I had to sandpaper the tip of jack as it refused to enter into the plug (it was ok after this treatment :-).
Regarding the actual question, apply any type of material that you find suitable around the sleeve part of the connector, near the base, so that it makes a cushion with the required size to stop the jack in the proper position. Wire, cotton string, duct date may all be a make do solution, however a wire will probably be best as you can adjust the thickness of the cushion by the number of turns and it will not deform (so much) with use.
Another alternative is to find a steel washer size that fits into the jack and does not protrudes from the handling part. With a bit of luck one or two washers (depending on thickness) glued in position will do the trick.
If you're a handy man you can find a bit of material (e.g. plastic, or even wood) and make your own washer with the exact size specification to fit the jack and make it stop in the precise required position.
Answer: Cut a washer out of something non-conductive (actually, wouldn't matter if it WAS conductive I guess) and slip it over the shaft of the plug.
Comment: Tidiest may be a short bit cut from a plasic tube.


How to add '.option("overwriteSchema", "true")' to saveAsTable() in SparkR

Question: How can I save an R dataframe with SparkR::saveAsTable() again under the same name as an already existing table after changing columns?
I am working with R on databricks and saved an R dataframe table_x as table in the database, using this:
data_x <- SparkR::createDataFrame(table_x)
SparkR::saveAsTable(data_x, tableName="table_x", mode = "overwrite")

Later I added columns to the table and changed some column names as well. When I try to save it again, it does not work and the error message says there is a schema mismatch, even if I removed table_x from the database with "drop table".
If I try
data_x <- SparkR::createDataFrame(table_x)
SparkR::saveAsTable(data_x, tableName="table_x", mode = "error")

the error message says: "The associated location (...) is not empty and also not a Delta table."
So even if I dropped table_x, its location is not empty and its table schema is still there?
Below the error message it says:
To overwrite your schema or change partitioning, please set:
'.option("overwriteSchema", "true")'.

How can I do this in SparkR::saveAsTable? The RDocumentation site says that additional options can be passed to the method, but how exactly would I do this?
Answer: Very simple, you just add this argument overwriteSchema = "true" to the function SparkR::saveAsTable. Check out the documentation for ... argument of SparkR::saveAsTable function, which says "additional option(s) passed to the method".
If you are not aware of how ... argument works, check Hadley Wickham's book on Advanced R, https://adv-r.hadley.nz/functions.html#fun-dot-dot-dot
The observation here is that, by default, all the Spark dataframes are saved in Delta format instead of Parquet, and Delta format enforces schema consistency as one of many benefits.
Worked out example:
data_x <- SparkR::createDataFrame(iris)
SparkR::saveAsTable(data_x, tableName="table_x", mode = "overwrite")

# modify dataframe
data_x_mod <- data_x |>
  SparkR::withColumn("col_x", SparkR::lit(1)) |>
  SparkR::withColumn("Sepal_Length", data_x$Sepal_Length * 2)

# try to overwrite at same location - results in ERROR
SparkR::saveAsTable(data_x_mod, tableName="table_x", mode = "overwrite")
# Error in handleErrors(returnStatus, conn) :
#  org.apache.spark.sql.AnalysisException: A schema mismatch detected when writing to the Delta table ...

# Add mergeSchema or overwriteSchema option as needed - executes WITHOUT ERROR
SparkR::saveAsTable(data_x_mod, tableName="table_x", mode = "overwrite", overwriteSchema = "true")


How do i show that $w=z^2$ maps $|z-1|\leq 1$ onto $R=2(1+\cos \theta)$?

Question: How do i show that $w=z^2$ maps $|z-1|\leq 1$ onto the cardioid $R=2(1+\cos \theta)$?
Comment: That should be $\lvert z-1\rvert = 1$, not $\le 1$. Write $z = 1+e^{i\varphi} = (e^{-i\varphi/2}+ e^{i\varphi/2})e^{i\varphi/2} = \left(2\cos \frac{\varphi}{2}\right)e^{i\varphi/2}$.
Answer: Mr. Fischer has given a very effective hint in his comment. The following approach will give the same result.
$z = x + iy$. So the circle $|z - 1| = 1$ can be written as $(x-1)^2 + y^2 = 1$ or by $x = 1 + \cos(\theta), y = \sin(\theta)$.
$z^2 = x^2 - y^2 + 2ixy$. Write $u = x^2 - y^2$ and $v = 2ixy$. Represent $u$ and $v$ by function of $\theta$. $u = 1 + 2\cos(\theta) + \cos(2 \theta)$ and $v = 2\sin(\theta) + \sin(2\theta)$ in simplified form.
$r^2 = u^2 + v^2$ and simplifying get $r^2 = 4(1 + cos(\theta))^2$ and so $r$.
Now see the fact that the function $z^2$ is entire and so conformal. Hence answer.