Add new widget upon clicking floating action button in flutter

Question: I am a beginner in Flutter. I am trying to add a new list item widget to screen when floating action button is pressed. How do I achieve this?
I am trying to create a list of items. When the floating action button is clicked, a dialog box is prompted and user is asked to enter details. I want to add a new list item with these user input details.
This is my input_page.dart file which I am calling in main.dart
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';

class MedPage extends StatefulWidget {
  @override
  _MedPageState createState()=> _MedPageState();
}

class _MedPageState extends State<MedPage> {

  Future<String>createAlertDialog(BuildContext context) async{

    TextEditingController customController= new TextEditingController();
    return await showDialog(context: context,builder: (context) {
      return AlertDialog(
        title: Text("Name of the Pill"),
        content: TextField(
          controller: customController,
        ),
        actions: <Widget>[
          MaterialButton(
              elevation: 5.0,
              child: Text("OK"),
              onPressed: (){
                Navigator.of(context).pop(customController.text.toString()); // to go back to screen after submitting
              }
              )

        ],
      );
    });
  }

  @override
  Widget build(BuildContext context) {

    return Scaffold(
      appBar: AppBar(
        title: Text('My med app'),
      ),
      body: Center(

        child: Column(
          mainAxisAlignment: MainAxisAlignment.spaceBetween,
          crossAxisAlignment: CrossAxisAlignment.stretch,
          children: <Widget> [
            Expanded(
              child: ListView(
                padding: const EdgeInsets.all(8),
                children: <Widget>[
                  ReusableListItem(Color(0xFFd2fddf),"Name 1"),

                  ReusableListItem(Colors.orange,"Name 2"),

                  ReusableListItem(Color(0xFF57a1ab), "Name 3"),
                ],

              ),
            ),

          ],
        ),

      ),

      floatingActionButton: FloatingActionButton(
        onPressed: (){
          print("Clicked");
          createAlertDialog(context).then((onValue){
            print(onValue);
            setState(() {

            });

          });

        },
        child: Icon(Icons.add),
      ),
    );

  }
}

class ReusableListItem extends StatelessWidget {
  ReusableListItem(this.colour,this.pill);

  Color colour;
  String pill;

  @override
  Widget build(BuildContext context) {
    return Container(
      height: 50,
      margin: const EdgeInsets.all(8),
      decoration: BoxDecoration(
          color: colour,
          borderRadius: BorderRadius.circular(15.0)

      ),
        child: Center(
            child: Text(pill)
        ),

    );
  }
}

Answer: You don't need to change much in your code, maintain a variable that stores the values entered to be able to show them in the list. You should use Listview.builder() in order to dynamically render the items.
Here's your code:
class MedPage extends StatefulWidget {
  @override
  _MedPageState createState() => _MedPageState();
}

class _MedPageState extends State<MedPage> {
  List<String> items = [];

  Future<String> createAlertDialog(BuildContext context) async {
    TextEditingController customController = new TextEditingController();
    return await showDialog(
        context: context,
        builder: (context) {
          return AlertDialog(
            title: Text("Name of the Pill"),
            content: TextField(
              controller: customController,
            ),
            actions: <Widget>[
              MaterialButton(
                  elevation: 5.0,
                  child: Text("OK"),
                  onPressed: () {
                    Navigator.of(context).pop(customController.text
                        .toString()); // to go back to screen after submitting
                  })
            ],
          );
        });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('My med app'),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.spaceBetween,
          crossAxisAlignment: CrossAxisAlignment.stretch,
          children: <Widget>[
            Expanded(
              child: ListView.builder(
                itemBuilder: (context, index) {
                  return ReusableListItem(Color(0xFFd2fddf), items[index]);
                },
                itemCount: items.length,
              ),
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: () {
          print("Clicked");
          createAlertDialog(context).then((onValue) {
            // print(onValue);
            setState(() {
              items.add(onValue);
            });
          });
        },
        child: Icon(Icons.add),
      ),
    );
  }
}

class ReusableListItem extends StatelessWidget {
  ReusableListItem(this.colour, this.pill);

  final Color colour;
  final String pill;

  @override
  Widget build(BuildContext context) {
    return Container(
      height: 50,
      margin: const EdgeInsets.all(8),
      decoration:
          BoxDecoration(color: colour, borderRadius: BorderRadius.circular(15.0)),
      child: Center(child: Text(pill)),
    );
  }
}

Answer: Firstly you need to use ListView.builder() rather than ListView because you have dynamic content. Also you need to hold your items in a list.
// create a list before
ListView.builder(
    itemCount: list.length,
    itemBuilder: (BuildContext context, int index) {
     return Text(list[index]);
    }
  )

When you click on FloatingActionButton() you will call AlertDialog() method.
FloatingActionButton(
        onPressed: (){
AlertDialog(
    content:  Form(), // create your form here
    actions: [
// add a button here
]
   )
})

This method will show a dialog(you will add a form inside of the dialog). When the user completes the form(after clicking the button) you will add a new object to the list and update the state with setState({})
onPressed: (){
setState({
// add new object to the list here
});
Navigator.pop(context); // this will close the dialog
}


VBA to find column by name and reference values from it in a formula

Question: I'm new to vba, and i only know what i've had to use (not much) and i've tried to frankenstein something together by copying all of your helpful suggestions in other threads, but i'm making a mess of it.

I need to find a column by name (won't always be in the same place)
I need to fill the first open column with a formula that references the column i just found (won't always be in the same place)
If i can write a header name, that'd be swell.
Sub Labels()
Dim NextEmptyCol As Long
Dim lastRow As Long

SearchV = Range("A1:DD1").Find(What:="team_id", LookIn:=xlValues, LookAt:=xlWhole, _
MatchCase:=False, SearchFormat:=False).Column

lastRow = ActiveSheet.Range("B" & Rows.Count).End(xlUp).Row

NextEmptyCol = Cells.Find("*", [A1], , , xlByColumns, xlPrevious).Column + 1
Range(lastRow & Cells(Rows.Count, "A").End(xlUp).Row).Formula = "=b1+c1"

End Sub

I have no idea what i'm doing - this is cobbled from other forum threads, but i thought SearchV would find the column, lastRow would find how many cells needed to be filled, and NextEmptyCol would put them in the  next column. (b1+c1 is a just a placeholder formula - i didn't know how to refer to the column i found in the first bit.

What a mess.
Comment: the range for `Range(lastRow & Cells(Rows.Count, "A").End(xlUp).Row).Formula` will mess up! `lastRow` is numeric like `5`. But `Cells(Rows.Count, "A").End(xlUp).Row` will also be numeric like `7`. This way `Range()` will get `57` as address (which is not a valid ref)!
Answer: If I got you right, you want to go for the column after the last used, and fill it with a formula which is =[column B] + [column with search term]. This should do it:
Sub DoFomula()

  Dim SearchV As Long
  Dim myRange As Range

  'column to write in / offset for headers
  Set myRange = Cells(1, Columns.Count).End(xlToLeft).Offset(1, 1)

  'height we need
  Set myRange = myRange.Resize(Range("B" & Rows.Count).End(xlUp).Row - 1, 1)

  'column we do look for
  SearchV = Application.Match("team_id", Rows(1), 0)

  'input formula
  myRange.FormulaR1C1 = "=RC2 + RC" & SearchV

  'your header
  myRange.Offset(-1).Resize(1).Value = "My New Header"

End Sub

Should be self explaining, but if you still have any questions, just ask :)
Comment: It looks like in the formula line, you reference the SearchV cell by using quotes to step outside the formula. If i need to reference that cell several times in the same formula, how do i do that? I tried repeating what you had done, but i got a  syntax error.
Comment: Can you show how the formula needs to look like and what you have tried? Right now I don't know how to explain it properly without knowing what you are aiming for.
Comment: Nevermind. Figured it out. Was missing an & sign. You guys are the best
Comment: This is what i ended up with, if you're curious. '"=IF(RC" & SearchV & "<300,0,IF(RC" & SearchV & "<500,1,IF(RC" & SearchV & "<800,2,IF(RC" & SearchV & "<1300,3,IF(RC" & SearchV & "<2500,4,IF(RC" & SearchV & "<5000,5,IF(RC" & SearchV & "<7500,6,IF(RC" & SearchV & "<10000,7,8))))))))"'
Answer: I believe this might be what you are looking for:
Sub subLabels()

Dim NextEmptyCol As Long
Dim lastRow As Long
Dim rngFound As Range

'Change the sheet name in the next line (if necessary) to the correct one
With Worksheets("Sheet1")
    Set rngFound = .Range("A1:DD1").Find(What:="team_id", LookIn:=xlValues, LookAt:=xlWhole, _
        MatchCase:=False, SearchFormat:=False)

    'Proceed only if "team_id" was found
    If Not rngFound Is Nothing Then
        lastRow = .Range("B" & .Rows.Count).End(xlUp).Row
        NextEmptyCol = .Cells.Find("*", [A1], , , xlByColumns, xlPrevious).Column + 1
        .Cells(lastRow, NextEmptyCol).Formula = "=B" & lastRow & "+C" & lastRow
    Else
        MsgBox "Column 'team_id' not found." & Chr(10) & "Aborting!"
    End If
End With

End Sub

Here are the changes:

I added the reference to a particular sheet instead ActiveSheet in case the macro runs while another sheet is active. Otherwise, that will mess up your file. You might have to change the sheet name there.
Exchanged the method Range for Cells in the line where you are setting the formula.
Include the possibility that the column "team_id" could not be found.
Made the formula modular. Otherwise, the formula in any row would be =B1+C1. Instead you probably want it to be =B25+C25 in row 25. If that's not the case then you can easily change it back.

Note: don't hesitate to mark any key-word in the code and then press F1 to learn more about it. For example: highlight the word cells and then press F1 to learn the difference between cells and range.
Comment: So, if by chance instead of B1+C1, i wanted to do `=IF(AB10<300,0,IF(AB10<500,1,IF(AB10<800,2,IF(AB10<1300,3,IF(AB10<2500,4,IF(AB10<5000,5,IF(AB10<7500,6,IF(AB10<10000,7,8))))))))' where AB is the "team_id" column we found....
Comment: In that case I'd recommend something like this: `Replace("=IF(AB##<300,0,IF(AB##<500,1,IF(AB##<800,2,IF(AB##<1300,3,IF(AB##<2500,4,IF(AB##<5000,5,IF(AB##<7500,6,IF(AB##<10000,7,8))))))))", "##", lastRow)`
Comment: Thanks for all your help. I figured out how to get the cell references for the team_id columns, except that when they are output, it has the actual column name, instead of the letter reference. (I used rngFound)
Answer: Your problem is that you're referencing a range with two rows: lastRow & Cells(Rows.Count, "A").End(xlUp).Row. I think you want Range(NextEmptyCol & lastRow).Formula = "=B1+C1"
Comment: Ok, that's super helpful, it's making more sense. One question, if i take the & lastRow out of the last line, will it fill the whole column?
Comment: Sorry couldn't get back to you in time. @Ralph's answer seems to do everything you wanted.


DATASTAGE ODBC query get current_timestamp

Question: Estou a fazer um job no datastage e quando insiro a query no ODBC preciso do current timestamp, sendo que me passa sempre para null ao escrever na tabela.
Já utilizei CURRENT_TIMESTAMP, (CURRENT_TIMESTAMP), '#CURRENT_TIMESTAMP#', etc..
Segue a query.
(INSERT INTO dev_dw.compress_log VALUES ('#ORIGIN_TABLE#' , '#DESTINATION_TABLE#', '#CONDITION#' , CURRENT_TIMESTAMP )).

Alguém consegue ajudar-me?

Criei uma table em parquet no hive, ao executar o job não consigo inserir o current_timestamp na tabela que criei.

INSERT INTO dev_dw.compress_log VALUES ('#ORIGIN_TABLE#' ,
    '#DESTINATION_TABLE#', '#CONDITION#' , CURRENT_TIMESTAMP )

Comment: Está um pouco confuso cara, você consegue explicar um pouco melhor?
Comment: Qual o banco, SQL Server?
Comment: Criei uma table em parquet no hive, ao executar o job não consigo inserir o current_timestamp na tabela que criei.
Answer: Boa noite!
Você pode fazer um insert passando uma coluna nova a partir de um transformer com a função CurrentTime()


Move data to top or bottom w/ Javascript

Question: Ok just got assigned to a new project and have never dealt with Javascript before. Right now the data is moved up and down the grid with the code below. I need to add to the case statement a "top" and "bottom" that will the user the move the selected row to either the top of the grid or the bottom. I am trying to google search answers but not having much luck. Any advice on where I could continue my search would be helpful.
Ok new issue. I added some code that I created. It is in the case statement under "top" and "bottom" The code I added allows me to select an item and move it to the very top of the list. But moving it to the bottom and IE tells me that the script is taking to long. ANy Advice?
Never mind I got the code working. Below is the working code.
function move(direction) {
try {
    if (CSAdmin.selectedItem.element &&
CSAdmin.selectedItem.ID) {
        var row = CSAdmin.selectedItem.element;
        var node =
    getNode(CSAdmin.selectedItem.type, CSAdmin.selectedItem.ID);
        if (node) {
            var sibling = null;
            var rowSibling = null;
            var parent = node.parentNode;
            var rowParent = row.parentNode;
            switch (direction) {
                case "up":
                    {
                        sibling = node.previousSibling;
                        if (sibling) {
                            parent.removeChild(node);
                            parent.insertBefore(node, sibling);
                        }
                        //Move the table row
                        rowSibling = row.previousSibling;
                        if (rowSibling) {
                            rowParent.removeChild(row);
                            rowParent.insertBefore(row, rowSibling);
                        }
                        break;
                    }
                case "down":
                    {
                        sibling = node.nextSibling;
                        if (sibling) {
                            parent.removeChild(sibling);
                            parent.insertBefore(sibling, node);
                        }
                        //Move the table row if it isn't the insert row
                        rowSibling = row.nextSibling;
                        if (rowSibling && rowSibling.id.indexOf("new", 0) == -1) {
                            rowParent.removeChild(rowSibling);
                            rowParent.insertBefore(rowSibling, row);
                        }
                        break;
                    }
                    case "top":
                    {
                        sibling = node.previousSibling;
                        while (sibling) {
                            if (sibling) {
                                parent.removeChild(node);
                                parent.insertBefore(node, sibling);
                                sibling = node.previousSibling;
                            }
                        }
                        //Move the table row
                        rowSibling = row.previousSibling;
                        while (rowSibling) {
                            if (rowSibling) {
                                rowParent.removeChild(row);
                                rowParent.insertBefore(row, rowSibling);
                                rowSibling = row.previousSibling;
                            }
                        }
                        break;
                    }
                case "bottom":
                    {
                        sibling = node.nextSibling;
                        while (sibling) {
                            if (sibling) {
                                parent.removeChild(sibling);
                                parent.insertBefore(sibling, node);
                                sibling = node.nextSibling;
                            }
                        }
                        //Move the table row if it isn't the insert row
                        rowSibling = row.nextSibling;
                        while (rowSibling && rowSibling.id.indexOf("new", 0) == -1) {
                            if (rowSibling && rowSibling.id.indexOf("new", 0) == -1) {
                                rowParent.removeChild(rowSibling);
                                rowParent.insertBefore(rowSibling, row);
                                rowSibling = row.nextSibling;
                            }
                        }
                        break;
                    }
            }
        }
    }
}
catch (err) {
    throw new Error("Error moving row:" + err.description);
}

Comment: I really don't understand your question. Please add an example of what you're trying to achieve/what you already have.
Comment: What I have is a datagrid. If you select a row and hit either Up or Down button you can move a row up or down one row at a time. I want to add a Move To Top and Move To Bottom button that will move that row either to the top of the grid or to the bottom. The code above currently will either move it up or down one row at a time.
Answer: The variable names in your existing code are kind of confusing (you don't even tell us what node is), but this is the basic premise for making it work. I used jQuery to shortcut the work in this example, but the logic is sound and is not reliant on jQuery.
http://jsfiddle.net/qukDJ/
//move to top
var currentRow = $(this).closest('tr')[0];
var firstRow = currentRow.parentNode.childNodes[0];
currentRow.parentNode.insertBefore(currentRow, firstRow);

Moving to the bottom is basically the same thing except you want lastRow and insertAfter.
var lastRow = currentRow.parentNode.childNodes[currentRow.parentNode.childNodes.length - 1];
currentRow.parentNode.insertAfter(currentRow, lastRow);

Comment: Sorry for the confusion this is not my code. It belongs to a developer no longer here. I am trying to figure it all out myself. From what I can tell node is slimpy the selectedItem type and ID. I am basing this only on this line var node =getNode(CSAdmin.selectedItem.type, CSAdmin.selectedItem.ID);
Comment: This is not the exact right answer but it was the next based thing it allowed me to figure out what the javascript was doing and allowed me to realize how to do it correctly.


Under what conditions can we find a basis of a Lie algebra such that the adjoint representation acts by skew-symmetric matrices?

Question: My question might be very silly as I've only recently started to learn about Lie algebras.
Given a Lie algebra $\mathfrak{g}$, and a vector space basis $\{e_i|i=1 \ldots \dim \mathfrak{g}\}$, we can write the adjoint action $\text{Ad}_x: \mathfrak{g} \to \mathfrak{g}: y \mapsto [x,y]$ in matrix form, where the $i,j$th entry of the matrix $[\text{Ad}_x]$ equals $\lambda_i$ if $[x,e_j] = \sum_{k}\lambda_k e_k$. I am looking for necessary and sufficient conditions on our Lie algebra $\mathfrak{g}$ such that there exists a basis in which all of these matrices $\{ [\text{Ad}_x]| x \in \mathfrak{g} \}$ are skew-symmetric.
If the condition turns out to be either simplicity of compactness of the associated simply connected Lie group, I would be very happy.
Comment: Look up the Killing form. In case the real Lie algebra is that of a compact semisimple Lie group, then minus the Killing form is positive definite. Since the Killing form is preserved by the adjoint action, this implies the statement you were hoping for in case the Lie algebra is that of a compact semisimple Lie group.
Comment: @Malkoun: Although I believe you're ultimately right with that sufficient criterion, I actually have a hard time filling in the details without using quite a bit of Lie theory. The problem is that I see the automorphisms $Ad(g)$ for $g$ in the corresponding Lie *group* are orthogonal wrt the Killing form, and I guess that makes the $ad(x)$ for $x$ in the Lie *algebra* (which are not even invertible as linear maps) skew-symmetric (at least for certain choices of basis). But it seems like you have an easier argument which does not go through the group? If so, could you elaborate?
Comment: @TorstenSchoeneberg, I will write an answer, explaining my comment in more detail.
Answer: $\DeclareMathOperator{\tr}{tr}\DeclareMathOperator{\ad}{ad}$I will expand my comment here. I am assuming (though I am not sure if this is the case) that the OP is interested in real Lie algebras. If $\mathfrak{g}$ is the Lie algebra of a compact semisimple Lie group, then with respect to any orthonormal basis of $\mathfrak{g}$ with respect to (minus) the Killing form, the matrix representing $\operatorname{ad}_x \in \mathfrak{gl}(\mathfrak{g})$ is skew-symmetric, for any $x \in \mathfrak{g}$.
Define the Killing form $B(-,-): \mathfrak{g} \times \mathfrak{g} \to \mathbb{R}$ by:
$$B(x,y) = \tr(\ad_x \circ \ad_y).$$
I claim that for any $z \in \mathfrak{g}$, $B([z,x],y) + B(x,[z,y]) = 0$. For short, it is common to say that the Killing form is $\ad$-invariant.
$$ \begin{align*} & B([z,x],y) + B(x,[z,y]) \\
    = & \,\tr(\ad_{[z,x]} \circ \ad_y)+
    \tr(\ad_x,\ad_{[z,y]}) \\
    = & \,\tr([\ad_z,\ad_x] \circ \ad_y) + \tr(\ad_x \circ \, [\ad_z, \ad_y]) \\
    = & \,0 \end{align*}$$
by the cyclic property of the trace ($\tr(BCA) = \tr(ABC)$). I have also used the fact that $\ad_{[x,y]} = [\ad_x,\ad_y]$.
There is another fact from Lie theory that we need, namely that if $\mathfrak{g}$ is the Lie algebra of a compact semisimple Lie group, then its Killing form is negative definite.
Let us now choose an orthonormal basis of $\mathfrak{g}$, say $x_1,\ldots,x_n$, with respect to minus the Killing form. Note that, for any $x \in \mathfrak{g}$, we have
$$ -B(\ad(x)(x_i),x_j) = -B([x,x_i],x_j) = B(x_i, [x,x_j]) = B(x_i, \ad_x(x_j)).$$
Hence the matrix representing $\ad_x$ with respect to the orthonormal basis $x_1,\ldots,x_n$ is skew-symmetric.
Comment: Thank you very much!
Comment: "De rien". I did not know that you were mostly interested in complex Lie algebras. As @Torsten Schoeneberg remarked, since any complex semisimple Lie algebra $\mathfrak{g}$ has a "compact real form" (i.e. there is "real slice" of $\mathfrak{g}$ which is the Lie algebra of a real compact semisimple Lie group), using the same basis as in my answer would indeed show the claim you want, that any $ad_x$ is represented by a complex skew-symmetric matrix.
Answer: Malkoun's beautiful answer shows that for a semisimple real Lie algebra, a sufficient criterion for such a basis to exist is that the Lie algebra is anisotropic, i.e. what in the real case is commonly called "compact", i.e. the corresponding Lie groups are compact.
I just want to point out that for semisimple real Lie algebras $\mathfrak g$, this is also necessary (as alluded to by Tsemo Aristide in the answer to this related question). Namely, if a real semisimple Lie algebra is not anisotropic (compact), then it contains elements $x \neq 0$ which are ad-diagonalisable, i.e. such that $ad(x)$ has non-zero real eigenvalues. However, all eigenvalues of real skew-symmetric matrices are purely imaginary, so there can be no basis of $\mathfrak g$ in which all $ad(x)$ are skew-symmetric.
Incidentally, Malkoun's answer can also be used to show that somewhat surprisingly, the result is true for all complex semisimple Lie algebras $\mathfrak g$. Namely, it is a classical cornerstone of the theory that for each such $\mathfrak g$ there exists a (unique) compact real form, i.e. an anisotropic real semisimple Lie algebra $\mathfrak g_0$ such that the scalar extension $\mathbb C \otimes\mathfrak g_0$ is isomorphic to $\mathfrak g$. Now take an orthonormal (real) basis $e_1, ..., e_n$ of $\mathfrak g_0$ w.r.t. its Killing form as per Malkoun's answer, and the matrices of all the adjoints to those elements will be skew-symmetric. Now all elements of $\mathfrak g$ are $\mathbb C$-linear combinations of elements of $\mathfrak g_0$, hence the matrices of all $ad(x), x \in \mathfrak g$ in the basis $1 \otimes e_1, ..., 1 \otimes e_n$ will be skew-symmetric (complex) matrices. (Note that what I wrote in my second paragraph does not contradict this, as complex skew-symmetric matrices can have all eigenvalues one could want, as long as they add up to $0$.)
Comment: Thank you very much for your time!
What is wrong with skew-symmetric complex matrices? I am indeed working with complex Lie algebras, and I needed a simple criterion for the existence of an intertwining isomorphism between the adjoint and the coadjoint representation. If I haven't messed up somewhere, I think this should be exactly skew-symmetricity of the action of $\mathfrak{g}$ on itself.
Comment: Yes, the argument I provided also shows that if you have a complex semisimple Lie algebra $\mathfrak{g}$, then there is a basis of $\mathfrak{g}$ with respect to which $ad_x$ is represented by a complex skew-symmetric matrix. However, regarding your last comment, I want to remark that being skew-hermitian is a reality condition, so it may not be suitable for complex semisimple Lie algebras (I think).
Comment: @LukasRollier, having an ad-invariant non-degenerate complex symmetric bilinear form on the adjoint representation of a complex Lie algebra $\mathfrak{g}$ does I think provide an intertwining isomorphism between the adjoint and co-adjoint representation. Moreover, I think that for a complex Lie algebra, the Killing form is non-degenerate iff the Lie algebra is semisimple, but please double check. I think it is called Cartan criterion maybe. It remains to check whether for a complex non semisimple Lie algebra, there exists or not such an intertwining isomorphism.
Comment: You're both right, I retreat my comment. It's just that sesquilinear forms over $\mathbb C$ feel more common than bilinear ones, but of course these can be studied too.
Comment: @Malkoun, you're right about the Cartan criterion. It would remain to be checked for non semisimple Lie algebras, were it not that I'm not researching those. What I'm really working towards is a study of trivalent categories, building on work of Morrison, Peters and Snyder. It was mentioned very briefly in their paper that complex simple Lie algebras provide such categories, but as I knew nothing about Lie theory, I needed help. Thanks to you both for providing :D
Answer: The problem has haunted me for the last couple of weeks, and I want to add another answer:
Let's assume our Lie algebra $\mathfrak g$ is of finite dimension $n$ over a field $k$. The computation in Malkoun's answer shows that the question is equivalent to:

Does there exist a (symmetric, nondegenerate) $ad$-invariant bilinear form on $\mathfrak g$ that in some basis is represented by the identity matrix $Id_n$, i.e. such that the corresponding quadratic form is equivalent to $q_{st} (x_1, ..., x_n) =x_1^2 + x_2^2 + ... + x_n^2$?

Now let $\mathfrak g$ be absolutely simple over a field $k$ of characteristic $0$. Then it's well-known that if $b: \mathfrak g \times \mathfrak g \rightarrow k$ is any $ad$-invariant, nondegenerate bilinear form, then there is $c \in k^*$ such that $b(x,y) = c \kappa(x,y)$ for all $x,y \in \mathfrak g$, where $\kappa(\cdot, \cdot)$ is the Killing form. So our question becomes whether or not the Killing form, up to scalar multiple, is represented by $Id_n$.
(NB. For symmetric BLFs/ quadratic forms one usually looks at the equivalence relation on $M_n(k)$ given by
$$ A \simeq B :\Leftrightarrow \exists P \in M_n(k): A= P^TBP$$
and says A, B are congruent. Here, we are looking at the "coarser" equivalence relation $$ A \sim B :\Leftrightarrow \exists c \in k^*, P \in M_n(k): A=cP^TBP$$
This comes up a lot in the study of classical Lie algebras. I think Jacobson called two such matrices "cogredient", but I don't think that has become standard nomenclature. Elman/Karpenko/Merkurjev in The Algebraic and Geometric Theory of Quadratic Forms call it "similar". For now I will just say "congruent up to scalars".)
Now to decide whether this is the case for a given absolutely simple Lie algebra over a specific field can become quite intricate, and can depend a lot on both the Lie algebra and arithmetic of the field. The answers given for $\mathbb R$ and $\mathbb C$ are, in way, very good cases.
First off, the assertion is true if $k$ is quadratically closed (in particular, if $k$ is algebraically closed), because then, $q_{st}$ is the only quadratic form up to congruence (we don't even need scalar multiples there).
Over $\mathbb R$, via Sylvester's law of inertia, the only scalar multiples of $q_{st}$ up to congruence are $\pm q_{st}$, which are up to congruence the only definite quadratic forms there; i.e. our question is equivalent to deciding whether the Killing form is definite, and it is well-known that it is iff $\mathfrak g$ is a compact form.
For other fields, however, things might get tough. Let's just look at the
Easiest example $\mathfrak g:= \mathfrak{sl}_2(k)$
Since this Lie algebra (of dimension $n=3$) contains non-trivial $ad$-nilpotent elements, the Killing form contains isotropic vectors. A necessary condition for our assertion is therefore that there are $(0,0,0) \neq (x,y,z) \in k^3$ such that $x^2+y^2+z^2=0$. Actually, the theory of quadratic forms also shows that this is sufficient. Indeed, one can do this with explicit matrix computations: W.l.o.g. assume there are $x \in k, y \in k^*$ with $x^2+y^2+1=0$. Then
$$e_1 := \pmatrix{-\frac{1}{y}&\frac{x}{y}\\\frac{x}{y}&\frac{1}{y}} , e_2 := \pmatrix{-\frac{x}{y}&-\frac{1}{y}\\-\frac{1}{y}&\frac{x}{y}} , e_3 := \pmatrix{0&1\\-1&0}$$
form a basis of $\mathfrak{sl}_2(k)$ which is "orthonormal" with respect to $-\frac{1}{8}$ times the (isotropic) Killing form. (Note e.g. the special case that $k$ contains a square root of $-1$ called $i$: Then we can set $x=0$, $y=i$ and get $$e_1=\pmatrix{i&0\\0&-i}, e_2 = \pmatrix{0&i\\i&0}, e_3 = \pmatrix{0&1\\-1&0}$$
a standard basis of the compact real $\mathfrak{su}_2$ as well as $\mathfrak{sl}_2(\mathbb C)$, matching previous result for those fields.)
But for which fields there exist such $x,y (,z)$ is a non-trivial problem in general! E.g. while all extensions of some $p$-adic field $\mathbb Q_p$ for odd primes $p$ do, and also all seven quadratic extensions of $\mathbb Q_2$, the $2$-adic numbers $\mathbb Q_2$ themselves do not, and for other extensions of $\mathbb Q_2$ the answer seems to be hard to find.
Meaning that e.g. all $\mathfrak{sl}_2(\mathbb Q_p)$ for odd $p$ have bases for which all matrices representing the adjoint action are skew-symmetric, but there is no such basis for $\mathfrak{sl}_2(\mathbb Q_2)$.
Further, we can easily see that for any number field $k$ which can be embedded into $\mathbb R$, $\mathfrak{sl}_2(k)$ has no such basis (this generalises to any non-anisotropic form over any formally real field); certainly over many other number fields $k$ (trivial example $k=\mathbb Q(i)$), $\mathfrak{sl}_2(k)$ does have such a basis; but for example $\mathfrak{sl}_2(\mathbb Q(\sqrt{-7}))$ does not, as it embeds into $\mathbb Q_2$.
Now $\mathfrak{sl}_2$ can have other forms over fields, the classification of which is equivalent to that of division quaternion algebras over those fields. If $k$ is a finite extensions of some $\mathbb Q_p$, up to isomorphism there is only one, just like over $\mathbb R$. We call it
$\mathfrak{su}_2$ (over a $p$-adic field $k$)
Its elements are of the form $$\pmatrix{r\sqrt{a}&s+t\sqrt{a}\\b(s-t\sqrt{a})&-r\sqrt{a}}$$ where $r,s,t \in k$, and $a,b$ are chosen such that the quadratic form $X_0^2-aX_1^2-bX_2^2+abX_3^2$ is anisotropic (up to congruence, there is exactly one anisotropic quadratic form in four variables over $k$). One can check that the Killing form of $\mathfrak{su}_2$ is anisotropic again, and up to scaling is congruent to the unique (up to scaling) anisotropic quadratic form in three variables over $k$, which is $X_0^2-aX_1^2-bX^2$.
But our previous results showed that for all $p \neq 2$ (and also for any quadratic extension of $\mathbb Q_2$), the form $q_{st}$ is isotropic over $k$, so it cannot be congruent up to scaling to that Killing form! (Whereas conversely for odd degree extensions of $\mathbb Q_2$, it is the unique-to-scaling anisotropic form with $a=b=-1$). This means that for all $\mathbb Q_p$ ($p$ odd) the forms of $\mathfrak{sl}_2$ behave exactly the other way than over $\mathbb R$: The compact one cannot be written as skew-symmetric matrices, but the split one can!

Finally, note that although some of the above arguments generalise, some of them do not even generalise to $\mathfrak{sl}_3$: At some steps above I implicitly used that quadratic forms in three variables are still relatively easy to handle (in particular, up to scaling there is exactly one isotropic one). Another way to think of the $\mathfrak{sl}_2$-forms is in terms of isos of split or non-split forms of $\mathfrak{sl}_2$ and $\mathfrak{so}_3$, respectively. Any non-zero map between these is an isomorphism, for dimension reasons, so one is compact iff the other is. This line of argument would also break down already for forms of $\mathfrak{sl}_3$. For higher dimensions, one needs to know more about the Witt groups of specific fields than I do.
Comment: Interesting. I wonder what happens in characteristic $p$ as well. I suspect it to be very rare to happen, that there is a basis with respect to which the $ad_x$ are skew-symmetric, for all $x$ in the Lie algebra, over fields of characteristic $p$. I could be wrong though.


is there a shortcut for code folding on eclipse?

Question: I'm looking for a way to close/open foldable blocks of code using the Keyboard. I found some shortcuts in old answers but none of them seem to work on the version of Eclipse that I use (2020-09).
Comment: Please edit your question to specify a particular version of Eclipse, since in two years your version will no longer be "the newest version".
Comment: I also used that Eclipse feature for a while. Then I realized that whenever I felt the need to use it, the file simply had too many lines. For me, that is a file that I have to work on a lot and that has more than 200 lines. So if your file has more than 200 lines, and you need to work in it often, you probably implemented something in a not-so-good manner. The only time big source code files may exist is when you have to include a lot of datasets / enum cases / step-by-step-sequences. But then you should hardly ever work in that file. Use delegation to helper classes, or inheritance or enums
Comment: Most common code-smell with big files is copy/past code duplication with minor changes, that actually should be solved by parametrization or different object component models.
Answer: You can use Cmd (on macOS) or Alt (on Windows) and the subtract and divide on numeric keypads. If you don't have an external keypad then you can remap it yourself by going to preferences, and then keys, and you can enter your own.
If you want to expand/collapse all of them, you can hold shift down as well.


APR Problem - How much finance should I take out based on a credit of £550?

Question: I'm having a problem putting my mathematics degree to good use. A friend of mine wants me to work out the following problem for him:
A motorbike costs £14,000. I will receive a "sweetener" from the motorcycle company of £550 if I take out any amount of finance (£14K, £10K, or £5K - they will give you £550 however much finance you take out.)
With 5.9% APR, he wants to know how much finance he can take out, so that the cost of the finance is mitigated by the £550 sweetener, i.e. the finance costs him nothing.
I came up with the following solution:
$5.9%$ APR is equivalent to $0.01616%$ DPR (daily percentage rate)
Hence your interest rate is 100.01616% per day, or 1.0001616 as a decimal.
So we can write $b(1.0001616)^{365n}$ to express the total amount you will be paying back based on the amount borrowed, $b$, for $n$ number of years.
If we subtract the original amount borrowed $b$ from this, we will have how much interest we are paying for the finance:
$b(1.0001616)^{365n}-b = b(1.0001616^{365n}-1)$
And we want this to be the same as the sweetener so it effectively costs nothing, hence:
$b(1.0001616^{365n}-1) = 550$
which when rearranged, gives me  $b=2840$
However, when I use an APR calculator, I'm told if I borrow £2840 for 36 months at 5.9%, the cost of the loan is £265... not the £550 I was expecting it to be.
Can anyone tell me where my solution has gone wrong? any help is appreciated.
Thanks!
Doug
Answer: For 3 years the composite interest is 9.1%. If you finance £6040 you are going to pay £6590 in three years, paying £183.06 per month.
Comment: Please explain how you arrived at 19%. Also please explain how you arrived at £2931.
And why when I use an online APR calculator does it say that £6000 (approx.) borrowed at 5.9% APR for 3 years will cost £550 (approx.) which doesn't agree with your figure?
Comment: I overlooked the question, now the figures are correct.
Comment: Cool. I suppose, technically you should factor in the future value of the 550, but with rates so low, it won't matter much.


Three vectors giving resultant 0, but not forming a triangle?

Question: There's this statement in one of my books that says,

In general, for any non-zero vectors $\overrightarrow{a}$, $\overrightarrow{b}$, and $\overrightarrow{c}$, one may note thst although $\overrightarrow{a}+ \overrightarrow{b}+\overrightarrow{c}=0$, they may not always represent the sides of a triangle.

This seems confusing. If I take three vectors, and consider just two of them, I'll get a resultant vector, which can only be cancelled out by another vector parallel to it in the opposite direction, implying that all three have to lie on a triangle. That's the general consensus on three non-zero vectors giving zero resultant that I know of, too.
Is there a case I'm missing here?
Comment: Maybe they don't count it as a "triangle" if $a, b, c$ are on the same line?
Comment: I agree with Yuval. Such "naming" claim also highly depends on the context. If the underlying field for the vectors isn't $\mathbb{R}$, personally I would not call such triple a triangle.
Comment: @Yuval: That explains it nicely. Wonder why the book didn't specify this, because it seemed almost intentionally vague, like there were lots of cases left out. Don't get me wrong; this answers the question, but are you sure there aren't any more cases?
Answer: OP: "@Yuval, that explains it nicely. Wonder why the book didn't specify this, because it seemed almost intentionally vague, like there were lots of cases left out... are you sure there aren't any more cases?"

Suppose that $\mathbf a,\mathbf b,\mathbf c$ aren't collinear (i.e., aren't
parallel).
Then, since $\mathbf a,\mathbf b,\mathbf c$ are nonzero and sum to
$\mathbf 0,$ no pair is collinear; in particular, $\mathbf a
   \nparallel \mathbf b.$
Thus, $0^{\circ}<\text{angle between }\mathbf a \:\&\: \mathbf b
   <180^{\circ}.$
Since $\mathbf a+\mathbf b+\mathbf c=\mathbf 0,$ the three vectors
form a triangle.

Therefore, if the three vectors don't form a triangle, then they
must be collinear. In this case, the three vectors in fact form a degenerate triangle.
Comment: Nice, concise answer. I hope I haven't missed out on any logic here, though, because it seems trivial, atleast looking at the cases we've considered.


Volume of a Cone using Cylindrical Coordinates

Question: I'm aware of the usual method for calculating the volume by expressing the integrals for $dr$ and $dz$ in terms of $z$ to get the correct answer but when I attempted to solve it expressing everything in terms of $r$ I can't quite seem to get it.
Trying to calculate the volume of a cone of radius $R$ and height $h$:
If we try to express everything in terms of $r$ then using similar triangles we obtain $r=\frac{zR}{h}$, now for integration limits $r:\frac{zR}{h}\to R$, $z: 0\to h$ and $\theta:0\to 2\pi$ so the integral becomes
$$
\int_{0}^{2\pi}\int_{0}^{h}\int_{\frac{zR}{h}}^{R}r\, drdzd\theta=\pi\int_{0}^{h}(R^2-\frac{z^2R^2}{h^2})dz=\frac{\pi R^2}{h^2}\int_{0}^{h}(h^2-z^2)dz=\frac{\pi R^2}{h^2}(h^3-\frac{h^3}{3})\\
\int_C r\, drdzd\theta=\frac{2\pi R^2h}{3}
$$
which is double the correct answer of $\frac{1}{3}\pi R^2h$
Answer: Let $(\rho,z,\phi)$ be the cylindrical coordinate of a point $(x,y,z)$. Let $r$ be the radius and $h$ be the height. Then $z \in [0,h], \phi \in [0,2\pi], \rho \in [0,rz/h]$. The volume is given by
$$\begin{align*}
\iiint_C\,dV
&= \int_0^{2\pi} \int_0^h \int_0^{rz/h} \rho\,d\rho\,dz\,d\phi \\
&= 2\pi \int_0^h \left. \frac{\rho^2}{2} \right|_0^{rz/h}\,dz \\
&= \pi \int_0^h \frac{r^2 z^2}{h^2}\,dz \\
&= \frac{\pi r^2}{h^2} \frac{h^3}{3} = \frac{\pi r^2 h}{3}
\end{align*}$$
as desired.
Your integral gives the volume of the inverse of a cone. That is, the part of a cylinder remained when a cone is removed from it.
Comment: @Craig Try drawing an upside down cone with vertex at the origin. It might help clearing things up a bit.
Comment: Oh, yea that makes sense since it starts at $\rho=0$ for every integral. I'm confused now as to why if we integrate with respect to z first like here:
https://www.math.washington.edu/~aloveles/Math324Fall2013/f13m324TripleIntegralExamples.pdf
they don't have 0 in the lower limit for z but instead rh/a
Comment: @Craig The order of integration does not matter. See Fubini's theorem.
Comment: I'm referring to the limits they used for their integration, they have $\int_{0}^{2\pi}\int_{0}^{R}\int_{hr/R}^{R}r\, dzdrd\theta$ although I can't comprehend why the lower limit isn't 0 with respect to the $dz$ integration.


Convergence in $L^p(\lambda)$ of mean of backward shifted integrable functions

Question: Let $(\Omega, \mathcal{A}, \mu)$ be a $\sigma$-finite measure space. I want to prove the following (Exercise 7.1.2 in Klenke's Probability book):

Let $p \in (1, \infty), f \in \mathcal{L}^p(\lambda)$, where $\lambda$ is the Lebesgue measure on $\mathbb{R}$. Let $T: \mathbb{R} \to \mathbb{R}, x \to x + 1$. Show that,
\begin{align}
\frac{1}{n}\sum_{k = 0}^{n - 1}f\circ T^k \to 0 \hspace{0.2cm} \text{as} \hspace{0.2cm} n \to \infty \hspace{0.2cm} \text{in} \hspace{0.2cm} L^p(\lambda).
\end{align}

I am lost on how to prove this. I think I am at least able to prove that the sequence is a Cauchy sequence and thus converges, but even there I am not sure. Here is my attempt:
Let $S_n = \frac{1}{n}\sum_{k = 0}^{n - 1}f\circ T^k$. Then
\begin{align}
\|S_n - S_{n - 1}\|_p &= \|\frac{1}{n} f \circ T^{n-1} - \frac{1}{n(n-1)}\sum_{k = 0}^{n -2}f \circ T^k \|_p \\
& \leq \frac{1}{n} \|f \circ T^{n - 1}\|_p + \frac{1}{n(n - 1)}\sum_{k = 0}^{n -2} \|f \circ T^k \|_p\\
&= \frac{2M}{n} \to 0,
\end{align}
where $M = \|f\circ T^k\|_p$. Here I am obviously assuming $\|f\circ T^i\|_p = \|f\circ T^j\|_p$ for all $i, j \in \{0, \ldots, n- 1\}$, which I think is true, but can only provide intuitive (as opposed to formal) justification (Since $f \in \mathcal{L}^p (\lambda)$, then the integral of the function should not change by shifting it around horizontally. I guess the idea would be to invoke the translation invariance of the Lebesgue measure?).
I have two questions then:
(1) Is my proof that the sequence converges in $L^p(\lambda)$ correct? Is my above assumption correct? If yes, what would be the formal justification?
(2) How would one prove that the limit is $0$?
Thanks much in advance!
Answer: With regards to the questions of the OP
(1) your approach is not entirely correct since having $\|S_nf-S_{n-1}f\|_p\xrightarrow{n\rightarrow\infty}0$ does not imply that the sequence $S_nf$ is Cauchy in $L_p$
(2) Looking at the material presented in your textbook, the idea it seems is to show that $\{|S_nf|^p:n\in\mathbb{N}\}$ converges to $0$ in measure and also that is uniform integrable, at least for a collection of functions $f$ that is dense in $L_p$.
Here I outlined the ideas in more detail. The tricky part is proving uniform integrability of $\{|S_nf|^p:n\in\mathbb{N}\}$. I suggest an approach at the end. Finally, I should mentioned that applying tools from Ergodic Theorem (not covered in the textbook until much later and then only in the context of probability measures) give a short solution to the problem, in particular the Dunford-Hopf-Schwartz) ergodic theorem which states:

Theorem (Dunford--Hopf--Schwartz). Let $(S,\mathscr{B},\mu)$ a $\sigma$-finite measure space. Suppose $T$ is a positive $\mathcal{L}_1(\mu)$-$\mathcal{L}_\infty(\mu)$ contraction.  Define $A_n=\frac1n\sum^{n-1}_{k=1}T^k$. If $1\leq p<\infty$, then

For any  $f\in\mathcal{L}_p(\mu)$, $Af:=\lim_nA_nf$ exists $\mu$-a.s., and $Af\in \mathcal{L}_p(\mu)$.
The map $A:f\mapsto Af$  is a positive $\mathcal{L}_1(\mu)$--$\mathcal{L}_{\infty-}(\mu)$ contraction and  $AT=TA=A$.
If $1<p<\infty$, $\|A_nf-Af\|_p\xrightarrow{n\rightarrow\infty}0$.

If $\mu(S)<\infty$, then:

$A$ is an $\mathcal{L}_\infty(\mu)$ contraction.
$\|A_nf-Af\|_1\xrightarrow{n\rightarrow\infty}0$ for all $f\in\mathcal{L}_1(\mu)$

In the case of the OP, the operator is $Tf(x)=f(x+1)$. Clearly $Tf\geq0$ when $f\geq0$, and $\|Tf\|_p\leq\|f\|_p$  (in fact equal) for any $1\leq p\leq \infty$ (This means that $T$ is an positive $L_1-L_\infty$ contraction).

Fix $1<p<\infty$, and as before,  $S_nf(x):=\frac1n\sum^{n-1}_{k=0}f(x+k)$ for any function $f\in L_p$.
Recall that the space $\mathcal{C}_{00}(\mathbb{R})$ of continuous functions of compact support is dense in $L_p(\lambda)$.
For $g\in C_{00}(\mathbb{R})$ $S_ng(x)=\frac1n\sum^{n-1}_{k=1}g(x+k)\xrightarrow{n\rightarrow\infty}0$ and so $S_ng$ also converges in measure to $0$.
Showing that $\|S_ng\|_p\xrightarrow{n\rightarrow\infty}0$ for all $g\in C_{00}(\mathbb{R})$ would imply the result for all $f\in L_p$. Indeed, given $f\in L_p$ and $\varepsilon>0$ choose $g\in C_{00}(\mathbb{R})$ such that $\|f-g\|_p<\varepsilon$.
Then
$$\|S_nf\|_p\leq \|S_n(f-g)\|_p+\|S_ng\|_p<\varepsilon+\|S_ng\|_p$$
whence we obtain
$$\limsup_n\|S_nf\|_p\leq\varepsilon, \qquad\varepsilon>0$$
Thus, $\|S_nf\|_p\xrightarrow{n\rightarrow\infty}0$.

Edit:
After more thinking I realized that we can do without having to check for uniform integrability and instead directly prove that
$\|S_ng\|_p\xrightarrow{n\rightarrow\infty}0$ for all $g\in\mathcal{C}_{00}(\mathbb{R})$.
The support of $g$ is contained in some intercal of the form $I:=(k,m]$ for $k,m\in\mathbb{Z}$. Splitting $I$ in $m-k$ of length $1$ we write $g=\sum^{m-k-1}_{\ell=0}g\mathbb{1}_{(k+\ell,k+\ell+1]}$.
Let $\phi_{\ell}:=g\mathbb{1}_{(k+\ell,k+\ell+1]}$.
Since the operator $S_n$ is linear, we have that
$S_ng=\sum^{m-k-1}_{\ell=0}S_n(\phi_\ell)$
The advantage of breaking $g$ in pieces of $\phi_\ell$ of support of length one is that successive integer translations yield functions whose supports whose interiors are  pairwise disjoint, for
$$\operatorname{supp}(\phi_\ell\circ T^j)=[k+\ell-j,k+\ell+1-j].$$
Then
$$\|S_ng\|_p\leq\sum^{m-k-1}_{\ell}\|S_n\phi_\ell\|_p$$
and by translation invariance, for each $0\leq \ell < m-k$
\begin{align}
\|S_n\phi_\ell\|^p_p&=\frac{1}{n^p}\int\Big|\sum^{n-1}_{j=0}\phi_\ell\circ T^k\Big|^p=\frac{1}{n^p}\int\sum^{n-1}_{j=0}|\phi_\ell\circ T^k|^p=\frac{1}{n^{p-1}}\|\phi_\ell\|^p_p
\end{align}
Putting things together, we obtain that
$$\|S_ng\|_p\leq n^{\tfrac1p -1}(m-k)\|g\|_p\xrightarrow{n\rightarrow\infty}0$$
Answer: Here is an alternative solution.
The operator norm of $S_n$ is bounded above by $1$. Let $g$ be function in $L^p$. Then
$$ \|S_n f\| \le \|S_n g\|+ \|S_n (f-g)\|\le \|S_n g \| + \|f-g\|.$$
Thus we will be done if we can show that for every $\epsilon>0$ we can find $g$ such that (i)  $\|f-g\|<\epsilon/2$ and (ii) $\|S_n g\|< \epsilon/2$.
(i) For an integer $M>0$, let $g_M= f {\bf 1}_{[-M,M)}$. By dominated convergence, for $M$ large enough $\|f-g_M\|<\epsilon/2$.
Moreover, $g_M$ is the sum of $2M$ functions $h_j$, $j=-M,\dots,M-1$ where $h_{j} = f {\bf 1}_{[j,j+1)}$. Each of these functions is supported on an interval of length $\le 1$.
(ii) Because of the last fact, for each $j$, the functions  $h_j, h_j \circ T,\dots$ have non-overlapping supports, and so (that's the key):
$$\|S_n h_j\|^p = n*n^{-p} \|h_ j\|^p \quad \Rightarrow \quad \|S_n h_j\|= n^{1/p-1} \|h_j\|.$$
Thus,
$$\|S_n g_M \|=\|S_n \sum h_j\| \le\sum \|S_n h_j\|  \le n^{1/p-1} \sum_{j=-M}^{M-1} \|h_j\|.$$
Since $p>1$, $n^{1/p-1}\to 0$, and therefore we can choose $n$ large enough so that $\|S_n g_M\|\le \epsilon/2$.
$\Box$
Comment: Thanks! Maybe I'm misunderstanding here, but why are the $h_j=g \mathbf{1}_{[j, j+1)}$  rather than $h_j=f \mathbf{1}_{[j, j+1)}$?
Comment: right. corrected!


Format of $_POST variables into prepared statement

Question: This is probably a dumb question but if i am inserting data using MySQLi and prepared statements.
Is it possible to put in one line different conditions?
For example if text are Active change to Inactive and if text are Inactive change to Active. Is it possible to do in only one line?
$Status = ($_POST['status'] == "Inactive" ? "Inactive" : "Active");

Comment: Are you doing anything else in that query, or just toggling the active/inactive? Show your query, please.
Answer: If all you're doing is toggling the status between Active and Inactive, then you can do it in a single query, using CASE WHEN.
UPDATE myTable
SET status = (CASE WHEN status = 'Inactive' THEN 'Active'
                   ELSE 'Inactive'
              END)

WHERE id = ?

Comment: You can also use the `IF()` function which is often shorter: `IF(status='Inactive','Active','Inactive')`.
Comment: @Qirel thanks. I think is better to do this condicional with the update.
Answer: Both of the following should be legal:
$Status = ($_POST['status'] == "Inactive" ? "Inactive" : "Active");
$stmt->bind_param("s", $Status);

$stmt->bind_param("s", ($_POST['status'] == "Inactive" ? "Inactive" : "Active"));

WARNING : this 2nd version yields the "Only variables should be passed by reference" strict standards warning.  Consider your situation carefully when choosing this approach.

... But note that your expression,
($_POST['status'] == "Inactive" ? "Inactive" : "Active")

will yield Inactive when $_POST['status'] is already Inactive, which is the opposite of the desired behavior you described in your question.  I believe you want:
($_POST['status'] == "Inactive" ? "Active" : "Inactive")

EDIT 1
Regarding your comment question, "but, not's possible to put something else for example: if ($_POST['status'] == 'Inactive'){ $Status = ($_POST['status'] == "Inactive" ? "Inactive" : "Active"); else ...", something like that is legal, but doesn't make too much sense.  That literally is the same thing as:
if ($_POST['status'] == 'Inactive') {
    if ($_POST['status'] == "Inactive") {
        $Status = "Inactive";
    } else {
        $Status = "Active";
    } else {
        ...
}

Comment: Can I put this with if else statement for example in the same line?
Comment: @GuifIf : if I understand your question correctly, no : PHP will not evaluate an `if ... else` statement down to a value; `bind_param()` (like all functions) is expecting a set of values for its parameter list, so you can only use variables that contain a value or expressions that evaluate to a value
Comment: but, not's possible to put something else for example: ```if ($_POST['status'] == 'Inactive'){ $Status = ($_POST['status'] == "Inactive" ? "Inactive" : "Active"); else.```..
Comment: Note that your second snippet would yield warnings - "*Only variables should be passed by reference*" - https://stackoverflow.com/questions/23843383/mysqli-strict-standards-only-variables-should-be-passed-by-reference?rq=1
Comment: @GuifIf : the ternary operator (`condition ? true-result : false-result`) is already a kind of `if ... else` statement; it doesn't make much sense to combine an `if ... else` with a ternary evaluation the way that you have; I will edit my answer to illustrate ...
Comment: @Qirel : thanks!  I've added a cautionary note to that effect


Python: Converting python2 to python3

Question: I am trying to use
https://github.com/iandees/mongosm/blob/master/insert_osm_data.py
this package. It seems like it is written in Python2. I have converted all the way to next(context). However, I am getting name 'long' is not defined.
Is there any way that I can define this somewhere? How can I define 'long' and I have no idea what this is for even for Python2 Script (which worked fine somehow).
Answer: long() is basically renamend to int() in Python 3.
Please see https://www.python.org/dev/peps/pep-0237/ for details.
So, either do a search of long and replace with int, or define it
long = int

somewhere at the beginning of your file.
Answer: You should convert all code to Python3
https://www.google.com/search?channel=fs&client=ubuntu&q=Converting+python2+to+python3
https://docs.python.org/3/library/2to3.html

Python: Converting python2 to python3


Embedded image doesn't show up in email body

Question: I'm using nodemailer to send email with embedded images, but the images doesn't show up in message body in some email client app (eg thunderbird).
I'm suspecting this happened because of Content-Transfer-Encoding is set to quoted-printable. So it add 3D characters (which is encoding for = character) in src property of img element:

How to turn off quoted-printable encoding permanently in nodemailer ? I've tried to set textEncoding: 'base64' in message options, but it seems nodemailer ignore it.
let message = {
  from: {
    name: 'Someone',
    address: 'firstname.lastname@example.com'
  },
  to: {
    name: sender,
    address: emailTo.toLowerCase()
  },
  subject: 'Purchased Tickets',
  html: 'Some text

Some more text

',
  attachments: ticketList.map((t, i) => ({
    filename: `ticket${i + 1}.png`,
    content: t.qrCode,
    cid: `${email@example.com`
  })),
  encoding: 'base64',
  textEncoding: 'base64'
}

transporter.sendMail(message, (err, info) => {
  if (err) {
    console.log(err)
  } else {
    console.log(info)
  }
})

Answer: I have the same problem in some of my emails, but it does not appear to be related to Content-Transfert-Encoding.
Indeed, the images are displayed on some of my mails with
Content-Transfer-Encoding: quoted-printable

but not all...
Comment: I have tried it with some email client apps, and apparently some displays it and some don't.


CSS for 100% Height

Question: Edit: Answered. transform: scaleY() works acceptably well for my purposes. Sorry for the trouble!
I'm creating some code for a CSS animation triggered with JavaScript. This animation will be applied to a div element, and I want to know a way that I can input a value representing the full height of the div (div height will vary). I can trigger it just fine with the to value set to height: 100px, but that's not what I want.
@keyframes slideIn {
    from {height: 0;}
    to {height: 100%;}
}

Is there any way to represent the full height of an element in CSS?
Comment: you know that the parent needs to have a specified height for this to work? If the parent is body then you need to specify body's height to 100% as well (html tag also) > `html, body {height:100%;}`
Comment: You can do multiple ways to achieve the goal. Try using `flex` or `position: absolute; top: 0; bottom: 0`. otherwise animate the `max-height` property or set a fixed height for parent.
Comment: I'm inserting a div element into the body and I want it to slide in from `0` height to its full height.
Comment: Never mind, never mind, I'll just hack something together in a script tag.
Comment: By full height you mean `auto` which is not animatable, for that you can use the `max-height` property to animate the div where max-height would be a height greater than the maximal height a div will reach. have a look at [how to transition height from 0 to auto](http://stackoverflow.com/questions/3508605/how-can-i-transition-height-0-to-height-auto-using-css)
Comment: Also, what about trying the `scaleY()` transform property?
Comment: If it helps to see my code, here it is: https://www.khanacademy.org/computer-programming/simple-post-ui-w-markdown-work-in-progress/6647559692353536
Comment: Lines `27` and `295` are of interest.
Comment: It would help to create a simplified version of the code, not hundreds of lines most of which have nothing to do with the problem.
Comment: Anyway, the height of the body is not automatically the height of the window. If you want it to be, put `html, body {height:100%}` in your stylesheet.
Comment: That doesn't seem to make any difference.
Answer: The height of post div is not fixed, it's dynamic. 100% makes the div fill the total height of its parent. What you really mean is animating a div from 0 height to auto height, which is not animatable at the moment. You can try other approaches like max-height or scaleY() for a pure CSS solution. Alternatively you can run the calculations with JavaScript.
Max-height
@keyframes slideIn {
    from {max-height: 0px;}
    to {max-height: 1000px;}
}

Drawbacks

Transition time inconsistencies
Limits box height

Vertical scale
@keyframes slideIn {
    from {transform: scaleY(0);}
    to {transform: scaleY(1);}
}

Drawbacks

Does not "slide" the space around div
May override other transform effects
Comment: Ah, I see. Thank you for this, I'm very naive and I appreciate your help a lot. Sorry if I was trouble, the latter solution works well as long as the transition length is not too slow.
Answer: If you don't (want to) know about the element height up front and 'grow' it to become the height it would normally occupy, you cannot use height: 100% as this would make the element occupy 100% of its parent element height.
You can, however tell it may use N<unit> in size (where N<unit> is an arbitrary value like 10000px or 1000em, or whatever floats your boat).
@keyframes slideIn {
  from {max- height: 0;}
  to {max-height: 100%;}
}

But in order for this to work, you will also need some other properties set (outside the animation):
height: auto;
overflow: hidden;

@keyframes woot {
  from { max-height: 0; }
  to   { max-height: 100em; }
}

.example1 {
  max-height: 1em;
  overflow: hidden;
  transition: max-height 5s;
}
.example1:hover {
  max-height: 100em;
}

.example2 {
  overflow: hidden;
  animation: woot 5s infinite alternate;
}

     Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam et volutpat lectus, ac hendrerit dui. Donec a vehicula nunc. Integer eleifend blandit libero vitae porttitor. In auctor odio a diam commodo dignissim. Donec vitae nisi fermentum, hendrerit augue at, sagittis erat. Mauris at consectetur sem. Duis ligula metus, fermentum vel vulputate vitae, lacinia nec turpis. Sed nec nibh porttitor, congue neque nec, laoreet purus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Curabitur facilisis mi imperdiet dignissim suscipit. Proin dictum varius velit eu eleifend. Vivamus imperdiet eleifend tortor, sit amet mollis leo auctor a. Sed condimentum nisi tortor, quis interdum sapien dignissim sed. Sed interdum, justo a ultrices sodales, risus nunc ultricies eros, et tempor nisl elit sed eros.

     Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam et volutpat lectus, ac hendrerit dui. Donec a vehicula nunc. Integer eleifend blandit libero vitae porttitor. In auctor odio a diam commodo dignissim. Donec vitae nisi fermentum, hendrerit augue at, sagittis erat. Mauris at consectetur sem. Duis ligula metus, fermentum vel vulputate vitae, lacinia nec turpis. Sed nec nibh porttitor, congue neque nec, laoreet purus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Curabitur facilisis mi imperdiet dignissim suscipit. Proin dictum varius velit eu eleifend. Vivamus imperdiet eleifend tortor, sit amet mollis leo auctor a. Sed condimentum nisi tortor, quis interdum sapien dignissim sed. Sed interdum, justo a ultrices sodales, risus nunc ultricies eros, et tempor nisl elit sed eros.

There is one catch though, you will need to exaggerate the max-height value to the point where you are always certain the contents will fit, and it is the full value of said max-height that counts for the duration of the animation, not just the visible portion.

CSS for 100% Height


Como aumentar a quantidade de casas decimais depois da vírgula?

Question: Como eu posso fazer o campo exibir 0,0000 por exemplo ? Por padrão ele só seta 2 casas, não estou conseguindo aumentar.
Exemplo:
import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import javafx.scene.control.*;
import java.math.BigDecimal;

public class CampoMonetario extends TextField {

    public CampoMonetario() {
        setValor("0,00");
        //setStyle("-fx-text-fill: red;");

        lengthProperty().addListener(new ChangeListener<Number>() {
            @Override
            public void changed(ObservableValue<? extends Number> observable, Number oldValue, Number newValue) {
                String value = getText();
                value = value.replaceAll("[^0-9]", "");
                value = value.replaceAll("([0-9]{1})([0-9]{14})$", "$1.$2");
                value = value.replaceAll("([0-9]{1})([0-9]{11})$", "$1.$2");
                value = value.replaceAll("([0-9]{1})([0-9]{8})$", "$1.$2");
                value = value.replaceAll("([0-9]{1})([0-9]{5})$", "$1.$2");
                value = value.replaceAll("([0-9]{1})([0-9]{2})$", "$1,$2");
                setText(value);

                textProperty().addListener(new ChangeListener<String>() {
                    @Override
                    public void changed(ObservableValue<? extends String> observableValue, String oldValue, String newValue) {
                        if (newValue.length() > 16) {
                            setText(oldValue);
                        }
                        if (newValue.length() > 4) {
                            if (newValue.substring(0, 1).contains("0")) {
                                setText(newValue.substring(0, 1).replace("0", "") + getText().substring(1, getText().length()));
                            } else if (newValue.substring(0, 2).contains("0.")) {
                                setText(newValue.substring(0, 1).replace("0", "") + getText().substring(1, getText().length()));
                            }
                        }
                    }
                });
            }
        });

        focusedProperty().addListener(new ChangeListener<Boolean>() {
            @Override
            public void changed(ObservableValue<? extends Boolean> observableValue, Boolean out, Boolean in) {

                if ((new BigDecimal(getValor() + "0")).compareTo(BigDecimal.ZERO) <= 0 && in) {
                    setText("");
                } else if (getText().equals("") && out) {
                    setText("0,00");
                }
                final int length = (getText().length());
                if (length > 0 && length < 4) {
                    setText(getText() + "00");
                    lengthProperty();
                }
            }
        });
    }

    public void setValor(Object valor) {
        if (!(valor + "").contains("nul")) {
            setText(String.valueOf(valor).replace(".", ",").replace("", ""));
            requestFocus();
        } else {
            setText("0,00");
        }
    }

    public BigDecimal getValor() {
        return ((!getText().isEmpty()) ? new BigDecimal(getText().replace(".", "").replace(",", ".")) : BigDecimal.ZERO.setScale(2));
    }
}

Comment: Mas se a proposta do seu campo é ser um campo monetário duas casa é suficiente pois o limite de centavos é 99. Ou não?
Answer: Só aumentar o Scale do bigdecimal, como abaixo, por exemplo passando de um double pra um scale 12.
double d = ...
BigDecimal db = new BigDecimal(d).setScale(12, BigDecimal.ROUND_HALF_UP);

Comment: onde eu vou aplicar esse db ?


What to do with random effects correlation that equals 1 or -1?

Question: Not so uncommon occurrence when dealing with complex maximal mixed models (estimating all possible random effects for given data and model) is perfect (+1 or -1) or nearly perfect correlation among some random effects. For the purpose of the discussion, let's observe the following model and model summary
Model: Y ~ X*Cond + (X*Cond|subj)

# Y = logit variable
# X = continuous variable
# Condition = values A and B, dummy coded; the design is repeated
#             so all participants go through both Conditions
# subject = random effects for different subjects

Random effects:
 Groups  Name             Variance Std.Dev. Corr
 subject (Intercept)      0.85052  0.9222
         X                0.08427  0.2903   -1.00
         CondB            0.54367  0.7373   -0.37  0.37
         X:CondB          0.14812  0.3849    0.26 -0.26 -0.56
Number of obs: 39401, groups:  subject, 219

Fixed effects:
                 Estimate Std. Error z value Pr(>|z|)
(Intercept)       2.49686    0.06909   36.14  < 2e-16 ***
X                -1.03854    0.03812  -27.24  < 2e-16 ***
CondB            -0.19707    0.06382   -3.09  0.00202 **
X:CondB           0.22809    0.05356    4.26 2.06e-05 ***

The supposed reason behind these perfect correlations is that we have created a model which is too complex for the data that we have. The common advice that is given in these situations is (e.g., Matuschek et al., 2017; paper) to fix the overparameterized coefficients to 0, because such degenerate models tend to lower the power. If we observe a marked change in fixed effects in a reduced model, we should accept that one; if there is no change, then there is no problem in accepting the original one.
However, let's presume that we are not interested only in fixed effects controlled for RE (random effects), but also in the RE structure. In the given case, it would be theoretically sound to assume that Intercept and the slope X have non-zero negative correlation. Several questions follow:

What to do in such situations? Should we report the perfect correlation and say that our data is not "good enough" to estimate the "real" correlation? Or should we report the 0 correlation model? Or should we maybe try to set some other correlation to 0 in hope that the "important" one will not be perfect anymore? I don't think there are any 100% correct answers here, I would mostly like to hear your opinions.
How to write the code which fixes the correlation of 2 specific random effects to 0, without influencing the correlations between other parameters?
Comment: A radical alternative is to regularize the model, i.e. fit a Bayesian model with somewhat informative priors on the random effects structures (e.g. via `blme`, `MCMCglmm`, `rstanarm`, `brms` ...)
Comment: Package nlme gives you fine control regarding the variance-covariance matrix of random effects. I've never really needed this myself, but I would reread *Mixed-Effects Models in S and S-PLUS* (Pinheiro and Bates, 2000) if I did.
Comment: @BenBolker Ben. I am not sure that is a radical idea, as fitting an unregularized model might be the more radical way to fit a model ;)
Comment: Thank you guys all for great answers... Unfortunately, I was offline for couple of days, but I am back.
Answer: Singular random-effect covariance matrices
Obtaining a random effect correlation estimate of +1 or -1 means that the optimization algorithm hit "a boundary": correlations cannot be higher than +1 or lower than -1. Even if there are no explicit convergence errors or warnings, this potentially indicates some problems with convergence because we do not expect true correlations to lie on the boundary. As you said, this usually means that there are not enough data to estimate all the parameters reliably. Matuschek et al. 2017 say that in this situation the power can be compromised.
Another way to hit a boundary is to get a variance estimate of 0: Why do I get zero variance of a random effect in my mixed model, despite some variation in the data?
Both situations can be seen as obtaining a degenerate covariance matrix of random effects (in your example output covariance matrix is $4\times 4$); a zero variance or a perfect correlation means that the covariance matrix is not full rank and [at least] one of its eigenvalues is zero. This observation immediately suggests that there are other, more complex ways to get a degenerate covariance matrix: one can have a $4\times 4$ covariance matrix without any zeros or perfect correlations but nevertheless rank-deficient (singular). Bates et al. 2015 Parsimonious Mixed Models (unpublished preprint) recommend using principal component analysis (PCA) to check if the obtained covariance matrix is singular. If it is, they suggest to treat this situation the same way as the above singular situations.
So what to do?
If there is not enough data to estimate all the parameters of a model reliably, then we should consider simplifying the model. Taking your example model, X*Cond + (X*Cond|subj), there are various possible ways to simplify it:

Remove one of the random effects, usually the highest-order correlation:
X*Cond + (X+Cond|subj)

Get rid of all the correlation parameters:
X*Cond + (X*Cond||subj)

Update: as @Henrik notes, the || syntax will only remove correlations if all variables to the left of it are numerical. If categorical variables (such as Cond) are involved, one should rather use his convenient afex package (or cumbersome manual workarounds). See his answer for more details.
Get rid of some of the correlations parameters by breaking the term into several, e.g.:
X*Cond + (X+Cond|subj) + (0+X:Cond|subj)

Constrain the covariance matrix in some specific way, e.g. by setting one specific correlation (the one that hit the boundary) to zero, as you suggest. There is no built-in way in lme4 to achieve this. See @BenBolker's answer on SO for a demonstration of how to achieve this via some smart hacking.

Contrary to what you said, I don't think Matuschek et al. 2017 specifically recommend #4. The gist of Matuschek et al. 2017 and Bates et al. 2015 seems to be that one starts with the maximal model a la Barr et al. 2013 and then decreases the complexity until the covariance matrix is full rank. (Moreover, they would often recommend to reduce the complexity even further, in order to increase the power.) Update: In contrast, Barr et al. recommend to reduce complexity ONLY if the model did not converge; they are willing to tolerate singular covariance matrices. See @Henrik's answer.
If one agrees with Bates/Matuschek, then I think it is fine to try out different ways of decreasing the complexity in order to find the one that does the job while doing "the least damage". Looking at my list above, the original covariance matrix has 10 parameters; #1 has 6 parameters, #2 has 4 parameters, #3 has 7 parameters. Which model will get rid of the perfect correlations is impossible to say without fitting them.
But what if you are interested in this parameter?
The above discussion treats random effect covariance matrix as a nuisance parameter. You raise an interesting question of what to do if you are specifically interested in a correlation parameter that you have to "give up" in order to get a meaningful full-rank solution.
Note that fixing correlation parameter at zero will not necessarily yield BLUPs (ranef) that are uncorrelated; in fact, they might not even be affected that much at all (see @Placidia's answer for a demonstration). So one option would be to look at the correlations of BLUPs and report that.
Another, perhaps less attractive, option would be to use treat subject as a fixed effect Y~X*cond*subj, get the estimates for each subject and compute correlation between them. This is equivalent to running separate Y~X*cond regressions for each subject separately and get the correlation estimates from them.

See also the section on singular models in Ben Bolker's mixed model FAQ:

It is very common for overfitted mixed models to result in singular fits. Technically, singularity means that some of the $\theta$ (variance-covariance Cholesky decomposition) parameters corresponding to diagonal elements of the Cholesky factor are exactly zero, which is the edge of the feasible space, or equivalently that the variance-covariance matrix has some zero eigenvalues (i.e. is positive semidefinite rather than positive definite), or (almost equivalently) that some of the variances are estimated as zero or some of the correlations are estimated as +/-1.
Comment: @RockyRaccoon Yes, I think it's better to use/report the estimated correlation parameter but here we are talking about the situation when we arguably *cannot* estimate it because it converges to 1. That's what I would write in a paper: "The full model converged to solution with corr=1 so following advice in [citations] we used a reduced model [details]. The correlation between random effect BLUPs in this model was 0.9." Again, when you are not including the correlation, you are not constraining the model to treat them as uncorrelated! You are simply not *modeling* this correlation explicitly.
Comment: What my example shows is that for `(Machine||Worker)` `lmer` estimates one variance more than for `(Machine|Worker)`. So what `lmer` does for `||` with factors cannot be described by 'this only removes correlations between factors, but not between levels of a categorical factor.' It alters the random effects structure in a somewhat weird way (it expands `(Machine||Worker)` to `(1|Worker) + (0+Machine|Worker)`, hence the additional variance). Feel free to change my edit. My main point is that in this statements the distinction between numerical and categorical covariates needs to be made clear.
Comment: No, also does not work with binary variables, see for yourself: `machines2 <- subset(Machines, Machine %in% c("A", "B"));
summary(lmer(score ~ Machine + (Machine  || Worker), data=machines2))`. It in general does not work with factors due to this expansion and the way `R` deals with factors in `model.matrix`.
Comment: @amoeba: I think you made an interesting point by suggesting to turn to `ranef` values for studying the correlation between random effects. I am not too deep into this topic, but I know that it is usually not recommended to work with extracted values of `ranef`, but rather with estimated correlations and variances. What's your opinion on that?

Plus, I don't know how one would explain to the reviewer that the correlation in the model was not postulated, but we still calculate the correlation of the extracted values. That doesn't make sense
Comment: I have one more question: do close-to-zero variances and perfect and close-to-perfect correlations of random effects imply something about the real value of the parameters? For example, do -1 correlations imply that the real correlation is at least negative and/or that it is at least non-zero? More concretely, if we try to estimate the correlation which is 0 in reality, is it possible that we would get -1 estimation?
Comment: @RockyRaccoon I would expect the optimization algorithm to get stuck at the boundary solution of -1 correlation if real correlation is strong and negative (i.e. relatively close to -1). Similarly to how I would expect to see random variance estimated as 0.0 if the real variance is small.
Comment: Ok, that would also be my guess... Bates et al. (2015; https://arxiv.org/abs/1506.04967) mention on p. 4 regarding the +-1 correlation or 0-variance estimations:  "...This corresponds to estimates of zero random-effects variance
in a model with random-intercepts only or a correlation of ±1 in a model with correlated random
intercepts and slopes. However, already a three-by-three correlation matrix will not usually show
boundary values like these, even when it is singular." Do they suggest here that boundaries can be around some value other than 1, or 0 for variances? It's not so clear...
Comment: Did you read the 3rd paragraph of my answer? That's *exactly* what I was talking about there.
Comment: Sorry, my mistake. Was reading it some time ago and I just commented as I was reading Bates et al. (2015).
Comment: Also, is there any other way of "testing" whether a covariance matrix is degenerate (singular), apart from the mentioned PCA methodology from Bates et al (2015)? I am talking about the situations where you have 4x4 or larger matrices, where degeneration does not have to yield +/-1 correlation or 0 variance.
Comment: @RockyRaccoon Why do you need another way? You need to extract covariance matrix with `VarCorr()` and then get eigenvalues with `eigen()`. It's 1 line of code.
Comment: Indeed, you are right.
Comment: One question from a bit different perspective: I got the impression that the greatest problem with singular fits, +/-1correlation, overfitted models etc. is that fixed effects test lose power. What if the maximal model yielded all significant fixed effects. Would you say that then it is justified to keep the maximal model, although it is overfitted, i.e. it has singular fit?
Answer: I agree with everything said in amoeba's answer which provides a great summary of the current discussion on this issue. I will try to add a few additional points and otherwise refer to the handout of my recent mixed model course which also summarizes these points.

Suppressing the correlation parameters (options 2 and 3 in amoeba's answer) via || works only for numerical covariates in lmer and not for factors. This is discussed in some detail with code by Reinhold Kliegl.
However, my afex package provides the functionality to suppress the correlation also among factors if argument expand_re = TRUE in the call to
mixed() (see also function lmer_alt()). It essentially does so by implementing the approach discussed by Reinhold Kliegl (i.e., transfomring the factors into numerical covariates and specify the random-effects structure on those).
A simple example:
library("afex")
data("Machines", package = "MEMSS") # same data as in Kliegl code

# with correlation:
summary(lmer(score ~ Machine + (Machine  | Worker), data=Machines))
# Random effects:
#  Groups   Name        Variance Std.Dev. Corr
#  Worker   (Intercept) 16.6405  4.0793
#           MachineB    34.5467  5.8776    0.48
#           MachineC    13.6150  3.6899   -0.37  0.30
#  Residual              0.9246  0.9616
# Number of obs: 54, groups:  Worker, 6

## crazy results:
summary(lmer(score ~ Machine + (Machine  || Worker), data=Machines))
# Random effects:
#  Groups   Name        Variance Std.Dev. Corr
#  Worker   (Intercept)  0.2576  0.5076
#  Worker.1 MachineA    16.3829  4.0476
#           MachineB    74.1381  8.6103   0.80
#           MachineC    19.0099  4.3600   0.62 0.77
#  Residual              0.9246  0.9616
# Number of obs: 54, groups:  Worker, 6

## as expected:
summary(lmer_alt(score ~ Machine + (Machine  || Worker), data=Machines))
# Random effects:
#  Groups   Name         Variance Std.Dev.
#  Worker   (Intercept)  16.600   4.0743
#  Worker.1 re1.MachineB 34.684   5.8894
#  Worker.2 re1.MachineC 13.301   3.6471
#  Residual               0.926   0.9623
# Number of obs: 54, groups:  Worker, 6

For those not knowing afex, the main functionality for mixed models is to provide p-values for the fixed effects, e.g.,:
(m1 <- mixed(score ~ Machine + (Machine  || Worker), data=Machines, expand_re = TRUE))
# Mixed Model Anova Table (Type 3 tests, KR-method)
#
# Model: score ~ Machine + (Machine || Worker)
# Data: Machines
#    Effect      df        F p.value
# 1 Machine 2, 5.98 20.96 **    .002
# ---
# Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

summary(m1)
# [...]
# Random effects:
#  Groups   Name         Variance Std.Dev.
#  Worker   (Intercept)  27.4947  5.2435
#  Worker.1 re1.Machine1  6.6794  2.5845
#  Worker.2 re1.Machine2 13.8015  3.7150
#  Residual               0.9265  0.9626
# Number of obs: 54, groups:  Worker, 6
# [...]

Dale Barr from the Barr et al. (2013) paper is more cautious in recommending reducing the random-effects structure than presented in amoeba's answer. In a recent twitter exchange he wrote:

"reducing the model introduces unknown risk of anticonservativity, and should be done with caution, if at all." and
"My main concern is that people understand risks associated with model reduction and that minimizing this risk requires a more conservative approach than is commonly adopted (eg each slope tested at .05)."

So caution is advised.

As one of the reviewers I can also provide some insight on why we the Bates et al. (2015) paper remained unpublished. Me and the other two reviewers (which signed, but will remain unnamed here) had some criticism with the PCA approach (seems unprincipled and there is no evidence that it is superior in terms of power). Furthermore, I believe all three criticized that the paper did not focus on the issue of how to specify the random-effects structure, but also tries to introduce GAMMs. Thus, the Bates et al (2015) paper morphed into the Matuschek et al. (2017) paper which addresses the issue of the random-effects structure with simulations and the Baayen et al. (2017) paper introducing GAMMs.
My full review of the Bates et al. draft can be found here. IIRC, the other reviews had kind of similar main points.
Comment: @amoeba It is important to note that the approach using `dummy()` only works with the default treatment contrasts and not when the random-effects use sum-to-zero contrasts (which one should use in case the model has interactions). You can e.g., see that if you compare the variance components in the example above for the `lmer_alt` call with the `mixed` call.
Comment: OK. Then I might insert some small edits/updates in it to clarify some of the points that you are making. Regarding Bates preprint it might very well be suboptimal in various respects. But I do fully agree with Bates et al. that singular covariance matrices are *exactly the same problem* as the correlations of +1/-1. Mathematically, there is just no difference. So **if** we accept that perfect correlations compromise power, **then** we must be very wary of the singular cov. even in the absence of explicit simulations showing it. I disagree that it's "unprincipled".
Comment: @amoeba `lmer_alt` basically works exactly like `lmer` (or even `glmer`) with the only difference that it allows the `||` syntax. So I am not sure why you would want to avoid `afex` at all costs. It should even work without attaching (i.e., `afex::lmer_alt(...)`).
Comment: @amoeba What it does is basically the approach described in the code by Reinhold Kliegl (i.e., expanding the random effects). For each random effects term of the formula it creates a model matrix (i.e., converts the factors into numerical covariates). This model.matrix is then `cbind` to the data. Then the random-effects term in the formula are replaced with a new one in which each of the newly created columns is concatenated with a +. See lines 690 to 730 in https://github.com/singmann/afex/blob/master/R/mixed.R
Comment: Regarding categorical variables to the left of `||`, this is a really important point, thanks for bringing it up and explaining it to me (I edited my answer to reflect it). I like this functionality of `lmer_alt` in `afex`. I will just mention here for completeness that to get the same output with vanilla `lmer` call without any additional preprocessing one can e.g. specify `(1+dummy(Machine,'B')+dummy(Machine,'C') || Worker)`. This clearly becomes very cumbersome when categorical variable has many levels.
Comment: That's a good point.
Comment: @Henrik: Thank you for complementing the answer of @amoeba. I think it added the info about the `afex` functionality. I will accept the answer from @amoeba because everything relevant is summarized there.
Comment: @Henrik: I got the following error: `Error in model.matrix.default(tmp_random[[i]], data = data) :
  model frame and formula mismatch in model.matrix()
In addition: Warning messages:
  Due to missing values, reduced number of observations to 22211
  Due to missing values, set_data_arg set to FALSE.` Do you know what is the problem? I was trying to model GLMM with `afex`
Comment: @RockyRaccoon That sounds like a bug that I have never seen. Please post a reproducible example to either github (https://github.com/singmann/afex/issues) or the afex forum (http://afex.singmann.science/) or send it via mail. Otherwise I will be unable to help.
Comment: Henrik, it would be great to have your input here: https://stats.stackexchange.com/questions/345842. It's about how to specify models with no correlations between levels of a categorical factor, so directly related to this answer and to `lmer_alt`. I have just posted an answer in that thread and would much appreciate if you could take a look. Cheers.
Answer: I too have had this problem when using maximum likelihood estimation - only I use Goldstein IGLS algorithm as implemented through the MLwiN software and not LME4 in R. However in each and very case the problem has resolved when I have switched to MCMC estimation using the same software. I have even had a correlation in excess of 3 which resolved when I changed estimation. Using IGLS, the correlation is calculated post estimation as the covariance divided by the product of the square root of the product of the associated variances - and this does not take account of the uncertainty in each of the constituent estimates.
The IGLS software does not 'know' that the covariance implies a correlation and just calculates estimates of a constant, linear, quadratic etc variance function. In contrast the MCMC approach is built on the assumption of samples from a multivariate normal distribution which corresponds to variances and covariances with good properties and full error propogation so that the uncertainty in the estimation of the covariances is taken into account in the estimation of the variances and vice versa.
MLwin beings the MCMC estimate chain with IGLS estimates and the non-negative definite variance covariance matrix might need to be altered by changing the covariance to zero at the outset before starting the sampling.
For a worked example see
Developing multilevel models for analysing contextuality, heterogeneity and change using MLwiN 3, Volume 1 (updated September 2017); Volume 2 is also on RGate
https://www.researchgate.net/publication/320197425_Vol1Training_manualRevisedSept2017
Appendix to Chapter 10


Url and HTML inspect gives different outcomes with BeautifulSoup

Question: When I copy the url of a facebook page and create a BeautifulSoup object, it gives me a text that is not actually the posts on the pages. Namely
text = requests.get('https://www.facebook.com/toyota').text
soup = BeautifulSoup(text, 'lxml')
soup.get_text()

returns '\n\n\n\n\n\n\n\n\n\n\n\nToyota USA - Ana Sayfa\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n'.
However, when I inspect that Facebook page and copy HTML element and follow similar steps, I get what I want. So
html_inspected = Copied HTML Element
soup = BeautifulSoup(html_inspected)
soup.get_text()

returns the actual text on the Facebook page I want. My question is am I supposed to inspect and copy the HTML every time I want to get the actual content in a page? Isn't there any shortcut for getting posts and comments on a Facebook page without inspecting every time?
Comment: Check your `soup` if you find expected elements - website deals with `javascript` so you should not find any rendered information.
Comment: First be aware that Facebook doesn't allow you to scrape them. The short cut is to use Facebook API which is the allowed way to get data from Facebook
Answer: As pointed out by @HedgeHog, this could be a JavaScript  issue.
The simplest solution would be to use a ready-made scraper library for the task:
from facebook_scraper import get_posts

for post in get_posts('toyota', pages=1):
    print(post['text'][:50])

Alternatively, you could use Selenium:
from selenium.webdriver import Firefox
from selenium.webdriver.common.by import By

with Firefox() as driver:
    driver.get('https://www.facebook.com/toyota')
    elem = driver.find_elements(By.XPATH, '//div[@dir="auto"]')

    for item in elem:
        print(item.text)

Comment: Are you really sure? Could it not be that there is no other content delivered, cause website deals with `javascript`?
Comment: Yes, that's perfectly possible.
Comment: I'll update my answer.
Comment: I'm aware of facebook_scraper, but the problem with that library is that it scrapes posts from the latest to the older ones. So, when I want to extract information about posts between 2010 and 2020,  facebook_scraper starts scraping from 2022 regardless, and then it automatically stops scraping for some reason before even coming to posts in 2020. Another downside is that facebook_scraper takes a lot of time to scrape a page.
Comment: If that's the case, then, maybe scraping with Selenium would be a better choice.


First steps learning to predict financial timeseries using machine learning

Question: I am trying to get a grasp on how to use machine learning to predict financial timeseries 1 or more steps into the future.
I have a financial timeseries with some descriptive data and I would like to form a model and then use the model to predict n-steps ahead.
What I have been doing so far is:
getSymbols("GOOG")

GOOG$sma <- SMA(Cl(GOOG))
GOOG$range <- GOOG$GOOG.High-GOOG$GOOG.Low

tail(GOOG)

           GOOG.Open GOOG.High GOOG.Low GOOG.Close GOOG.Volume GOOG.Adjusted     sma range
2013-05-07    863.01    863.87   850.67     857.23     1959000        857.23 828.214 13.20
2013-05-08    857.00    873.88   852.91     873.63     2468300        873.63 834.232 20.97
2013-05-09    870.84    879.66   868.23     871.48     2200600        871.48 840.470 11.43
2013-05-10    875.31    880.54   872.16     880.23     1897700        880.23 848.351  8.38
2013-05-13    878.89    882.47   873.38     877.53     1448500        877.53 854.198  9.09
2013-05-14    877.50    888.69   877.14     887.10     1579300        887.10 860.451 11.55

Then I have fitted a randomForest model to this data.
fit <- randomForest(GOOG$GOOG.Close ~ GOOG$sma + GOOG$range, GOOG)

Which seems to fit surprisingly well:
> fit

Call:
 randomForest(formula = GOOG$GOOG.Close ~ GOOG$sma + GOOG$range,      data = GOOG)
               Type of random forest: regression
                     Number of trees: 500
No. of variables tried at each split: 1

          Mean of squared residuals: 353.9844
                    % Var explained: 97.28

And tried to use it to predict:
predict(fit, GOOG, n.ahead=2)

But this prediction ofc did not work.
I try to predict the Close, should I lag the other variables by as many steps as I want the prediction, before fitting the model?
Probably a lot of other stuff I should take into account as well but these are really my first steps trying out machine-learning.
Answer: My first observation is that you did not lag the inputs relative to the closing price and that is why you observed such good fit. The SMA (simple moving average) uses the closing price in its calculation and the high low range encompasses the closing price, so using them to predict the closing price imparts a look ahead bias. My opinion is that if you are trying to predict the closing price two days ahead you should build your model with inputs that are lagged from the closing price by at least two days. Some of the inputs may be lagged by more than two days, but I would start simple and try and use just a handful of inputs.
As far as your objective to predict closing price, I think that closing prices are too noisy to be used as target variables and using them will lead to overfitting or optimization of the wrong objective. Instead I would start by smoothing the closing price with a moving average and then predicting the direction of price change over the next two days. For example i might replace the close with a 5 day SMA of the close and then code the price change of the SMA as 1 if it was positive over the next two days and 0 otherwise. Because the output variable is now coded as a 1 or a 0 this is a good problem to try and solve with the random forest function you were using. You could also try some other classification algoritms like logistic regression, neural networks, and SVMs and maybe combine a few into an ensemble to improve your performance. This is still a difficult problem to solve without overfitting, but it is a step in the right direction. Another word of caution is that your final model could have amazing accuracy at classifying the next two days as either positive or negative, but still lose money because it classified a few large moves incorrectly.
I would also recommend building your model on more than one security so that the machine learning algorithm does not hone in on the idiosyncrasies of one stock. I would start with at least 5 stocks that are not highly correlated to eachother.
Trading on the Edge by Guido Deboeck is a good place to start for exploring the applications of machine learning to financial time series prediction. It's an older book so it is way behind the technology we have available today but it is a good start. I would also recommend New Trading Systems and Methods by Kaufman and Expert Trading Systems by John Wolberg.
Comment: Hi, created some features and made a model: http://pastie.org/7958695 I think I got the mechanics figured out. This model is however all too curvefitted. Have you done anything similar and had any real use of it? Would be cool to talk more.
Comment: I'm always open to more detailed conversations. My contact info is on my profile. Shoot me an e-mail if you would like.
Comment: Did some simple regressions to predict returns which should work ok, compared to predicting a price-value. Besides predicting return have to predict a large enough return to beat spread&commish etc. Would be cool to talk more, couldnt find your mail in your profile. Mine is in the pastie I posted in the previous comment.
Comment: ok, thank you for this good answer.
The example was more for me to understand how to set up the data so randomForest can use it.
But now I know I need to lag the inputs to get it working.

Would it make any sense to have the price of the SMA in your example as output variable, or would it have to be as 0 or 1? \n

Thanks also for the book recommendations will familiarize myself with them.
Comment: You could definitely use the price of the SMA as the output variable; the problem just becomes more complex. Instead of predicting a direction you are now trying to predict an exact price value. In my experience this is a very difficult thing for a machine algorithm to do well out-of-sample. But if you are just familiarizing yourself with the process of setting up time series data for prediction this may be a good place to start, especially if it is more intuitive to you than directional prediction. You could also try and predict the price at which two moving averages will cross.


Do the rows used in row operations during LU factorisation matter?

Question: A method I have seen for finding the LU factorisation of a matrix is that U is the row echelon form of A. The row operations we perform on A to get to U must involve replacing $R_i$ by $R_i - kR_j$ where $R_i$ denotes the ith row. The matrix L keeps track of these multipliers k.
I am trying to find an LU factorisation for the following matrix:

We know that L will take the following form:

The rest of L will be populated by the multipliers used in transforming A to row echelon form U.
We can find the row echelon form of A in the following method.

The multiplier used to obtain a zero in row 2, column 1 was k = 1.5. So, row 2, column 1 of L should be 1.5.
The multiplier used to obtain a zero in row 3, column 1 was k = 2. So, row 3, column 1 of L should be 2.
The multiplier used to obtain a zero in row 3, column 2 was k = -2. So, row 3, column 2 of L should be -2.
So, L should be:

Overall, we should be able to write A as:

However, this is not correct! Actually doing this matrix multiplication does not give the right answer for A
If, instead, the first row operation I perform is $R_2 \to R_2 - 3R_1$, I don't run into the problem I just described, and the matrix multiplication works out fine:

So, my question is, do the row operations I use in finding the row echelon form of A affect the correctness of LU factorisation?
Or, have I just made a silly mistake somewhere above?
Answer: When making the LU-decomposition you should always use the number on the diagonal to make the numbers underneath it zero.
In your case, instead of using $R_3$ to eliminate $R_2$, you should have used $R_1$, since the $2$ is on the diagonal. This would result in $R_2\leftarrow R_2-3R_1$.
Sometimes the number on the diagonal (also called the pivot), is zero. In this case you should multiply $A$ with a permutation matrix which switches two rows. Then you would get $PA=LU$.
Comment: Thanks for your reply! Does this rule apply to general Gaussian elimination/row echelon form problems too, or just row reduction to row echelon form in the context of LU decomposition?
Comment: This is only necessary with LU decomposition, gaussian elimination can be done with any row.
Comment: Perfect thanks for clearing that up.


Assume that $f \in L^1(\mathbb{R})$ $ g(x)=e^{2 i \pi x}$ compute $ f*g$

Question:
Assume that $f \in L^1(\mathbb{R})$, compute $ f*g$
  with $ g(x)=e^{2 i \pi x}$

Thm1 let $f \in L^1(R)$ and $g\in C^p(R)$. Assuming $g^k$ is bounded for $k=0,1,\dots ,p$
1) $f*g \in C^p(R)$
2) $(f*g)^k=f*g^k$  for $k=1,\dots ,p$
The composition
$$ (f*g) = \int f(t-\tau)y (\tau)d \tau$$
def of $L^1(R)$ is space meausrable where
$$ \int |f(t)|dt < \infty $$
know that $$ e^{i2 \pi x} = \cos(2 \pi x )+ i\sin(2 \pi x)$$
convolution is commutative $ f*g =g*f.$

$$\begin{aligned}
(f*g) &= \int e^{i 2 \pi (t-\tau) } f(\tau) d \tau = ?
\end{aligned} $$
want to say it is $(e^{i2 \pi} *f)$ distorting the first theorem but x is not an integer . Feel like doing countour integration in complex analysis but idk
Answer: $$(f*g)(t) = \int_{\Bbb R} e^{i 2 \pi (t-\tau) } f(\tau) d \tau
      = e^{i 2 \pi t }  \int_{\Bbb R}  e^{-i 2 \pi\tau } f(\tau) d \tau = e^{i 2 \pi t }\hat f(1) $$
where the Fourier transform is given by $$\hat f(x)=  \int_{\Bbb R}  e^{-i 2 \pi x\tau } f(\tau) d \tau$$