Including files in Clojure without Leiningen: Could not locate file

Question: Trying to do a simple clojure program without that includes a function defined in another file. It is unable to find the file in the same directory.
helloInclude
    src
        main.clj
        importer.clj

main.clj
(ns main
  (:gen-class)
(:require [importer]))

(defn -main
  [& args]
  (println "Hello, World!"))

(-main)
(println (func 5 6))

importer.clj
(ns importer
  (:gen-class))

(defn func
    [a b]
    (+ a b 1))

Executed from within the src directory with:
java -jar "Path\To\Clojure\clojure-1.8.0\clojure-1.8.0.jar" -i main.clj

Returns error:
Exception in thread "main" java.io.FileNotFoundException: Could not locate importer__init.class or importer.clj on classpath., compiling:(C:\Users\jamesjenkinson\Clojure\helloInclude\src\main.clj:1:1)
        at clojure.lang.Compiler.load(Compiler.java:7391)
        at clojure.lang.Compiler.loadFile(Compiler.java:7317)
        at clojure.main$load_script.invokeStatic(main.clj:275)
        at clojure.main$init_opt.invokeStatic(main.clj:277)
        at clojure.main$init_opt.invoke(main.clj:277)
        at clojure.main$initialize.invokeStatic(main.clj:308)
        at clojure.main$null_opt.invokeStatic(main.clj:342)
        at clojure.main$null_opt.invoke(main.clj:339)
        at clojure.main$main.invokeStatic(main.clj:421)
        at clojure.main$main.doInvoke(main.clj:384)
        at clojure.lang.RestFn.invoke(RestFn.java:421)
        at clojure.lang.Var.invoke(Var.java:383)
        at clojure.lang.AFn.applyToHelper(AFn.java:156)
        at clojure.lang.Var.applyTo(Var.java:700)
        at clojure.main.main(main.java:37)
Caused by: java.io.FileNotFoundException: Could not locate import__init.class or import.clj on classpath.
        at clojure.lang.RT.load(RT.java:456)
        at clojure.lang.RT.load(RT.java:419)
        at clojure.core$load$fn__5677.invoke(core.clj:5893)
        at clojure.core$load.invokeStatic(core.clj:5892)
        at clojure.core$load.doInvoke(core.clj:5876)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5697)
        at clojure.core$load_one.invoke(core.clj:5692)
        at clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
        at clojure.core$load_lib.invokeStatic(core.clj:5736)
        at clojure.core$load_lib.doInvoke(core.clj:5717)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:648)
        at clojure.core$load_libs.invokeStatic(core.clj:5774)
        at clojure.core$load_libs.doInvoke(core.clj:5758)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:648)
        at clojure.core$require.invokeStatic(core.clj:5796)
        at clojure.core$require.doInvoke(core.clj:5796)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at src.main$eval3$loading__5569__auto____4.invoke(main.clj:1)
        at src.main$eval3.invokeStatic(main.clj:1)
        at src.main$eval3.invoke(main.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:6927)
        at clojure.lang.Compiler.eval(Compiler.java:6916)
        at clojure.lang.Compiler.load(Compiler.java:7379)
        ... 14 more

Also tried explicitly calling classpath to be the current directory with:
java -classpath . -jar "Path\To\Clojure\clojure-1.8.0\clojure-1.8.0.jar" -i main.clj

I am unable to use Leiningen for this. How should I properly reference the file?
Edit:
Changed namespaces to top level.
Comment: try removing "src." from both namespaces' declarations. (if src is your root folder) else, if "helloInclude" is root, rewrite `(require [import])` to `(require [src.import])`
Comment: src is the root folder. Removed the `src` from both declarations, still doesn't work.
Comment: you have a wrong namespace syntax. try to rewrite it like this: main: `(ns main (:gen-class) (:require [import]))` , import: `(ns import (:gen-class))` . Also i would rename `import` to something else, since there is `import` in clojure already
Comment: See edits, still not being found.
Comment: there is no "root folder" at all here, because no directory on disk is on the classpath
Answer: I hope I'm getting you correctly and we have the similar problem which I seem to have solved. My current directory structure is (forgive the humorous nature of things):
.
├── chikaka
│   ├── main.clj
│   └── tools
│       └── dog.clj
├── core.clj
└── deps.edn

The key for me was deps.edn file which had the following contents:
{:paths ["."]}

Source: https://clojure.org/reference/deps_and_cli#_paths
When you add paths to the deps file, each is added the to glorious CLASSPATH that Java likes so much. Then you will have requires like this working for example:
(ns chikaka.main
  (:require [chikaka.tools.dog :refer [bark]]))

Answer: require only finds files that are under your classpath. If you want to be able to use require, the path to your code relative to the classpath has to match the package and name of the namespace. The -cp arg to java can add directories to the classpath, and the current directory is not implicitly in the classpath.
If the file is just import it must be directly in the top directory, and its package must be changed so it is a top level namespace. To require it with its current location and package, you must use src.import instead of import.
Also, top level namespaces are not recommended.
Finally, the proper option to run a -main is -m.
+$ ls
import.clj  main.clj
+$ cat import.clj
(ns import)

(defn foo
  []
  (println "hello"))
+$ cat main.clj
(ns main
  (:require [import])
  (:gen-class))

(defn -main
  [& args]
  (import/foo))
:$ java -cp ~/bin/clojure.jar:. clojure.main -m main
hello

Comment: This is being executed from the `src` folder. I've changed the namespaces  to remove the `.src`, still with no luck.
Comment: Also tried executing from the parent folder, both with main in `src` and with main in the parent, with the appropriate namespaces. Also unable to find the file.
Comment: also, the file name must match the namespace - if the ns is `import` the file must be `import.clj` not `importer.clj`
Comment: Yeah, I changed it from import due to leetwinski's comment about import already being a word in clojure. I forgot to edit the namespace on the snippet, but had done it in code. Still no success.
Comment: wait, your current directory isn't even in the classpath
Comment: @JamesJenkinson I have updated the answer to show a working version of what you are attempting, and describing the remaining issues.
Comment: On further reflection - I recommend using `load-file` if you are interested in having a simple top level file file that simply runs with top level side effects. Constructs like `require`, `ns`, `gen-class` and setting the classpath are all designed for reusable and architected code, and are not needed when using `load-file`. Either use `ns` and the classpath and `require` etc. with a proper non-top-level namespace, and ensure you have no top level side effects in your files, or just use `load-file`, but there is no need to mix these approaches.


Como deixar a janela com fundo transparente no Kivy

Question: Olá, eu estou tentando fazer um programa que crie uma janela sem bordas e com o fundo transparente.
A primeira parte já consegui:
Window.borderless = True

Mas ainda não consegui deixar o fundo transparente. Alguém sabe como fazer isso?
Answer: Vitor utiliza essa função de exemplo:
from kivy.config import Config

Config.set('graphics', 'position', 'custom')

Config.set('graphics', 'fullscreen', 'fake')

Config.set('graphics', 'top', '0')

Config.set('graphics', 'left', '0')

Config.set('graphics', 'resizable',  True)

from kivy.app import App

from kivy.uix.label import Label

from kivy.core.window import Window

Window.size = (1000, 500)

from kivy.uix.widget import Widget

from kivy.uix.boxlayout import BoxLayout

from kivy.uix.textinput import TextInput

from kivy.uix.button import Button

from kivy.uix.floatlayout import FloatLayout

from kivy.graphics import Color, Ellipse, Line,Rectangle

from kivy.uix.screenmanager import ScreenManager, Screen

from kivy.uix.relativelayout import RelativeLayout

from kivy.uix.image import Image

import win32gui

import win32con

import win32api,pywintypes

import win32gui

import win32con

import win32api,pywintypes

class Program(App):

    def on_start(self):

        hwnd = win32gui.FindWindow(None, "pencere")

        win32gui.SetWindowLong(hwnd, win32con.GWL_EXSTYLE,win32gui.GetWindowLong(hwnd,win32con.GWL_EXSTYLE) | win32con.WS_EX_LAYERED | win32con.WS_CHILD | win32con.WS_EX_TRANSPARENT |win32con.WS_EX_TOPMOST)

        win32gui.SetLayeredWindowAttributes(hwnd, win32api.RGB(255, 0, 128),40, win32con.LWA_ALPHA)

    def build(self):

        sm = ScreenManager()

        duzen = FloatLayout()

        screen = Screen(name='deneme ekran 1')

        self.title = "pencere"

        yazi = Label(text = self.title,pos=(10,10),size=(100,35),size_hint=(None,None),halign="center",bold=True,color=(1,1,1,1))

        buton = Button(text="",size_hint=(None,None),size=(26,26))

        buton.pos = (0,0)

        buton.background_normal = "minimize.png"

        buton.background_down = "minimize.png"

        with Window.canvas:

            Color(0,1,0,1)

            Rectangle(pos=(10, 10), size=(100, 40))

            duzen.add_widget(yazi)

        return duzen

Program().run()

Usei isso quando comecei, retirei isso do reddit.
Segue referência:
Link
Comment: Vou ver se consigo fazer alguma coisa coisa com isso... De qualquer forma, obrigado pela ajuda!


Select all element within specific class if its not found in a selector

Question: I want to set a width of span element within 'select2-container' class if its outside any table
I try with this code but it doesn't work it will select all span elements even if it's inside a table.
body *:not(table) .select2-container {
    width: 100% !important;
}

I also try with body *:not(td) .select2-container {... but it also not worked.
Answer: you need to redifine class one more time,
.select2-container {
    width: 100% !important;
}

table .select2-container {
    width: unset;
}

Comment: It doesn't work the select2 make to element a fixed width at creating span.
so at making  width unset will ignore the constant width.


Replace note name with variable - lilypond

Question: nl = \break % it's true, ok!

a = \axyz % this is wrong with output time adds 1 2 4 8 16 ..
\axyz16 \axyz4 % etc.

Is it possible to build variable to replace notes? Maybe with Scheme?
I would like to markup some notes within info like this:
a = \az^\markup \translate #'(-1 . 0) "+5"
b = \bz^"+12"

etc.
Please any ideas?
Something with Scheme function like
(lambda (la-la? (x? y?))) newLa

Comment: I'm somehow having trouble trying to understand what you're asking. Could you include some more specifics?
Comment: I just search possibllity make variable for note names or chord names to use in score.
If I write \var16 or \var4 or \var[time] this is not avaliable usally
Comment: [link](http://images.vfl.ru/ii/1646228937/ef822c6b/38285254.png)
Comment: Did I get you right? You need to store a chord and reuse it with different durations.
Maybe all you need is the shorthand (q) for repeating a chord?
4 q8 q8
You could also write a lilypond-function using \shiftDurations.
Comment: \shiftDurations - Thanks! I will study that. q - I know
Answer: If I understand you correctly you want to be able to have a variable with music and potentially some additional stuff like TextScripts.
You can use functions like this
var =
#(define-music-function (dur) (ly:duration?)
   #{
     a $dur ^\markup \translate #'(-1 . 0) "+5"
   #})

\displayMusic {
  \var4 \var2 \var8. \var16
}

The one disadvantage of this is that you cannot use this without specifying a duration.
You can also use music function to replace certain notes by different music like this:
mapMyNote =
#(define-music-function (music) (ly:music?)
   (define (walk music)
     (let* ((elts (ly:music-property music 'elements))
            (elt (ly:music-property music 'element)))
      (if (music-is-of-type? music 'note-event)
          (if (= (ly:pitch-notename (ly:music-property music 'pitch)) 5) ; a
              #{
                \transpose #(ly:make-pitch -1 0) #(ly:make-pitch (ly:pitch-octave (ly:music-property music 'pitch)) 0)
                 $(ly:music-property music 'duration) ^"+5"
              #}
              (begin
               (if (not (null? elt))
                   (ly:music-set-property! music 'element (walk elt)))
               (ly:music-set-property! music 'elements (map walk elts))
               music))
          (begin
           (if (not (null? elt))
               (ly:music-set-property! music 'element (walk elt)))
           (ly:music-set-property! music 'elements (map walk elts))
           music))))
    (walk music))

\mapMyNote { a a' }

EDIT: @semiono Aha. Seeing this is about microtonality: Something like this might also be useful:
t=
#(define-event-function (ct) (number?)
   #{
   -\tweak self-alignment-X #CENTER
   -\tweak parent-alignment-X #CENTER
   -\tweak font-size #-3.4
   -\markup \bold #(format #f (if (integer? ct) "~@d" "~@f") ct)
   #})

{ c' d' e'\t2 f'^\t-2 g'\t2.3 }

Comment: [link](https://images.vfl.ru/ii/1646256558/38f5f094/38291067.png)
Really useful for me! I do microtonal experiments and put the value of cents in the mark :)
Comment: Thank you again! But e\t2 or e^"+2" practically does not reduce the text except for alignment. Originally I did this aa\aax ab\abx with aax = \aa translate ~"+5" etc.
Nevertheless Lisp/Scheme great tool :)
Comment: @semiono Something like this can be useful if you need a) a larger amount of cent values or b) if you ever want to change the formatting of these.


uploaded image convert to small image with php

Question: Hello i have an issue about php. In my php files i have 1 part for image upload.
It also translates and uploads the image I uploaded in the same small size. But the reduction does not work for the .png extension while working for the .jpg extension.
<code>$uploaddir = "storage/products/screenshot/"; //Screenshot upload directory

$valid_formats = array("jpg","jpeg","png"); // Valid image formats

$countScreen = 0;
foreach ($_FILES['item_screen']['name'] as $name => $value) {
    $filename = stripslashes($_FILES['item_screen']['name'][$name]);

    $size = filesize($_FILES['item_screen']['tmp_name'][$name]);
    //Convert extension into a lower case format
    $ext = getExtension($filename);
    $ext = strtolower($ext);
    //File extension check
    if (in_array($ext, $valid_formats)) {

        if($ext=="jpg" || $ext=="jpeg" )
        {
            $uploadedfile = $_FILES['item_screen']['tmp_name'][$name];
            $src = @imagecreatefromjpeg($uploadedfile);
        }
        else if($ext=="png")
        {
            $uploadedfile = $_FILES['item_screen']['tmp_name'];
            $src = @imagecreatefrompng($uploadedfile);
        }
        else
        {
            $src = @imagecreatefromgif($uploadedfile);
        }

        list($width,$height)=getimagesize($uploadedfile);

        $newwidth=800;
        $newheight=($height/$width)*$newwidth;
        $tmp=imagecreatetruecolor($newwidth,$newheight);

        $newwidth1=222;
        $newheight1=($height/$width)*$newwidth1;
        $tmp1=imagecreatetruecolor($newwidth1,$newheight1);

        @imagecopyresampled($tmp,$src,0,0,0,0,$newwidth,$newheight,$width,$height);
        @imagecopyresampled($tmp1,$src,0,0,0,0,$newwidth1,$newheight1,$width,$height);

        $random1 = rand(9999, 100000);
        $random2 = rand(9999, 200000);
        $image_name =  $first_title . '_' . $random1 . $random2 . '.' . $ext;
        $image_name1 = 'small_' .$first_title . '_' . $random1 . $random2 . '.' . $ext;

        //$newname = $uploaddir . $image_name;

        $filename = $uploaddir . $image_name;
        $filename1 = $uploaddir . $image_name1;
        $pic_name = $image_name1;

        @imagejpeg($tmp,$filename,100);
        @imagejpeg($tmp1,$filename1,100);

        @imagedestroy($src);
        @imagedestroy($tmp);
        @imagedestroy($tmp1);

        //Moving file to uploads folder
        if (!move_uploaded_file($_FILES['item_screen']['tmp_name'][$name], $filename)) {
            $errors[]['message'] = $lang['ERROR_UPLOAD_IMG'];
        }
        if ($countScreen == 0)
            $image_name2 = $image_name;
        elseif ($countScreen >= 1)
            $image_name2 = $image_name2 . "," . $image_name;

    } else {
        $errors[]['message'] = $lang['ONLY_JPG_ALLOW'];
    }
    $countScreen++;
}
</code>
Github gist: https://gist.github.com/barangndz/e911ec01494bcfc594145c9edf3611d6
Comment: 1. What is your question 2. Please read: [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve)
Comment: Perhaps you missed `[$name]` in `imagecreatefrompng` case.
Answer: I think you need to try with this method:
<code>if($ext=="jpg" || $ext=="jpeg" || $ext=="png" )
{
    $uploadedfile = $_FILES['item_screen']['tmp_name'][$name];
    $src = @imagecreatefromjpeg($uploadedfile);
}
else
{
    $src = @imagecreatefromgif($uploadedfile);
}
</code>
Comment: It's work thank you. I try this method befora except deleting this:

        else if($ext=="png")
        {
            $uploadedfile = $_FILES['item_screen']['tmp_name'];
            $src = @imagecreatefrompng($uploadedfile);
        }
Comment: Welcome, if you feel great, please vote me and follow my website: http://datainflow.com/programming/


Analyzing clustered (repeated measures) data with very few clusters

Question: What is the most appropriate method for analyzing clustered data (i.e., repeated measurements nested within subjects) when there are very few clusters (e.g., 3 subjects)?
Here's a brief description of the study:

There are 3 subjects, each experiencing 5 conditions.
Within each condition, there are 12-15 repeated measures.
The measures are: the number of responses on two alternatives (B1 and B2) and number of reinforcers earned for each type of response (R1 and R2).

The question is how subjects allocate responses to the two alternatives (B1 and B2) based on the number of reinforcers earned under those alternatives (R1 and R2).
The image below provides examples of some possibilities.
Comment: I don't understand your research question. Please can you explain the study design and describe all of the variables. What is the outcome/response/dependent variable, and what are the independent variables, and how are they all measured ?
Comment: The dependent variable is the number of responses (B1=number of responses on option 1, B2=number of responses on option 2). The independent variable is the number of reinforcers delivered for each response option. In the five conditions, reinforcement (food) is concurrently available on the two options, but the schedule of reinforcement for each response differs across conditions. In the control condition, the schedule of reinforcement is the same for both options. In the other conditions, reinforcement is available more frequently for one option (e.g., every 12s for B1 vs. every 60s for B2).
Comment: How are B1 and B2 related ? Are these seperate outcomes ? If you had more subjects what kind of model would you fit ?
Comment: B1 and B2 are separate outcomes. B1=moving to one location and B2=moving to a second location. I think a linear mixed-effects model might be suitable if there were more subjects.
Answer: From the description in the question and the comments, it appears that a mixed effects model with random intercepts for subjects would be the preferred analysis model for these data. However, given the small number of subjects, this is not appropriate within the usual frequentist paradigm. So, I would suggest fitting fixed effects for subjects (ie just a linear model, within random effects), which will also control for the non-independence of observations within subjects.
I might also consider a Bayesian mixed model approach with random effects for subjects if I knew what intra-class correlation was likely, in order to set a reasonable prior on the variance of the random intercepts for subject (this might be available from prior studies).
Comment: Thank you, Robert. Regarding your suggestion to fit fixed effects for subjects, are you suggesting fitting a separate model for each subject? How does that control for the non-independence of observations within subjects?
Comment: No. I mean including subject as a fixed effect instead of a random effect. Both methods control for non independence but when the number of subjects is high it is preferable to use random effects.


Compare growth rate for different sized samples

Question: I have 11 years of membership data for a denomination of churches and I want to find the top 20 growth rate "trend breakers".
In my current method I have:

calculated % change from the previous year for all churches
calculated 3yr, 5yr and 10yr averages
determined the overall mean for each of these
filtered out below avg scores for 10y, 5y, and then 3y

This stil leaves me with a list of more than 20.
I have two questions:

Is there a more scientific way to identify the top twenty trendbreakers?
On what basis can I filter out small churches with explosive but short-term growth that inflates their growth average?
Answer: It's good that you already recognize that growth can more easily be explosive off a small baseline.
I'd recommend that you start with a plot. Plot growth rates on the vertical axis (best to use geometrical means of year-over-year growths, which is just the 10th root of the overall 11-year growths for those churches where you have data for all 11 years) against initial membership on the horizontal axis. Then you can remove smaller churches based on the plot and finally pick the top 20 of the remainder.


Problemas ao criar um evento

Question: Estou tentando criar um evento mas obtenho o erro abaixo:

E minha query:
CREATE EVENT insert_count
ON SCHEDULE
EVERY 50 SECOND
COMMENT 'bla bla bla bla bla bla.'
DO BEGIN
INSERT INTO teste (nome, conectado, desconectado)
SELECT parceiro.nomeP AS 'Parceiro',
SUM(di.connectedOr = 'true') AS 'ClientesConectados',
SUM(di.connectedOr = 'false') AS 'ClientesDesconectados'
FROM vpn AS di
JOIN parceiro
ON parceiro.codParceiro = di.idpartner
WHERE di.comName REGEXP 'skytef-'
GROUP BY di.idpartner
END

Answer: Conforme documentação sobre Create Event, alterei seu scripts adicionando o comando Delimiter, pois assim temporiamente o MySql ignora o ;, para executar o comando em um bloco, algo assim.
Mais detalhes consulte Defining Stored Programs.
Espero que te ajude.
delimiter |
CREATE EVENT insert_count
    ON SCHEDULE
       EVERY 50 SECOND
    COMMENT 'bla bla bla bla bla bla.'
    DO
    BEGIN
      INSERT INTO teste (nome, conectado, desconectado)
        SELECT parceiro.nomep AS 'Parceiro'
              ,SUM(di.connectedor = 'true') AS 'ClientesConectados'
              ,SUM(di.connectedor = 'false') AS 'ClientesDesconectados'
          FROM vpn AS di
          JOIN parceiro
            ON parceiro.codparceiro = di.idpartner
         WHERE di.comname regexp 'skytef-'
         GROUP BY di.idpartner;
    END |

delimiter;

Comment: Erro SQL (1064): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'delimiter' at line 1

Problemas ao criar um evento


Programatically accessing block settings

Question: Is it possible in Drupal 7 to load block visibility settings programatically in module? I'm rendering block programatically and I want to include checking those settings, so it still will be possible to hide block from UI. Property of "status" and "visibility" are not enough, since it's possible that the block won't be placed in any region at all.
Answer: Sure, block_load gives you the contents of the whole row from the block table:
$block = block_load('module', 'delta');

$region = $block->region;
$weight = $block->weight;
$custom = $block->custom;
$visibility = $block->visibility;
$pages = $block->pages;
$title = $block->title;

And so on.
Comment: This is the only thing this object contains:
`bid -> 187,
  module -> block,
  delta -> 20,
  theme -> galeco,
  status -> 0,
  weight -> 0,
  region -> ,
  custom -> 0,
  visibility -> 0,
  pages -> ,
  title -> ,
  cache -> -1,
  i18n_mode -> 1`

Visibility and Status values are already computed, so if the block is turned off they will give me 0, even if the rest of visibility settings state that the block should be displayed. That's why I neeed granural acess to settings. I stated that  status and visibility properties are not enough in my initial post.
Comment: What are _the rest of the visibility settings_? Do you mean "Roles" and "Content type"? Those are in the `block_role` and `block_node_type` tables respectively
Comment: By visibility settings I mean visibility settings as shown here http://prntscr.com/hzdjku The actual settings, not computed value whether block should be displayed or not. In this case I'm mostly interested in block_roles if that's what stores data from Roles tab in Visibility settings. Is there a function to load it per block, or should I just get it from database by query?
Comment: Just get it from the DB, there's no helper in the api


Final step in proving that the only invariant subspaces of $\mathbb{R}^2$ are $\mathbb{R}^2$ and $0$

Question: There's this example in my book that goes as follows:
Let $T$ a linear operator on $\mathbb{R}^2$ represented in the standard ordered basis by the matrix $$\begin{bmatrix}0&-1\\1&0\end{bmatrix}$$
Then the only subspaces of $\mathbb{R}^2$ which are invariant under $T$ are $\mathbb{R}^2$ and $0$. For any other invariant subspace $W$ would necessarily have dimension 1. But if $W$ is the subspace spanned by some non-zero vector $\alpha$ then the statement that $W$ is invariant under $T$ means that $T\alpha=c\alpha$ for some real number $c$.
So far so good, but here comes the step I don't understand.
But this is impossible with $\alpha\neq0$ because one can easily verify that for any $c$ the operator $(T-cI)$ is invertible.
My question is, what does it matter that $T-cI$ is invertible, and why does that make things impossible? Additionally, are there alternative ways to prove such a statement?
Comment: Do you know about eigenvalues?
Comment: I have an _extremely limited_ knowledge of them. I'm able to recognise what they are when explained but have never worked with them. That being said, the next paragraph goes in to "characteristic values" which seem suspiciously similar.
Comment: They're the same thing. "Characteristic value" is a translation of the term. I asked because if you know about them, it's obvious how invertibility matters, but it sounds like the discussion in your question leads to that, so you can't really use that.
Comment: Viewed differently, $T$ is a rotation of the plane by 90 degrees. A subspace is a line through the origin. When you rotate a line by 90 degrees, it cannot be mapped to itself! Do observe that this argument works only in 2D. In 3D a rotation would be about an axis, and the axis is surely mapped to itself no matter how much you rotate.
Answer: $Tα=cα$ implies $Tα-cα=0$, i.e. $(T-cI)α=0$. Now if $T-cI$ is invertible, we can just multiply the last equation from the left with the inverse of $T-cI$ and we get $α=0$.
Answer: There is an elementary alternative proof. Suppose $T(\alpha) = c \alpha$. Write $\alpha = (a, b)$. Then you find that $-b = ca$ and $a = cb$. The only real possibility is $a = b = 0$.


AWS Fargate + Application Load Balancer SSL Termination

Question: I'm trying to configure ECS Fargate behind an Application Loader Balancer (ELBv2), and I would like to terminate the TLS/SSL connections on the ALB, and send HTTP traffic (port 80) to the Fargate images, which listen on port 80.
This is the diagram of what I'm trying to do:

I am currently able to send HTTP traffic from the internet to Fargate, but I have not found a way to terminate the TLS connection at the Application Loader Balancer and forward HTTP traffic to ECS. Is this possible?
I am hosting a fairly static application with no sensitive user data,
hence I am not particularly worried about data in transit between the
ALB and ECS.

Any help appreciated!
Answer: Yes it is possible.

Create a SSL certificate for your domain in ACM (Amazon Certificate Manager).
Create a HTTPS Listener in the ALB that will listen on port 443 and configure it to use the above SSL Certificate.
Open the ALB Security Group to permit inbound traffic on port 443.

That's it, pretty much. The ALB must have one Listener per port, so if you are accepting traffic on both HTTP/80 and HTTPS/443 you'll need 2 listeners.
The Target Group doesn't change, it will still connect to your Fargate containers over HTTP.
Hope that helps :)
Comment: As an aside, you can also configure the ALB to redirect any traffic arriving on 80 to go to 443: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#redirect-actions
Comment: @MarkEntingh Are you sure that the the return traffic goes thriugh NAT and not through the ALB? That'd be quite unusual, perhaps if you're using some obscure protocol over multiple ports that may be the case. Normal HTTPS or WebSocket traffic simply goes back through the same ALB and is therefore TLS protected between the client and ALB in both directions.
Comment: What about encrypting the response back to the client via TLS? How would you set that up? In my case, the ALB is on a public subnet, ECS is on a private subnet, and my NAT sends traffic from ECS back to the client


String from address in VB.NET

Question: In my VB.NET application I use a function from a .dll that shall give me a string.
When calling the function the Callback gives me the lenght of that string and an address instead of the string itself.
I tried to get the string with this code but I think it was meant for VB classic, not for VB.NET:
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByVal Destination As Object, ByVal Source As Object, ByVal Length As Long)
Private Declare Function lstrlenA Lib "kernel32" (ByVal lpString As Long) As Long

Private Function pvToString(ByVal lPtr As Long) As String
    If lPtr <> 0 Then
    pvToString = String(lstrlenA(lPtr), 0)
    Call CopyMemory(ByVal pvToString, ByVal lPtr, Len(pvToString))
    End If
End Function

I get 4 Errors and I don't know how to get rid of them:

'.' expected (for line 5 for String)
'String' is a class type and can not be used as an expression (for line 5)
Expression expected (for line 6 for ByVal)
Expression expected (for line 6 for ByVal)

Sorry, if my question is stupid. I'm a starter in programming and VB.NET
Comment: What I can help you with: Remove the `ByVal`s.
Comment: Full example of how to use this is here: http://www.pinvoke.net/default.aspx/urlmon/CopyMemory.html
Comment: After looking at a VB6 documentation, the `String()` function just creates a string filled with the specified amount of a certain character. You should be able to remove this line without any problem.
Answer: First about errors:
1 and 2:  use new String(' ',len) instead.
3 and 4: remove call and ByVal
and all Long variables and argument must be declare as integer expect lstrlen that should be declare as IntPtr.
anyway you should forget all. if you have a pointer (IntPtr) and you want to convert it to string, depending to your string type you can easily use:
    dim myString as string= System.Runtime.InteropServices.Marshal.PtrToStrXXXX(MyPSZString)
Comment: I used your Marshal suggestion. Now I get Chinese characters instead of Roman characters. That's quite weird because the source string is in Roman letters.:D Any ideas?
Comment: I used Marshal.PtrToStrAuto()
Comment: Did `PtrToStrAuto` resolved problem? try BStr and Uni versions. can you insert your unmanaged function signature?
Comment: PtrToStrAuto didn't solve the problem. Same thing with Uni. BStr gives me an error saying "You tried to read or write in protected memory. Mostly this is a hint that other memory is spoilt." (translated from German into Englisch)
Comment: The Documentation about the function I'm trying to use says this:
"BOOL XC_CommGetProjectNameFromPLC(ULONG ulChannel )

Function: Get the projectname from plc. If there is no project loaded  the funtions returns 1 in Arg3 . The stringpointer in Arg4 points to a 0x00.
Returnwert: TRUE  if Success
FALSE if not STATE_IDLE or COMM_BAD_CHANNEL
Callback: Arg1: STATE_PINF
  Arg2: Channelnumber
  Arg3: Stringlength +1  (terminating 0)
  Arg4: Pointer to String
"
Comment: So I did this:

    Public Shared Function XC_CommGetProjectNameFromPLC(ByVal ulChannel As Integer) As Integer
    End Function
Answer: Looks like your declaration should be something like
Private Declare Function lstrlenA Lib "kernel32" (ByVal lpString As Long) As String

instead of
Private Declare Function lstrlenA Lib "kernel32" (ByVal lpString As Long) As Long

Sorry have no access to visual studio right now to test this. But looks like this is where it is going wrong.
PS: declaration might be wrong for the other function too. try removing ByVal
Comment: why -vote? This might be a better answer for you. http://stackoverflow.com/questions/3921852/how-to-get-the-string-from-an-address-specified-by-a-long-in-vb
Comment: You probably got a downvote because what you're saying is incorrect. Infact it should actually be `Integer`: http://www.pinvoke.net/default.aspx/kernel32.lstrlen - **Also**, the link you gave him is almost exactly the same code that he uses, meaning it won't work.
Comment: Also, `ByVal` doesn't make the declaration incorrect. If you don't use `ByVal` it will be added by default by the compiler. I think those two points explain the downvote.

String from address in VB.NET


non-normal residuals in ARIMA

Question: I am trying to fit an ARIMA model and I have already evaluated a few variations which I finally selected ARIMA(1,1,3) model. The residuals seems to be uncorrelated and all the lags are significant. However, in this model and even in all the others I tried, the normality condition for residuals is always violated and they look like this when plotted against normal distribution.
Should I transform my data somehow? I have already used natural log and first differencing in order to make the data stationary or can I ignore the assumption when I have a lot of observations (1,5M) ?
Comment: I have read in some works, that the normality of the residuals is imortant so that the t-statistics of the AR and MA terms are valid. As I understood, if the residuals wont be normally distributed I may mistakenly exclude some significant term or include some insignificant one. But I am not sure and therefore I came here to ask :)
Comment: I found two conflicting opinions here on Cross Validated: [Aksakal says](http://stats.stackexchange.com/questions/96598/what-if-the-residual-series-from-arima-not-looks-normal/96602#96602) that MLE is not reliable if the normality assumption is violated. [Rob J. Hyndman says](http://stats.stackexchange.com/questions/79400/does-arima-require-normally-distributed-errors-or-normally-distributed-input-dat/79793#79793) that non-normality is not that big of a problem. I am sure Rob J. Hyndman is very experienced and authoritative, I just wonder what the explanation for his observation is.
Comment: @ChristophHanck, your statements about asymptotics make sense in a linear regression setting. However, ARIMA models may be more sensitive to non-normality since the normality assumption is used in the maximum likelihood estimation (but not in OLS) and the ARIMA likelihood is quite complicated. Could you give an explanation or a reference supporting your opinion? I share the doubts of m3d1v0...
Comment: @RichardHardy, I'd first say it sure is true that asymptotics are often better guide in finite samples when errors are normal than if they come from, say, some weird skewed distribution. Second, I meant to refer to that MLEs are often also quasi-MLEs in that they often consistently estimate parameters of interest (with also a large-sample normal approximation) even if the likelihood is misspecified in that a wrong error distribution is chosen. A reference would be [Fuller (1996)](http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471552399.html), Theorem 8.4.1 (p. 432).
Comment: I understand that. My main worry is whether quasi-MLE works for ARIMA. As you just noted, *MLEs are* ***often*** *also quasi-MLEs*, so I wonder if ARIMA belongs to the case where the normal MLE is a quasi-MLE. Would only the true erorr distribution matter when answering this question (I suspect a skewed error distribution would be a killer), or does it also depend on the model (like ARIMA, multiple linear regression etc)?
Comment: You are right about arIma, and indeed Fuller's theorem is about stationary time series. At least, White's (1982 Ecma) paper does not cover this case directly, but then, his are not necessary conditions. In short, I do not know. My hunch would be that it should also work, given that coefficients are typically estimated superconsistently in nonstationary time series models, but that really is just a guess.
Comment: Actually, I was referring to stationary ARMA as well (the integratedness was not my concern, although perhaps it should have been?). Would you say (or at least guess) that MLE should work fine as a quasi-MLE when applied on a stationary ARMA?
Comment: I got a short comment by Rob J. Hyndman where he says *MLE with a Gaussian likelihood is asymptotically equivalent to least squares estimation. Minimizing the sum of squared errors should work ok for almost all distributions* [here](http://stats.stackexchange.com/questions/79400/does-arima-require-normally-distributed-errors-or-normally-distributed-input-dat/79793?noredirect=1#comment274002_79793). So perhaps you were right all along.
Comment: Why do you expect that errors should be normally distributed, whether or not the other checks you mention look good? There is no particular reason they should be.
Comment: Under very strict conditions, normality of the errors is important to get that t-statistics are indeed t-distributed. These are not met here anyhow, so that you need to rely on asymptotics to get a distribution for your t-statistics. Now, if you rely on asymptotics, normality of the errors is not (that) important anymore.
Answer: Your QQplots could indicate $t$-distributed error terms might fit better. You could try to fit an ARIMA-model with $t$-distributed innovation terms, and see if the fit is very different from the fit you have now. I have done such things with the bugs software, there are certainly other options.
Answer: Before you do an ARIMA model you have to check if the data is be stationary and if any seasonality should be defined using autocorrelation (ACF) and partial correlation functions(PACF). The auto correlation should follow the 95% confidence bands. Stationary data is detected using a run sequence plot or auto correlation.
If it is not stationary you might have to detrend it. My guess is it was not stationary.
Comment: The wording of the answer may be a bit confusing, so let me make a couple of remarks. (1) Statistically significant ACF or PACF values do not mean that the underlying series is nonstationary. (2) There are multiple different forms of nonstationarity, and different methods to detect them. (Augmented) Dickey-Fuller test mentioned in the comment above is one option. Interpreting autocorrelations is less straightforward.
Comment: Yes, that is exactly what I did. First I took a log transform because I am using exchange rate data and I wanted to smoothen a bit the excesive pikes. Then I run a Dickey-Fuller test which suggested that my series is not stationary, hence I differenced it by 1 and then it looked much beter visually and it passed the Dickey-Fuller. I also tried to regres the differenced values on various trends(linear,quadratice...) but all options turned to be insignificant. Therefore I think my data are stationary. Moreover, when I was evaluating the ARIMAs, there was no autorroelation in the residuals.
Comment: @m3d1v0 One doesn't/shouldn't take logs to "smoothen a bit excessive peaks" that is the role of pulse Indicators.. This post speaks to the role of power transforms of which a log is one choice http://stats.stackexchange.com/questions/18844/when-and-why-to-take-the-log-of-a-distribution-of-numbers . You have to carefully select the appropriate remedy for the problem at at hand. Transformations are like drugs ... some are good for you and some are not.
Answer: If the residuals contain pulses or level shifts this can lead to "non-normality" . Try detecting Interventions and add them as necessary. Another way residuals can exhibit non-normality is if there is a deterministic change in error variance suggesting Weighted Least Squares OR if the model's parameters are not constant over time suggesting data segmentation..

non-normal residuals in ARIMA


How do I use FDR in this situation?

Question: Firstly, I am not a statistician so apologies if attempt of a solution is ridiculous.
I am trying to come up with an algorithm that estimates a vector $v$ thats contaminated with noise
Suppose we have vector $v$ (which has entries $1, \ldots, l$) of $0's$ and $1's$. I want this function to be independent of how $v$ is distributed so no inference of its distribution should be used, but for now lets assume its uniform (also each entry of the vector is independent)
Now I make $n$ copies of this vector and then add Gaussian noise with mean 0 and variance 1 to each copy (independently) lets call them $x_1, \ldots x_n$. I want to use $x_1 \ldots x_n$ to try and recover $v$
I want to try and estimate the original using false discovery rate but I'm not quite sure how I would apply it to this situation
Suggest I'm trying to estimate the $i$th entry of $v$. According to wikipedia (BH procedure) I would have a collection of null hypothesis, I'm a little unconfident in thinking they're
$H_1 =$ the $i$th entry is $0$ with added Gaussian noise of mean $0$
$H_2 = $ the $i$th entry is $1$ with added Gaussian noise of mean $0$
Then wikipedia (Benjamini–Hochberg procedure) says I should have some $p$-values, but now I'm a little lost, p-values corresponding to what data? My best guess is that they're
$p_1$ is the probability of getting data at least as extreme as $\{x_i(1), \ldots,x_i(l)\}$ under the assumption of $H_1$
$p_2$ is the probability of getting data at least as extreme as $\{x_i(1), \ldots,x_i(l)\}$ under the assumption of $H_2$
(I think I would use a t-test to calculate these p-values with dgrees of freedom $n-1$?)
Now I am meant to order them, for simplicity lets assume they're already ordered
Lets set false discovery rate $\alpha = 0.05$.
Suppose $k=1$ is the largest $k$ such that $P_k \leq \frac km \alpha = 0.025k$ and so then I reject $H_1$. Is it now a fair to say that my best guess at $v(i)$ is that it is $1$?
What should I infer if all or none $k$ are such that $P_k \leq \frac km \alpha$
Does this look correct to you?
I appreciate any feedback
Thanks
Comment: @ssdecontrol I shall discuss it with him, I agree, FDR isn't the best technique here. Either way it was my first time using FDR and a learnt a lot trying to use it!
Comment: "I am trying to come up with an algorithm that estimates a vector  thats contaminated with noise" -- sounds like a statistician to me!
Comment: I held off writing a more complete answer because it's a little unclear to me what your data and analysis goal are. Let $v$ be a deterministic mapping $v: \{1,\dots,l\} \rightarrow \{0, 1\}$, and you observe $v + N$ where $N \sim Gaussian(0, 1)$ and want to recover $v$, correct? And for each element $i$ of $v$ you conduct an hypothesis test of $H_0:\ v_i = 0$ and $H_A:\ v_i = 1$?
Comment: @ssdecontrol Yes that's correct $B$ is a vector known to contain $0$'s and $1$'s and I observe $B+G$, I want to estimate $B$ using FDR. It's part of a supervised side project I am working on and my supervisor advised me to use FDR. I, like you, am struggling to work out how it can be applied to help me in this situation :(
Comment: I edited more detail into my comment
Comment: Correct, I can observe $v+N$, and I observe it $n$ independent times. Yes that's right, they're the hypothesis's I came up with (I couldn't think of any others)
Comment: Why are you trying to use FDR? Is there a particular reason? There are other, more established, methods of matrix recovery.
Comment: @Chris yes and I've used many other methods to do so. I've basically been given a list of methods to experiment with, this was one of them
Comment: @Henry, Ah, I just read your comment above; my apologies.
Comment: @Henry, is $n$ potentially large in size, or is it limited?
Comment: @ChrisC I have had some success with $n \approx 10$ with other methods, and what I've been using while trying to come up with something using FDR
Comment: @HBeel did you ask your supervisor for clarification? Maybe you can mention how most FDR control algorithms aren't really equipped for this problem
Answer: Here's my shot, it might be incorrect, so hopefully more people will chime in.
Given a $g\times1$ vector $v$ : {$1, ..., g$}  $\to$ {$1, 0$}, and an error vector $e$ ~ $Z(0,1)$, creating $n$ copies of $v+e$ results in a $g \times n$ matrix named $V$.
We want to recover $v$, knowing the distribution of $e$.  Each row $i$ of $V$ either has true $\mu = 0$ or $\mu = 1$, which we can assess by a simple $Z$ test for $\hat{\mu} = 0$ because we know the distribution of the noise.
We are interested solely in true differences larger than 0, so we decide to do a one-tailed Z test.  This makes:
$H_0:$ $\mu_i = 0$
$H_1:$ $\mu_i > 0$
Setting $\alpha = 0.05$, critical $z = 1.6449$ for a one tailed $Z$ test.

Computing a $Z$ test P value for each row $V_i$ gives us a $g \times 1$ vector of P values, of which we must correct for multiple testing through FDR.
Because we set $\alpha = 0.05$, the chance of truly accepting the null is 0.95 for one test, and 0.95$^n$ for $n$ uncorrelated tests.  Obviously this must be corrected for.  One method would be Bonferroni, another would be FDR.  Bonferonni correction controls family wise error rate, wheras FDR controls the false discovery rate.
Ordering your P values (but obviously maintaining their index values) and allowing for false discovery rate = 0.05, you would declare a test a significant result if $P_i < \frac{i}{g} \alpha$. This would mean that the effect overcomes the multiplicity (with 5% error).  If you have no values with $P_i < \frac{i}{g} \alpha$, it means that none of your values have been succesfully recovered.
However, the original one tailed $Z$ test only has $\beta - 1 = 0.26$ at $\alpha = 0.05$, so only $\approx$ 26% of your true 1 values can be recovered.  If $g$ is small, then perhaps noise overwhelmed signal in all your cases.
There are definitely better ways, but I hope this helped a little bit.
Comment: The difficulty I have with this answer, and indeed with the whole problem, is that I can't seem to think of how one might tighten $H_1$ from a range (including an infinity of values we don't care about) to a point, the single value we do care about. The BH procedure, in my understanding, is fundamentally "Fisherian" in that is works by manipulating the magnitude of p values, while the problem is fundamentally "Neymanian-Pearsonian."
Comment: It is an explicit comparison of two well-defined hypothesis that are complements in an arbitrary space, rather than the comparison of an explicit null hypothesis with an implied alternative in Euclidean space. Since $\alpha$ is a concept from the NP paradigm, it should be possible to control the FDR without appealing to p values that depend monotonously on an unbounded test statistic. That is, it should be possible to tighten up $H_1$ from a range to a point, and still apply some kind of FDR control. But I don't know how any of the FDR procedures I know (eg BH) can do this
Comment: (+1 for the nice answer)
Comment: @ssdecontrol, I completely agree with everything you mentioned, and I find myself having the same problem with my answer. I am completely sure that $H_1$ should not be a range, however, I was also at a loss of how to formulate it.  The insistence on using FDR in this problem makes it significantly more difficult. Thank you for the upvote, by the way.
Comment: It also seems that having an alternative hypothesis that is "too large" biases the analysis. What about $H_0: P_i \leq 0.5$ and $H_1: P_i > 0.5$? Then you can go full Fisherian and say "further from 0.5 is greater evidence in that direction." It's a hack but it works in this case. Actually I bet this is a general principle: as long as you can embed your hypotheses in a metric space such that they are linearly separable, you can define your null hypothesis in terms of a cutoff and construct a test statistic accordingly. I think the machine learning camp already does this
Comment: Thanks for your answer (although I must admit I haven't quite got from start to finish working through it yet), and also your input @ssdecontrol . I agree it's an awkward setting to work with.
Comment: @ChrisC I have implemented what I had hoped for. I didn't really understand what the pValues were corresponding too and how to order them. Your answer is excellent, thanks!
Comment: @ssdecontrol, that's really interesting, and I'll have to try it out and see if it helps the power! Thanks for the really interesting discussion! HBeel, I'm glad we were of help; hope everything works out for you!

How do I use FDR in this situation?


Normal Distribution and Iterated Logarithm

Question: Let $X_n$ be independent $N(0, \sigma^2)$-distributed random variables with partial sum $S_n := \sum_{k=1}^n X_k$, $n \geq 1$. Then I read the following results.
$$
\sum_{k = 1}^n \mathbb P (S_n > \epsilon\sqrt{n\log\log n}) \sim \sum_{k = 1}^n \frac{\sigma}{\epsilon\sqrt{\log\log n}} (\log n)^{-\frac{\epsilon^2}{2\sigma^2}} = \infty,
$$
for any $\epsilon > 0$. I do not know how these results are derived. To be specific,

How to get the first approximation, please?
How to know the second series does not converge, please? I tried to apply ratio test, but I got the limit is 1 which is not helpful. Could anyone help me, please? Thank you!

Update of the first question:
By the answer provided below, one has $$\mathbb P(S_n > \epsilon\sqrt{n\log\log n}) = \mathbb P\left(Z > \frac{\epsilon}{\sigma} \sqrt{\log\log n}\right) \leq \frac{1}{\sqrt{2\pi}}\frac{\sigma}{\epsilon\sqrt{\log\log n}} (\log n)^{-\frac{\epsilon^2}{2\sigma^2}}.$$ Then how to get $$\lim_{n \to \infty} \frac{\mathbb P\left(Z > \frac{\epsilon}{\sigma} \sqrt{\log\log n}\right)}{\frac{\sigma}{\epsilon\sqrt{\log\log n}} (\log n)^{-\frac{\epsilon^2}{2\sigma^2}}} = 1,$$ please? Thanks.
Answer:
Using independence, we obtain that
$$\mu\{S_n>\varepsilon\sqrt{n\log\log n}\}=\mu\{N>\varepsilon\sqrt{\log \log n}/\sigma\},$$
where the distribution of $N$ is standard normal. So we are reduce to estimates on the tail function of a standard normal distribution. We can use the equivalent
$$\mu\{|N|>x\sqrt 2\}\overset{x\to +\infty}{\sim}\frac 1{x\sqrt{\pi}}\exp(-x^2).$$
Notice that for each positive $p$ the series $\sum_{n=1}^\infty(\log n)^{-p}$ is divergent. Indeed, $\sum_{n=N+1}^{2N}(\log n)^{-p}\geqslant N(\log(2N))^{-p}$.
Comment: Sorry, I updated my question according to your answer. Could you elaborate more on both 1 and 2, please? To be specific, how to see $\sum (\log n)^{-p}$ is divergent and then what?
Comment: @20824 I've edited. Is it clearer?


Simple neural network in tensorflow -> shape problem

Question: I have two data list (x1, y1), (x2, y2) ...
I do not know the equation between x and y. So, I tried to use neural network to find it.
The hyperbolic.txt file has (x1, y1), (x2, y2) ...
The codes are below, but it does not work.
ValueError: Cannot feed value of shape (30,) for Tensor 'Placeholder:0', which has shape '(?, 1)'

I guess the shape of np_poses_x, np_poses_y might be wrong, but I cannot come up with how to change it.

I think both of them should have (30,1) shape.

import tensorflow as tf
import numpy as np
from tqdm import tqdm
import random

class datasource(object):
    def __init__(self, xx, yy):
        self.x = xx
        self.y = yy

def get_data(directory, dataset):
    xx = []
    yy = []

    with open(directory+dataset) as f:
        for line in f:
            p0,p1 = line.split()
            p0 = float(p0)
            p1 = float(p1)
            xx.append((p0))
            yy.append((p1))

    return datasource(xx, yy)

def gen_data(source):
    while True:
        indices = list(range(len(source.x)))
        random.shuffle(indices)
        for i in indices:
            yval = source.x[i]
            xval = source.y[i]
            yield xval, yval

def gen_data_batch(source, batch_size):
    data_gen = gen_data(source)
    while True:
        x_batch = []
        y_batch = []

        for _ in range(batch_size):
            _x, _y = next(data_gen)
            x_batch.append(_x)
            y_batch.append(_y)

        yield np.array(x_batch), np.array(y_batch)

X1 = tf.placeholder(tf.float32, shape=[None, 1])
Y = tf.placeholder(tf.float32, shape=[None, 1])

W1 = tf.Variable(tf.random_normal([1, 50], stddev=0.01))
L1 = tf.nn.relu(tf.matmul(X1, W1))

W2 = tf.Variable(tf.random_normal([50, 256], stddev=0.01))
L2 = tf.nn.relu(tf.matmul(L1, W2))

W3 = tf.Variable(tf.random_normal([256, 1], stddev=0.01))
model = tf.matmul(L2, W3)

cost = tf.reduce_mean(tf.square(model-Y))
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(cost)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

datasource = get_data('', 'hyperbolic.txt')

max_iterations = 100000
batch = 30
data_gen = gen_data_batch(datasource, batch)
for i in range(max_iterations):
    np_poses_x, np_poses_y = next(data_gen)
    feed = {X1: np_poses_x, model: np_poses_y}
    sess.run(optimizer, feed_dict=feed)
    np_loss = sess.run(cost, feed_dict=feed)
Answer: You got it right, you need to feed your network a (N,1) tensor and not a (N,) tensor.
The easiest solution might be to add the new dimension on the numpy side, using either np.newaxis (which is None) or the np.reshape function.
So you can apply this in gen_data_batch, replacing yield np.array(x_batch), np.array(y_batch) by yield np.array(x_batch)[:, np.newaxis], np.array(y_batch)[:, np.newaxis] for example.
You can also add this new axis on np_poses_x and np_poses_y :
feed = {X1: np_poses_x.reshape((len(np_poses_x), 1)),
        model: np_poses_y.reshape((len(np_poses_y), 1))}

Comment: Thank you for your reply !!, If i change to [:, None], those shape are (30,1). However error occurs at sess.run << TypeError: Fetch argument  has invalid type , must be a string or Tensor. (Can not convert a GradientDescentOptimizer into a Tensor or Operation.)
like this.
Comment: @Wooni I haven't touched TF in a long time... but I suspect you should `sess.run` on `train` instead of `optimizer`
Comment: OMG.. my big mistake. model -> Y is the answer :)


How performs the function $ z^{1/2}$ in the complex plane?

Question: How performs the function $ z^{1/2}$ in the complex plane?
Thanks for your help
I know it's a multi-valued function and that we must be careful with the branch on which it is defined, one of my questions is what is the main branch?, Where is continuous and which is defined?
Comment: Who's they?....
Comment: See [Riemann surface](http://en.wikipedia.org/wiki/Riemann_surface).
Comment: I just want to see their behavior in two dimensions
Answer: The principal square root function uses the nonpositive real axis as a branch cut (Wikipedia).
Answer: It is defined on $\mathbb{C}\backslash{(-\infty, 0]}$ (which is the main branch of the complex Logarithm) and is continuous everywhere.
Comment: I'm more familiar with the convention that it is defined everywhere but discontinuous on the negative real axis (which gets sent to the positive imaginary axis), as in lhf's answer.


How to sort top result on virtual memory on Redhat Linux?

Question: There is an interactive key "M" to sort by memory, which seems to be sorting on resident memory. Is there a way to sort on virtual memory? I happen to be working on Redhat Linux, but the question is not specific to this distribution.
Answer: You get a list of things you can sort by by pressing O. Virtual segment size is option o, so you can get what you want by pressing Oo<Enter>.
Answer: Try pressing F or O in top, that should allow you to select the column you want to sort by.
Comment: then hit enter (other keys are possible) to get back to the initial view
Comment: Letter [O], `-mem` or `-vsize`, [enter] works on MacOS.
Answer: top -u test -o VIRT
Where -u is the users whose processes you are interested in and -o is the field to filter on, yields something like


Chamar função Python que já esteja rodando

Question: Tenho o seguinte cenário.
Tenho uma função que consome muita RAM e gostaria de deixa-la rodando em background para cada vez que eu for chamar ela não ter que carregar novamente todos os dados em memória, e quando eu precisar do seu resultado eu realizo a chamada dela.
No caso eu preciso informar um parâmetro e ela deve me retornar um resultado, como fazer isso em Python?
Lembrando que esse que irá processar eu preciso que ele esteja rodando e quando o outro for se comunicar com ele seja só chamar esse modulo que já está rodando.
****Bom como você disse deixa eu exemplificar melhor.*****
Eu tenho um arquivo de  1,5 GB que contém varios textos e eu subo ele em memória, e realizo consulta nesses textos. O que eu não quero é que a cada consulta eu tenha que carregar esse arquivo.
Eu quero que ele fique rodando de fundo enquanto eu navego nas outras telas que eu tenho(Estou usando Django, criei varias páginas e uma dessas páginas permite eu realizar consultas nesse arquivo) pois várias telas vão usar esse arquivo de maneira diferente, então eu gostaria de ter um processo rodando sempre e quando eu precisar de um retorno eu chamo esse processo e ele retorna o que eu desejo
Answer: Você está praticamente descrevendo um dos conjuntos de vantagens de se usar programação orientada a objetos.
Apesar de que em Python é possível sim, ter uma "função" que fica com um estado (os dados) carregados e você consiga, de fora dela, mandar executar um trecho e receber um valor de volta, com o uso de "generators", esse não é um uso muito comum para funções.
Para objetos, no entanto, é bem natural. Infelizmente você não deu exemplo do que quer fazer. Claro que se o problema for demora do tempo para ler ou produzir os dados, ou não ter cópias repetidas dos dados, esse método ajuda. Se apenas uma cópia dos dados já te traz problemas de memória, não tem muito o que fazer.
Vamos supor que o que você precisa seja ler dados de um arquivo do Excel, e aí ter uma chamada que consulta dados que estavam nessa planilha: você pode ter uma classe que lê o arquivo no inicializador (o método __init__, e guarda os dados em memória, e um método que consulta uma dada célula:
import pandas as pd

class Planilha:
    def __init__(self, caminho_do_arquivo):
         self.dados = pd.read_excel(caminho_do_arquivo)

    def consulta(self, linha, coluna):
         return self.dados.loc[linha, coluna]

E você cria uma instância da classe planilha, que lê o arquivo uma vez, e a cada consulta de dados, pode chamar o método "consulta". (Esse exemplo é bem tosco, já que você poderia usar o objeto retornado por .read_excel, que é um pandas DataFrame diretamente, e nem precisaria da classe planilha - mas se precisar de processamento mais especializado em cima dos dados contidos na planilha, já faz sentido um design desses.
De qualquer forma, tudo o que descrevi até aqui se aplica a código que é executado para inicializar os dados - e retorna o controle pra você, mas mantém uma referência a esses dados inicializados.
Se for realmente como você descreve de esse código ter que "ficar rodando de fundo" e só responder quando for chamado (talvez com um resultado parcial?), aí é outra história - você vai precisar usar threads ou processos separados. Mas não dá pra saber sem exemplo. Pode ser que "o que fica rodando" fique melhor se for um processo "worker" acrescentando dados a um banco de dados, e você possa usar uma função normalzinha só pra consultar esse banco. O ideal é colocar em uma próxima pergunta os dados que você tem, o que você quer, e que abordagens você ja tentou, com código mesmo.
Comment: Editei a minha pergunta colocando mais detalhes
Comment: Se é o seu caso de uso, então o exemplo que eu dei, da classe é a resposta. Só em vez de ler a planilha com a chamada ao Pandas, você lê o seu arquivo de texto.
Comment: mas a solução ideal aí é colocar esses dados num banco SQL, em ve de deixar em memória: aí você não consome toda essa memória só pra agilizar o acesso aos dados. Deixe isso pra uma segunda etapa: aproveite agora pra estudar classes.


"Wrong Sign" On Regression Coefficients - Hierarchical Multiple Linear Regression

Question: I am analyzing my data on the relationship between spirituality and negative emotional states (depression, anxiety, and stress) using a hierarchical multiple linear regression. Everything seemed to be "okay" with the results until I checked the signs of the regression coefficients!! The results are showing a positive relationship between depression and a spirituality factor which includes having a reason for living. Virtually all of the research in this area shows that there should be a negative relationship between these two factors, i.e. that higher levels of depression are correlated with lower levels of having a reason for living (without any interaction terms or moderating variables).
Here are the steps that I have completed so far in this three-part data analysis (1st data analysis - Principal components analysis / 2nd Part - EFA (Exploratory Factor Analysis) / 2nd data analysis - hierarchical multiple linear regression):

Created spirituality questionnaire and collected data from sample (n = 189).
Used a principal components analysis to identify how many factors to extract in EFA.
Derived four factors from EFA.
Factors included log transformed questions which changed the directionality of the questionnaire (i.e. questions which were "positively" worded became negatively worded).
All questions on questionnaire were standardized (transformed to z-scores) to correct for directionality problem.
Negative emotional states were examined for normality. Two of the three negative emotional states had to be log transformed.
Ran 3 hierarchical multiple linear regressions with DV = Either log depression, log anxiety, or stress and IVs = Age (first "block"), four spirituality factors (second "block") using enter method.
Noticed that signs were incorrect, so I compared log transformed versions of DVs to non-transformed versions AND standardized DV (z-score) versions (see this article: The Independent Sign Bias: Gaining Insight from Multiple Linear Regression [PDF] --> pg. 1)
IV having to do with meaning in life had correct sign in non-transformed version of depression HOWEVER it looks like there is a collinearity issue between two of the spirituality IVs with the non-transformed version which may affect the signs!!! (Condition index > 15 and Variance proportions: .81 and .40)
Looked at bivariate correlations to see if the signs were correct, and oddly enough the signs were accurate (this was only true for the non-transformed versions of the hierarchical multiple linear regressions). None of the correlations were above .6.
Removed Age as an IV to see if that made a difference. It did not.
Removed spirituality IVs one at a time with stepwise method rather than enter method and this only increased the amount of collinearity between the variables that remained in the final model.

I would greatly appreciate any suggestions on how to correct for the wrong signs and/or if I did anything incorrectly in my three-part data analysis.
Answer: There are several things in your description that are a bit confusing, for example you state that taking the log transform reverses the direction of the coding, but the log by itself does not reverse coding.
Your main question seems to be that when you look at individual pairwise correlations the sign of the correlation is as expected, but some of the signs of the slopes in a multiple regression are opposite what you expect.  This is not uncommon since the interpretation of slopes is much more complex in multiple regression models.
Consider this example (I read this recently, I don't get the credit for thinking of it):  Collect data on the change in various peoples pockets, the variables to collect are the total value of the change (y), the total number of coins (x1), and the total number of coins that are not quarters, or if using non-US coins then number of coins not the highest common coin carried (x2).  Generally x1 and x2 will both be positivly correlated with y, but if you do a multiple regression using both x1 and x2 then the slope on x2 will be negative because to increase the number of non-quarters without changing the total number of coins we need to trade quarters for other coins of lesser value which decreases y.  You could have something similar happening with your data, does it really make sense to increase the religeous variable without the others changing?  What is often more meaningful is to compare predicted outcomes for what would be considered common combinations of your predictor variables.
Answer:
Factors included log transformed questions which changed the
  directionality of the questionnaire (i.e. questions which were
  "positively" worded became negatively worded).

How does taking the log do this? Also, a questionnaire does not have directionality, questions do. For a factor anlaysis, though, it is fine to have some negatively worded and some positively worded questions.

All questions on questionnaire were standardized (transformed to
  z-scores) to correct for directionality problem.

Standardizing via z-scores does nothing about directionality

Negative emotional states were examined for normality. Two of the
  three negative emotional states had to be log transformed.

Why? The independent variables do not need to be normally distributed.

IV having to do with meaning in life had correct sign in
  non-transformed version of depression HOWEVER it looks like there is a
  collinearity issue between two of the spirituality IVs with the
  non-transformed version which may affect the signs!!! (Condition index

15 and Variance proportions: .81 and .40)

Collinearity can certainly do this.

Looked at bivariate correlations to see if the signs were correct, and
  oddly enough the signs were accurate (this was only true for the
  non-transformed versions of the hierarchical multiple linear
  regressions). None of the correlations were above .6.

What? The regressions were not transformed, the variables were. It's unclear why you transformed the variables or how this changed the signs. Also, high correlations are not necessary for collinearity.

Removed Age as an IV to see if that made a difference. It did not.
Removed spirituality IVs one at a time with stepwise method rather
  than enter method and this only increased the amount of collinearity
  between the variables that remained in the final model.

Don't use stepwise, please. If some variables are collinear there are various things you can do:
1) Remove one or more of them
2) Use a biased but lower variance method of regression, such as ridge regression
3) Combine the variables into a smaller number of variables
Answer: If you visualized the data and it confirms what you found in the regression then why do you think the signs are wrong?  Your study may merely have found the opposite result.  Consider your likelihood of being published in your particular field and then ponder whether you now believe that the effect in the literature you read is unbiased.
Alternatively, I'm not sure how you're measuring spirituality and whether prior research directly measured having a reason for living but it seems to me that if a component of spirituality is believing things might be far more grand in some sort of an after life then you might have a poor proxy for a reason for living.


Why does this RegEx have to be surrounded in parenthesis to match?

Question: I started with this RegExp:
 //g

And I wanted it to match both of these lines:

In my first attempt, I changed file to (file|virtual) like so:
//g

But, that didn't match anything. After a little playing around, I found that adding parenthesis around everything would get a match:
/()/g

I don't understand why though - why didn't the first modification match, and why do the extra parenthesis fix it?
Answer: I figured it out - it was the \1 back refference near the end - it was supposed to match the opening (['"]) with the same quote at the end. But when I added an extra capture group, then it was looking for a second (file|virtual) at the end of the file path.
The fix was to change it to \2.
This is the corrected RegExp:
//g


How to decide touch location in silverlight WP7?

Question: Is there a general way to know x and y with the reference is the upper left corner of the screen?
because when I'm trying to get x and y when I'm dragging an image, I get them according to the corner of the image, not the canvas that contains the image.
Answer: You can see this.
How do you find the screen position of a control in silverlight on WP7?
Comment: thanks Cong Tran,, but I'm talking about silverlight, not XNA.
Comment: This should be a comment, not an answer


Here is a linear algebra problem from an interview and I have no clue.

Question: Matrix C of size n$\times$n is symmetric . Zero is a simple eigenvalue of C. The associated eigenvector is q. For  $\epsilon$>0, the equation $Cx+\epsilon x=d$ in x, where x and d are n-dimensional Column vectors and d is known, has a solution that depends on  $\epsilon$. Call this solution  $x(\epsilon)$. Express $ \lim\limits_{\epsilon \to 0^+} \epsilon x(\epsilon)$ in terms of vectors q and d.
I have some inspirations:

When $\epsilon \to 0$, $\epsilon q$ also comes to $0$. So we have: $C(x+q)+\epsilon (x+q)=d$
Since C is a symmetrix matrix, we use diagonalization like: $U\Lambda U^Hx+\epsilon x=d$

But to be honest, I have no clue on this and it bothers me a lot. I can't figure out how to use these conditions we have.
Thank you for reading my question!
Could you give me some clues on this problem?
Comment: What have you tried?
Comment: @snowman no problem, that's quite normal, but it's good to add some of your own thoughts to the question (as I see you have done) to help others know roughly where your thought process is at so we can help you better! Also, some users think questions that only have a problem statement seem low-effort and may be tempted to close them, so adding your own efforts show people that you've really thought about it before asking. :)
Comment: @YiFan Sure, thanks!
Comment: Do not use images to convey information not otherwise available in your post. [See here for why it is bad practice to do so](https://math.meta.stackexchange.com/a/34123/742).
Comment: @YIFan To be honest, I tried some methods but I don't think they make sense. I never meet problems like this before.
Comment: @ArturoMagidin
Of course, I think you are right.
Answer: I assume here that $C$ is a real symmetric matrix.
HINT: We assume $q$ is a unit vector. Apply the Spectral Theorem to write
$$C+\epsilon I = \epsilon qq^\top + \sum_{i=1}^{n-1} (\lambda_i + \epsilon) v_iv_i^\top,$$
where $q,v_1,\dots,v_{n-1}$ form an orthonormal basis. Then
$$(C+\epsilon I)^{-1} = \frac1{\epsilon} qq^\top + \sum_{i=1}^{n-1} \frac1{\lambda_i + \epsilon} v_iv_i^\top.$$
Of course, we assume $\epsilon>0$ small enough so that no $\lambda_i+\epsilon$ becomes $0$.
Can you finish now?
Comment: Thanks a lot. I wrote an answer based on your hint:
https://math.stackexchange.com/users/1057706/snowman
Answer: Thanks @Ted Shifrin! Here is my answer:
First,
$(C+\epsilon I)x = d$, so $x=(C+\epsilon I)^{-1}d$ and then $\epsilon x=\epsilon (C+\epsilon I)^{-1}d$
As said by Ted,
$$(C+\epsilon I)^{-1} = \frac1{\epsilon} qq^\top + \sum_{i=1}^{n-1} \frac1{\lambda_i + \epsilon} v_iv_i^\top.$$
So we have:
$$\epsilon x=\epsilon(C+\epsilon I)^{-1}d = qq^\top d + \sum_{i=1}^{n-1} \frac{\epsilon}{\lambda_i + \epsilon} v_iv_i^\top d$$
The final result would be:
$$\lim\limits_{\epsilon\to 0^+}\epsilon x = qq^\top d$$


segmentation fault, address out of bounds

Question: I'm writing a function that takes a string of input from the user until newline is hit, then parses the string into each word and puts them into an array.
The problem I get is when you first start the program, if you enter in whitespace as input, it gets a segmentation fault. If you put in anything else, it runs fine. Then if you put just whitespace as input after that, it runs fine as well.
Here is a snippet of the function:
#define MAXARGS 10

char buf[100];
char cmd[MAXARGS][100];
char *bp = NULL;
int i, n;

for (i = 0; i < MAXARGS; i++)
    strcpy(cmd[i], "");

fputs(" >> ", stdout);
fgets(buf, sizeof(buf), stdin);
bp = buf;

for (i = 0; i < MAXARGS; i++)
{
    sscanf(bp, "%99s%n", cmd[i], &n);
    bp += n;
    while (*bp == ' ') // segfault is here
        bp += 1;
}

I'm on Windows running MinGW with MSYS.
When I run it through GDB I get:
Program received signal SIGSEGV, Segmentation fault.
0x0040152d in command () at main.c:46
46                              while (*bp == ' ')
(gdb) print bp
$1 = 0x47291c <Address 0x47291c out of bounds>
(gdb) print *bp
Cannot access memory at address 0x47291c

...but I'm not familiar with GDB so I don't know what else to check
Answer: sscanf should return the number of matched items, if it can't match anything it will return 0 and n will be left with what it had before, and you end up using n to increment a pointer.
Check what sscanf returns before using n and don't use it if it returns 0
Answer: Print n. I suspect it gets a really unexpected value (perhaps negative ? although that's unlikely).
Comment: @hugoblack, you are relying on side effects, you should check the result of sscanf and never use `n` if the result is not valid.
Comment: So it does! Something in the 2 million range... so I ended up trying to compare something at a memory address that didn't exist. I initialized n to 0 and it fixed the problem. Thank you!


Why is an illegal shift treated as a false start at the end of a half in American football?

Question: On 2 November 2015 near the end of the Seahawks-Cowboys game, the Cowboys were down by 1 but had possession of the ball. They had  no timeouts and next to no time left on the clock. Hurrying to get a play off, they committed an illegal shift by failing to get set before snapping the ball. The result of the play was a sack, but the sack was negated because in the last minute of a half an illegal shift is treated as a false start. In other words, no free play is given for the defense. Why is this? Sure, it aids the offense in a potentially viewer-friendly, comeback situation, but it's not hard to think of a jillion other rules that would similarly aid the offense.
Comment: I don't see anything in the rules remotely agreeing with Peirara here.  He's a bit more specific in his [tweet](https://twitter.com/MikePereira/status/660979281556668416), which suggests that the players were required to be set and weren't (which would make it a false start, I suppose, since players who ought to be set were moving?).  Very curious.
Answer: It's possible Pereira was referring to the section referring to illegally conserving time:

SECTION 7 - ACTIONS TO CONSERVE TIME

ARTICLE 1. ILLEGAL ACTS

A team is not permitted to conserve time inside of one minute of either half by committing any of these acts:

a foul by either team that prevents the snap (i.e., false start, encroachment, etc.)
intentional grounding
an illegal forward pass thrown from beyond the line of scrimmage
throwing a backward pass out of bounds
spiking or throwing the ball in the field of play after a down has ended, except after a touchdown
any other intentional foul that causes the clock to stop.

Penalty: For Illegally Conserving Time: Loss of five yards unless a larger distance penalty is applicable.
When actions referred to above are committed by the offensive team while time is in, officials will run 10 seconds off the game clock before permitting the ball to be put in play on the ready-for-play signal. The game clock will start on the ready-for-play signal. If the offensive team has timeouts remaining, it will have the option of using a timeout in lieu of a 10-second runoff, in which case the game clock will start on the snap after the timeout. The defense always has the option to decline the 10-second runoff and have the yardage penalty enforced, but if the yardage penalty is declined, the 10-second runoff is also declined.

These aren't explicitly false starts, though, and not all are even dead ball fouls.
There's nothing in the actual NFL Rulebook that refers to converting an illegal shift into a false start.  If it's true, and I have no reason to think Pereira would be wrong about something like this, it's either in the referee's supplement (which is not freely available) or an interpretation of something I'm not seeing.  (I don't see, either, where it says that the offense must get set on certain plays differently in the last minute, which he also refers to in his tweets.  The offense must always be set for one second - see 7-4-6 - so I don't see how the last minute is special.)
The complete false start rule is rule 7-4-2, and reads:

ARTICLE 2. FALSE START

It is a False Start if the ball has been placed ready for play, and, prior to the snap, an offensive player who has assumed a set position charges or moves in such a way as to simulate the start of a play, or if an offensive player who is in motion makes a sudden movement toward the line of scrimmage. Any quick abrupt movement by a single offensive player, or by several offensive players in unison, which simulates the start of the snap, is a false start.

There are then some specific items clarifying what is/isn't a false start for particular positions, but nothing about the last minute.
And for illegal shifts, rule 7-4-8:

ARTICLE 8. SHIFTS

The offensive team is permitted to shift and have two or more players in motion multiple times before the snap. However, after the last shift, all players must come to a complete stop and be in a set position simultaneously for at least one second.
If any eligible backfield player goes in motion (one at a time) after the last shift and comes to a complete stop, there is no requirement for a full second pause before a second player can legally go in motion.
However, if the first player has not come to a complete stop when the second player goes in motion, it is another shift and requires another simultaneous stop for at least one second by all players.
It is also an illegal shift if a player under or behind center goes in motion and fails to come to a complete stop for at least one full second prior to the snap, or before a second player goes in motion.
Note: The offensive team must present a legal formation both before and after a shift.
Penalty: For an illegal shift: Loss of five yards.

What I suspect, though, is that this was just badly worded.  Rule 7-4-6 is probably what controls here, and while it doesn't have a specific penalty associated with it, it clearly would be a dead ball foul.

ARTICLE 6. COMPLETE STOP

All offensive players are required to come to a complete stop and be in a set position simultaneously for at least one second prior to the snap.

That describes more aptly what it sounds like was the problem: if the players were never set properly to begin with, it's not a shift.  I'm not sure why a penalty isn't explicitly listed here, but it would be similar to a false start, so why not call it that. It's also not clear if it's a dead ball foul (like false start) or a live one, but dead ball makes sense to me.


How apply foreach loop for every line of string

Question: I have a string:

Coffee

Tea

Milk

Milk

Milk

Milk

Milk last

and I want to interate over all li-tags (in order to add some text between the tags):
    for (char c: innerText.toCharArray()) {

        if(c == '<'){
            // process the text but in this way
           //I iterate over whole text as a one line
        }
    }

Comment: Do you want the contents or the tags too?
Comment: Is this HTML ? Can the processing be done on the client?
Comment: Yes, it is simple html, it is a standalone app.
Answer: If you don't want to use a parser, you can use regular expressions for this as follows:
String regex = "
((?:\\w|\\s)+)
";
String list = "..."; //your list

Matcher m = Pattern.compile(regex).matcher(list);

while (m.find())
{
    String ofInterest = m.group(1);
    //doSomething
}

((?:\\w|\\s)+) is a capturing group that takes characters from [A-Za-z_0-9] or any whitespace, where the + indicates that there must be at least one character.  Use a * instead if there could be no characters in between.
m.group(1) simply extracts what is between
 and
, where the 1 cooresponds to the first capturing group.
You can replace what's inside the capturing group with whatever you consider to be valid.
Here's a tutorial on Regular Expressions.  Good luck!
Comment: Thanks guys, stackoverflow is great ! ! !
Comment: @MrPhi Glad I could help.  If you don't know regular expressions already, I highly suggest you go through the tutorial.  They are really helpful with a lot of tasks.  Good luck!
Comment: Exactly, i have a problems with text processing, I do not know when use DOM, when SAX, when RegEX, There are lots of possibilities and it is easy to get lost in all of them. The question is only part of greater problem to solve, for now thanks to the Q&A I get a way in which should I will go. Thanks one more time all of you!
Comment: Could you give me a hint whether your solution is the best for my main problem: http://stackoverflow.com/questions/19903402/calling-recursion?noredirect=1#comment29612056_19903402 ?
Comment: @MrPhi I'm about to pass out, not sure if it's because I'm extremely tired or not, but I'm not entirely sure what your other question is asking.
Answer: Use JSoup for parsing HTML text or DOM parser to parser the XML string. Then you can pass the parsed value to the for-each loop.
Answer: Try this
String str ="
Coffee

Tea

Milk

Milk

Milk

Milk

Milk last
";
    String[] temp =str.replaceAll("
", "").split("
");
    String result ="";
    for (String value: temp){
        result = result + "
";
        // update the value here if needed
        result = result + value;
        result = result+ "
";
    }

     System.out.println(result);

Answer: Try:
    String str
            = "
Coffee
"
            + "
Tea
"
            + "
Milk
"
            + "
Milk
"
            + "
Milk
"
            + "
Milk
"
            + "
Milk last
";
   String[] products= str.substring(4, str.length()-5).split("

");
    for (String prod : products) {
        System.out.println(prod);
    }

Output:
Coffee
Tea
Milk
Milk
Milk
Milk
Milk last

Answer: it shows that you are parsing some html elements,
jsoup is one the great java html parsers,
http://jsoup.org/cookbook/extracting-data/selector-syntax
Comment: Yes, you are right but I do not want to use from some frameworks, I want to seize the Java to the maximum ;-)
Answer: As the OP is wanting to add some text between the tags I suggest that he use jquery instead of a server side solution.
$( "li" ).each(function( index ) {
  str = $(this).text();
  $(this).text(str + " Other text");
 });

Comment: What is OP ? Thanks but it is standalone app.
Comment: OP = Short for Original Poster. Used on online message boards and forums.
Answer: Have a try with the following code:
String text = "
Coffee

Tea

Milk

Milk

Milk

Milk

Milk last
";
    text = text.replaceAll("
", "");
    String[] results = text.split("
");
    // The first one is empty, remove it
    for (int i = 1; i < results.length; i++) {
        System.out.println(results[i]);
    }

Output in Console is as follows:
Coffee
Tea
Milk
Milk
Milk
Milk
Milk last

Answer: It can all be done on one line:
for (String option : innerText.replaceAll("^.*?
|
.*?$", "").split("
\\s*
")) {
   System.out.println(option);
}


Feature module not exporting component

Question: My simple app has a root module named AppModule and a feature module named HomeModule. I'm trying to create a route for a component named HomeComponent (which is part of HomeModule) but all I get is
Module "HomeModule" has no exported member 'HomeComponent'

app.routing.ts
import { Routes, RouterModule } from '@angular/router'
import { HomeComponent } from './Home/home.module.ts' // HomeComponent not found

const appRoutes: Routes = [
  { path: '', component: HomeComponent }
];

export const appRoutingProviders: any[] = [

];

export const routing = RouterModule.forRoot(appRoutes);

home.module.ts
import { NgModule }      from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { HomeComponent }  from './home.component.ts';

@NgModule({
  imports:      [ BrowserModule ],
  declarations: [ HomeComponent ],
  exports:      [ HomeComponent ]
})

export class HomeModule { }

home.component.ts
import { Component } from '@angular/core';

@Component({
  selector: 'home',
  template: '
Hello World
'
})

export class HomeComponent { }

Why is HomeModule not exporting its component?
Comment: Does my answer solve your issue?
Answer: Ref1:http://angularjs.blogspot.com/2016/08/angular-2-rc5-ngmodules-lazy-loading.html
Ref2:https://angular.io/docs/ts/latest/guide/ngmodule.html#!#imports
Issue 1: NgModel Import/Export
import { HomeComponent } from './Home/home.module.ts' // HomeComponent not found

home.module.ts exports HomeModule, not HomeComponent.
@NgModule exports make Angular2 features of those export components available to the importing module. features can be template directives, selector, injectable service, etc.
Home component feature is its selector. So importing HomeModule into app.module will make HomeComponent selector home available to any app.module's component.
To have HomeModule export HomeComponent explictly, index.js and index.d.ts are needed. (Inspired by Fabio Antunes answer)
Use selector home
To use this, exports: [ HomeComponent ] is required in home.module.ts.
app.module.ts
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { AppComponent } from './app.component';

import { HomeModule } from './Home/home.module';

import { routing } from './app.routing';

@NgModule({
    imports: [
        BrowserModule,
        HomeModule,
        routing],
    declarations: [
        AppComponent],
    bootstrap: [AppComponent]
})
export class AppModule { }

app.component.ts using selector home
import {Component} from '@angular/core';

@Component({
    selector: 'app-component',
    template: `

{{title}}

        <home></home>`
})
export class AppComponent {
    title = 'APP';
}

Use HomeComponent
To use HomeComponent directly, we need to add index.js and index.d.ts into ./Home
./Home/index.js
exports.HomeModule = require('./home.module').HomeModule;
exports.HomeComponent = require('./home.component').HomeComponent;

./Home/index.d.ts
exports * './home.module';
exports * from './home.component';

Then import Home Module like npm package
app.routing.ts
// We are importing the module directory, not ./Home/home.module.ts
import { HomeComponent } from './Home';

import { Routes, RouterModule } from '@angular/router'

const appRoutes: Routes = [
  { path: '', component: HomeComponent }
];

export const appRoutingProviders: any[] = [

];

export const routing = RouterModule.forRoot(appRoutes);

Issue 2: Routing (to child module)
To route to a module, the child module need to have its own router setup.
Use loadChildren in parent routing(app.routing.ts).
Use RouterModule.forChild in child routing(home.routing.ts).
app.module.ts
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { AppComponent } from './app.component';

import { HomeModule } from './Home/home.module';

import { routing } from './app.routing';

@NgModule({
    imports: [
        BrowserModule,
        HomeModule,
        routing],
    declarations: [
        AppComponent],
    bootstrap: [AppComponent]
})
export class AppModule { }

app.component.ts
import {Component} from '@angular/core';

@Component({
    selector: 'app-component',
    template: `

{{title}}

      Home

    <router-outlet></router-outlet>`
})
export class AppComponent {
    title = 'APP';
}

app.routing.ts
import { Routes, RouterModule } from '@angular/router';

const appRoutes: Routes = [
    {
        path: '',
        redirectTo: '/home',
        pathMatch: 'full'
    },
    {
        path: 'home',
        loadChildren: './Home/home.module'
    }
];

export const appRoutingProviders: any[] = [

];

export const routing = RouterModule.forRoot(appRoutes);

Home/home.module.ts
import { NgModule }      from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { HomeComponent }  from './home.component';

import { routing } from './home.routing';

@NgModule({
  imports: [
    BrowserModule,
    routing],
  declarations: [
    HomeComponent]
})

export class HomeModule { }

Home/home.routing.ts
import { Routes, RouterModule } from '@angular/router';

import { HomeComponent } from './home.component';

const homeRoutes: Routes = [
    {
        path: '',
        redirectTo: '/home',
        pathMatch: 'full'
    },
    {
        path: 'home',
        component: HomeComponent
    }
];

export const appRoutingProviders: any[] = [

];

export const routing = RouterModule.forChild(homeRoutes);

Comment: For a moment I figured that defining `path: 'home', loadChildren: './Home/home.module'` along with the child routing `path: 'home', component: HomeComponent` would yield URLs like `example.com/home/home`, but apparently not. Even more, when NOT repeating the `home` path in `Home/home.routing.ts`, then requesting `example.com/home` fails. (Just for future readers; I'm sure you know, @John.)
Answer: Feature modules should import CommonModule instead of BrowserModule
Answer: When setting up a route where your app.route.ts will load your HomeModule using loadChildren there are two things you must do. Either:
export default class HomeModule() {} // note *default*

or in your app.route.ts you need to add #HomeModule:
import { Routes, RouterModule } from '@angular/router';

const appRoutes: Routes = [
    {
        path: '',
        redirectTo: '/home',
        pathMatch: 'full'
    },
    {
        path: 'home',
        loadChildren: './Home/home.module#HomeModule' <-- or like this
    }
];

export const appRoutingProviders: any[] = [

];

export const routing = RouterModule.forRoot(appRoutes);

If you plan to add additional routes to the home.route.ts it might be a good idea to define children property like:
import { Routes, RouterModule } from '@angular/router';

const homeRoutes: Routes = [
    path: "",
    component: HomeComponent,
    children: [
        {
            path: "",
            component: HomeComponent
        },
        {
            path: ":id",
            component: HomeDetailsComponent"
        }
    ]
]

What you have to remember in the above case is that your main RouterOutlet in your app.component.ts (if it is called like that), is not going to serve the route such as http://some_url/Home/1 in order to load HomeDetailsComponent.
The thing is that the route served by HomeDetailsComponent is a child route of HomeComponent and as such HomeComponent's template or templateUrl must have it's own <router-outlet></router-outlet> defined, othwerise you will get an error message Cannot find primary outlet to load 'HomeDetailsComponent'.
Answer: @yourfriendzak, just remove the '.ts' from below statement:
import { HomeComponent } from './Home/home.module.ts' // HomeComponent not found

Hope it helps...

Feature module not exporting component


Transform Student's T distribution to Beta distribution

Question: I want to calculate the inverse CDF of the Student's T distribution, but I only have an accurate routine for the Beta distribution available. I have tried to make a transformation, but I'm getting the wrong answers out. Could somebody check my reasoning?
Let $T$ have a Student's $T$ distribution with $n$ degrees of freedom. Given a probability $p$, we want to find $t$ such that
$$
P(T < t) = p
$$
Let us suppose without loss of generality that $t \geq 0$, i.e. $p \geq 1/2$, since otherwise we can use symmetry to frame the problem this way. First, we transform to a $F_{1,n}$ distribution:
$$
P(T < t) = 1 - P(T > t) = 1 - \frac{1}{2} P(|T| > t) = \frac{1}{2} \left( 1 + P(|T| < t) \right) = \frac{1}{2} \left( 1 + P(T^2 < t^2) \right)
$$
Noting that $T^2$ is distributed as $F_{1,n}$ (see here), we can rearrange to get:
$$
P(F_{1,n} < t^2) = 2p - 1
$$
Next we use the fact that if $B$ has a $Beta(\frac{\alpha_1}{2}, \frac{\alpha_2}{2})$ distribution, then $\frac{\alpha_2 B}{\alpha_1 (1 - B)}$ has an $F_{\alpha_1, \alpha_2}$ distribution (see here), to transform our expression (with $\alpha_1 = 1, \alpha_2 = n$):
$$
P \left( \frac{nB}{1 - B} < t^2 \right) = 2p - 1
$$
We use $B > 0$ to rearrange the inequality:
$$
P \left( B < \frac{t^2}{n + t^2} \right) = 2p-1
$$
Letting $F_B$ be the CDF of $B$, we set $x = F_B^{-1}(2p - 1) = \frac{t^2}{n + t^2}$ and we can solve the original problem for t:
$$
t = \sqrt{ \frac{xn}{1 - x} }
$$
Where the positive square root is taken because of our assumption that $t > 0$. Is this correct?
Comment: Presumably you want $T$ to have a Student's $t$ distribution with $n$ (rather than $t$) degrees of freedom.  Your claim is that $F_{t_n}^{-1}(p) =  \sqrt{ n\dfrac{F_{Beta(1/2, n/2)}^{-1}(2p-1)}{1 - F_{Beta(1/2, n/2)}^{-1}(2p-1)} }$ at least when $\frac12 \le p \lt 1$.  Then  similarly $F_{t_n}^{-1}(p) =  -\sqrt{ n\dfrac{F_{Beta(1/2, n/2)}^{-1}(1-2p)}{1 - F_{Beta(1/2, n/2)}^{-1}(1-2p)} }$  when $0 \lt p \le \frac12$. That seems to be correct, for the reasons you give.   Though note that calculating the Beta distribution quantile function is not simple.
Comment: Thanks @Henry . Yes I did mean $n$ df, I have updated the question accordingly
Answer: Yes, your approach is generally correct if you want to generate realizations of a Student's $t$-distributed random variable $T$ from a suitable Beta-distributed random variable $X$.  You would presumably use a third, auxiliary random variable, say $Y \sim \operatorname{Bernoulli}(1/2)$; e.g.,
$$X \sim \operatorname{Beta}(1/2, n/2), \quad Y \sim \operatorname{Bernoulli}(1/2), \\ T = (2Y-1) \sqrt{\frac{nX}{1-X}} \sim \operatorname{StudentT}(n).$$
The reason for the additional Bernoulli variable is so that you can generate negative realizations of $T$, not just the positive ones.
While your derivation employs the "inverse CDF" method, this doesn't actually yield a closed-form inverse CDF for the Student's $t$ distribution, since there isn't a simple closed form for the inverse of the beta CDF.  It only shows how, if you have access to a random number generator that can generate realizations from a beta distribution, you can use it to get Student's $t$ realizations via a simple transformation.
Comment: Thanks. I won't be generating realisations of the random variable - the context is that I'm using the NAG library, and the Student's T inverse CDF is only accurate to about 1e-5, but their Beta routine allows greater precision. Under the hood their [Beta implementation](https://www.nag.com/numeric/nl/nagdoc_27.3/clhtml/g01/g01fec.html) uses a modified version of Newton-Raphson
Comment: You only need your $Y$ if you want random $t_n$-distributed numbers and have a generator for $\operatorname{Beta}(1/2, n/2)$ distributed numbers (perhaps found with rejection sampling as with `rbeta()` in R).  But I read the question as looking for the inverse CDF, so the equivalent of basing `qt()` on `qbeta()` in which case you do not need your $Y$


Do players get yellow cards when they remove their jersey during celebrations?

Question: If a player receives a first caution (yellow card) during the game and the same player scores a goal and removes his jersey will he be dismissed from the field of play and shown a red card?
Answer: Yes, it happen numerous of times.
It's like every other yellow card,
look at this from FIFA laws of the games:

Celebration of a Goal
While it is permissible for a player to demonstrate his joy when a goal has been
  scored, the celebration must not be excessive.
  Reasonable celebrations are allowed, but the practice of choreographed
  celebrations is not to be encouraged when it results in excessive time-wasting
  and referees are instructed to intervene in such cases.
A player must be cautioned if:

in the opinion of the referee, he makes gestures which are provocative,
  derisory or inflammatory
he climbs on to a perimeter fence to celebrate a goal being scored
he removes his shirt or covers his head with his shirt
he covers his head or face with a mask or other similar item

Leaving the field of play to celebrate a goal is not a cautionable offence in itself
  but it is essential that players return to the field of play as soon as possible.
  Referees are expected to act in a preventative manner and to exercise common
  sense in dealing with the celebration of a goal.

Two examples from the FA Premier League:
- Arjen Robben of Chelsea
- Ross Wallace of Sunderland
Answer: Yes. The players usually gets booked for celebrating the goal by removing his shirt due to the disciplinary reasons. And the referees spare no mercy to issue the second yellow card (first card in the earlier match) & send the player off. There have been no complaints from the players & the managers about this.