Odds ratio for an $I \times J$ table

Question: I am trying to find a general expression for the odds ratio in a two-way table. I know for a $2 \times 2$ table odds ratio can be expressed as:
$${\rm OR} = \frac{\pi_{11}\pi_{22}}{\pi_{12}\pi_{21}}.$$
I am trying to find a general expression of the odds ratio for an $I \times J$ table. Is there any way to write the OR in terms of $\pi_{ij}$?
Answer:
Is there any way to write the OR in terms of $\pi_{ij}$? (Emphasis added.)

There is no single odds ratio for an $I \times  J$ contingency table. As Agresti says (page 54, Categorical Data Analysis, 2nd edition)

For $I \times J$ tables, it is rarely possible to summarize association by a single number without some loss of information.

For rows $a$ and $b$ and columns $c$ and $d$, you can write the odds ratio as a generalization of the $2 \times 2$ formula:
$$\frac{\pi_{ac} \pi_{bd}}{\pi_{bc} \pi_{ad}}. $$
You can describe all such odds ratios in terms of a set of $(I-1)(J-1)$ odds ratios. For me, the simplest is to set cell $IJ$ as a reference ($b=I,d=J$ in the above) for the $(I-1)(J-1)$ combinations of rows and columns not involving either row $I$ or column $J$.
There are summary single measures of association of nominal variables in $I \times J$ tables, explained for example in Sal Mangiafico's R Handbook
. Cramér's V is frequently used; its Wikipedia page has links to other such measures.
Comment: Thanks, it was helpful.
Comment: $OR= \frac{\pi_{00} \pi_{ij}}{\pi_{i0}\pi_{0j}}$, where $\pi_{00}$ is the reference cell, and $i=1,2, \ldots, I$, and $j=1,2, \ldots, J$. Does this make any sense? Can I write it like this?
Comment: @Dihan close, but you have to be careful with your indexing. With your setting $\pi_{00}$ for the reference cell and with $i$ and $j$ each running from 1 to $I$ and $J$ respectively, you might seem to have an $(I+1) \times (J +1)$ contingency table. Indexing of contingency table rows and columns typically starts with 1, not 0.
Comment: Yes, I understand. I am working on a paper, in that paper, the author starts indexing from 0 instead of 1, which makes me think like this. Though I do not like to start with 0. Thanks for your feedback.

Odds ratio for an $I \times J$ table


Show that $f(z) = c \sin (\pi z)$

Question:
Let $f: \mathbb{C} \to \mathbb{C} $ be an entire function  such that  $f(0)=0, f(z+1) = -f(z)$ and $|f(z)| \leq e^{\pi |\text{Im}(z)|}$ for all $z \in \mathbb{C}$. Show that $f(z) = c \sin (\pi z)$, for some $c \in \mathbb{C}$.

I'm trying to use Liouville's theorem.
If $z \notin \mathbb{Z}$, then:
$$\bigg|\frac{ f(z)}{(e^{i\pi z}- e^{-i\pi z})} \bigg| =
\frac{|f(z)|}{|e^{i\pi z}- e^{-i\pi z}|} \leq
\frac{e^{\pi |\text{Im}(z)|}}{|e^{i\pi z}- e^{-i\pi z}|}  \leq
\frac{e^{\pi |\text{Im}(z)|}}{|e^{i\pi z}| - |e^{-i\pi z}|} $$
$$= \frac{e^{\pi |\text{Im}(z)|}}{e^{-\pi \text{Im}(z)} - e^{\pi \text{Im}(z)}} =  \frac{1}{e^{-\pi (\text{Im}(z) + |\text{Im}(z)|)} - e^{\pi (\text{Im}(z) - |\text{Im}(z)|)}}$$
I'm stuck here, I don't know how to prove that the last term is bounded.
And I don't know how to proceed when  $z \in \mathbb{Z}$.
Could somebody help me out?
New approaches are welcome.
Comment: @JackD'Aurizio We don't need anything remotely as hard as Paley-Wiener here. $\lvert \sin (\pi z)\rvert \geqslant \frac{1}{4} e^{\pi \lvert \operatorname{Im} z\rvert}$ for $\lvert \operatorname{Im} z\rvert \geqslant 1$ plus periodicity suffices.
Comment: Note that $|e^{i\pi z}|=e^{-\pi \text{Im}z}$. Also, your last expression is wrong, $-e^{\pi(\text{Im}z-|\text{Im}z|)}$ should read $-e^{\pi(\text{Im}z+|\text{Im}z|)}$. To show this is bounded, just divide into cases $\text{Im}z \ge 0$ and when it is negative
Comment: I think you bounded by infinite from above after the second inequality. The denominator is cero if $z \in \mathbb{R}$.
Comment: This question is almost a duplicate of https://math.stackexchange.com/questions/2197387/characterization-of-the-sinus/2197477 - the claim is a consequence of the Paley-Wiener theorem.
Comment: @MarkViola Good point, though not precisely zero...yet surely a constant. Deleting.
Comment: Of course, that is why I did not mark the question as a duplicate, but still, the claim holds under much weaker assumptions.
Answer: Note that $f$ and $g \colon z \mapsto \sin (\pi z)$ are both periodic with period $2$. Also, $g$ has simple zeros at the integers, and $f$ also vanishes at the integers. Thus $q = f/g$ has removable singularities at the integers, and considering them removed, $q$ is an entire function with period $2$. By continuity, $q$ is bounded on $K = \{ z = x + iy : \lvert x\rvert \leqslant 1,\, \lvert y\rvert \leqslant 1\}$. By periodicity, $q$ is bounded on $\{ z : \lvert \operatorname{Im} z\rvert \leqslant 1\}$.
It remains to see that $q$ is bounded on $\{ z : \lvert \operatorname{Im} z\rvert \geqslant 1\}$. Since $\sin \overline{w} = \overline{\sin w}$, we have $\lvert g(\overline{z})\rvert = \lvert g(z)\rvert$, hence it suffices to look at the lower half-plane. Now if $y \geqslant 1$ and $x\in \mathbb{R}$, then
$$\lvert e^{i(x-iy)} - e^{-i(x-iy)}\rvert = e^y\lvert e^{ix} - e^{-ix-2y}\rvert \geqslant e^y (1 - e^{-2y}) \geqslant (1 - e^{-2})e^y \geqslant \frac{1}{2}e^y,$$
whence $\lvert \sin (\pi z)\rvert \geqslant \frac{1}{4} e^{\pi\lvert \operatorname{Im} z\rvert}$ for $\lvert \operatorname{Im} z\rvert \geqslant 1/\pi$. And this yields $\lvert q(z)\rvert \leqslant 4$ for $\lvert \operatorname{Im} z\rvert \geqslant 1/\pi$. Together with the boundedness on the strip $\lvert \operatorname{Im} z\rvert \leqslant 1$, we see that $q$ is globally bounded, hence constant by Liouville's theorem.

Show that $f(z) = c \sin (\pi z)$


Python monotonically increasing memory usage (leak?)

Question: I'm using this simple code and observing monotonically increasing memory usage. I'm using this little module to dump stuff to disk. I observed it happens with unicode strings and not with integers, is there something I'm doing wrong?
When I do:
>>> from utils.diskfifo import DiskFifo
>>> df=DiskFifo()
>>> for i in xrange(1000000000):
...     df.append(i)

Memory consumption is stable
but when I do:
>>> while True:
...     a={'key': u'value', 'key2': u'value2'}
...     df.append(a)

It goes to the roof. Any hints?  below the module...

import tempfile
import cPickle

class DiskFifo:
    def __init__(self):
        self.fd = tempfile.TemporaryFile()
        self.wpos = 0
        self.rpos = 0
        self.pickler = cPickle.Pickler(self.fd)
        self.unpickler = cPickle.Unpickler(self.fd)
        self.size = 0

    def __len__(self):
        return self.size

    def extend(self, sequence):
        map(self.append, sequence)

    def append(self, x):
        self.fd.seek(self.wpos)
        self.pickler.dump(x)
        self.wpos = self.fd.tell()
        self.size = self.size + 1

    def next(self):
        try:
            self.fd.seek(self.rpos)
            x = self.unpickler.load()
            self.rpos = self.fd.tell()
            return x

        except EOFError:
            raise StopIteration

    def __iter__(self):
        self.rpos = 0
        return self

Comment: @piotr: a 'leak' is when the memory is still claimed but is inaccessible to the application. If python can still use the memory but hasn't decided to free it, say it's lying stale in a cache somewhere, then it isn't a leak.
Comment: When you do `for i in xrange('1000000000')` you'll get a TypeError.
Comment: Why not use `shelve`? http://docs.python.org/library/shelve.html
Comment: How are you measuring memory consumption?  Are you aware that Python rarely (almost never) returns memory to the OS?
Comment: @S.Lott sort of, but then it should stabilize at some point right? one thing is not returning and the other is leaking...
Comment: @piotr: "sort of, but" is not a helpful answer.  How are you measuring memory consumption?
Comment: @Graham Lee If that' was the case the memory would stabilize, where is all that memory going?
Comment: @piotr: not necessarily. A cache can monotonically grow too.
Comment: @S.Lott then why this simple program seems to release memory?:    map(lambda x: None, range(10000000))
    sys.stdin.read()
Comment: @piotr: How are you measuring memory consumption?
Answer: The pickler module is storing all objects it has seen in its memo, so it doesn't have to pickle the same thing twice. You want to skip this (so references to your objects aren't stored in your pickler object) and clear the memo before dumping:
def append(self, x):
    self.fd.seek(self.wpos)
    self.pickler.clear_memo()
    self.pickler.dump(x)
    self.wpos = self.fd.tell()
    self.size = self.size + 1

Source: http://docs.python.org/library/pickle.html#pickle.Pickler.clear_memo
Edit:
You can actually watch the size of the memo go up as you pickle your objects by using the following append function:
def append(self, x):
    self.fd.seek(self.wpos)
    print len(self.pickler.memo)
    self.pickler.dump(x)
    self.wpos = self.fd.tell()
    self.size = self.size + 1

Comment: Yes, it does. When you call `self.pickler.dump(x)`, the pickler object does something like `self.memo.append(x)`. As you go through your `while True:` loop in your example code, you are creating thousands of objects which your pickler object is keeping references to, meaning they are kept in memory and not gotten rid of by the GC. Calling `self.pickler.clear_memo()` essentially causes the pickler to do `self.memo = []`, getting rid of any references to the objects and allowing the GC to get rid of them.
Comment: This made a huge difference in memory consumption, from 1.5G to 11M
Comment: That doesn't explain the increase in memory, since the object being pickled again is the same.
Comment: @poitr - I've edited my answer with some code which will allow you to watch the size of the memo increase as you pickle things.
Comment: Just bear in mind that this might cause your pickle to be larger - nothing will be picked by reference.
Answer: To add to the answer by combatdave@:
I just bypassed the terrible memo caching in pickle since clearing the memo on the reader side seems impossible and was an apparently unavoidable memory leak. Pickle streaming seem to be designed for reading and writing moderately sized files, not for reading and writing unbounded streams of data.
Instead I just used the following simple utility functions:
def framed_pickle_write(obj, stream):
    serial_obj = pickle.dumps(obj)
    length = struct.pack('>I', len(serial_obj))
    stream.write(length)
    stream.write(serial_obj)

def framed_pickle_read(stream):
    data = stream.read(4)
    length, = struct.unpack('>I', data)
    serial_obj = stream.read(length)
    return pickle.loads(serial_obj)


NoClassDefFound of MasterNotRunning when using janusgraph backend HBase

Question: I am using janusgraph, with storage backend HBase. Currently I am trying to add vertices to the database. The part of the code is
public class Graph {
  private static JanusGraph graph = JanusGraphFactory.open("conf/jg.properties");

  public static JanusGraph getGraph() {
    return graph;
  }
  public static void addVertex() {
    for (int i=0; i<5; i++) {
        graph.addVertex("test", i);
    }
    graph.tx().commit();
  }
}

with the main function calls
Graph.addVertex();

The error is
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:56)
at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1376)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:80)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 16 more

I am using janusgraph 0.2.0, in maven  0.2.0
and hbase 1.2.0, java 1.8
I set storage-hostname=127.0.0.1 in jg.properties, so is it a dependency error? Where is exactly the MasterNotRunning?
Comment: `Could not find implementation class` means: A java class is instantiated by reflection (see java.lang.*reflect*.InvocationTargetException in stack trace) at runtime. If the class is not present (class or containing jar not in classpath) then you will get this error.
Comment: janusgraph 0.3 is not released yet. Please downgrade to 0.2 according to [version-compat](https://mbrukman.github.io/docs.janusgraph.org/0.3.0-SNAPSHOT/version-compat.html)
Comment: @Rcordoval I changed it to `0.2.0` and the same problem remains
Comment: Looks like you are missing the janusgraph-hbase jar file
Comment: @Litchy look at this [pom.xml](https://github.com/JanusGraph/janusgraph/blob/e1b087566bbc85ecfadaef9455c4f8358ae88947/janusgraph-hadoop-parent/janusgraph-hadoop-2/pom.xml)
Comment: @nos I added `janusgraph-hbase` and the error becomes `java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/MasterNotRunningException`, I set `storage-hostname=127.0.0.1`, how could this happen?..
Comment: @Litchy There seems to be another dependency to `org/apache/hadoop/*` that you have to resolve.
Comment: @Würgspaß I also added another `janusgraph-hadoop` and the problem remains. I have updated the error and the dependencies up to now above
Comment: After adding the dependencies the error becomes class not found of `MasterNotRunning`
Comment: @Litchy If org/apache/hadoop/ is missing, you are missing some hadoop jar files. Are you using some form of dependency management (e.g. maven) ? If you are doing all this manually, you have quite a bit of job in front of you to track down all the correct dependencies and the proper versions of jar files that work together.
Comment: @nos I am using maven, adding dependencies `janusgraph-core,janusgraph-hbase,janusgraph-hadoop`, after adding `janusgraph-hbase` the error becomes above
Comment: @Rcordoval do you mean that I need to add all the dependencies in this `pom.xml` to my `pom.xml`?
Comment: no, cross check versions...
Answer: I have found the solution, but still curious.
I search the class MasterNotRunning and found this is in package org.apache.hadoop.hbase and finally make a guess to add org.apache.hbase & hbase-client into dependency. And the error disappears.
However, I still could not find the reason of this error. If some code in dependency jar uses the class MasterNotRunning then it should import the hbase-client jar, then this should be already in the dependency. How can this code pass the compiling and finally throws an exception during the running time.
Just add, I uses export->runnable jar file to get my jar, so all the dependencies should be added into it.


SQL VarChar to Date

Question: hi
i am trying to convert a VarChar date field (e.g. 20100320) to a real date field like

'dd/mm/yyyy' (e.g. 20/03/2010).

I have tried two ways:
a)
 (SELECT    MIN(CAST(A.DateOfAction AS Date)) AS Expr1
           FROM      ResAdm.Action A
           WHERE    (A.PersonID = P.PersonID))

     AS 'Period From',

b)
(SELECT    MIN(CONVERT(DATE, A.DateOfAction, 103)) AS Expr1
           FROM      ResAdm.Action A
           WHERE    (A.PersonID = P.PersonID))

     AS 'Period From',

both producing the result like

yyyy-mm-dd (e.g. 2010-03-20)

but i want the result like

dd/mm/yyyy (e.g. 20/03/2010)

any help will be appreciated.
thanks.
Answer: Try this:
select convert(varchar(8), convert(datetime, min(a.DateOfAction), 112), 103)

Your problem is that once you have a date format, SQL Server will dump it out in its default date format, which you've discovered is yyyy-mm-dd.  You need to convert from date to varchar to get the format you want.  But to convert from date, you need to first convert to date!  112 is the format for yyyymmdd, and 103 is the format for dd/mm/yyyy, so this is why you need these formats.  (Books Online reference for date formats)
Answer: Declare @date nvarchar(100)
set @date = '20100320'
select convert(varchar, CONVERT(datetime, @date, 109), 103)

You can use
convert(varchar, CONVERT(datetime, A.DateOfAction, 109), 103)

Comment: not only is it is lazy to just use "varchar", but it is a very bad habit as well.  When you use just "varchar" without a defined length your actually get a "varchar(30)" with may be more or less than you actually need.  if it is less then you will have silent truncation and a fun time debugging.
Comment: Declare @date nvarchar(10)
set @date = '20100320'
select convert(varchar(10), CONVERT(datetime, @date, 109), 103)

SQL VarChar to Date


Inconsistent Accessibility of Nested object Declarations in Kotlin

Question: Consider the following Kotlin code:
object Domain {
  val name = "local"
  val location = object {
     val city = "Pittsburgh"
     val state = "Pennsylvania"
  }
}

While this definition is valid and compiles, the following line fails:
val x = Domain.location.city // Error:(30, 27) Kotlin: Unresolved reference: city

However, if we rewrite the above definition as follows:
object City {
    val city = "Pittsburgh"
    val state = "Pennsylvania"
}
object Domain {
    val name = "local"
    val location = City
}

val x = Domain.location.city    // works fine

My question: is this really correct behavior according to the language spec? This doesn't seem sensible or consistent. It makes it inconvenient to declare complex singleton object declarations without breaking each child object out into a top-level declaration. It seems like the compiler is creating anonymous types when this syntax is used, however, type of the assigned value is always object when the definition is in a nested context. This almost seems like a type inference bug in the compiler. What am I missing?
Comment: Upon further review, it appears that the intended way to create nested object declaration is as follows:
`object Foo {
   val x;
   object Bar { val y }
 }`
However, since the other syntax exists and produces, somewhat useless behavior, it still seems like it should either compile into the equivalent code, or be disallowed.
Answer: What creating an anonymous object does is create a class that extends Any. Since no type information can be given except the superclass (in this case, Any), all fields not declared in that superclass will not be visible.
By creating your named object Bar, the class Bar is generated, as are the references to the fields, which is why they are visible in that case.
Comment: I understand the basic premise of object; the weirdness is that if you declare them as I did in my second example above (two top-level objects with the second referencing the first) sufficient type information is retained to allow references to the fields of the inner object. This seems inconsistent and unexpected. Also, while the alternate syntax works and is less verbose than breaking objects up, it has the downside that code that uses reflection to traverse the members of the anonymous objects doesn't see the nested object properties (since they're not properties but actually nested types).
Comment: I'm not sure why you think that referncing a top level object would not retain enough information to allow accessing its fields. Since it is a named object, it will have a generated class file, keeping all field information and allowing access. When you create an anonymous class, the generated class file keeps no information beyond the supertype information.


Anchors (URL) instead of text (URL)

Question: Trying to achieve the following logic:
If URL in text is surrounded by paragraph tags (Example:
URL
), replace it in place to become a link instead: Click Here
The original file is a database dump (sql, UTF-8). Some URLs already exist in the desired format. I need to fix the missing links.
I am working on a script, which uses Beautifulsoup. If other solutions are make more sense (regex, etc.), I am open to suggestions.
Comment: Please provide some examples of what you've already done, note any issues you're having.
Answer: You can search for all p elements that has a text starting with http. Then, replace it with a link:
for elm in soup.find_all("p", text=lambda text: text and text.startswith("http")):
    elm.replace_with(soup.new_tag("a", href=elm.get_text()))

Example working code:
from bs4 import BeautifulSoup

data = """

http://google.com

https://stackoverflow.com

"""

soup = BeautifulSoup(data, "html.parser")
for elm in soup.find_all("p", text=lambda text: text and text.startswith("http")):
    elm.replace_with(soup.new_tag("a", href=elm.get_text()))

print(soup.prettify())

Prints:

I can imagine this approach break, but it should be a good start for you.

If you additionally want to add texts to your links, set the .string property:
soup = BeautifulSoup(data, "html.parser")
for elm in soup.find_all("p", text=lambda text: text and text.startswith("http")):
    a = soup.new_tag("a", href=elm.get_text())
    a.string = "link"
    elm.replace_with(a)

Comment: Alexander, thank you for your prompt response. I have tested your solution. It works fairly well. One more quick question if you don't mind. How do I decorate the anchors with a static text, so they do not appear empty? For instance, a href="http://google.com>Click Here instead of just empty tags?

Anchors (URL) instead of text (URL)


The probability of an observation belonging to one or other sample

Question: Assume I asked 10 children and 10 adults whether they like my cake. 9 kids did and 1 didn't, while for the adults it was the reverse. Using these data, can I say that any random person who likes my cake is more likely to be a child than an adult? Is it possible to put an exact probability on this guess?
Answer: Let $C$ denote being a child and $C'=A$ being an adult. Similarly, let $L$ denote liking the cake. Then, based on the data, we can estimate the following probabilities for the population:
$$P(L|C)=0.9, \ \ P(L|C')=0.1$$
On your sample we have $P(C)=P(C')=0.5$ since number of adults and children are the same. But, you can also assign prior probabilities to these groups, e.g. probability of a randomly chosen person being a child is $0.3$ for example. But, this is not inferred from the data.
What we're interested in is $P(C|L)$, which can be expanded via Bayes Rule:
$$P(C|L)=\frac{P(L|C)P(C)}{P(L)}=\frac{P(L|C)P(C)}{P(L|C)P(C)+P(L|C')P(C')}$$
Once agreed on priors, the rest is substitution.
Comment: Fantastic, thank you very much!


How do I tell whether an executable has run or not?

Question: I have an executable I am calling from a third party program.  I am calling it using an administrator account with "run as a service" privileges, and the third party program logs report that the executable ran "successfully" with a runtime of  0.000 seconds (though the start & end times shown by the third party program are a couple hundred milliseconds apart).  However, the executable is supposed to output a jpg, which is not happening.
If I run the executable from the command line, it works fine (outputs jpg - runtime is about 20 seconds).  If I run it from a windows scheduled task with no users logged on, it works fine.  Also, I can run other executables (tried notepad.exe) from the third party program successfully.
So, my question is...How can I monitor the executable to see if it was actually run, whether permissions issues were encountered, etc.?
Answer: I use perfmon and set a counter that includes "All instances" of the Process counter.  Also, take a look at SysInternal's Process Monitor, which is very verbose, unless you know what you're looking for.  For instance, use the filter:
Process *contains* 3rdparty.exe Include.
Comment: Thanks for the ideas!  I tried perfmon first, but it seems it only shows current active processes, correct?  Part of my problem is that this executable fires & then closes in milliseconds, so I have to have something that holds onto history.  I'm also looking through Process Monitor.  Your suggested filter worked great.  I'm currently combing through it trying to figure out where it gets hung up.
Comment: Set up a Counter Log, when you add "All Instances," it keeps track of all instances of processes running throughout the time the counters are being recorded (whether they exist when the counter starts or stops is irrelevant).  Process Monitor will definitely tell you what's going on, remember to Filter menu> Drop Filtered Events or it will happily eat your page file alive.
Answer: Does the service account have permission to read and write & execute in the proper folders?
You can check the permissions by hand.
Comment: Yes, I've given the service account appropriate permissions, and I've successfully run a different executable from the same folder.
Comment: It sounds like that user can't write to the destination.  This is what tkrabec is saying.  Use Effective Permissions on the security tab.
Answer: Process Monitor (procmon) by SysInternals/Microsoft will show you all kinds of stuff.  You can have it filter on process name -- that way you'll see what it's doing and (probably) see where it fails to access something.
Do make sure to set a filter - it's overwhelming to see everything taking place no the system :)


O que pode ser feito para melhorar a performance de um CSS muito grande (384KB)

Question: Meu site possui uma folha de estilos, já minificada, com tamanho total de 384KB.
Se possível, o que posso fazer para melhorar o desempenho?
Pensei em comprimir com GZIP, o que deixaria a folha com 59KB, e até dividir essa folha em 2 folhas de 29,5 KB cada. Isso ajuda?
Comment: Se o Gzip reduziu esse tanto o tamanho, é porque tem coisa repetida que provavelmente não precisava ser (afinal, é assim que os compressores funcionam...). É meio difícil responder assim "no ar" o que pode ser feito, mas se você postasse sua folha no pastebin ou em outro site desse tipo nós poderíamos dar uma olhada e, quem sabe, ajudar a identificar alguns anti-padrões... P.S. Dividir a folha em duas ou mais não ajuda, pelo contrário, atrapalha, e eu concordo com o Bacco sobre a questão do gzip.
Comment: Provavelmente organizar os estilos é o melhor caminho. Grande chance do problema ser o jeito que a própria folha foi feita. Pode até existir uma razão verdadeira pra uma folha ter tudo isso, mas eu apostaria mais em não ser esse o caso. E o gzip quem tem que fazer é o servidor, automaticamente na negociação do protocolo, se estiver devidamente configurado.
Comment: Faço das minhas palavras as do Bacco e mgibsonbr. Provavelmente a melhor maneira é ajeitar sua folha, se possível, claro.
Comment: Você já deu uma olhada na aba `Audits` do Developer Tools no Chrome? É exibido uma série de itens do que pode ser feito para melhorar a performance da sua página
Comment: É que eu juntei várias folhas em uma só. Por isso tem esse tamanho todo. (ps: o site é responsivo, então aumenta mais o tamanho da folha porque tem mais linhas). A folha está correta, o problema não é esse. O Tamanho se deve apenas a várias folhas unificadas. Sobre o gzip, eu já sei que é feito pelo servidor, e eu já utilizo ._.
Comment: Eu só sei que gzipado fica 59KB porque fiz testes via servidor.
Comment: Acho que se você uniu varias folhas ctza existe varios componentes em comum podendo unificar no css, mas se fazer isso dependendo pode perder a legibilidade.
Comment: Olha isso, talvez ja tenha visto, http://browserdiet.com/pt/
Answer: Um quantidade de estilos como essa deve se encaixar em uma ou mais das seguintes categorias...
Redundância de atributos
Pode ser que haja muitos atributos repetidos em diversas classes. Por exemplo, imagine que você esteja formatando os títulos assim:
h1 { font-weight: bold; font-size: 24px; ... }
h2 { font-weight: bold; font-size: 20px; ... }
h3 { font-weight: bold; font-size: 16px; ... }
...

Nesse caso, poderia agrupar os atributos comuns assim:
h1, h2, h3 { font-weight: bold; ... }
h1 { font-size: 24px; ... }
h2 { font-size: 20px; ... }
h3 { font-size: 16px; ... }

Estilos muito específicos
Outro problema comum é usar classes sempre muito específicas. Já vi alguns sistemas com uma classe para cada elemento usando o id como seletor.
Imagine algo como o seguinte:
#link1 { A; B; C; D }
#link2 { A; B; C; D }
#link3 { A; B; C; D; E }

(Sendo A, B, C, D e E estilos quaisquer)
Se você tiver várias classes CSS com estilos parecidos, separe o que é igual e coloque numa nova classe, então use composição de estilos no atributo class dos elementos da página separando o nome das classes por espaços em branco.
Exemplo:
.link { A; B; C; D }
.link-maior { E }

Meu Link

No exemplo acima, a classe .link poderia conter os atributos gerais dos links, enquanto a classe .link-maior teria uma formatação específica para links maiores.
Falta de padrão
Um CSS grande também pode ter origem em falta de padronização, onde cada componente de cada tela é um pouco diferente dos demais.
Neste caso o ideal é refatorar as telas, padronizando-as em um conjunto de estilos unificado.
Ferramenta de geração
Outra possibilidade é que alguma ferramenta está gerando os estilos com muito lixo, sendo que vários dos cenários acima podem ocorrer simultaneamente.
Conheço uma ferramenta desse tipo onde todos os componentes da tela são posicionados de forma absoluta. Como os posicionamentos e estilos são únicos para cada elemento da página, o sistema gera um arquivo CSS com uma classe para cada elemento com sua posição e formatação.
O ideal seria jogar fora essa ferramenta e fazer as telas do jeito certo. Caso não seja possível, ao menos divida os estilos de cada tela em arquivos separados.
Sistema realmente grande e complexo
Por último, pode ser que o sistema seja realmente grande o suficiente para justificar um estilo tão grande.
Neste caso, a melhor estratégia seria dividir o arquivo de forma que ele não seja carregado como um todo.
Coloque as classes principais usadas em todas as telas num arquivo principal, depois divida o restante dos estilos em diferentes arquivos seguindo algum critério relacionado ao uso do sistema, como grupos de telas que geralmente são usadas ao mesmo tempo, tipos de usuário (administrador, normal, ...) ou ainda com um CSS individual para cada tela.
O tamanho total dos estilos não vai diminuir, mas o carregamento dos CSSs será diluído entre várias requisições, melhorando um pouco a experiência do usuário ao acessar o sistema.
Não se esqueça de comprimir
O Autor da Pergunta (AP) afirmou que o arquivo já está comprimido ou "minificado", o que consiste na remoção de todos os caracteres desnecessários como espaços em branco e quebras de linha.
Mas para deixar a resposta completa, não podemos esquecer de que aplicar a "minificação" é um passo elementar para a otimização do tempo de resposta de qualquer aplicação não trivial.
Comment: +1. Tirando o último caso (dado o tamanho do CSS), acho que tem um pouco de tudo atrapalhando a folha em questão.
Comment: Faltou mencionar espaços em branco e comentários que são desnecessários e podem ser removidos automaticamente por ferramentas como [YUI Compressor](http://yui.github.io/yuicompressor/css.html). Tem até uma [versão online](http://refresh-sf.com/yui/) dele.
Comment: @BrunoAugusto Verdade. Mas não falei porque o AP disse que o dele já está comprimido.
Comment: Sim, sim, falei mais pra deixar a resposta mais completa até mesmo a título de referência futura.
Comment: Sim @utluiz é um site bem complexo, possui bastaaaantes elementos, paginas... É normal um site ter pelo menos 59KB de CSS em todo o site? Ou ainda tá muito? asd


OLS estimate of a linear model with dummy variable

Question: I know a regression of y on x (dummy variable) and a constant term can be represented in the following form:

On the other hand OLS estimator can be presented in the following form:

I need to see how these two equations for Beta estimation are related to one another.
Comment: This looks like a [routine textbook style question](http://stats.stackexchange.com/tags/self-study/info). Tell me (you can work it out) - what's the value of $\bar x$? of $x_i-\bar x$ (for the two possible cases)? of $\bar y$ in terms of $\bar y_1$ and $\bar y_0$?
Comment: Yes this is a textbook question and your hint helped. Thanks!
Comment: If you got the question out, would you like to post an answer?
Comment: could you briefly talk about how to get the estimation for beta1 from the original regression model? how do you know beta_hat = y1_bar - y0_bar ?
Comment: Hi @Glen_b, I'm struggling to get an answer to a similar question. This time, it's the case of a simple linear regression with a three level categorical variable. Can you please provide me with any guidance? Here is the link of the question: https://stats.stackexchange.com/questions/546082/least-squares-estimates-of-simple-linear-regression-with-three-level-categorical
Answer: The model that we have is:

Knowing that X is a dummy variable, we can get the following:

Using the above information, we can substitute for components of OLS estimation and by simplifying we get:


iterate over array of objects and fetch only specific key to form a new array of object in react js

Question: I have below object. I want to iterate over it and need to fetch only key's values of ability and value key.
var object = [{
    ability: "Strong",
    name: "STRENGTH",
    value: "customer has 10 properties",
    type: "MultiChoice"
  },
  {
    ability: "Good",
    name: "STRENGTH",
    value: "customer has 5 properties",
    type: "MultiChoice"
  },
  {
    ability: "average",
    name: "STRENGTH",
    value: "customer has 3 properties",
    type: "MultiChoice"
  }
]

So my expected output should be like below.
[{
    ability: "Strong"
    value: "customer has 10 properties"
  },
  {
    ability: "Good"
    value: "customer has 5 properties"
  },
  {
    ability: "average"
    value: "customer has 3 properties"
  }
]

Then I again want to use above object to iterate over it and bind its values to dropdown. So my drop-down will display "value" to end user and DB will have "ability". I am trying below code snippet but its not giving expected output. Something I am doing wrong here.

var object = [{
    ability: "Strong",
    name: "STRENGTH",
    value: "customer has 10 properties",
    type: "MultiChoice"
  },
  {
    ability: "Good",
    name: "STRENGTH",
    value: "customer has 5 properties",
    type: "MultiChoice"
  },
  {
    ability: "average",
    name: "STRENGTH",
    value: "customer has 3 properties",
    type: "MultiChoice"
  }
]

let multiChoiceResult = [];
let ability, value
Object.entries(object).map(([key, value]) => {
  ability = value.ability;
  value = value.value;
  multiChoiceResult.push(ability, value);
});
console.log(multiChoiceResult);
Comment: I made you a snippet. Please fix the console error - you do not have `rangesetDeduced, rangesetValue)`
Comment: Just to note - your object is not an object, it's an array.. of objects! .. and actually calling an object 'object' might get a tad confusing.
Comment: @mplungjan code snippet fixed. Can you please help what wrong I am doing here.
Comment: I added commas on the objects
Comment: I updated my answer with a select
Answer: Your solution expects an object as input. But object actually is an array. You can simply use the map function on the given items as follows:

var object = [
  {
    ability: "Strong",
    name: "STRENGTH",
    value: "customer has 10 properties",
    type: "MultiChoice"
  },
  {
    ability: "Good",
    name: "STRENGTH",
    value: "customer has 5 properties",
    type: "MultiChoice"
  },
  {
    ability: "average",
    name: "STRENGTH",
    value: "customer has 3 properties",
    type: "MultiChoice"
  }
];

console.log(
    object.map((item) => ({ ability: item.ability, value: item.value }))
)
Comment: Thanks for this code. It has worked fine and I got an object in required format. like this. [
0 :  { ability: "Strong" ,value: "customer has 10 properties" }
1 :  { ability: "Good",   value: "customer has 5 properties" }
2 :  { ability: "average", value: "customer has 3 properties" }
] But how can I use this object to bind the values to dropdown. There I am stuck really.
Comment: @Nilesh You are welcome. Would be great if you'd accept it as the answer to your problem.
Comment: I need to show value's value to end user and its ability value will be saved in back end.
Comment: Since I haven't seen any code in that regards, it's hard to help you with that. How about you give it a try with the step you got further now and in case you are stuck again, you can ask a new question for that precise problem.
Comment: @Nilesh I added a select to my answer
Answer: As in your code snippet, it's an array of objects. So you can use the map function to iterate it.
object.map((item) => ({ ability: item.ability, value: item.value }));

Answer: The map can be vastly simplified
https://ariya.io/2013/02/es6-and-object-literal-property-value-shorthand

var object = [{ ability: "Strong", name: "STRENGTH", value: "customer has 10 properties", type: "MultiChoice" },
  { ability: "Good", name: "STRENGTH", value: "customer has 5 properties", type: "MultiChoice" },
  { ability: "average", name: "STRENGTH", value: "customer has 3 properties",  type: "MultiChoice" }]

let multiChoiceResult = Object.values(object)
  .map(({ability, value}) => ({ability, value}));

console.log(multiChoiceResult);

For a drop down you can do

var object = [{ ability: "Strong", name: "STRENGTH", value: "customer has 10 properties", type: "MultiChoice" },
  { ability: "Good", name: "STRENGTH", value: "customer has 5 properties", type: "MultiChoice" },
  { ability: "average", name: "STRENGTH", value: "customer has 3 properties",  type: "MultiChoice" }]

let multiChoiceResult = Object.values(object)
  .map(({ability, value}) => `<option value="${ability}">${value}</option>`);

document.getElementById("abilities").innerHTML+=multiChoiceResult.join("")
<select id="abilities" name="abilities">
  <option value="">Please select</option>
</select>


$\lim_{n\to\infty}n^2(\int_{0}^1 \sqrt[n]{x^n+(1-x)^n}\,dx-\frac{3}{4})$

Question: I saw this problem in a problem book. and I have no idea how to find the limit here.
$$\lim\limits_{n\to\infty}n^2\left(\int\limits_{0}^1 \sqrt[n]{x^n+(1-x)^n}\,dx-\frac{3}{4}\right)$$
I know that$$\lim\limits_{n\to\infty}\int\limits_{0}^1 \sqrt[n]{x^n+(1-x)^n}\,dx =\frac{3}{4}$$

proof $$  \int _{0} ^ {\frac{1}{2}}  \sqrt[n]{(1-x)^n}dx   + \int _{{\frac{1}{2}}} ^ 1  \sqrt[n]{x^n}dx= \frac{3}{2} \leq \int _0 ^ 1  \sqrt[n]{x^n+(1-x)^n}dx \leq  \int _{0} ^ {\frac{1}{2}}  \sqrt[n]{2(1-x)^n}dx   + \int _{{\frac{1}{2}}} ^ 1  \sqrt[n]{2x^n}dx= \sqrt[n]{2}\frac{3}{2}$$  since $\lim_{n \to \infty }\sqrt[n]{2}=1 $ thenby squeeze theorem $\lim\limits_{n\to\infty}\int\limits_{0}^1 \sqrt[n]{x^n+(1-x)^n}\,dx =\frac{3}{4}$

by attempt was to use the fact that
$I<\sqrt[n]{2}\frac{3}{4}$
so
$$I-\frac{3}{4}<\frac{3}{4}(\sqrt[n]{2}-1)$$
$$n^2(I-\frac{3}{4})<n^2(\frac{3}{4}(\sqrt[n]{2}-1))$$
but $$\lim\limits_{n\to\infty}n^2(\frac{3}{4}(\sqrt[n]{2}-1)) =\infty $$
and here I got stuck
Comment: I calculated it for really some really large $n$ using Mathematica. This also gave the next term in the asymptotic expansion as $\zeta(3)/8$, so it seems there's some zeta function stuff going on.
Comment: With some fancy changes of variables, you can get the equivalent limit $\lim_{n\rightarrow\infty} n^2\int_0^1((1+u^n)^{1/n}-1)/(1+u)^3du$. That binomial term looks more tractable now.
Comment: This problem has been asked today.
Comment: Numerically it looks to be $\pi^2/48$. No idea where that comes from.
Comment: what's the name of the book?@Ahmed
Comment: @Ahmed barak Allahu fik
Comment: where ? If you can ,help me with this problem
Comment: how did you get that answer ? also The answer is correct
Comment: this is impressive but how did you get that what substitution did you apply? and this would solve the problem
Comment: @
Sine of the Time
 sharpening mathematical analysis skills https://link.springer.com/book/10.1007/978-3-030-77139-3
Answer: Define $I_n$ by
\begin{align*}
I_n
&:= \int_{0}^{1} \sqrt[n]{x^n + (1-x)^n} \, \mathrm{d}x - \frac{3}{4}.
\end{align*}
Using the symmetry,
\begin{align*}
I_n
&= 2 \int_{0}^{\frac{1}{2}} (1 - x)\left(1 + \left(\frac{x}{1-x}\right)^n \right)^{1/n} \, \mathrm{d}x - \frac{3}{4} \\
&= 2 \int_{0}^{\frac{1}{2}} (1 - x) \left[ \left(1 + \left(\frac{x}{1-x}\right)^n \right)^{1/n} - 1 \right] \, \mathrm{d}x.
\end{align*}
Now we substitute $x = \frac{1}{2} - \frac{t}{n}$. Then
\begin{align*}
I_n
&= \frac{1}{n} \int_{0}^{n/2} \left(1 + \frac{2t}{n}\right) \underbrace{\left[ \left(1 + \left(\frac{1 - 2t/n}{1 + 2t/n}\right)^n \right)^{1/n} - 1 \right]}_{[\blacksquare]} \, \mathrm{d}t.
\end{align*}
To analyze the behavior of this limit, we note that $\frac{1-x}{1+x} \leq e^{-2x}$ and $(1 + x)^{\alpha} \leq 1 + \alpha x$ for all $x \geq 0$ and $\alpha \in [0, 1]$. From this, we get
$$ 0 \leq [\blacksquare] \leq \frac{1}{n} e^{-4t}. $$
This allows us to invoke the dominated convergence theorem, concluding that
\begin{align*}
&\lim_{n\to\infty} n^2 I_n \\
&= \int_{0}^{\infty} \lim_{n\to\infty} \left(1 + \frac{2t}{n}\right) n [\blacksquare] \mathbf{1}_{\{t \leq n/2\}} \, \mathrm{d}t \\
&= \int_{0}^{\infty} \log(1 + e^{-4t}) \, \mathrm{d}t \\
&= \frac{\pi^2}{48}.
\end{align*}
Answer: Let's denote $\displaystyle I(n)=\int\limits_{0}^1 \sqrt[n]{x^n+(1-x)^n}\,dx$, then
$$I(n)\overset{x=\frac t2+\frac12}{=}\frac12\int_0^1(1+t)\sqrt[n]{\left(1+\frac{1-t}{1+t}\right)}dt\overset{x=\frac{1-t}{1+t}}{=}2\int_0^1\frac{\sqrt[n]{(1+x^n)}}{(1+x)^3}dx$$
-this is the convenient form obtained by @eyeballfrog
$$=2\int_0^1\frac{e^\frac{\ln(1+x^n)}n}{(1+x)^3}dx=2\int_0^1\frac{dx}{(1+x)^3}\left(1+\frac1n\ln(1+x^n)+\frac1{2n^2}\ln^2(1+x^n)+...\right)\tag{1}$$
$$=\frac34+\frac2n\int_0^1\frac{\ln(1+x^n)}{(1+x)^3}dx+o\big(\frac1{n^2}\big)$$
In turn,
$$I_2(n)=\frac2{n}\int_0^1\frac{\ln(1+x^n)}{(1+x)^3}dx\overset{t=x^n}{=}\frac2{n^2}\int_0^1\frac{\ln(1+t)\,t^{\frac1n-1}}{(1+t^{\frac1n})^3}dt$$
The integrand is dominated by $\displaystyle f(t)=\frac{\ln(1+t)}t$. Taking the limit under the integral sign
$$\lim_{n\to\infty}n^2I_2(n)=\frac14\int_0^1\frac{\ln(1+t)}tdt=\frac{\pi^2}{48}$$

Addendum
We can evaluate third asymptotic term ($\sim\frac1{n^3}$). We have to take the third term from (1):
$$J_1=\frac1{n^2}\int_0^1\frac{\ln^2(1+x^n)}{(1+x)^3}dx\overset{t=x^n}{=}\frac1{n^3}\int_0^1\frac{t^{\frac1n-1}\ln^2(1+t)}{(1+t^\frac1n)^3}dt$$
$$=\frac1{8n^3}\int_0^1\frac{\ln^2(1+t)}tdt+o\big(\frac1{n^3}\big)=\frac{\zeta(3)}{32n^3}+o\big(\frac1{n^3}\big)\tag{2}$$
We also have to extract the part $\sim\frac1{n^3}$ from the second term in (1):
$$J_2=\frac2n\int_0^1\frac{\ln(1+x^n)}{(1+x)^3}dx\overset{IBP}{=}-\frac1n\frac{\ln(1+x^n)}{(1+x)^2}\,\bigg|_0^1+\int_0^1\frac{x^{n-1}}{(1+x)^2(1+x^n)}dx$$
$$\overset{x=e^{-t}}{=}-\frac{\ln 2}{4n}+\int_0^\infty\frac{dt}{(e^{nt}+1)(1+e^{-t})^2}$$
$$\overset{x=nt}{=}-\frac{\ln 2}{4n}+\frac1n\int_0^\infty\frac{dx}{(e^x+1)(1+e^{-\frac xn})^2}$$
Dropping exponentially small corrections, we are allowed to decompose $e^{-\frac xn}$ into the series.
$$J_2=-\frac{\ln 2}{4n}+\frac1n\int_0^\infty\frac{dx}{(e^x+1)(2-\frac xn+\frac{x^2}{2n^2}-...)^2}$$
$$=-\frac{\ln 2}{4n}+\frac1{4n}\int_0^\infty\left(1+\frac xn-\frac{x^2}{2n^2}+\frac34\frac{x^2}{n^2}+O\big(\frac1{n^3}\big)\right)\frac{dx}{e^x+1}$$
$$=\frac{\pi^2}{48}\frac1{n^2}+\frac1{16n^3}\int_0^\infty\frac{x^2}{e^x+1}dx+O\big(\frac1{n^4}\big)\tag{3}$$
Taking the term $\sim\frac1{n^3}$ from (2) , together with (3) we get third asymptotic term:
$$I_3=\frac{\zeta(3)}{32n^3}+\frac1{16n^3}\int_0^\infty\frac{x^2}{e^x+1}dx=\frac{\zeta(3)}{8n^3}\tag{4}$$
It means that
$$\boxed{\,\,I(n)=\frac34+\frac{\pi^2}{48n^2}+\frac{\zeta(3)}{8n^3}+O\big(\frac1{n^4}\big)\,\,}$$
Given the symmetry of several first asymptotic terms, we may suppose that a concise form for full asymptotics exists.
Comment: Maybe related, https://link.springer.com/article/10.1007/s11139-019-00235-z.
Comment: Nice ! When I introfuce the newt term $a=0.150259630131738$. Cheers & (+1)
Comment: @Ahmed $o(1/n^2)$ means the term that declines more rapidly than $\frac1{n^2}$. It can be, for example, $\frac{C}{n^3}$ at $n\to\infty$
Comment: @Svyatoslav what does $o\big(\frac1{n^2}\big)$ mean , I see it very often but I don't know what it is. is $o\big(\frac1{n^2}\big)$ is somekind of special function ?
Answer: This is not an answer.
$$I_n=-\frac{3}{4}+2\int_{0}^{\frac 12} \sqrt[n]{x^n+(1-x)^n}\,\,dx$$
Computing them (for $10 \leq n \leq 1000$ by step of $10$) and using a quick and dirty linear regression
$$n^3\,I_n=a + b\,n$$ gives (long decimal numbers being truncated), with $R^2=0.9999999998917$,
$$\begin{array}{|llll}
\hline
 \text{} & \text{Estimate} & \text{Std Error} & \text{Confidence
   Interval} \\
 a & 0.148594 & 2.54\times 10^{-4} & \{0.148090,0.149097\} \\
 b & 0.205619 & 4.36\times 10^{-7} & \{0.205618,0.205620\} \\
\end{array}$$
$b=\frac {\pi^2}{48}$ is "obvious" (thanks to Wolfram Alpha).
For this value of $b$, re-tuning $a$ gives, at the level of $95$%
$$0.149414\leq a \leq 0.149974$$
Using
$$a=0.14969420124943726324726381376809257943038859$$ gives
$$I_{2000}=\color{red}{5.14229}013\times 10^{-8}\qquad \text{instead of} \qquad  5.14229654\times 10^{-8}$$  Searching the $ISC$, nothing "simple" came out.
Now, considering
$$n^3\,I_n=a + \frac {\pi^2}{48}\,n- \frac {b}{n}$$
$R^2=0.9999999999995880$ starts to be impressive
$$\begin{array}{|llll}
\hline
 \text{} & \text{Estimate} & \text{Std Error} &
   \text{Confidence Interval} \\
 a & 0.150313 & 8.49 \times 10^{-6} & \{0.150296,0.150330\} \\
 b & 0.119254 & 6.64 \times 10^{-4} & \{0.117936,0.120572\} \\
\end{array}$$ gives
$$I_{2000}=\color{red}{5.14229}712\times 10^{-8}\qquad \text{instead of} \qquad  5.14229654\times 10^{-8}$$
No luck searching the $ISC$.


Tornado GoogleOAuth2Mixin yields an access token instead of a user object

Question: I'm trying out Tornado's GoogleOAuth2Mixin with the example code shown in the documentation. According to the docs, the function get_authenticated_user "Handles the login for the Google user, returning a user object." (emphasis mine), but the object I receive is a dictionary like so:
{
    u'access_token': u'...',
    u'expires_in': 3600,
    u'id_token': u'...',
    u'token_type': u'Bearer'
}

While this is helpful, and I know how to make another HTTP request with the access token to retrieve information about authenticating user, it seems like a bit of a stretch of the imagination to call this dictionary a "user object".
Is this really what I'm supposed to expect, and I'm now meant to go and get the user data myself with another request, or is something going wrong?
Answer: This function has never worked as documented, although now that it has existed so long in its current form the documentation, and not the code, will have to change. GoogleOAuth2Mixin.get_authenticated_user returns only the authentication data and you'll have to make a separate request for further details.


Services module: What is 'Index' operation?

Question:
D7 / Services 3.2
retrieve / create / update / delete are very straightforward, make senses to me that matching 'CRUD', I don't understand what 'index' here does?
Thanks.
Answer: 'Index' in services module will list out an entity(node, users, taxonomy..)  in descending Order of its created time, vid, name etc based on the arguments that are passed. The common arguments for the Index are

page  - Page number of results to return (in pages of 20).
fields - The fields you want returned.
parameters - An array containing fields and values used to build a sql WHERE clause indicating items to retrieve. (array)
pagesize - number of items to be returned. (Integer)
In case of retrieve, it returns the complete entity object of the passed 'entity_id'

UPDATED:
Sample URLS for Index is

http://domain/endpoint/node?fields=nid,vid&parameters[nid]=7&parameters[uid]=1&page=2
http://domain/endpoint/taxonomy_term?fields=tid,name&parameters[tid]=7&parameters[vid]=1
http://domain/endpoint/user?fields=uid,name,mail&parameters[uid]=1
http://domain/endpoint/node?pagesize=3

For documentation you can see Services: An API for remote applications.
Comment: How do I construct the 'index' argument in uri? e.g page argument. Is it something like:  http://///page/3 ? Is there any documentation on the web? (i am having hard time searching for docs)
Comment: @ gilzero I have updated the answer.
Comment: @DineshKumar Now if I want to create a pagination in js, I will need to know the total count of data. How can I find that?
Answer: services module provides some endpoints with resources.
index will return all the nodes depends on the resource you selected.
in this case suppose node is a resource and URL looks like this - http://domain/endpoint/node with a GET Request will return all the nodes.


Jackson json property (string) to instance

Question: Assuming I have the following JSON:
{
  "property": "123:1234"
}

How do I use Jackson annotations to ensure that the string value of "property" is de-serialized to a self-defined class rather than a String object?
I went through their documentation and I was unable to find this particular feature.
Thanks in advance.
Comment: do you want your property to be stored in some object with fields? Maybe delimeted by `:` ?
Comment: @michalk Yes. The string is used as an identifier within our system and I'd like to wrap it in a class dedicated to those identifiers.
Answer: First thing first define your class that needs to be used:
@JsonInclude(JsonInclude.Include.NON_NULL)
public class JsonTest{
@JsonProperty("property")
private String property;

    //define your getters and setters for the field

Then you can use the ObjectMapper class from jackson:
  public static <T> T extractObjectFromJson(String jsonText, Class<T> type) {
    try {
        return new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false).reader().forType(type)
                .readValue(jsonText);
    } catch (Exception e) {
        //Manage your exception here
    }
    return null;
}

So you can just call the method extractobjectFromJson(//Your JSON String, JsonTest.class)  to get your JSON deserialized.
Answer: You could create custom deserializer for your field. Assuming you want to map it to SomeClass object :
public class SomeClass {

    @JsonDeserialize(using = CustomPropertyDeserializer.class)
    private Properties property;

    public Properties getProperty() {
        return property;
    }

    public void setProperty(Properties property) {
        this.property = property;
    }
}

You annotate your field that you want to deserialize customly with @JsonDeserialize annotation passing custom deserializer.
Your deserializer could look like this :
public class CustomPropertyDeserializer extends StdDeserializer<Properties> {

    public CustomPropertyDeserializer() {
        super(Properties.class);
    }

    @Override
    public Properties deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
        String valueAsString = p.getValueAsString();
        String[] split = valueAsString.split(":");

        return new Properties(split[0], split[1]);
    }
}

And custom property class :
public class Properties {
    private String first;
    private String second;

    public Properties(String first, String second) {
        this.first = first;
        this.second = second;
    }

    public String getFirst() {
        return first;
    }

    public void setFirst(String first) {
        this.first = first;
    }

    public String getSecond() {
        return second;
    }

    public void setSecond(String second) {
        this.second = second;
    }
}

For testing it :
    public static void main(String[] args) throws IOException {
        String s = Files.lines(Paths.get("src/main/resources/data.json")).collect(Collectors.joining());

        ObjectMapper objectMapper = new ObjectMapper();

        SomeClass someClass = objectMapper.readValue(s, SomeClass.class);

        System.out.println(someClass.getProperty().getFirst());
        System.out.println(someClass.getProperty().getSecond());

    }

The output is then :
123
1234

So all the custom logic how to map your String to some class that you define could be placed in deserialize method of your custom deserializer.


Why won't jQuery's .text() recognize \n?

Question: I am trying to print html text on a page.
Right now I am doing
$("#generate_html_button").click(function(){
    var generated_html_text = $("#visual_form").html();
    $("#generated_html_text").text(generated_html_text);//generated_html_text is just a
 element
  });

If I print the generated_html_text to the console it will come out looking like

            dropdown

            <select class="form-control">
              <option value="option1">Option 1</option>
              <option value="option2">Option 2</option>
            </select>

the console will use the proper forming. When I use .text() to print the text to my page, it ignores the \n and just prints it in one long line.

 dropdown

 <select class="form-control"> <option value="option1">Option 1</option> <option value="option2">Option 2</option> </select>

Most suggest using .replace(), but when I tried generated_html_text = generated_html_text.replace(/\n/g, '
'); but that will produce the same result just with  everywhere without the formatting, and I am trying to avoid having
s in my output. I have also tried changing .text() to .html() but then it just renders the elements. Why won't .text() recognize the existing \n?
Answer: To make whitespace significant, try the CSS white-space: pre-wrap on your container element.
Answer: From http://php.net/manual/en/function.nl2br.php
<?php
echo nl2br("foo isn't\n bar");
?>

will output :
foo isn't

bar


Python3, Selenium and Chrome portable

Question: probably someone can help with Windows, Python, Selenium and using Chrome webdriver with ChromePortable.
I've defined a new folder
c:\myproject
Wihtin this folder the wedriver is located:
c:\myproject\driver\chromedriver.exe
and also the Chrome Portable
c:\myproject\chromeportable\chrome.exe
Now I would like to build a simple Python script which opens - let say - stackoverflow.com.
On a computer, where google is installed it isn't a problem like
from selenium import webdriver
driver = webdriver.Chrome("c:\myproject\driver\chromedriver.exe")
driver.get("https://stackoverflow.com")

However, how to change the script if no google chrome is installed and google chrome portable should be used?
Any idea?
Thank you very much in advance and have a nice day
Andreas
Answer: Use Options class:
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = "c:\myproject\chromeportable\chrome.exe"
# you may need some other options
#options.add_argument('--no-sandbox')
#options.add_argument('--no-default-browser-check')
#options.add_argument('--no-first-run')
#options.add_argument('--disable-gpu')
#options.add_argument('--disable-extensions')
#options.add_argument('--disable-default-apps')
driver = webdriver.Chrome("c:\myproject\driver\chromedriver.exe",
            options=options)

Comment: One more question!
Google Chrome portable recorgnize not click() events. Any idea? If I use the same code with local installed chome browser (none portable version) ist works.
Thank you very much in advance
Best regards

Andreas
Answer: from selenium.webdriver import Chrome
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = "E:\soft\GoogleChromePortableDev\App\Chrome-bin\chrome.exe"
from bs4 import BeautifulSoup
from time import sleep
from tqdm import tqdm
browser = Chrome('C:\webdriver\105_0_5195_19\chromedriver.exe')

    PS C:\parsing\sel\mvi> & C:/Python310/python.exe c:/parsing/sel/mvi/s1.py
c:\parsing\sel\mvi\s1.py:14: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  browser = Chrome('C:\webdriver\105_0_5195_19\chromedriver.exe')
Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\selenium\webdriver\common\service.py", line 71, in start
    self.process = subprocess.Popen(cmd, env=self.env,
  File "C:\Python310\lib\subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Python310\lib\subprocess.py", line 1438, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] Cannot find the specified file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\parsing\sel\mvi\s1.py", line 14, in <module>
    browser = Chrome('C:\webdriver\105_0_5195_19\chromedriver.exe')
  File "C:\Python310\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 69, in __init__
    super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "C:\Python310\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 89, in __init__
    self.service.start()
  File "C:\Python310\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home

PS C:\parsing\sel\mvi>

Windows 10
Python 3.10.5

Python3, Selenium and Chrome portable


Non-trivial presheaf of abelian groups with trivial stalks

Question: This is a problem 1.13 of Tennison Sheaf Theory.
"Construct a topological space $X$ and presheaf $F$ of abelian groups on $X$ with the following properties.
(a) for any open $U\subset X$, $F(U)\neq\{0\}$
(b) for all $x\in X$, the stalk $F_x=\{0\}$
(If you cannot, prove that it is impossible.)"
Since $F$ is contravariant functor from category of topological space $X$'s open set to abelian group category, I would like $F$ acting on morphism for any distinct two open sets $V\subset U$, $F(U)\to F(V)$ as trivial map and $F(U)\to F(U)$ is just standard $1_{F(U)}$ map. Then the stalk at any point of $X$ is trivial. This is a presheaf.
However, pictorially, I draw picture of presheaf over $X$ as some kind of vector bundle over $X$ and I suspect that there will be non-trivial stalk somewhere. I should not draw presheaf as vector bundle which is totally different from presheaf.
Which conclusion is correct?
Comment: When you imagine the presheaf as a vector bundle, how do you imagine the restriction maps?
Comment: @k.stm I would shrink the size of open set and look at the vector bundle over $X$ restricted to $U$ open. Then keep restricting to $V\subset U$ to see non-trivial sections over $V$. If I cannot shrink the size of open set, then I have to conclude that the stalk has to agree with class of some element of the section in that open set.
Comment: @k.stm Am I having a wrong picture here?
Answer: Your example is correct (assuming no element $x\in X$ has a smallest open neighborhood $U$, since then the stalk at $x$ would be $F(U)$ which is nontrivial).   To the extent that this example violates your intuition, that means your intuition is incorrect.  A presheaf is just any functor on the category of open subsets of $X$, and this is a very flexible notion.  It doesn't have to look anything remotely like the "sections" of some kind of "bundle".  That intuition is only really appropriate for sheaves.
Comment: No, or at least I don't have one.  That's the entire point: presheaves _aren't_ "geometric", only sheaves are.
Comment: Is there any geometric visualization of presheaf which corresponds to this example? It seems this construction is quite arbitrary
Comment: I see. Thanks a lot.


Diameter of a triangle

Question: If $T\subseteq \mathbb R^2$ is a generic plane triangle, I want to find its diameter
$$d=\sup\{\lvert| x-y \rvert|: x,y\in T\}$$
Intuitively I think that $d$ is the length of the longest edge of $T$. How can I formally prove this?
Answer: Given any triangle $T$, in fact any bounded closed convex subset of $\mathbb{R}^2$.
The map
$$T^2 \ni (x,y) \mapsto |x-y|^2 \in \mathbb{R}$$
is a continuous function on $T^2$ bounded from above. Since $T^2$ is compact, the map achieves its maximum on some $(u, v) \in T^2$. i.e.
$$\sup \{\;|x-y|^2 : x, y \in T\;\} = | u - v |^2.$$
If either $u$ or $v$ is not an extremal point
of $T$, say $u = \frac12 ( u_1 + u_2 )$ where $u_1, u_2 \in T$, then by substituting $x_1$ by $u - v$ and $x_2$ by $\frac12 ( u_1 - u_2 )$ into
parallelogram identity:
$$| x_1 + x_2 |^2 + |x_1 - x_2|^2 = 2 ( |x_1|^2  + |x_2|^2 )$$
one find at least one of $|u_1 - v|^2$ and $|u_2 - v|^2$ is greater than $|u-v|^2$. This contradicts with the role of $u, v$ that maximize $|u - v|^2$.
As a result, the $u, v$ that maximize $|u-v|^2$ are both extremal points and
$$\sup\{\;|x-y|^2 : x,y \in T\;\} = \max\{\;|u-v|^2 : u, v \in T, \text{ both extremal}\;\}$$
A triangle has 3 extremal points, i.e. its 3 vertices, and hence its diameter is the length of its longest edge.
Comment: This is very elegant! If I interpret it correctly then  you are essentially using that $\Vert x-y \Vert^2$ is a *convex* function of both arguments, and therefore attains its maximum at extremal points on a convex set.
Comment: @MartinR yup. that's the idea.
Comment: One could also argue that $\Vert x-y \Vert$ (without the square) is convex, because of the triangle inequality.
Comment: @MartinR yes, you can. However, for Euclidean spaces, the squared distance is slightly better because it is strictly convex.
Answer: [intuitive answer]
If $y$ is fixed and if $x$ is a point in interior, we draw a small ball $B(x,r)$.
Then there exists a point $x'\in \partial B(x,r)$ such that
$$ d(x',y) =d(x,y)+r $$

Diameter of a triangle


What is $\int_0^{\pi/2}\cos^{10}x (\sin 12x )\,dx\;$?

Question:
$$\int_0^{\pi/2}(\cos^{10}x )(\sin 12x )dx$$

I am unable to figure out the trick this question expects me to use. Its not a simple by parts question.
If I denotes the integral then I is also equal to $\int_0 ^{\pi/2}(\sin^{10}x) (\cos 12 x)dx$
Adding the two I's , doesn't yield anything special.
How do I go about solving this problem? What's the trick to immediately recognize the method the integration questions expect me to use?
Comment: One quick suggestion:  When writing titles, please include some text, instead of solely mathjax.  With text, a user can right click on the text to open your question in a new tab, which I invariably do for every question that I visit.  When a title is all and only mathjax, there is no way to do this; if I try right clicking on only a mathjax title, my only options are to view the source/formatting.  No way to open the question in a new tab.
Comment: It's possible to write $\sin(12x)$ as $Q_{12}(\cos x) \sin x$ for some polynomial $Q_{12}$, so then you could do a substitution $u = \cos x$.
Comment: *Mathematica* gives the answer instantly:  $1/11$.
Comment: @amwhy I agree with you, but you can also do Ctrl+ click to open in new tab.
Answer: Hint: the integrand can be written as
$$\begin{align}
  & ={{\cos }^{10}}\left( x \right)\sin \left( 11x+x \right) \\
 & ={{\cos }^{10}}\left( x \right)\left( \sin \left( 11x \right)\cos \left( x \right)+\cos \left( 11x \right)\sin \left( x \right) \right) \\
 & ={{\cos }^{11}}\left( x \right)\sin \left( 11x \right)+{{\cos }^{10}}\left( x \right)\cos \left( 11x \right)\sin \left( x \right) \\
 & =-\frac{1}{11}\frac{d}{dx}\left[ {{\cos }^{11}}\left( x \right)\cos \left( 11x \right) \right] \\
\end{align}$$
Answer: I'd suggest rewriting this with exponentials using Euler's formula; then you just need to multiply it out and trivially integrate the exponentials. It's perhaps not the most elegant way to do it, but it's straightforward. The binomial theorem does most of the work of multiplying it out for you.
Answer: Hint :
\begin{eqnarray*}
\frac{d}{dx}(\cos^{11}x \cos(11x)) =-11\cos^{10} x \sin x \cos(11 x) - 11 \cos^{11} x \sin(11x).
\end{eqnarray*}
Now use the $\sin$ addition formula.
Answer: $$I = \int \cos^{10}x(\sin 12 x)dx$$
$$I = \Im\int \bigg[e^{i(12 x)}\cdot \bigg(\frac{e^{ix}+e^{-ix}}{2}\bigg)^{10}\bigg]dx$$
$$=\frac{1}{2^{10}}\Im\int \bigg[e^{i(2x)}\cdot \bigg(e^{i(2x)}+1\bigg)^{10}\bigg]dx$$
Now put $e^{i(2x)}+1=t,$ Then $\displaystyle 2ie^{i(2x)}dx = dt\Rightarrow e^{i(2x)}dx = \frac{dt}{2i}$
So $$=\Im\bigg[\frac{1}{2^{10}i}\int t^{10}dt\bigg] = \frac{1}{2^{11}\cdot 11}\Im\bigg[\bigg(\frac{e^{2ix}+1}{e^{ix}}\bigg)^{11}\cdot e^{11ix}\bigg]+\mathcal{C}$$
$$=-\frac{1}{11}\Im\bigg[\cos^{11}(x)\cdot \bigg(\cos 11 x+i\sin 11 x\bigg)\bigg]+\mathcal{C}$$
So $$I = -\frac{1}{11}\cos^{11}(x)\cdot \cos (11x)+\mathcal{C}$$
So $$\int^{\frac{\pi}{2}}_{0}\cos^{10}x(\sin 12 x)dx = -\frac{1}{11}\bigg[\cos^{11}(x)\cdot \cos(11)\bigg]\Bigg|^{\frac{\pi}{2}}_{0}=\frac{1}{11}.$$
Answer: Hint:
$$
\begin{align}
\sin x\sin y &= \dfrac12(\cos(y - x) - \cos(y + x)) \\
\cos x\cos y &= \dfrac12(\cos(y + x) - \cos(y - x)) \\
\sin x\cos y &= \dfrac12(\sin(y + x) - \sin(y - x))
\end{align}$$
Also,
$$\begin{align}
\cos^{10}x\sin 12x &= (\cos^2x)^5\sin 12x \\
&= \left(\dfrac12(\cos 2x + 1)\right)^5\sin 12x \\
&= \dfrac1{32}\left(\cos^52x + 5\cos^42x + 10\cos^32x + 10\cos^2x + 5\cos 2x + 1\right)\sin 12x
\end{align}$$
and so on. Using the product-to-sum formulas above and linearity, you should be able to solve the integral.
Answer: The solution holds in a word: linearise the expression, starting from Euler's formulæ. Here's how to start:
\begin{align}
\cos^{10}x\,\sin 12 x&=\Bigl(\frac{\mathrm e^{ix}+\mathrm e^{-ix}}{2}\Bigr)^{\!10}\,\frac{\mathrm e^{12ix}+\mathrm e^{-12ix}}{2i}\\
&=\frac{\mathrm e^{10ix}+10\mathrm e^{8ix}+45\mathrm e^{6ix}+\dotsm}{2^{10}}\cdot\frac{\mathrm e^{12ix}+\mathrm e^{-12ix}}{2i}
\end{align}
Can you proceed?


Accessing the PC audio input stream

Question: I would like to be able to access the computers incoming audio input stream which is connected to a radio receiver. I would like to continuously read of the incoming frequencies. I don't really know where to start with this. Any help on the subjuect is highly appreciated. How can I go about accessing the computers hardware in e.g C# or C++? Is there any language which offers APIs for this purpose? I know programming in C, C++, Java and C#, but if there is any other language where it is easier to accomplish then that is preferable.
Comment: What platform, operating system, etc ?
Answer: I find the JUCE library best for this purpose. It is very cross-platform C++ and allows you to easily get a real time stream in from all manner of audio devices with minimal fuss. The download contains a demo application with an audio demo. You can copy this code to give you a starting point.
The library is free to use in open source software and it has a very modestly priced commercial licence.
Comment: Thanks. I will dvelve into JUCE right away. This might be exactly what I need.
Comment: @yms I fail to understand your downvote.  The license is not indicated as a concern, so your comment and downvote seem irrelevent.  Upvoted to cancel.
Comment: @Andy It was wrong/misleading information. Answers in SO are read by many programmers, not just the original poster of the question. I have changed my vote to +1 though, since the answer was fixed and that was the purpose of my downvote anyway.
Comment: @yms Understood, I didn't see the answer until after the edit.

Accessing the PC audio input stream


Cython parallel OpenMP for Black Scholes with NumPy integrated, serial code 10M options 3.5s, parallel?

Question: Here is the Black (Black Scholes less the dividend) option pricing model for options on futures written in Cython with actual multi-threading, but I can't run it. (NOW FIXED, SEE LATER POST BELOW FOR ANSWER).  I am using Python 3.5 with Microsoft Visual Studio 2015 compiler.  Here is the serial version that takes 3.5s for 10M options: Cython program is slower than plain Python (10M options 3.5s vs 3.25s Black Scholes) - what am I missing?
I attempted to make this parallel by using nogil but after compiling, I cannot access the internal function CyBlackP.  There are several issues with this (at least on Windows).  1) Cython when generating the OpenMP code assumes you are beyond v2.0 but Microsoft Visual Studio 2015 is stuck on the old version which requires signed iterators.  The workaround I have is after first attempting to build the code, it will error out, then open the output CyBlackP.cpp file in Microsoft Visual Studio 2015, search for size_t __pyx_t_2 (line 1430), then change it to ssize_t __pyx_t_2, and change the next line from size_t __pyx_t_3 to ssize_t __pyx_t_3 to get rid of signed/unsigned errors, and compile again.  2) You can't directly go from NumPy arrays into the function as nogil only works on pure C/C++ functions, so I have several helper functions to convert the NumPy array inputs into C++ vector format, pass those to a C++ function, then convert the returned vector back to a NumPy array.  I'm posting the parallel code here for others to use and I'm sure someone out there can figure out why I can't access the parallel function from Python - the non-parallel version was accessed like this from CyBlackP.CyBlackP import CyBlackP.
Code is here with steps on how to build.  First file save as CyBlackP.pyx
[note the exposed function to Python here is CyBlackP, which converts the NumPy input arrays into C vectors through the helper functions, then passes the C vectors to the C function CyBlackParallel, which runs with nogil and OpenMP.  The results are then converted back to a NumPy array and returned from CyBlackP back to Python]:
import numpy as np
cimport cython
from cython.parallel cimport prange
from libcpp.vector cimport vector

cdef extern from "math.h" nogil:
    double exp(double)
    double log(double)
    double erf(double)
    double sqrt(double)

cdef double std_norm_cdf(double x) nogil:

    return 0.5*(1+erf(x/sqrt(2.0)))

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)
cdef CyBlackParallel(vector[double] Black_PnL, vector[double] Black_S, vector[double] Black_Texpiry, vector[double] Black_strike, vector[double] Black_volatility, vector[double] Black_IR, vector[int] Black_callput):
    cdef int i
    N = Black_PnL.size()

    cdef double d1, d2

    for i in prange(N, nogil=True, num_threads=4, schedule='static'):
        d1 = ((log(Black_S[i] / Black_strike[i]) + Black_Texpiry[i] * (Black_volatility[i] * Black_volatility[i]) / 2)) / (Black_volatility[i] * sqrt(Black_Texpiry[i]))
        d2 = d1 - Black_volatility[i] * sqrt(Black_Texpiry[i])
        Black_PnL[i] = exp(-Black_IR[i] * Black_Texpiry[i]) * (Black_callput[i] * Black_S[i] * std_norm_cdf(Black_callput[i] * d1) - Black_callput[i] * Black_strike[i] * std_norm_cdf(Black_callput[i] * d2))

    return Black_PnL

cdef vector[double] arrayToVector(np.ndarray[np.float64_t,ndim=1] array):
    cdef long size = array.size
    cdef vector[double] vec
    cdef long i
    for i in range(size):
        vec.push_back(array[i])

    return vec

cdef vector[int] INTarrayToVector(np.ndarray[np.int64_t,ndim=1] array):
    cdef long size = array.size
    cdef vector[int] vec
    cdef long i
    for i in range(size):
        vec.push_back(array[i])

    return vec

cdef np.ndarray[np.float64_t, ndim=1] vectorToArray(vector[double] vec):
    cdef np.ndarray[np.float64_t, ndim=1] arr = np.zeros(vec.size())
    cdef long i
    for i in range(vec.size()):
        arr[i] = vec[i]

    return arr

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)
cpdef CyBlackP(ndarray[np.float64_t, ndim=1] PnL, ndarray[np.float64_t, ndim=1] S0, ndarray[np.float64_t, ndim=1] Texpiry, ndarray[np.float64_t, ndim=1] strike, ndarray [np.float64_t, ndim=1] volatility, ndarray[np.float64_t, ndim=1] IR, ndarray[np.int64_t, ndim=1] callput):
    cdef vector[double] Black_PnL, Black_S, Black_Texpiry, Black_strike, Black_volatility, Black_IR
    cdef ndarray[np.float64_t, ndim=1] Results
    cdef vector[int] Black_callput
    Black_PnL = arrayToVector(PnL)
    Black_S = arrayToVector(S0)
    Black_Texpiry = arrayToVector(Texpiry)
    Black_strike = arrayToVector(strike)
    Black_volatility = arrayToVector(volatility)
    Black_IR = arrayToVector(IR)
    Black_callput = INTarrayToVector(callput)
    Black_PnL = CyBlackParallel (Black_PnL, Black_S, Black_Texpiry, Black_strike, Black_volatility, Black_IR, Black_callput)
    Results = vectorToArray(Black_PnL)

    return Results

Next code piece save as setup.py for use by Cython:
try:
    from setuptools import setup
    from setuptools import Extension
except ImportError:
    from distutils.core import setup
    from distutils.extension import Extension

from Cython.Distutils import build_ext
import numpy as np

ext_modules = [Extension("CyBlackP",sources=["CyBlackP.pyx"],
              extra_compile_args=['/Ot', '/openmp', '/favor:INTEL64', '/EHsc', '/GA'],
              language='c++')]

setup(
    name= 'Generic model class',
    cmdclass = {'build_ext': build_ext},
    include_dirs = [np.get_include()],
    ext_modules = ext_modules)

Then from a command prompt, type: python setup.py build_ext --inplace --compiler=msvc to build.
Any help on getting access to this function is appreciated, not sure why I can't seem to locate it after compiling.  I can import CyBlackP or from CyBlackP import * but I can't get to the actual function to calculate the option values.
Here is a realistic NumPy test script to use if you want to test this Cython function:
BlackPnL = np.zeros(10000000)
Black_S=np.random.randint(200, 10000, 10000000)*0.01
Black_Texpiry=np.random.randint(1,500,10000000)*0.01
Black_strike=np.random.randint(1,100,10000000)*0.1
Black_volatility=np.random.rand(10000000)*1.2
Black_IR=np.random.rand(10000000)*0.1
Black_callput=np.sign(np.random.randn(10000000))
Black_callput=Black_callput.astype(np.int64)

Comment: *"You can't directly go from NumPy arrays into the function as `nogil` only works on pure C/C++ functions"* - you can pass in existing numpy arrays  either as typed memoryviews (e.g. `double[:] array`) or using the old "`np.ndarray[...] array`" syntax without acquiring the GIL, you just can't instantiate them or access their Python methods within a `nogil` block.
Comment: A NumPy array is essentially a Python wrapper around a memory buffer that can be exposed directly to C/C++ functions - there's no conversion needed. Take a look at the [documentation here](http://docs.cython.org/src/userguide/memoryviews.html). I'm not sure if this is directly related to the problem you're describing, but getting rid of all of those unnecessary helper functions would greatly simplify your code and make debugging it easier.
Comment: you cannot `import` a `cdef` function in Python. Expose it as a `def` function. `cdef` functions can only be `cimport`ed from Cython.
Comment: So even though the function converts the NumPy arrays into C++ vectors before the C++ only function with `nogil` processes them, you're saying the cpdef function won't be exposed at all to Python?  The compiler won't compile a `nogil` block if it touches a Python object whereas this compiles.
Comment: Alright upvoted your comments, attempting to try memory views on my strided arrays after some reading...
Comment: @dashesy The function he's referring to is `cpdef`ed, so at first glance it ought to be importable from Python.
Comment: Yes it is `cpdef`'d but can't import it... tried using `dir` on it but all I can see is: `CyBlackP.__spec__
Out[40]: ModuleSpec(name='CyBlackP', loader=None, origin='namespace', submodule_search_locations=_NamespacePath(['C:\\Users\\slezm0\\Documents\\Python Scripts\\CyBlackP']))`
Comment: @ali_m got it working then tried to implement memory views per your suggestion.  Much cleaner code as you mentioned (and way faster) although the return value for BlackPnL shows:  - do you know how to convert this back to a numpy array?
Comment: You can call `np.asarray(...)` on the output
Comment: Thanks @ali_m for the suggestions!  I just figured that one out as well :)
Answer: Okay I figured out what was wrong using dependency walker http://www.dependencywalker.com/ on the CyBlackP.cp35-win_amd64.pyd file generated by Cython.  It showed that 2 DLLs were not found: msvcp140_app.dll and vcomp140_app.dll which are just the x64 versions of MSVC OpenMP and CRT C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\x64\
Microsoft.VC140.OpenMP\vcomp140.dll and C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\x64\Microsoft.VC14
0.CRT\msvcp140.dll renamed with _app inserted, and copied to the \CyBlackP\ project directory.  I also updated my setup.py like this which gets rid of the annoying import statement (now just from CyBlackP import CyBlackP):
try:
    from setuptools import setup
    from setuptools import Extension
except ImportError:
    from distutils.core import setup
    from distutils.extension import Extension

from Cython.Distutils import build_ext
import numpy as np
import os

module = 'CyBlackP'

ext_modules = [Extension(module, sources=[module + ".pyx"],
              extra_compile_args=['/Ot', '/favor:INTEL64', '/EHsc', '/GA', '/openmp'],
              language='c++')]

setup(
    name = module,
    cmdclass = {'build_ext': build_ext},
    include_dirs = [np.get_include(), os.path.join(np.get_include(), 'numpy')],
    ext_modules = ext_modules)

Comment: `%timeit CyBlackP(BlackPnL, Black_S, Black_Texpiry, Black_strike, Black_volatility, Black_IR, Black_callput)
1 loop, best of 3: 2.88 s per loop` - verified in the task manager this is going to 4 cores as stated in the code.
Comment: Ok so the problem was `name= 'Generic model class'` because that is the name of the module to import from. Missing dlls should not change importing logic, they should result in exceptions (or crashes) but nothing to do with import.
Comment: Not so sure as that name worked fine using the serial version of the code... and you didn't have to call it either.
Comment: Actually after converting my code to memoryviews per @ali_m suggestion, the module hid itself in another layer down again.  To fix I deleted the `name = module` from above and it imported fine.
Comment: BTW the code with memoryviews only takes ~0.65s so a huge improvement over the original 2.88s.


Ormlite Inner Join em cinco tabelas

Question: Eu quero criar alguns JOIN em cinco tabelas como este exemplo:
SELECT * FROM TableA AS TA
    INNER JOIN TableB AS TB ON TA.Id = TB.IdTA
    INNER JOIN TableC AS TC ON TB.IdTC = TC.Id
    INNER JOIN TableD AS TD ON TC.Id = TD.IdTC
    INNER JOIN TableE AS TE ON TD.IdTE = TE.Id
WHERE TE.Id = 30085

Obs: As TableB e TableD são tabelas N-M
Estou um pouco confuso de como fazer isso no Ormlite do Android, alguém poderia me ajudar com essa query ou me fornecer um passo a passo?
Comment: `Primeiro:`, não colocar `*`, ou seja, discriminar os campos, `Segundo:` se as TableB e TableD existe uma junção entre elas e se for necessário coloque a tabela intermediária com duas comparações `on` **comparação** e `and` **comparação** . Obs: seria uma das formas talvez depende muito do que você deseja comparar !!!
Answer: Segue resposta adaptada daqui:
Vai ser algo mais ou menos assim (adaptando pro seu exemplo):
// Começa a primeira query
QueryBuilder<TableA, Integer> TableAQb = TableADao.queryBuilder();
QueryBuilder<TableB, Integer> TableBQb = TableBDao.queryBuilder();
// Join entre TableA e TableB
TableAQb.join(TableBQb);
//TERCEIRA QUERY
QueryBuilder<TableC, Integer> TableCQb =
    TableCDao.queryBuilder();
TableCQb.where().gt("mixedvalue", 100);
// join TableC
TableCQb.join(TableAQb);
List<TableC> TableCQb.query();

Seguindo assim para as demais tabelas.
Lembrando que há suporte para "queries cruas", inclusive o método Dao.queryRaw() onde você pode utilizar sua própria query. Segue documentação.
Comment: Ae! Consegui fazer para as 5 tabelas... Obrigado!

Ormlite Inner Join em cinco tabelas


Java Random access Map

Question: I'm looking for a opensource library that has an implementation of a random access map.
I need a map that maintains its hash index but also indexes the values in insertion order like LinkedHashmap except you don't have to iterate through it to find eg. element 2. Something like this:
Map m = new ArrayMap();
m.put("0", "v0");
m.put("1", "v1");
m.put("2", "v2");
m.put("3", "v3");
then:
assertEquals("v2", m.get("2"));
assertEquals("v2", m.getAtIndex(2));

The idea is that both types of lookups must be fast.
A quick google didn't find anything, I didn't see it in Guava or commons collections (I may have overlooked it).
I don't really have the time to implement it properly right now.
Comment: LinkedHashMap [supports](http://download.oracle.com/javase/6/docs/api/java/util/LinkedHashMap.html#get%28java.lang.Object%29) `m.get("2")`
Comment: is it static or dynamically updated?
Comment: Are you asking for something where both 'm.get(1)' and 'm.get("key")' are O(1)? And is the 'order' within the map the order of the keys?
Comment: @DJClayworth yes and yes
Answer: If your Map can be immutable, you can do this:
ImmutableMap<String, String> map = ...
String v2 = map.entrySet().asList().get(2).getValue();

asList() on the entrySet() for a regular ImmutableMap just uses the map's own array of entries directly, so it's random access and fast.
Comment: I imagine people will see all those method calls and figure it's slow, but no, it should be quite efficient.
Answer: If your map is static or if it's just updated ocasionally you could use the values() method of the LinkedHashMap which provides the list of values of the map as a Collection<T>. Then you can convert the set to an array with toArray(T[] a) and you will have constant access to the elements, of course this will waste some memory to store the additional references but you are asking two specific good complexities so a memory trade-off is necessary. This will be fine if you don't need to get values while you add them in a mixed way.
The only different way would be to implement a LinkedHashMap by yourself by using an array instead that a linked list to store insertion order, but you will have to care about doubling the capacity of the arraylist when needed to keep performance enough fine.
Answer: You want to look up values two different ways.  The simplest/fastest way to do this is to maintain two collections, one Map and one ArrayList.
private final Map<String, String> map;
private final List<String> list;

public void put(String key, String value) {
   map.put(key,value);
   list.add(value);
}

public String get(String key) {
   return map.get(key);
}

public String get(int index) {
   return list.get(index);
}

Comment: I'd lean towards this, but of course put() returns any previous value in the Map interface, so it (and remove()) are a bit more complicated in terms of keeping the inner list up to date.  If additions were infrequent enough compared to retrievals, you could regenerate the whole array on each put/remove as others are suggesting.
Answer: I think that LinkedHashMap is exactly what you need.
You can use it as map:
map.get("v2");

and as list:
new ArrayList(map.values().values()).get(2);

If you want to can write your own 5 lines long class that implements this logic.

Java Random access Map


Can I style my ngx-bootstrap element from my components .css file?

Question: I'm using a ngx-bootstrap 'alert' component. And I'd like to style the alert message, but adding the styles to the compnents .css file doesn't do anything.
question - ngx-bootstrap says to style do this below but I'd like to know if it can be done from the component .css file?
https://valor-software.com/ngx-bootstrap/#/alerts#local-styling

styles: [
    `
  :host >>> .alert-md-local {
    background-color: #009688;
    border-color: #00695C;
    color: #fff;
  }
  `
  ]

ex.

.alert {
  border-radius: 0px !important; // doesn't do anything
  margin-bottom: 0px !important; // doesn't do anything
}
<alert [type]="alert.type" [dismissible]="dismissible" [dismissOnTimeout]="alert.timeout" class="text-center" style="margin-bottom: 0px;">{{ alert.msg }}</alert>
Comment: Try `alert { border... }` With no dot `.` cose `` Is not an class. Learn more about targeting in CSS.
Comment: that is incorrect. .alert IS a class that is part of that bootstrap component...Learn how ngx-bootstrap converts to it's appropriate element.
Answer: You can use ng-deep for this. Though it is mentioned in the docs that it will be deprecated soon, there's a lot of discussion going on about a good alternate solution for the same. Here's the GitHub issue.
For now, ng-deep still works.
:host ::ng-deep .alert {
    border-radius: 0px;
    margin-bottom: 0px;
}

Comment: well, based on my OP I don't even need ::ng-deep as I show above
Comment: I'm just looking to see if there is a pure .css solution


Excluding Fields in Django Class Based Views Not Working

Question: Boy, it feels like I've tried EVERYTHING here, and I just can't get this form to render properly (e.g. with a subset of fields). Here's the relevant items (extra items removed):
models.py:
class Response(models.Model):
    public = models.BooleanField(default=False)
    question = models.CharField(max_length=255, default='', blank=True)

class ResponseForm(ModelForm):
    class Meta:
        model = Response
        fields = ['question']

views.py:
class ResponseCreate(CreateView):
    model = Response
    fields = ['question']

response_form.html:
<form action="" method="post">{% csrf_token %}
    {{ form.as_p }}
    <input type="submit" value="Create" />
</form>

I've read every bit of docs, and they suggest using exclude, excludes, field, fields in permutations of tuple, or list, and no matter what the "public" field seems to show up every time! Thoughts?
Thanks for your help!
Comment: If you add a comma in your form, `fields = ['question',]`, does that change things?
Comment: @ScottWoodall That would make a difference only for tuple type, not list
Answer: It looks like you need to specify form_class on your view:
class ResponseCreate(CreateView):
    model = Response
    form_class = ResponseForm

Answer: Just I want to mention that you don't really need a ModelForm to be created separately.

Generic views really shine when working with models. These generic
  views will automatically create aModelForm, so long as they can work
  out which model class to use

You can just use this.
class Response(models.Model):
    public = models.BooleanField(default=False)
    question = models.CharField(max_length=255, default='', blank=True)

class ResponseCreate(CreateView):
    model = Response
    fields = ['question']

Answer: You can use exclude field in your class form:
class ResponseForm(ModelForm):
    class Meta:
        model = Response
        exclude = ['field_to_exclude']

class ResponseCreate(CreateView):
    model = Response
    form_class = ResponseForm


Build java project with parameters using maven

Question: I want to build a Java project using Maven with parameters, which change in every build. One parameter is for example a key which is checked inside of the program.
The parameters should not be able to be read out once the project is build. Tried different Approche including plugins from org.codehaus.mojo… but having problems "plugin execution not covered by lifecycle"....
    /****************************/
    /**read property values     */
    /****************************/
    //Create a new property list
    Properties properties = new Properties();
    //create a input strem
    InputStream inputStream = null;
    //try to read the property file
    try {
        String filename ="restApi.properties";
        inputStream = Main.class.getClassLoader().getResourceAsStream(filename);
        if(inputStream==null) {
            System.out.println("Unable to read required properties");
        }
        properties.load(inputStream);
        System.out.println(properties.getProperty("property_mainUrlValue"));
    } catch (IOException ex) {
        ex.printStackTrace();
    } finally {
        if (inputStream != null) {
            try {
                inputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

my pom.xml
<properties>        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <property_mainUrlValue>property_mainUrlValue</property_mainUrlValue>
<properties>

<build>
    <sourceDirectory>src</sourceDirectory>
    <pluginManagement>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.7.0</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>properties-maven-plugin</artifactId>
            <version>1.0-alpha-2</version>
            <executions>
                <execution>
                    <phase>generate-resources</phase>
                    <goals>
                        <goal>write-project-properties</goal>
                    </goals>
                    <configuration>
                        <outputFile>${project.build.outputDirectory}/restApi.properties</outputFile>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
    </pluginManagement>
</build>

error shown in eclipse
     Unable to read required properties
     Exception in thread "main" java.lang.NullPointerException
at java.util.Properties$LineReader.readLine(Unknown Source)
at java.util.Properties.load0(Unknown Source)
at java.util.Properties.load(Unknown Source)
at itAsset.Main.main(Main.java:58)

Comment: Why are parameters changing for each build? What kind of parameters are you talking? And for what purpose are they used?
Comment: thought about to build a jar for a IOT device...appkey and url and thingName would be provided by the build sever...through installation routine i would transfer the program afterwards to the thing...
Comment: generic iot parameters such as mac &&& are checked during execution. Therefore the program would only work on that specific device..
Answer: I guess that you are searching for what Maven calls (for some reason which I do not understand) "filtering".
The basic idea is this:
You turn on the feature by including the following configuration into pom.xml:
...
<resource>
    <directory>src/main/resources</directory>
    <filtering>true</filtering>
</resource>
...

This causes that when you run mvn resources:resources -Dkey="mmm123", and you have a resource src/main/resources/restApi.properties containing line
appKey=${key}

then Maven will create a resource output in target/classes/restApi.properties which contains
appKey=mmm123

The detailed description is here: http://maven.apache.org/plugins/maven-resources-plugin/examples/filter.html


How did the term 'English' originate? (spin on a ball)

Question: Where does the term 'English' come from, with regards to an athlete applying a spin to a ball?  I've heard it mainly in billiards and basketball, but elsewhere too.
I have tried looking this up on Wikipedia. It does not give the story of the origin of the term.
Answer: According to this answer the story goes that an enterprising gentleman from these shores travelled to the United States during the latter part of the last century and impressed the Americans with a demonstration of the effect of 'side' on pool or billiard balls. His name was English.
Comment: Or more accurately, that answer notes that the OED *speculates* this is the origin.
Comment: It seems there's no other story out there...
Answer: Looking through several online dictionaries, my spouse found multiple references to the term "body English" morphing into an intended spin on a ball.  Given that automatic reaction when bowling or shooting pool when trying to get the ball to spin the way you want to, it makes sense.  However, the folksiness of it being named after a "foreigner" named English, quite charming, as the Brits would say.


Adding a validation handler to the registration form

Question: I have altered the registration form to change the thank you message and add the validation of some fields.
<code>function mymodule_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'user_register_form') {
    $form['actions']['submit']['#validate'][]  = 'checkout_nationality_form_validate';
    $form['#submit'][] = 'thank_on_submit';
  }
</code>
All code is working fine; my problem is the registration doesn't take the default Drupal validation. Why does this happen?
Answer: You need to change this line
<code> $form['actions']['submit']['#validate'][]  = 'checkout_nationality_form_validate';
</code>
to
<code> $form['#validate'][] = 'checkout_nationality_form_validate';
</code>
Answer: Your custom validation function should go into the $form['#validate'] array
<code>$form['#validate'] = 'checkout_nationality_form_validate';
</code>
I prefer using the implement hook_form_FORM_ID_alter instead of using an if statement in the hook_form.
<code>/**
 * Implements hook_form_form_id_alter().
 */
function MYMODULE_form_user_register_form_alter(&$form, &$form_state, $form_id) {
  $form['#validate'][] = 'checkout_nationality_form_validate';
}

/**
 * Custom validation function.
 */
function checkout_nationality_form_validate(&$form, &$form_state) {
  if($form_state['value']['my_field'] == 'is not valid') {
    form_set_error('my_field', t('Value not valid'));
  }
}
</code>


How to list all installed Jupyter kernels?

Question: Listing all the available environments is as simple as:
$ conda env list

Now how does one list the currently installed kernels, without having to go to the path:
$ ls /home/{{user}}/.local/share/jupyter/kernels/

Answer: With Jupyter installed you get the list of currently installed kernels with:
$ jupyter kernelspec list

python2     /usr/local/lib/python2.7/dist-packages/ipykernel/resources
testenv     /home/{{user}}/.local/share/jupyter/kernels/sparkenv

Answer: For those that come here because VSCode can't find the kernel although it is shown when using jupyter kernelspec list, try updating pyzmq.
pip install pyzmq --upgrade

Based on this answer