Proof: Adding additional regressor and the influence on the adjusted R^2

Question: I'm looking at the influence of an additional regressor in an OLS-model and on the adjusted $\bar{R}^2$. What I have to proove is that $\bar{R}^2$ rises if and only if the square of the respective t-statistic is bigger than 1. I found a solution to the proof in "Greene - Econometric Analysis, (Chapter 3, exercise 9)" and managed to replicate the steps more or less to get to the final result, which is as follows:
$\frac{b_k^2(x_k'M_1x_k)}{(s^2)}>1$,
where $b_k$ is the coefficient of the additional regressor in the long model and $s^2$ is its estimated variance. In my understanding the squared t-statistic of this regressor should just be
$\frac{b_k^2}{(s^2)}$.
How do I interpret the rest of the nominator $(x_k'M_1x_k)$? Is this even the right proof I'm looking for?
The complete solution I'm looking at is given in this PDF (p5, ex. 9):
pages.stern.nyu.edu/~wgreene/Text/Greene_6e_Solutions_Manual.pdf
Answer: Recall that the t-statistic on the $k$-th coefficient (for testing $\beta_k=0$ assuming homoskedasticity) in a linear regression is given by
$$
t=\frac{b_k}{\sqrt{s^2(X'X)^{-1}_{kk}}},
$$
where $(X'X)^{-1}_{kk}$ denotes the $k$-th diagonal element of $(X'X)^{-1}$, so that the squared t-ratio is
$$
t=\frac{b_k^2}{s^2(X'X)^{-1}_{kk}}.
$$
This is the same as your
$$
\frac{b_k^2(x_k'M_1x_k)}{s^2}=\frac{b_k^2}{s^2/(x_k'M_1x_k)}
$$
due to the Frisch-Waugh-Lovell theorem discussed in for example Chapter 3 of Greene's textbook, or more specifically/directly, as an application of the partitioned inverse lemma:
For $A\;(m \times m)$, $B\;(m \times n)$, $C\;(n \times m)$ and $D\;(n \times n)$ we have that, provided the respective inverses exist,
$$
\left(%
\begin{array}{cc}
A & B \\
C & D \\
\end{array}%
\right)^{-1}=
\left(%
\begin{array}{cc}
A^{-1} + A^{-1}B(D-CA^{-1}B)^{-1}CA^{-1}
& -A^{-1}B(D-CA^{-1}B)^{-1} \\
-(D-CA^{-1}B)^{-1}CA^{-1} & (D-CA^{-1}B)^{-1} \\
\end{array}%
\right).
$$
For $A=X_1'X_1$ (with $X_1$ the other regressors except $x_k$), $B=X_1'x_k$, $C=x_k'X_1$ and $D=x_k'x_k$, we can plug in for the lower-right element of the inverse to directly obtain that
$$
(D-CA^{-1}B)^{-1}=(X'X)^{-1}_{kk}=1/(x_k'M_1x_k).
$$


Proof concerning outer measure

Question: Assume that $X:=[a,b]$ is a fixed interval in $\mathbb R$ and let $m^*$ be the outer measure on $X$. Suppose that $A \subset X$ is a null set, i.e. $m^*(A)=0.$ Show that for every $B\subset X$,
$m^*(B \cup A)=m^*(B\setminus A)=m^*(B)$
I'm studying measure theory and looking for some direction on this proof. Thanks!
Comment: HINT: You will need to use monotonicity and subadditivity.
Answer: Take my hint from above if you would like just that. Below is a more complete solution.
$m^*(B\setminus A)\leq m^*(B)$ by monotonicity. $m^*(B)\leq m^*(B\cup A)$ by the same. $m^*(B\cup A)\leq m^*(B\setminus A)+m^*(A)$ by subadditivity.  Hence:
$$
m^*(B\setminus A)\leq m^*(B)\leq m^*(B\cup A)\leq m^*(B\setminus A)+m^*(A)=m^*(B\setminus A).
$$
So all the inequalities collapse into equalities.
Answer: $B \cup A = (B \setminus A) \cup A $, that is, it is the union of two disjoint sets so
$$m^*(B \cup A) = m^* (B \setminus A) + m^*(A)= m^*(B \setminus A). $$
Now since $A$ is a null set, $ B \setminus A= B$, hence $m^* (B \cup A)= m^*(B)$.
Comment: The outer measure of the union of two disjoint sets is not necessarily the sum of their outer measures: Take any non-measurable subset of $[0,1]$ as $A$, and $[0,1]\setminus A$ as $B$ for a counterexample. To get the equality in such generality you'd need more, for example, that $A$ and $B$ are not only disjoint but a positive distance apart.
Comment: Oh, you also say that $B\setminus A=B$. This is false in general. It is true if $A$ is disjoint from $B$, but false otherwise. "Null" means "of (outer) measure $0$". It is not the same as "empty".
Comment: Then $m^*(A \cup B) = m^*(A) + m^*(B) - m^*(A \cap B)$ does not hold as well?

Proof concerning outer measure


How to stop a thread when its in the wait()

Question:  while (!closingTime){

        depot.enterramp();
    }

This is to start the function as below
  synchronized (list) {

        while (list.size() == 0) {
            System.out.println(":: DEPOT\t:: " + "NO BUS FOUND IN THE APPROACHES DEPOT, WAITING THE BUS COMING");
            try {
                list.wait();

            } catch (InterruptedException iex) {
                iex.printStackTrace();
            }
        }

Currently my thread is waiting the notify()  [This code is in enterramp() ]
public class Clock extends Thread {
public void run() {
    try {
        Thread.sleep(15000);
        notifyTime();

    } catch (Exception e) {

    }
}

public synchronized void notifyTime() {
    System.out.println(":: CLOCK\t:: ALERT ALERT !!! DEPOT HAVE TO CLOSE IN 30 MINUTES, NO MORE ACCEPTING THE BUSES");
    closingTime = true;

    return;

}

This is the clock sleep for 15 seconds and make the closing time to true
 synchronized (list) {
            list.add(bus);
            list.notify();
        }

This is my part of notify it
The problem I facing is when the closing == true, but the thread is stucking in the wait() area, how do i can make the thread exit from the wait() when my closingtime == true ???
Comment: I believe that interrupting the thread is the correct way to go (`Thread.interrupt()`).
Comment: but the thread currently stuck in the wait(), I cant have any move until the thread been notified...
Comment: You need to notify the thread. Where do you call the last `synchronized`-block you have posted?
Comment: The first box of code is keep on go in the the second box of code, so when the closingtime is not true, the thread will keep wait() until invoked by the notify.
Answer: If you read the Object::wait documentation, it says the following:

Causes the current thread to wait until another thread invokes
  the notify() method or the notifyAll()method for this object, or some
  other thread interrupts the current thread, or a certain amount of
  real time has elapsed.

So you need to interrupt the thread.
Comment: How to interrupt it when the thread is stucking in the wait(), because of no bus keep on generating, therefore there will not have buses go into the 4th box of code which consist the notify(). – Huat Lee just now   edit
Comment: Make sure another thread has an instance to the thread you're running, and it interrupts it when required
Answer: a) Your exit condition (boolean closingTime ) AND the lock (on which synchronization occurs) should both be used by the waiting threads. Your code can wait while the list is empty AND the exit condition is false.
b) you are synchronizing on 2 monitors; the 'this' and the 'list'. If the critical zone was protected by just one dedicated monitor (it's somewhat ok to reuse the list), then adding to the list OR changing the closingTime exit condition could both use notify(). This means kicking the waiting thread out without interrupt, as it is intended for.
c) try to avoid notify(), use notifyAll() unless you really know why you should notify only one.
Answer: In your logic you should wait if there are no buses AND it's not closing time. But you check only first condition. I propose to notify the thread when you set closingTime to true. Make these changes:
while (list.size() == 0 && !closingTime) {

closingTime = true;
list.notifyAll();

Meanwhile on interrupt() topic: it does get you out of wait(), but its logic is very different. It is indended for outside control, like a signal to halt execution and exit. So, when you want to stop waiting, use notyfyAll(), when you want to BREAK execution, use interrupt().


Calculating the cumulative distribution function and the probability density function of an interval with ratio of a shorter and longer segment

Question: The interval $[0, 2]$ is divided into two parts by randomly marking a point in $[0, 1]$ according to the rectangular distribution. Let $X$ be the length ratio $L_1/L_2$ of the shorter segment $L_1$ to the longer segment $L_2$. Thus, $L_1$ is in the interval $[0, 1]$, and $L_2$ is greater than or equal to $1$, and also it is in the interval $[0, 2]$.
First calculate the cumulative distribution function then calculate the probability density function $f_X(x)$ of $X$.
Does somebody have some tips for me on how to even tackle this problem? I dont expect a solution just a way to atleast calculate the cumulative distribution function. For clarification its a university homework above our level and we are allowed to use external resources. I just need some help on how to even start, thanks.
Comment: It doesn't work too well to tell us you just want guidance; to receive a good answer; and then to *change your question* to inquire whether your work is correct.  That looks merely like you're trying to use this site to do and check your homework.
Comment: I used the help to come up with an answer and just wanted to see if i have done it the correct way. Im sorry if i used this platform the wrong way.
Answer: $L_1$ follows a Uniform $(0,1)$ and $L_2 = 2-L_1$.
So
$$X = \frac{L_1}{L_2} = \frac{L_1}{2-L_1}.$$
Use this, together with the definition of the cumulative distribution function $F_X(x)$, that for some value $x$
$$F_X(x) = \Pr(X\leq x).$$
Note: inside the probability, things can move around as long as, in the end, you keep the random variable(s) to the left of the inequality, and the fixed quantities to the right. Once you have a single random variable to the left of the inequality, then you have the cumulative distribution function of that variable.
Comment: @Ste0l Apart from our moderator intervention, the solution involves very basic mathematical operations, like differentiation. You should feel confident in doing them, since they do  not involve anymore anything from probability theory itself.
Comment: Hi, thanks alot for the help. I updated my question with the current solution. If you have time i would be happy for you to review it.


Autoplay images from SDCard

Question: Sorry Guys for all the vague questions. this is what i want to do. I want my app to randomly play images from SdCard and I want no user input after app is started. I have read lot of sdcard post but this is not same question. if this is possible please let me know how i go about this. I do not want any user input once the app is started it just plays images from sdcard. Thank you for your help. by the way i am trying to implement this using android.
Answer: Get the image paths into the sdcard and pass it to the gallery.
You can follow this link for few more information http://raivoratsep.com/114/android-gallery-tutorial-working-example/
suppose you get the path name as
String pathName = "/sdcard/images/abc.jpg";

You can set the image to the image view as
Drawable d = Drawable.createFromPath(pathName);

hope this may help you
Comment: That's my pleasure.
Hope this helps you.
Comment: Hi I am trying to work with this code, but it is giving me error when i set pathName to R.id.imageView1. and i do not know if this is the right way to set content view.
Comment: you need to set the image to the image view.
You are getting an error because Pathname is a string and R.id.imageView1 is an int.
you no need to set the pathName to R.id.imageView1 instead use Drawable d = Drawable.createFromPath(pathName);
and set it using the img.setImageDrawable(drawable) in the getView()
Comment: Did you get the paths of images from the sdcard?
Comment: well did you get the path of the images from the gallery.Say this please
Comment: I am not sure. I thought I set my own path like you have given example above. can we also set multiple paths.
Comment: ok i tried for an alternative(without gallery) and i will post it soon
Answer: To show a slide show of images from your sdcard have a look at  http://codinglookseasy.blogspot.in/2012/08/image-slide-show.html. Hope this helps you.
Comment: I follow the tutorial but did not work. It breaks when it starts. here is my logcat error.
Comment: hi
i tried it and it works too. the only problem is that when i try to rotate my phone to landscape view it crashes any solution. one more thing when i updated my sd card folder by putting some more images. it crashes.
Answer: try creating another layout-land folder with the same xml file as in the layout folder.
This will be used as the layout when the mobile is in the landscape mode.
And to prevent crashing try to use
System.gc();

in the ondestroy()
And i didnt find any difficulty by adding new images.
They work fine for me.

Autoplay images from SDCard


Omnipay PHPUnit Guzzle httpClient 404 Error - mock json

Question: I'm new to OmniPay, playing around with it and trying to make a simple custom gateway, and create a unit test with a mock json http response.
In the GatewayTest.php I set a mock http response:
public function testPurchaseSuccess()
{
    $this->setMockHttpResponse('TransactionSuccess.txt');

    $response = $this->gateway->purchase($this->options)->send();

    echo $response->isSuccessful();

    $this->assertEquals(true, $response->isSuccessful());
}

In the PurchaseRequest.php I'm trying to get it somehow:
public function sendData($data)
{
    $httpResponse = //how do I get the mock http response set before?

    return $this->response = new PurchaseResponse($this, $httpResponse->json());
}

So how do I get the mock http response in PurchaseRequest.php?
--- UPDATE ---
It turned out that in my PurchaseResponse.php
use Omnipay\Common\Message\RequestInterface;

//and...

public function __construct(RequestInterface $request, $data)
{
    parent::__construct($request, $data);
}

was missing.
Now with $httpResponse = $this->httpClient->post(null)->send(); in PurchaseRequest.php the assertions are OK, but when I use httpClient, Guzzle throws a 404 error. I checked Guzzle's docs and tried to create a mock response, but then again my assertions fail and 404 remains:
PurchaseRequest.php
public function sendData($data)
{
    $plugin = new Guzzle\Plugin\Mock\MockPlugin();
    $plugin->addResponse(new Guzzle\Http\Message\Response(200));

    $this->httpClient->addSubscriber($plugin);

    $httpResponse = $this->httpClient->post(null)->send();

    return $this->response = new PurchaseResponse($this, $httpResponse->json());

}

Any suggestions, how to get rid of the 404?
Answer: Ok, so here is what turned out to be the solution:
Original Problem
This was missing from my PucrhaseResponse.php:
use Omnipay\Common\Message\RequestInterface;

//and...

public function __construct(RequestInterface $request, $data)
{
    parent::__construct($request, $data);
}

PurchaseRequest.php:
public function sendData($data)
{
    $httpResponse = $this->httpClient->post(null)->send();

    return $this->response = new PurchaseResponse($this, $httpResponse->json());
}

Solving the 404 problem in the update
In order to prevent Guzzle from throwing exceptions I had to add a listener for request.error.
PurchaseRequest.php:
public function sendData($data)
{
    $this->httpClient->getEventDispatcher()->addListener(
        'request.error',
        function (\Guzzle\Common\Event $event) {
            $event->stopPropagation();
        }
    );

    $httpResponse = $this->httpClient->post(null)->send();

    return $this->response = new PurchaseResponse($this, $httpResponse->json());
}


VBA code works differently

Question: I have these 3 pieces of code, which work differently but deliver similar result. The aim is that I have in a separate column only the values from filled cells, and no blanks

This code copies only the results from the formulas in I10:I61. The formula returns also "", which I do not need.
With Worksheets("Calculator").Range("I10:I61")
    .Offset(, 0).SpecialCells(xlCellTypeFormulas, xlNumbers).Copy
    Worksheets("Graphs").Range("F2").PasteSpecial SkipBlanks:=True, Paste:=xlPasteValues
End With

This code copies values, where there are no formulas
With Worksheets("Calculator")
    Set VisRng = .Range("D10:D61").SpecialCells(xlCellTypeConstants)
    VisRng.Copy
    Worksheets("Graphs").Range("B2").PasteSpecial xlValues
End With

This is supposed to copy text results from formula. It does not work. The formula returns also "", which I do not need.
With Worksheets("Calculator").Range("K10:K61")
    .Offset(, 0).SpecialCells(xlCellTypeFormulas, xlTextValues).Copy
    Worksheets("Graphs").Range("A2").PasteSpecial SkipBlanks:=True, Paste:=xlPasteValues
End With

However, the last piece of code does not work. It should copy and paste only the filled cells and not the emty once(containing ""). Now it copies and pastes everything. How can I make it work? The result is:
K             A
Paris         Paris

Amsterdam      Amsterdam

I need it to be
A
Paris
Amsterdam

Comment: Your observation that they work differently is correct. But what is your question?
Comment: And so?? what is the question?
Comment: There are now more observations but still no question. Are you trying to ask how to make the third piece work? If so, please describe what it should do and what it does instead.
Comment: your reply is still not clear to me but check out my proposed solution posted as an answer
Comment: When you say it copies the cells containing `""` do you mean the formulas place an empty string result, like `=IF(A1=2,A1,"")`. If so, Excel treat this as text and you may need to use another method to collect only the cells you want to copy.
Comment: It copies the column and the cells, which contain a "" result in the original range. are left empty
Comment: It works like it should. It might not be what you expect, but that's because an empty string is [not the same](http://stackoverflow.com/q/1119614/11683) as blank.
Answer: Since excel treats formula results of "" as text, try this method instead.
Dim vData as Variant
vData = Worksheets("Calculator").Range("K10:K61")

Dim i as Integer
For i = Lbound(vData) to uBound(vData)
   If len (vData(i,1)) And Not IsNumeric(vData(i,1)) Then
     With Worksheets("Graphs")
         .Range("A" & .Rows.Count).End(xlUp).Offset(1).Value = vData(i,1)
     End With
   End If
Next

Comment: Still not the same. If the formula returns a string convertible to number (e.g. `="42"`), `SpecialCells(xlCellTypeFormulas, xlTextValues)` will properly see it, but `IsNumeric()` on it will be `True`. You probably want `and vartype(vData(i,1)) = vbstring`.
Comment: @GSerg yes and another difference is `xlCellTypeFormulas` takes only results of formulas (e.g. cell contains `="aaa"`) and no constant strings (e.g. cell contains `aaa`). But your code still does. Nontheless the question owner accepted the answer it should be clear that the code does not the same as number 3 of the question.
Comment: Just to comment: This does not replace the number 3 of the question. This copies **any** non-empty cell (including numeric values) instead of text values of formulas only!

VBA code works differently


Not able to curl an http rest api through a k8s service

Question: I have following deployments and service but still not able to access the service on my Windows machine with minikube installed on it.
# kubectl get deployment
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
helloworld-dep   3/3     3            3           9h

# kubectl get pods -o wide
NAME                             READY   STATUS    RESTARTS      AGE    IP           NODE       NOMINATED NODE   READINESS GATES
greet-api-demo                   1/1     Running   2 (23m ago)   6d9h   172.17.0.5   minikube   <none>           <none>
helloworld-dep-f8586dd84-28dxg   1/1     Running   1 (23m ago)   9h     172.17.0.6   minikube   <none>           <none>
helloworld-dep-f8586dd84-p4pg9   1/1     Running   1 (23m ago)   9h     172.17.0.7   minikube   <none>           <none>
helloworld-dep-f8586dd84-tgmcj   1/1     Running   1 (23m ago)   9h     172.17.0.8   minikube   <none>           <none>

# kubectl get services
NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
helloworld-service   NodePort    10.96.44.199   <none>        80:30008/TCP   4m1s
kubernetes           ClusterIP   10.96.0.1      <none>        443/TCP        191d

# kubectl logs helloworld-dep-f8586dd84-28dxg
Setting Active Processor Count to 8
Calculating JVM memory based on 11276448K available memory
For more information on this calculation, see https://paketo.io/docs/reference/java-reference/#memory-calculator
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -Xmx10889346K -XX:MaxMetaspaceSize=79901K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 11276448K, Thread Count: 50, Loaded Class Count: 11693, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 124 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
Picked up JAVA_TOOL_OPTIONS: -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=8 -XX:MaxDirectMemorySize=10M -Xmx10889346K -XX:MaxMetaspaceSize=79901K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.0.0)

2022-12-23T03:38:53.397Z  INFO 1 --- [           main] com.playground.k8s.K8sDemoApplication    : Starting K8sDemoApplication using Java 17.0.5 with PID 1 (/workspace/BOOT-INF/classes started by cnb in /workspace)
2022-12-23T03:38:53.403Z  INFO 1 --- [           main] com.playground.k8s.K8sDemoApplication    : No active profile set, falling back to 1 default profile: "default"
2022-12-23T03:39:14.389Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2022-12-23T03:39:14.648Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2022-12-23T03:39:14.650Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.1]
2022-12-23T03:39:15.883Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2022-12-23T03:39:15.888Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 21396 ms
2022-12-23T03:39:19.693Z  INFO 1 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
2022-12-23T03:39:23.699Z  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2022-12-23T03:39:25.262Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-12-23T03:39:25.397Z  INFO 1 --- [           main] com.playground.k8s.K8sDemoApplication    : Started K8sDemoApplication in 35.178 seconds (process running for 38.144)

Tried minikube ip, clusterip and node ip but none of them worked !
# curl http://192.168.49.2:30008/actuator
curl: (28) Failed to connect to 192.168.49.2 port 30008 after 21061 ms: Timed out

# curl http://10.96.44.199:30008/actuator
curl: (28) Failed to connect to 10.96.44.199 port 30008 after 21034 ms: Timed out

# curl http://172.17.0.6:30008/actuator
curl: (28) Failed to connect to 172.17.0.6 port 30008 after 21052 ms: Timed out

Answer: Not sure from where you are truing to hit the curl but you will be only able to curl those IP if truing inside of Kubernetes cluster.
Cluster IP is internal, you will be able to access it internal or internal services can use it to access it.
You can run one container extra to test this
kubectl run -it --rm --image=curlimages/curl curly -- sh

after command try curl <clusterIP> or curl helloworld-service
If you want to try from outside from Host machine you can check URL with
minikube service helloworld-service --url

try once curl localhost:30008/actuator
For node port you can get minikube IP with command
minikube ip

curl MinikubeIP:30008

Comment: did you try debugging with port-forward command if POD are working properly or not `kubectl port-forward helloworld-dep-f8586dd84-28dxg 80:8080`
Comment: you have to curl on localhost if you are doing portfoward. if tried `curl localhost:80` make sure your port are properly set i just set `80:8080` for test you can update it as per your service usage. your service running on 8080 ? set port properly and try. if after that curl give timeout there is issue with POD it's not responding.
Comment: I am hitting the curl command from Windows cmd prompt.  Using `minkube ip` on the same cmd prompt gives me '192.168.49.2' and `curl http://192.168.49.2:30008/actuator` returns `curl: (28) Failed to connect to 192.168.49.2 port 30008 after 21043 ms: Timed out` . Running `minikube service helloworld-service --url` returns
http://192.168.49.2:30008
Comment: `# kubectl port-forward helloworld-dep-f8586dd84-28dxg 80:8080
Forwarding from 127.0.0.1:80 -> 8080
Forwarding from [::1]:80 -> 8080`
This is what I see and running the curl command gives the same result. No luck
Comment: That worked !. Not sure why do we have to port forwarding using port-forward command
`# curl localhost:80/actuator
{"_links":{"self":{"href":"http://localhost/actuator","templated":false},"health":{"href":"http://localhost/actuator/health","templated":false},"health-path":{"href":"http://localhost/actuator/health/{*path}","templated":true}}}`
Comment: How can I get this working with the service port 30008 ? Has this something to do with Windows minikube ?
Comment: there is multiple ways to expose service if you want to use Node port which is 30008 you have to change service type to Node port. if for you port-forward worked which means your service is running on 8080 not on 80. did you check node port forwarding traffic to proper service port ?
Comment: ideally it should work with `Nodeip:nodeport` or `minikubeip:nodeport`.
Comment: Yes, the service is `NodePort` in .yml file and its running on 8080 for sure. But somehow the  `Nodeip:nodeport` or `minikubeip:nodeport` both are not working unless I use the `kubectl port-forward` option
Comment: due to driver issue or how you are starting the minikube also. port-forward is not proper way to expose service it's for debugging. Nodeport is fine else you can use the ingress as best practise suggest


How to do pagination in splash/scrapy?

Question: I am trying to get items from website which has no next page href only numbers.I cannot get all pages, only one page.How can I modify my code and why I cannot get all items with my code? Can anyone explain?
    import scrapy
from scrapy_splash import SplashRequest

class Hepsips4Spider(scrapy.Spider):
    #sayfaları otomatikleştir geçişi yapabilirsin
    name = 'hepsips4'
    allowed_domains = ['www.hepsiburada.com']
    script = '''

        function main(splash, args)
        assert(splash:go(args.url))
        assert(splash:wait(2))
        return {
        html = splash:html(),
        }
        end
          '''

    def start_requests(self):
        yield SplashRequest(url="https://www.hepsiburada.com/playstation-4-oyunlari-c-60003893?siralama=enyeni&sayfa=1",callback=self.parse,endpoint="execute",args={'lua_source': self.script})

    def parse(self, response):
        for i in range(1,25):
            yield SplashRequest(url=f"https://www.hepsiburada.com/playstation-4-oyunlari-c-60003893?siralama=enyeni&sayfa={i}",callback=self.parse_item,endpoint="execute",args={'lua_source': self.script})

    def parse_item(self,response):
        for row in response.xpath("//ul[@class='product-list results-container do-flex list']/li/div"):
             name =row.xpath(".//a/div/h3/@title").get()
             img =row.xpath(".//a/figure/div/img/@data-src").get()
             company="hepsiburada"
             link =row.xpath(".//a/@href").get()
             fulllink=f"https://www.hepsiburada.com{link}"
             price =row.xpath(".//a/div/div[2]/span/text()").get()
             #sepet indirimini düzeltmeye çalış
             platform='ps4'
             yield{
                'name': name,
                'image': img,
                'company': company,
                'full_link': fulllink,
                'price': price,
                'platform': platform

            }

Answer: script = '''

        function main(splash, args)
        assert(splash:go(args.url))
        assert(splash:wait(2))
        return {
        html = splash:html(),
        }
        end
          '''

Making waiting duration 2 fixed my problem.
Answer: import scrapy
from scrapy_splash import SplashRequest

class JbPracujSpider(scrapy.Spider):
name = 'jb_pracuj'
allowed_domains = ['www.pracuj.pl']

headers = {
    'Connection': 'keep-alive',
    'Cache-Control': 'max-age=0',
    'DNT': '1',
    'Upgrade-Insecure-Requests': '1',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36',
    'Sec-Fetch-User': '?1',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
    'Sec-Fetch-Site': 'same-origin',
    'Sec-Fetch-Mode': 'navigate',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9',
}

script = '''
    function main(splash, args)
        splash.private_mode_enabled = false
        assert(splash:go(args.url))
        assert(splash:wait(5))
        return {
            html = splash:html(),
        }
    end
        '''
def start_requests(self, headers=headers):
    yield SplashRequest(url='https://www.pracuj.pl/praca?rd=30&cc=5015%2c5016&pn=1', callback=self.parse, endpoint='execute', args={'lua_source': self.script, 'wait': 1})

def parse(self, response):
    for i in range(1,10):
        yield SplashRequest(url=f'https://www.pracuj.pl/praca?rd=30&cc=5015%2c5016&pn={i}', callback=self.parse_item, endpoint='execute', args={'lua_source': self.script, 'wait': 1})

def parse_item(self, response):
    for row in response.xpath("//li[@class='results__list-container-item']"):
        yield {
            'title': row.xpath(".//div[@class='offer-details__text']/h2/a[@class='offer-details__title-link']/text()|//div[@class='offer-details__text']/h2/button[@class='offer-details__title-link']/text()").get()
        }

and after execute this code i have a lot of None in my 'title' yield, but not everyone.

How to do pagination in splash/scrapy?


Acessar valor indice de um array

Question: Como faço pra pegar o valor do indice do array em cada interação, sem precisar de colocar a possição exemplo PermissionUsuarioFilter.UserPermissionId[0] preciso do indice dinamico e não estatico.

Preciso que a classe fique dessa forma
Comment: Será que não era para você estar fazendo `permissionSalve.PermissionId = item;` ?
Comment: Justamente, mas agora ele só esta salvando o ultimo item da lista no banco.
Comment: não vi o código todo mas, não entendo o pq desse `Deserialize`, se passar o parametro corretamente do javascript não precisa disseo, o método deveria aceitar `List` como parametro
Answer: No método .each( é posivel pegar a posição do array index e valor que ele contem value.
.each( function (index,value)

Na primeira interação index igual 0 e value igual {UserPermissionId : 7}
Assim você pode dar um value.UserPermissionId para obter o valor 7
Espero ter ajudado, boa sorte.
Comment: não da porque são dois arrays, um UserPermission e outro PermissionUsuario, quero comprar os valores identicos dos dois e marchar o "checked",true para os valores ==, o each não aceita dois arrays como parametro.
Comment: Ok, use o outro `.each()` para pode percorrer a lista filtrada, assim você poderá obter o resultado que deseja.

Acessar valor indice de um array


What kind of progression is this?

Question: Sometimes I listen to some songs and try to get the underlying features of it, and this time I've hit a bump.... I have no idea what is the progression on the following song:

It's TheFatRat and AleXa's "Rule the World" chorus part. If I transcribed it correctly, that's the song. In case the image doesn't show, the chords are Bb-F-G-C, on an F Major scale. So, from the little I know on music theory, this song's progression should be IV-I-II-V, right?
However, I haven't found this specific progression in my researches. Am I understanding this progression incorrectly, or does this progression actually exists? If it does, what's its name?
Comment: The song is in F and needs b-flat as signature. That's why Aaron's analysis is correct.

Btw.:

Your rhythmic notation lacks of transparency. Usually this rhythm should be notated by 2 tied 16th on the syncopation: *gate, top, space, up.*
Comment: I used a random notation online solution just to describe what I was thinking, I didn't aim for precise notation, at that moment. As for the signature, I had forgotten, thanks for pointing that out!
Answer: Allowing your transcription is correct, then yes, the progression is IV-I-II-V. However, since the II chord has a raised third (B-natural rather than B-flat), it would be better written as IV-I-V/V-V. V/V (read: five of five) means that chord functions as a dominant chord relative to the following C chord. This is called a "secondary dominant".
The progression itself doesn't have a name, but an ending on a dominant chord is called a "half cadence".
Comment: Or, an imperfect cadence, to the right of the pond.
Comment: @ASousa Good point. I've added a couple of links to Wikipedia definitions.

What kind of progression is this?


Uploaded image in Block Plugin keeps on getting deleted overnight

Question: I have a block plugin that allows users to upload an image to be used as a background. Below is what is inside my blockForm function.
<code>    $form = [
        'image' => [
            '#type' => 'managed_file',
            '#title' => 'Image',
            '#upload_location' => 'public://my_style/images',
            '#default_value' => $this->configuration['image'],
        ],
    ];
</code>
And my block submit function is as follows.
<code>public function blockSubmit($form, FormStateInterface $form_state) {
    $values = $form_state->getValues();

    foreach ($values as $index => $value) {
        $this->configuration[$index] = $value;
    }
}
</code>
Initially this works just fine. The image is uploaded and appears in the background of my block plugin. However overnight it's always deleted. I did see this post Set image as permanent in custom block plugin, temporary URI however I don't see anything different that would cause issues with my code against the one provided in that post. Is there something missing? Or is there anything wrong with how I'm performing the submit function.
I suppose I could try to check if the formstate value is a file and then load the file and then set it to permanent and then save it. But even if that works why would it be necessary in this case but not in others?
Answer: As you guessed its must to set permanent while saving the file. You need to include,
    $file = File::load( $fid );
    if ($file) {
      $file->setPermanent();
      $file->save();
    }

The files will be added to the {file_managed} table as temporary
  files. Temporary files are periodically cleaned. Use the 'file.usage'
  service to register the usage of the file which will automatically
  mark it as permanent.
Comment: Unfortunately after implementing this the issue wasn't resolved. Here's what I added to my block submit function.  public function blockSubmit($form, FormStateInterface $form_state) {
  $values = $form_state->getValues();

  foreach ($values as $index => $value) {
   $this->configuration[$index] = $value;

   $file = File::load($value);
   if ($file) {
    $file->setPermanent();
    $file->save();
   } else {

   }
  }
 }
Comment: Did you check what you get on $file? If you could, pls check the file_managed table with your $value against fid column and check if the status changed to 1.Also pls check  JayKandari's comment in this link https://drupal.stackexchange.com/questions/198135/how-do-i-save-uploaded-file-permanently-in-file-manged-table


Coupling: supremum of $X$,$Y$ random variables

Question: Here is an exercise from Resnick - Probability Path (3.18):
If $X$ and $Y$ are random variables on $(\Omega, \mathscr{B})$, show that
$$\sup_{A\in\mathscr{B}} |P[X\in A]-P[Y\in A]|\leq P[X\neq Y]$$
Now, I took the definition given by Resnick for $[X\in A]=${$\omega:X(\omega)\in A$}. Moreover, it looks like the set
$$\{\omega: X(\omega)\in A, Y(\omega)\notin A\} \subset \{\omega: X(\omega)\neq Y(\omega)\}$$
Therefore I was thinking to apply in any way the Fatou's lemma, but I really don't know how (perhaps I may use a sequence $A_i$ in order to apply $\sigma$-additivity, but I think it would be too restrictive).
As always, any help would be very appreciated, thanks.
Answer: $$[X\in A] \subset [Y\in A] \cup [X\neq Y]$$ so $$P[X\in A] \leq  P[Y\in A] +P[X\neq Y].$$ Hence $$P[X\in A]-P[Y\in A] \leq  P[X\neq Y] .$$ Similarly $$P[Y\in A]-P[X\in A] \leq  P[X \leq Y].$$ Putting these two together we get $$|P[X\in A]-P[X\in A]| \leq  P[X \neq Y] .$$
Now take supremum over $A$.
Comment: So since it is verified for any A, then it works also on the supremum, right?
Comment: @AkaiM Yes, that is right.


Dynamic default value for select list

Question: I am trying to make a reservation system for equipment in a laboratory.
People reserve equipment by creating "reservation" nodes that are tagged with the taxonomy term "Equipment".
In the "create reservation page" the referenced taxonomy field is a dropdown list.
On the other hand I have a table view with all the equipment with its corresponding taxonomy term.
I would like to have in a third column a link to the "create reservation page" and  I would like that in the "create reservation page"  the corresponding taxonomy term in the dropdown list is already selected.
Any idea how to succeed?
Answer: The Prepopulate module allows you to construct a special URL inside your table view that, when clicked, will prepopulate the correct taxonomy term on the "Create reservation page".
The special URL will looking something like this:
http://www.example.com/node/add/reservation?edit[field_equipment_term_reference]=123

You need to:

Download and install the Prepopulate Drupal module.
In your table view, add the Taxonomy Term ID field (if not already added).
In your table view, add a Global Text custom field.
In the Global Text field config screen, specify the text "create reservation page" and then expand the rewriting section, and turn the field into a link, with link destination of node/add/reservation?edit[field_equipment_term_reference]=[tid].

Please note:

the [tid] is the substitution token from the field added in step 2.
replace field_equipment_term_reference with the real machine name of the taxonomy term reference field you're using on the Reservation node.

Dynamic default value for select list


Is it valid to use ROC calculated during test/validation to interpret results of final production model?

Question: I've trained a binary classification model which outputs a "probability" between (0,1).
During testing and validation, I use the ROC to measure the performance of the model. Also, I use the ROC to determine the threshold on which to cutoff false vs true predictions (e.g. I set the target to under 15% FPR).
When creating a model for production, I thought it would be ideal to train on all the available data set (e.g. no test nor validation split). Now, without test or validation split I don't have a ROC for the final model so I am without a threshold to interpret the model's output.
Is it valid to use the ROC obtained during testing?
Should I calculate a new ROC of the final model over instances observed during training?
Is there something fundamentally wrong in my approach?
Answer: Yes, you can train on the whole data and report the test (or cross validation) ROC as the generalization error, but this ROC will probably be optimistic.
It would be better to estimate the generalization error on the model trained on the full data by doing cross validation. See this answer.
Comment: Thanks a lot @arinarmo for the answer. Very interesting.
Comment: If you feel like it helped, you should accept the answer. It's how we say thanks ;)


Conditional Probabilities Paradox

Question: I know which step is wrong in the following argument, but would like to have contributors' explanations of why it is wrong.
We assume below that weather forecasts always predict whether or not it is going to rain, so not forecast to rain means the same as forecast not to rain. We shall also assume that forecasts are not always right.
It is not generally true that the probability of rain when forecast is equal to that of its having been forecast to rain when it does rain. Indeed let us assume that
$$P(\text{R}|\text{F}_{\text R}) \neq P(\text{F}_{\text R}|\text{R}).$$
But, having been forecast to rain, it will either rain or not rain ($\bar{\text{R}}$), so
$$P(\text{R}|\text{F}_{\text R})+P(\overline {\text{R}}|\text{F}_{\text R})=1\ \ \ \ \ \ \mathbf{eq. 1}$$
Likewise, if it rains, it will either have been forecast to rain or (we are assuming) not forecast to rain ($\overline{\text{F}_{\text R}}$), so
$$P(\text{F}_{\text R}|\text{R})+P(\overline{\text{F}_{\text R}}|\text{R})=1 \ \ \ \ \ \ \mathbf{eq. 2}$$
But we know that "If rain then not forecast to rain" is logically equivalent to "If forecast to rain then no rain". So the corresponding conditional probabilities must be equal, that is
$$P(\overline{\text{F}_{\text R}}|\text{R})=P(\overline {\text{R}}|\text{F}_{\text R})\ \ \ \ \ \ \  \ \ \ \ \ \mathbf{eq. 3}$$
It follows immediately from $\mathbf {eqs 1,\ 2\ and\ 3}$ that
$$P(\text{R}|\text{F}_{\text R}) = P(\text{F}_{\text R}|\text{R}).$$
which is contrary to our hypothesis.
Answer: The statement "If $A$ then $B$" discusses the relation between $A$ and $B$ in the $A\subseteq B$ sense.
Not conditional events.
Let's throw a dice, $A$ is the result even, $B$ is the result $2$.
Following the steps of the original post, we can say
$$ \Pr(A\vert B)+\Pr(\bar{A}\vert B)=1,$$
and
$$ \Pr(B\vert A)+\Pr(\bar{B}\vert A)=1.$$
Moreover, we know that "if B then A" is equivalent to "if not A then not B", which means $B\subseteq A$ iff $\bar{A}\subseteq \bar{B}$.
But $\Pr(A\vert B)=1$ while $\Pr(B\vert A)=\tfrac{1}{3}$ which means that your next step is invalid.
Comment: So $\Pr(\overline {A}|B)=0$ and $\Pr(\overline {B}|A)=\tfrac 23$. This, I think, hits the nail on the head. Thank you.
Answer: The statement
"if $A$ then $\overline B$" implies "if $B$ then $\overline A$"
Is equivalent to the probability statement ...
$$\bigg(P(\overline B|A) = 1\bigg ) \implies  \bigg(P(\overline A|B) = 1 \bigg) \tag{1}$$
You can use the identities
$$P(A|B)=1-P(\overline A|B)
\\ \text{and}
\\P(B)P(A|B)=P(A)P(B|A)$$
To deduce the general relationship between $P(A|\overline B)$ and $P(\overline A| B)$
$$
 \begin{aligned}
P(B)P(A|B) &=P(A)P(B|A)
\\P(B) \left (1- P(\overline A|B) \right ) &=P(A) \left (1- P(\overline B|A) \right )
\end{aligned}$$
In this form, it is easy to see that $(1)$  must be satisfied, provided that neither $P(A)$ nor $P(B)$ are zero.
It can also be solved to give ...
$$ P(\overline A|B)=1-\frac{P(A)}{P(B)} \left (1- P(\overline B|A) \right )  $$
which is an identity provided that $P(B)$ is not zero.
Comment: Thank you for this elegant proof that $\left(P(\overline{B}|A) =1 \right) \implies \left (P(\overline {A}|B = 1 \right)$ using  Bayes'. As you say, this is clearly equivalent to the logical truth that $\left((\overline{B} \implies A  \right) \implies \left (\overline{A} \implies B  \right)$. And indeed your answer does show that my eq 3 doesn't hold generally, though it doesn't tackle head-on my faulty argument in favour of my eq 3.

Conditional Probabilities Paradox


Netbackup, Backup Selections Missing From Backup

Question: So I have a Linux server that is now gone, and that was being backed up by Netbackup 6.  When I go to restore, I don't see the folder listed.  I set the timeframe far back, and the policy to standard.  I do see some other folder like /etc and /var, but not /export/tools (What I am looking for).  This folder is listed as a backup selection for the policy that this server is under. I also see this folder backed up under other servers.
I don't think I added any exclusions, but since the client is gone, and /opt/openv/..whatever wasn't backed up, I don't think I can check that...?
Anything I can check to figure out why this might have not been backed up on the backup server?
Answer: I have no suggestions for you specifically. In general, if you haven't tested a restore, you're not doing backups properly. And always test a restore before you decommission a machine.
Answer: Was the /export directory hosted via NFS?
If so, check your policy to see if you had the Follow NFS Mounts option checked in the policy attributes.  If the /exports directory was mounted via NFS AND that option was not checked, then that policy may not have been backing up that directory.
Now, given the above, hopefully you were backing up the machine that the files were hosted on and they are available from there.
Otherwise, I am not sure that there is much you can do other than reviewing the logs for that client on the master server to see if it was processing that directory during backups.
Comment: Usually excluded backups will get an entry on the jobs activity slab, but will only be 32KB in size, or something similar. This is just the directory file/mount point being backed up, but not its contents.

If the /export directory is mounted via NFS and that Follow NFS mounts option is not checked in the policy, that is all it will do. You'll need to either check that option on the policy (which can be slow) or you need to be backing up that NFS server separately in another policy.
Comment: That makes sense, but it should have been source of NFS the exports which is why I am confused.  I have the data backed up from a manual backup I did, but I am wonder why it isn't it netbackup.
Comment: Do you know if the logs on the server make a log entry for exclusions that were listed on the client?


Show that: $\inf(A+B) = \inf(A)+ \inf(B)$

Question:
Let $A,B$ be non-empty, bounded subsets of $R$ and $A+B=\{a+b:a \in A,b \in B\}$. Show that: $$\inf(A+B)=\inf(A)+\inf(B)$$

This is what I have tried:
Suppose that $x \in A+B \Rightarrow x=a+b, \ a \in A, \ b \in B$. We know that $a\geq \inf A \text{ and } b\geq \inf(B)$ . So $x \geq \inf(A)+\inf(B) \Rightarrow\inf(A+B) \geq \inf(A)+\inf(B)$
If $x\in A \Rightarrow x \in A+B, \text{ so we conclude that } \inf(A) \leq  \inf(A+B)$ and if $x \in B \Rightarrow x \in A+B,\text{ so we conclude that } \inf(B) \leq  \inf(A+B)$. Therefore, we have that $\displaystyle \frac{\inf(A)+\inf(B)}{2} \leq \inf(A)+\inf(B) \leq \inf(A+B)$.
Could you tell me if it is right?
Comment: How do you know $x \in A \implies x \in A+B$?
Comment: I thougt that it would be equal to $a+0$..But we don't know that $0 \in B$,right?? :/
Answer: Your proof for $\inf (A+B)\ge \inf A+\inf B$ is right.
For the other direction. For any $\delta>0$, there exist $a\in A, b\in B$ s.t. $a<\inf A+\delta$ and $b<\inf B+\delta$, then $a+b<\inf A+\inf B+2\delta$, hence $\inf (A+B)\le \inf A+\inf B+2\delta$. Let $\delta$ go to zero.
Answer: I shall prove a small lemma.

If $x,y$ and $a$ be arbitrary real numbers such that the inequality $a \leq x \leq a+\frac {y}{n}$ holds for all integers $n \geq 1$, then $x=a$.

This is because if $x>a$,  then the number $\frac {y}{x-a}$ would be an upper bound for the set of positive integers, which is not the case.
Now, you have proved yourself that $\inf(A+B)\geq \inf(A)+\inf(B)$.
Also observe that for any positive integer $n$, $\exists a \in A$ such that $a \leq \inf(A)+\frac{1}{n}$ and $\exists b \in B$ such that $b\leq \inf(B)+\frac{1}{n}$.(Why?)
So, we have $\inf(A+B)\leq a+b \leq \inf(A)+\inf(B)+\frac {2}{n}$ $\forall n \in \mathbb N$
Thus we have $\inf(A)+\inf(B) \leq \inf (A+B) \leq \inf(A)+\inf(B) + \frac {2}{n}$ for all positive integers $n\geq1$. Thus from the lemma, we are forced to conclude that $\inf(A+B)=\inf(A)+\inf(B)$.


Can Drupal 8 Form #states API be used to expand/collapse

Question: Given a checkbox field with two options:

"Closed" (selected by default)
"Open"

I'd like to expand or collapse the  container of a different field (in our specific case an Address field). In HTML terms, this means adding the open HTML5 attribute to the  tag and adding aria-expanded=true to the  tag when the "Open" radio button value is clicked, and ensure it is removed when that radio option is not selected (which means by default at page load time as well).
I've found a few older semi-related issues and the documentation page for Drupal 8's drupal_process_states, but it's looking like this isn't possible with the Form #states API in Drupal 8. The docs do not include "open" in the list of valid states that may be applied to an element.
The valid states are:

enabled
disabled
required
optional
visible
invisible
checked
unchecked
expanded
collapsed

One would think that maybe the expanded and collapsed options could work, however this does not seem to be the case with my sample code below:
function mymodule_form_node_<type>_form_alter(&$form, FormStateInterface $form_state) {
  $form['field_address']['#states'] = [
    // Expand address field container when the Trigger field is 'Open'.
    'expanded' => [
      ':input[name="field_trigger"]' => [
        'value' => 'open'
      ],
    ],
  ];
}

Replacing 'expanded' state with 'open' in the above does not work, nor does the inverse logic using 'collapsed' state work:
function mymodule_form_node_<type>_form_alter(&$form, FormStateInterface $form_state) {
  $form['field_address']['#states'] = [
    // Collapse address field container when the Trigger field is 'Closed'.
    'collapsed' => [
      ':input[name="field_trigger"]' => [
        'value' => 'closed'
      ],
    ],
  ];
}

Answer: Unfortunately, I'm going to answer my own question:  expanded and collapsed states do work for the  type.
My problem was operator error.  I was adding the #states array at the level of the '#type' => 'container' but the container has a widget inside of it that holds the '#type' => 'details'.   Fun times with Form API!
The working code is this:
<?php
// In mymodule.module
use \Drupal\Core\Form\FormStateInterface;

function mymodule_form_node_<type>_form_alter(&$form, FormStateInterface $form_state) {
  $form['field_address']['widget'][0]['#states'] = [
    // Expand location field when the Type is 'On-site'.
    'expanded' => [
      ':input[name="field_trigger"]' => [
        'value' => 'open'
      ],
    ],
    // Collapse location field when the Type is 'Online'.
    'collapsed' => [
      ':input[name="field_trigger"]' => [
        'value' => 'closed'
      ],
    ],
  ];

  // Load edit form with Location field either open or closed, based
  // on the value of the Trigger field.
  $form['field_address']['widget'][0]['#open'] = ($form['field_trigger']['widget']['#default_value'] == 'open') ? 1 : 0;
}

Answer: Rather than using expanded/collapsed use open/close like below:
function mymodule_form_node_<type>_form_alter(&$form, FormStateInterface $form_state) {
  $form['field_address']['#states'] = [
    // Expand address field container when the Trigger field is 'Open'.
    'open' => [
      ':input[name="field_trigger"]' => [
        'value' => 'open'
      ],
    ],
    'close' => [
      ':input[name="field_trigger"]' => [
        'value' => 'closed'
      ],
    ],
  ];
}

Comment: I mentioned in passing on OP that I tried this and it didn't work.  I've tried again to be sure, and I can't make it work.  Are you sure there is code in Drupal 8 that works with open/close states?
Comment: Yes it's in Drupal8. See this link for demo https://files.fm/u/5fcyr5xd
Comment: Yes, open/close works in Drupal8.


200 balls - picking random from each

Question: Suppose there are two boxes each containing 200 balls: 100 red balls numbered from 1 to 100, and 100 blue balls numbered from 1 to 100. We pick one ball at random from each box.
(a) Given that at least one of the two balls picked is red, what is the probability that both balls are red?
(b) Given that at least one of the two balls is red and numbered 13, what is the probability that both balls are red?
a. The probability would be 1/2 for the first red ball, and for the second red ball, would it be 99/199?
b. the probability would be 1/200 for the first red ball, and for the second red ball, would it be 99/199?
Not sure how I would finish both of those problems though
Answer: (a) 1/3 (either the first ball is red, the second ball is red, or both balls are red, all with 1/4 probability, but both balls blue is excluded)
(b) hmmm, a bit more tricky. For n = 100, there are 200x200 possibilities, but only 200 + 200 - 1 of them have a red 13. (You have to reduce 1 to not count the case where both balls are red 13.)
Out of 399 possibilities, there are 200 where the second ball is blue, so I'd say 199/399.
Edit: and how coincidental, if you'd take n = 1, the formula (2n-1/4n-1) still holds.
Comment: why is it 199/399 and not 99/199? there's only 200 balls
Comment: There are two boxes with each 200 balls as I understand.
Comment: my bad, completely understand now! thank you very much.

200 balls - picking random from each


Python import MySQLdb error - Mac 10.6

Question: I downloaded and followed the install instructions for MySQL 5.5.8 (http://dev.mysql.com/downloads/mysql/) and for the MySQLdb python plugin. (http://sourceforge.net/projects/mysql-python/)
When I attempt to import MySQLdb to a python terminal, I get the below error:
Safira:~ yanigisawa$ python --version
Python 2.6.1
Safira:~ yanigisawa$ python -c "import MySQLdb"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/yanigisawa/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib
  Referenced from: /Users/yanigisawa/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
  Reason: image not found
Safira:~ yanigisawa$

I found question 4559699 on stack overflow, and followed the instructions there:
Safira:~ yanigisawa$ echo $DYLD_LIBRARY_PATH
/usr/local/mysql/lib
Safira:~ yanigisawa$ ls -l /usr/local/mysql/lib
total 363224
-rwxr-xr-x   1 root  wheel   3787328 Jan 18 19:41 libmysqlclient.16.dylib
-rw-r--r--   1 root  wheel  10035336 Dec  3 13:26 libmysqlclient.a
lrwxr-xr-x   1 root  wheel        23 Jan 14 22:01 libmysqlclient.dylib -> libmysqlclient.16.dylib
lrwxr-xr-x   1 root  wheel        20 Jan 14 22:01 libmysqlclient_r.16.dylib -> libmysqlclient.dylib
lrwxr-xr-x   1 root  wheel        16 Jan 14 22:01 libmysqlclient_r.a -> libmysqlclient.a
lrwxr-xr-x   1 root  wheel        20 Jan 14 22:01 libmysqlclient_r.dylib -> libmysqlclient.dylib
-rw-r--r--   1 root  wheel  88033760 Dec  3 13:22 libmysqld-debug.a
-rw-r--r--   1 root  wheel  84075304 Dec  3 13:26 libmysqld.a
-rw-r--r--   1 root  wheel      8472 Dec  3 13:26 libmysqlservices.a
drwxr-xr-x  13 root  wheel       442 Dec  3 13:53 plugin
Safira:~ yanigisawa$ md5 /usr/local/mysql/lib/libmysqlclient.16.dylib
MD5 (/usr/local/mysql/lib/libmysqlclient.16.dylib) = b8a78a93183720021b9dbe9ddad57111
Safira:~ yanigisawa$

And after cleaning all of the python egg files, and rebuilding, I still got the same error. Then I found bug 59006 (http://bugs.mysql.com/bug.php?id=59006) in MySQL 5.5.8 affecting Mac 10.6. I tried the work-around listed there, cleaned eggs and rebuilt, and I'm still getting this error.
Is there another work-around for this problem specific to python? Am I missing something else?
EDIT: Adding output from otool commands:
Safira:~ yanigisawa$ otool -L /Users/yanigisawa/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
/Users/yanigisawa/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so:
    libmysqlclient.16.dylib (compatibility version 16.0.0, current version 16.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)
Safira:~ yanigisawa$ otool -DX /usr/local/mysql/lib/libmysqlclient.16.dylib
/usr/local/mysql/lib/libmysqlclient.16.dylib
Safira:~ yanigisawa$

Answer: In my case, I solved by adding a couple of symbolic links as in http://ageekstory.blogspot.com/2011/04/installing-massive-coupon-on-mac-os-10.html
as following:

sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib
sudo ln -s /usr/local/mysql/lib /usr/local/mysql/lib/mysql
Comment: +1 this is what finally solved my problem after many hours, even after Ned Deily's edit seemed to work but didn't
Comment: this worked in solving the same problem seen in Eclipse/PyDev/Django/MySQL on macosx - thanks!
Answer: You could try running otool to find out exactly what library paths the MySQLdb C extension, _mysql.so is looking for:
$ otool -L /Users/yanigisawa/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so

and then the installed library name of the MySQL library file:
$ otool -DX /usr/local/mysql/lib/libmysqlclient.16.dylib

But you should not have to resort to setting DYLD_LIBRARY_PATH (or using install_name_tool to modify an executable); the need to set it is almost always a sign of a component that was built or installed incorrectly. In my experience, the MySQL project does not have a very good track record for consistency in their OS X builds and installers, as that issue you cite tends to confirm.
Trying to get a working combination of Python, MySQLdb, and MySQL libraries in OS X 10.6 is a very common cause of frustration: lots of questions about it here on SO and elsewhere.  There are a number reasons for that.  My advice is to go with a complete solution from one of the third-party distributors that will build and install compatible versions of everything you need.  And it should keep working if you need to upgrade components as time goes on.  I've had good success over the years with MacPorts; others prefer Fink or the newer HomeBrew.  With MacPorts, after installing the base files from the MacPorts .dmg, you can get everything built like so:
$ sudo port selfupdate  # ensure the port files are up-to-date
$ sudo port install py26-mysql

The MacPorts Python will be available at /opt/local/bin/python2.6.  If you are looking to run Django, there's a port for that as well.
EDIT: With the updated output from otool, you can see that there is a mismatch of library path names.  The MySQLdb extension is asking for a relative path name (no initial /), while the MySQL library advertises itself with an absolute path, normally what you want.  It seems to be the reverse of the MySQL issue 59006 but, without knowing exactly what steps you've performed up until now, it is really hard to say how you got to this state.  You may be able to use install_name_tool to patch the _mysql.so to have the absolute path name which would also eliminate any need to set DYLD_LIBRARY_PATH.  It's worth trying:
$ cd ~/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp
$ sudo install_name_tool -change libmysqlclient.16.dylib \
                                 /usr/local/mysql/lib/libmysqlclient.16.dylib \
                         _mysql.so

But that should not leave you with a warm and fuzzy feeling that you can reproduce the results when you need to update something.  There really is no virtue in trying to glue all these pieces from different sources into something that works.
Comment: I'm still new to the Mac OS. (about 2 months in now) I had hoped to get a "trial by fire" so to speak and get everything working more or less the manual way. However, if I can't get this working, I'll resort to using the mac ports option. Thanks for the suggestion.
Comment: The MacPorts solution eventually worked. It installed python 2.6.6 along side my 2.6.1 installation, and /usr/bin/python still pointed to 2.6.1, which caused the same error above to occur with my apache cgi scripts. I had to add a symlink to the 2.6.6 version, and everything started working fine. Thanks.
Comment: Glad to hear. But beware: in general, it's a very bad idea to overwrite anything in `/usr`, other than `/usr/local`. That's all managed by Apple as part of OS X. I'm not sure if you meant you overwrote `/usr/bin/python`.  Much better ways to handle multiple versions is to modify shell startup files to put `/opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin` at the front of `$PATH` or use an absolute reference when referring to Python, `/opt/local/bin/python2.6'.
Comment: Thx for the help. It turns out the bug is exactly MySQL bug 59006. It's a packaging bug. The solution at the top of that bug report works perfectly. You can actually `sudo install_name_tool -change libmysqlclient.xx.dylib /usr/local/mysql/lib/libmysqlclient.xx.dylib /usr/local/mysql/lib/libmysqlclient.xx.dylib` and DYLD_LIBRARY_PATH wouldn't be necessary anymore.
Comment: Hmm, I haven't tried it myself but shouldn't that be `sudo install_name_tool -id /usr/local/mysql/lib/libmysqlclient.xx.dylib /usr/local/mysql/lib/libmysqlclient.xx.dylib`?  You want to be changing the shared library identification name of the client lib, no?  And, of course, you want to first verify that the installed client lib name is wrong by using `otool -DX /usr/local/mysql/lib/libmysqlclient.xx.dylib`.  Ugh, the MySQL project really does not have a good track record with OS X packaging.
Comment: For this particular case, both works, don't know what the general case  should be tho.
Comment: I've been banging my head against a brick wall for hours (and have done so a few time previously). Your post solved my problem, specifically the `install_name_tool` bit in the edit. Many many thanks. I can't believe how difficult it is to install mysql with the python bindings on Snow Leopard, it's crazy.
Answer: I tried addressing the problem of an unfound dynamics library by compiling the MySQLdb module with static libraries (set static = True in site.cfg). But that generated the same error, as _mysql.o still was asking for the dynamic library. To fix that, I added libraries = [] to the block for if static in setup_posix.py.

Python import MySQLdb error - Mac 10.6


Twitter Bootstrap: Grid system floating

Question: I have the following simple html page. It should display Col1 to Col4 horizontally next to each other, but Col4 is in a below the others. Why? And how can I solve it?
I'm using bootstrap 2.2.2

    <link media="screen" rel="stylesheet" href="http://localhost:9000/assets/stylesheets/bootstrap.min.css">

Col1

Col2

Col3

Col4

Answer: Since you're trying to nest grid elements you need to specify the .row container as a child of your grid element container, this way your grid elements can properly align. The way you have it now the .row class won't be able to remove the extra gutter space it eliminates with a negative margin to fit all the columns within your container. You can read more about it here under the Nesting columns section.
Fixed markup:

    <link media="screen" rel="stylesheet" href="http://localhost:9000/assets/stylesheets/bootstrap.min.css">

Col1

Col2

Col3

Col4

Twitter Bootstrap: Grid system floating


A bright sounding piano or a mellow sounding one?

Question: When looking for a piano to buy, should I be taking note of the brightness of the sound of the piano?
What kind of piano sound would I be looking for for romantic period music and arranged music?
Basically, do I want a bright sounding piano or a mellow sounding one.
Answer: Your third question is a matter of opinion and can't be answered here. Pretty much the same with your second question. I'm pretty sure everyone would agree that the answer to the first question is "Yes". You should pay attention to the brightness, but how bright it should be is a matter of personal taste.
Important note: Pretty much all pianos get brighter in their first couple years as the hammer felt hardens, and they will more slowly continue to brighten over time, although string corrosion (fairly slow on most pianos) can darken the tone somewhat. So you should buy a piano that is a bit more mellow than you really want it to sound, unless you're buying a used piano.
Comment: The soundboard gradually bowing under the stress from the strings (especially on old and/or cheap pianos) is probably a more significant cause of the tone darkening than corroded strings - and unlike corroded strings, it is not repairable. On a grand piano that hasn't been "loved", just vacuuming the dust off the soundboard can make a difference! In general, re-voicing a piano to make the tone mellower is much easier than trying to make it brighter.
Answer: Rather than deciding between "bright" and "mellow," I suggest searching for a good sound in general and a long sustain. A good piano, one that's properly tuned, voiced, and regulated, will allow you the full spectrum of colors, from bright and brilliant to dark and lush, and the sound should last forever. Even if you prefer one of those two sounds now (bright vs mellow), a piano is a lifetime purchase, and your musical tastes may change over time. A good piano with those basic traits should give you all the colors you want, today and tomorrow.
Answer: I agree with Todd's answer; it's mostly a matter of personal preference.  However, it should be noted that a lot of the sound depends both on the style of the piano and the tuning.  The style may not be an adjustable factor depending on your circumstances.
An new upright piano will generally be more mellow than a grand, for example.  And like Todd says, they can get brighter with aging.  With uprights you will also be more likely to find that the soundboard softens after many years which will dull the tone.
Tuning an instrument a little higher will also give a brighter sound, as was seen historically with pitch inflation.  With a piano with multiple strings for each key, you can also offset the strings very slightly from each other to give the effect of a tack or ragtime piano (which is sort of bright and muddied at the same time).
Given the great electronic pianos we now have with key weighting and touch sensitivity, you might want to consider one of those — most come with several different timbre presets so you can select a brighter or mellower sound at will.  I really enjoy having the ability to select a harpsichord sound for certain pieces, myself.
Answer: It's really up to you. I like bright pianos as opposed to mellow ones. They are more happy sounding and when you play fast there is more dexterity in the sound. They are beautiful for romantic music. But for me they are more heart warming.
I'm a Bechstein lover and also love Yamaha because many of them are bright. But be careful not to go too bright so as to lose depth of tone. Pianos I can recommend: Bechstein, Zimmerman and Yamaha.


making a surrogate model of a function using feed forward networks

Question: I have been trying to learn a very simple function using feedforward network(function and configuration of network below). I sample 10-12 points in the range [0,1] to train the network.
def fun1D(x):
return np.power(6*x-2,2)*np.sin(12*x-4)

model used: 2 hidden layers with 60 units, optmizer=Adam(learning_rate =0.01). code using keras as below:
def setupANN(dim=1,units=60,loss='mse'):
model = Sequential()

## making the model graph, Stacking layers is done by .add():
model.add(Dense(units=units, input_dim=dim, activation='sigmoid'))
model.add(Dense(units=units, activation='sigmoid'))
model.add(Dense(units=1, activation= 'linear'))

optmiser = keras.optimizers.Adam(lr=0.01, beta_1=0.9, beta_2=0.999, epsilon=0.0001, decay=0.0)

# configure the model's learning process; loss and optimisation etc
model.compile(loss=loss,
              optimizer=optmiser, metrics=["mae"])

return model

Training: epochs = 10000, batch_size = 5
After training, I predict values in the range [0, 1] and plot them. For most of the cases, I am getting a good representation of the original function. However, When I plot the training loss vs epoch, I get a curve like this, which is quite annoying. I even run a hyperparameter search to tune the network parameters (no. layers, units, optmiser, epochs) but most of them get me same result. I know this is counter-intuitive as the number of training points may be low for this. But I want to know if someone has any idea to improve it, especially the strange plot of loss vs epoch.
Answer: This is something that I have observed with the Adam Optimizer on training too much. Below is the training loss from one of my experiments.

After several such results, I was able to conclude that this was a result of training too much. My guess is that because Adam works by optimizing the learning rate parameter-wise, when the model is nearing a minima, and Adam requires to update a parameter that has been rarely updated, the effective learning rate applied to it is quite high. This leads to a high training loss which is followed by immediate recovery, leading to the spike. This is what you tend to see from thereon.
The common suggestions are:

Stop early (1500 epochs/steps in your case).
Switch to SGD when nearing the minima. This leads to a smoother approach to the minima at the end.


How do I get the UUID for a current logged in user?

Question: In the Drupal 8 user's table, there is a UUID field. I'm trying to figure out if there is some method to easily access the UUID for a current user like you would when you get a UID for a user.
Thoughts?
Answer: You can load the UUID from entities with the method uuid(). If you have an account from the current user service then you have to load the user entity first:
$account = User::load($account->id());
$uuid = $account->uuid();

Answer: I ended up with something similar just now:
  /** @var \Drupal\user\Entity\User $user */
  $user = \Drupal\user\Entity\User::load(\Drupal::currentUser()->id());
  $uuid = $user->uuid();

Answer: If you're using JSON:API, you can get the UUID of the currently logged in user from /jsonapi, which returns a "me" link in the "meta" info:
{
  "jsonapi": {
    "version": "1.0",
    "meta": {
      "links": {
        "self": {
        "href": "http://jsonapi.org/format/1.0/"
      }
    }
  }
},
"data": [],
"meta": {
  "links": {
    "me": {
      "meta": {
        "id": "7ab54c06-e431-41a7-8f1f-a7c2165114a2"
        },
        "href": "http://example.lndo.site/jsonapi/user/user/7ab54c06-e431-41a7-8f1f-a7c2165114a2"
      }
    }
  },

If you need to get the UUID of a specific user from JSON:API and you have the Drupal UID, you can do this:
https://example.com/jsonapi/user/user?filter[uid][value]=123

Source


pROC versus ROCR

Question: This is a very basic question but I don't get why the following provides different results when applying pROC or ROCR, see plot.
Exp = c(1,0,1,1,1,1,1,0,0,1)
Pred = c(63.2,110.8,55.57,34.40,34.16,53.8,76.3,76.3,94.8,61.3)

# ########################## pROC ##########################
rocobj <- roc(response = Exp, predictor = Pred)
plot.roc(rocobj,main="pROC")

# ########################## ROCR ##########################
ROCRpred<-prediction(Pred,Exp)
plot(performance(ROCRpred,'tpr','fpr'),main="ROCR")

The interpretation of 1/0 should be same, why is it not?
Another question is what if I want to use say P/N for levels? Is there an order in which I have to define them?

Comment: The order of your factors matter and each package probably deals with them a little differently, if you get nonsense results just reverse the order for a "correct" plot.
Comment: Does a higher value for 'Pred' mean a higher probability for an 'Exp' with a 1 or a higher probability for an 'Exp' with a 0?
Comment: @Martijn it is the former, the higher 'Pred' the higher the probability for 'Exp'
Comment: @maciekj so the 110.8 prediction should correspond to a high probability for Exp=1 ?
Answer: The prediction function
If you do not specify which class is a positive case and a negative case then the prediction function needs to make up it's mind about positive and negative cases automatically. It does this as following:
> prediction

...

if (label.format == "ordered") {
    if (!is.null(label.ordering)) {
        stop(paste("'labels' is already ordered. No additional",
            "'label.ordering' must be supplied."))
    }
    else {
        levels <- levels(labels[[1]])
    }
}
else {
    if (is.null(label.ordering)) {
        if (label.format == "factor")
            levels <- sort(levels(labels[[1]]))
        else levels <- sort(unique(unlist(labels)))
    }
    else {
        if (!setequal(unique(unlist(labels)), label.ordering)) {
            stop("Label ordering does not match class labels.")
        }
        levels <- label.ordering
    }
    for (i in 1:length(labels)) {
        if (is.factor(labels))
            labels[[i]] <- ordered(as.character(labels[[i]]),
              levels = levels)
        else labels[[i]] <- ordered(labels[[i]], levels = levels)
    }
}

....

by specifying label.ordering = c(1,0) like
ROCRpred<-prediction(Pred, Exp, label.ordering = c(1,0))

you will get what you want.
Note that you can find help in R by typing help(prediction) and when you type just the name of the function prediction then you can see the function itself. (and of course you can replace this for any other function)

Conventions
You better use the following "conventions":

Use the 'higher' label for the positive class and the 'lower' label for the negative class.
Use the higher score a stronger tendency to the positive class. (currently you give the highest prediction score for the lowest class labels)

The following is a quote from the help file of the prediction function:

Since scoring classifiers give relative tendencies towards a negative
  (low scores) or positive (high scores) class, it has to be declared
  which class label denotes the negative, and which the positive class.
  Ideally, labels should be supplied as ordered factor(s), the lower
  level corresponding to the negative class, the upper level to the
  positive class. If the labels are factors (unordered), numeric,
  logical or characters, ordering of the labels is inferred from R's
  built-in < relation (e.g. 0 < 1, -1 < 1, 'a' < 'b', FALSE < TRUE). Use
  label.ordering to override this default ordering. Please note that the
  ordering can be locale-dependent e.g. for character labels '-1' and
  '1'.

So if you use
ROCRpred<-prediction(-Pred, Exp)

it works as well (in the sense that the curve is in the upper half, but note that there can still be a difference: prediction(-Pred, Exp) is not the same as prediction(Pred, -Exp), an image is shown later in this post).

Why did roc work but prediction not?
The roc function from the pROC package automatically determines the direction whether a higher score relates to a higher/lower probability of the positive class.
You still have to be very clear about the positive cases and negative cases though. You can get different results:
Comment: @maciekj could you write a more complete story. Something about binders and detecting poor ones (or is it detecting strong ones?). I am not sure what your Exp (how do you assign these numbers 0 and 1 to the cases) and Pred mean (what does 110.8 mean and how is it measured) and what your aim is with plotting an ROC curve. P/N refers to positive negative?
Comment: Martijn, thank you very much, an outstanding analysis!
Regarding the P/N assignmtent, we are dealing here with a poorly performing experimental setup, unable to detect so called strong binders (P) from the week ones. This can best be seen in the 110.8 case, where the exp assey misses it.
Comment: The assays measures binding between a molecule 'agent' and  a 'cell'.
For a given genotype (alleles with binding affinities) we get two values

1. EXPERIMENTAL results, 'Exp': P for 'positive' result meaning agent binds the cell or N for the 'negative' result.

2. IN SILICO prediction (ODE model for tumour size), 'Pred': simulated tumour size.

Both the experimental setup and model are imperfect. The assays clearly misses the strong binding subjects/phenotypes.
The model doesn't fully use the information about the binding strength due to gaps in bioinformatics.
Comment: Marijn, in the above I used 1 for 'P' and 0 for 'N'
Comment: Martijn, hope my explanation was clear. When you write 'You still have to be very clear about the positive cases and negative cases though. You can get different results' it's not up to me to assign the 1/0 lebels, they are given by experiment. The 'Pred' is more my job, is the model good or rubish. Right now it looks like it's the latter :(
The most important question is how do I set up pROC and ROCR so they provide cosistent results.
Comment: @maciekj So, you model tumour size and this can be a number like say 63.2 or 110.8. Is a larger tumor size associated with a higher or a lower probability that the agent binds the cell (exp = P/1)?
Comment: Apologies, I've used a misleading end point, it should rather be formulated as: 'Pred' is the number of activated cells. Meaning that the higher number of activated cells corresponds to higher probability for the binding between agnet and the cells.
Comment: So, you model the number of activated cells, which is a number like 63.2 or 110.8. Is a larger number of modeled activated cells associated with a higher or lower probability that the agent binds the cell (Exp = P/1)? Say you have a modeled value Pred=110.8 should you expect Exp=1 or Exp=0?
Comment: Yes there is this association, i.e. I expect Exp=1 for Pred=110.8
Comment: So for your three exp=0 you get your highest pred values 76.3, 94.8, 110.8 and for your seven  exp=1 you get your smallest values 34.40, 34.16, 55.57, 61.3, 63.2, 53.8, 76.3, which is nearly the most possible opposite result that you can get. What do you wish to learn from the ROC curve?
Comment: I have number of parameters I can vary in my ODE model, one goal is to optimize those parameters yielding the largest AUC of the ROC, anopther is to identify those which influence the AUC most. Also, sort of 'dose-response' curves, i.e. 'parameter-AUC' curves, could be plotted
Answer: The devil is in details and that is something that I also overlooked in the past.
The issue is that pROC (without notifying the user) will flip the curve if the AUC is much lower than 0.5. This is controlled by an argument named direction which by default is direction = "auto" I put the description at the end of my answer, but as a short answer, if your cases have higher values than controls (e.g cases are marked with 1 and controls with 0), then you should use direction = "<" and if it is the opposite, you should use direction = ">".
This is what the manual of pROC::roc says:

direction: in which direction to make the comparison?  "auto"
          (default): automatically define in which group the median is
          higher and take the direction accordingly.  ">": if the
          predictor values for the control group are higher than the
          values of the case group (controls > t >= cases).  "<": if
          the predictor values for the control group are lower or equal
          than the values of the case group (controls < t <= cases).
          You should set this explicity to ">" or "<" whenever you are
          resampling or randomizing the data, otherwise the curves will
          be biased towards higher AUC values.

pROC versus ROCR


machine learning models for identifying the functional transformations between two numerical sequences

Question: In the practice, there is a problem where the input can be a continuous line, like waveform, the output is also a continuous line, i.e., another waveform. In the sampling space, I can get 100 samples from waveform in the original continuous space. In other words, the problem can be transformed into finding the mapping function between a numerical sequence and another numerical sequence.
For instance, the input sequence is [-4.1288461e-16  -2.2452528e-15  -1.1717652e-14  -5.8685417e-14  -2.8203791e-13  -1.3006001e-12], the output sequence is [1.2080356e-01   0.0000000e+00   0.0000000e+00   0.0000000e+00   0.0000000e+00   0.0000000e+00 ]
In practice, we can have thousands of this kind of input-output pairs for training purposes. In the machine learning field, are there any specific models aiming to solve this kind of problem. My feeling is that neural network maybe a fit, but what kind of architecture is a better option. Here,  the output is not a categorical variables usually for classification problems.
Answer: The general term for this kind of problem, in which you want to predict one variable using another, and you have input-output pairs with which to train a model, is "supervised learning". Supervised learning is the most common application of statistics and machine learning. Machine-learning people like to call supervised learning of a continuous variable "regression", although to statisticians, this term refers to a certain family of models. In any case, there's a wide array of tools available for such problems, from linear regression to neural networks.


matrix expression for $e^{iA}Be^{iA}$ in terms of anticommutators?

Question: I'm familiar with the expression
$$e^{-iA}Be^{iA} = \sum_{n=0}^{\infty} \frac{i^n}{n!}[..[B,A],\dots A]_{n \; \rm times}$$
for square matrices $A$ and $B$ and was wondering if equivalently
$$e^{iA}Be^{iA} = \sum_{n=0}^{\infty} \frac{i^n}{n!}\{..\{B,A\},\dots A\}_{n \; \rm times},$$
or something similar?
Comment: Is the first equation really valid only for symmetric matrices?
Comment: I think the commutator is only defined for symmetric matrices
Comment: No, the commutator is defined for all square matrices: $[A, B] = AB-BA.$
Comment: Anyway, that's not what you question is about.
Comment: very sorry! I meant to say square, not symmetric, total brain lapsus!
Answer: Let $A,B$ be fixed square matrices of the same size, and define $F(t) = e^{itA}Be^{itA}.$ We then have
$$\begin{align}
F'(t) &= e^{itA}iABe^{itA} + e^{itA}BiAe^{itA} = e^{itA}(iAB+BiA)e^{itA} = e^{itA}i\{A,B\}e^{itA} \\
F''(t) &= e^{itA}iA(iAB+BiA)e^{itA} + e^{itA}(iAB+BiA)iAe^{itA} = e^{itA}i^2\{A,\{A,B\}\}e^{itA} \\
F'''(t) &= \cdots = i^3e^{itA}\{A,\{A,\{A,B\}\}\}e^{itA} \\
&\vdots \\
F^{(k)}(t) &= i^k e^{itA}(\{A,\bullet\}^kB)e^{itA}
\end{align}$$
so
$$
F(t)
= \sum_{k=0}^{\infty} \frac{t^k}{k!} f^{(k)}(0)
= \sum_{k=0}^{\infty} \frac{t^k}{k!} i^k (\{A,\bullet\}^kB)
$$
and
$$
e^{iA}Be^{iA} = F(1) = \sum_{k=0}^{\infty} \frac{1}{k!} i^k (\{A,\bullet\}^kB).
$$
Answer: Nice question and nice answers. There is a generalization that admits an even simpler proof that I think it's worth mentioning.
For (square) matrices $X,Y,B$,  let me define
$$
B(t):=e^{tX} B e^{tY}.
$$
Differentiating with respect to $t$ we see that $B(t)$ satisfies the following differential equation:
\begin{align}
\frac{dB}{dt}(t) &= XB(t) + B(t) Y \\
 &= \mathcal{K}_{X,Y}(B)& = XZ+ZY, \tag{1}
\end{align}
having defined the linear (super) operator
$$
\mathcal{K}_{X,Y}(Z) = XZ+ZY
$$
Now the solution of $(1)$ with initial condition $B(0)=B$ is  $
B(t) =  e^{t \mathcal{K}_{X,Y}} B$. We have then
$$
B(t) = e^{tX} B e^{tY} = e^{t \mathcal{K}_{X,Y}} B =\sum_{n=0}^\infty \frac{(t\mathcal{K}_{X,Y})^n}{n!} B .
$$
Set $Y=\pm X$ to obtain your formulas with commutators/anticommutators.
Comment: Thanks! I'm not sure what you mean. The solution of $\dot{B} = A B$ is $B(t)= e^{tA}B(0)$ also for non-commutative objects. In fact we never had to commute $A$ with $B$ or $B(0)$. In the post $A$ is $\mathcal{K}$
Comment: Ah, yes. ${}{}$
Answer: Define $L_A(B):=AB$ and $R_A(B):=BA$ and write $\mathrm{ad}_A(B):=[A,B]=(L_A-R_A)B$.
Then use the binomial theorem to expand $(\mathrm{ad}_A)^n$ (since $L_A,R_A$ commute):
$$ \begin{array}{lcl} \displaystyle \sum_{n=0}^{\infty} \frac{(i\,\mathrm{ad}_A)^n}{n!}B & = & \displaystyle \sum_{n=0}^{\infty} \frac{i^n}{n!}(L_A-R_A)^nB \\[5pt] & = & \displaystyle \sum_{n=0}^{\infty}\frac{i^n}{n!} \left[ \sum_{k+\ell=n} \binom{n}{k,\ell} L_A^k (-R_A)^{\ell} \right] B \\[8pt] & = & \displaystyle \sum_{k=0}^{\infty}\sum_{\ell=0}^{\infty}\frac{(iL_A)^k}{k!}\frac{(-iR_A)^{\ell}}{\ell!}B \\[8pt] & = & \displaystyle \left[\sum_{k=0}^{\infty}\frac{(iA)^k}{k!}\right]B\left[\sum_{\ell=0}^{\infty} \frac{(-iA)^\ell}{\ell!}\right] \\[6pt] & = & =\exp(iA)B\exp(-iA). \end{array} $$
The same thing works for $\{A,B\}$, just get rid of the minus sign in $L_A-R_A$.


notepad++ How to remove multi symbol comma,dot,Exclamation mark

Question: Example

Email me, new! responses ...to ?my posts
Email me new responses to my posts

I search multi time on google . I can replace it one by one but i want a formular remove all symbol one time.
I'm try search and replace with (.|,|?|!|...|) but it's doesn't work.
Answer: You have to escape specials characters and select the option "regular expression" in the replace window. This is your regex.
\.|\?|!|\,

Comment: Thank you for reply with image :D
Answer: If you want to remove all pnctuation marks, use unicode property \p{Punct}:

Ctrl+H
Find what: \p{Punct}+
Replace with: LEAVE EMPTY
check Wrap around
check Regular expression
Replace all
Comment: Yeah . But i don't want remove " ' " . So above answer is better . Thank you .


How do I use Scanner to read in a series of Strings from the keyboard, all on one line, and concatenate them

Question: How do I use Scanner to read in a series of Strings from the keyboard, all on one line, and concatenate them.
Here is the code I have so far:
import java.util.Scanner;

public class Exam12Practice {

   public static void main(String[] args)
   {
      Scanner input=new Scanner(System.in);
      String words="";
      System.out.println("enter a word");
      while(input.hasNext())
      {
         words = words.concat(input.next());
      }

      System.out.println(words);
   }
}

Answer: Your code already does what you are asking. To get it to work
Type in your words
Press Enter
Press CTRL-Z (^D on *nix systems)

Some points to note:
input.hasNext() will always return true for STDIN so just pressing Enter on its own won't work.
You could have used input.readLine() and split the words for your exercise.
Most people would probably prefer to use StringBuilder because of the improved performance it provides over String.concat.


angular js template with routing and translate refusing to work

Question: I am new to angular and it seems very interesting
I am creating a template for a promotional page and struggling to understand what is causing an issue on it - half of the script stopped working...
My fiddle is in here see >
fiddle example
or Plunker
I have working example without route...
Can you advice me how to fix this issue please
Comment: For a BIG code example, I think using http://plnkr.co/ is easier to work with.
Comment: Error: [ng:areq] Argument 'langSwitch' is not a function, got undefined
Comment: Hi, plunker example added.
And another working example without routing here http://fiddle.jshell.net/danilabelov/EkH52/
Answer: First of all, i'd suggest you to separate your project to different files based on best practises
Secondly, 'config, controller, factory' etc are methods of angular.module object. So you need to retrieve it first like
angular.module('<moduleName>').config(...)

or use some variable.
http://plnkr.co/edit/CK5R2cS7Zd9T9Kun9Epc?p=preview
Comment: thank you for explanation and for examples.

my original plan was to structure into modules but I ran into problem...


How pull innerHTML of element created by javascript function

Question: Question:
I have a div element with id of "tableholder" which is purely to hold the output on a javascript function that pulls data via ajax from a PHP file.
Within the tableholder element, is a list.js table that works if we simply include he php file instead of calling it via ajax and setting it via innerHTML = "".
How do I use the contents of innerHTML of the tableholder div when it has been created dynamically?
Code so far:
PHP File called:
    <?php

// Connection details for the database being called.
include 'connection.php';

// Putting the query to the database in a variable called "$assets".
$assets = "SELECT * FROM assetregister.getassets";
$assetPull = $conn->query($assets);
$output = "";

// Pulling all of the details from the database and displaying them on the page within the inidividual divs.
if ($assetPull->num_rows > 0) {
    $output .= "
";
    $output .= "<input class='search form-control mt-2' placeholder='Search...' />";
    $output .= "
";
    $output .= "<button class='sort btn' data-sort='model'>Sort By Model</button>";
    $output .= "<button class='sort btn ml-2' data-sort='domain'>Sort By Domain</button>";
    $output .= "<button class='sort btn ml-2' data-sort='location'>Sort By Location</button>";
    $output .= "
";
    $output .= "
";
    $output .= "
";
    $output .= "Model	";
    $output .= "Serial Number	";
    $output .= "Asset Number	";
    $output .= "Domain	";
    $output .= "Location	";
    $output .= "Type	";
    $output .= "
";
    while ($row = $assetPull->fetch_assoc()) {
        $output .= "
";
        $output .= "" . $row['modelName'] . "	";
        $output .= "" . $row['serialNumber'] . "	";
        $output .= "" . $row['assetNumber'] . "	";
        $output .= "" . $row['domain'] . "	";
        $output .= "" . $row['locationName'] . "	";
        $output .= "" . $row['type'] . "	";
    }
    $output .= "
";
    $output .= "
";
    $output .= "
";
    // If there is no rows in the table that is being called then they will display 0 Results... See below.
} else {
    $output .= "0 results";
}

echo $output;

$conn->close();
?>

 This works:

                <button class="btn btn-primary" onclick="assetSubmit()" style="width: 100%;">Add An Asset</button>
                <?php include 'scripts/getAsset.php'; ?>

This outputs the table and all of the list.js functions work fine as expected.
This Doesn't Work:

                <button class="btn btn-primary" onclick="assetSubmit()" style="width: 100%;">Add An Asset</button>

<?php include 'assets/layout/footer.php'; ?>

What I can assume:
From the various console.log lines and php echos I have put in to test the code, it would appear that the issue it that the javascript command for initiating the list.js table isn't able to find the "user" table that is created by the document.getElementById(name).innerHTML = data; in the populatetable() function. I know this because the folllowing shows blank:
console.log(document.getElementById(tableholder).innerHTML);
What am I doing wrong, or is it not possible to pull the innerHTML data of a dynamically created element?
Comment: What does `document.getElementById('tableholder')` display when it is logged?
Comment: I'm investigating this right now, but one thing I found just now: you're opening a new `` in each table row.
Comment: if I do console.log(document.getElementById('tableholder')); then I can see the entire tableholder element including the innerHTML. However, I can't do the same on a child element of tableholder. Also - I didn't notice the  thing but it works when directly ported to the page via php so I doubt this is the root cause of the issue.
Comment: I know what's happening; the ajax call is *asynchronous*, which means things aren't happening in the order you think they are.
Answer: The problem is that by the time you're calling List(), the ajax call hasn't finished yet, since it's asynchronous.
Try this instead:
var options = {
    valueNames: ['model', 'serialNumber', 'assetNumber', 'domain', 'location', 'type']
};
var userList;

$(document).ready(function() {
    populatetable('#tableholder');
});

function populatetable(id) {
    $.ajax({
        url: "scripts/getAssets.php",
        success: function(data) {
            $(id).html(data);
            userList = new List('users', options);
        }
    });
}

Here I'm using jQuery everywhere it's useful, and I'm calling List() only after the data is actually inserted.
Also note, that like I mentioned in my comment, you need to move the
 line outside the loop.
Comment: I have also moved the  tag and created a new row for each while loop as you mentioned. The answer works a treat!
Comment: Right, you were missing ` ... ` in both head and body of the table. Glad it's working :)


Finding an interpretation that makes a formula true

Question: I understand the basic rules of propositional and predicate logic, and what makes formulas true in a general sense, but I am stuck on how to answer specific questions that ask of "each of the formulas below, state an interpretation that makes it true and an interpretation that makes it false"
For instance, one of first questions asks me to state an interpretation that makes ∃x(F x ∧ ¬∃yGxy) true and one that makes it false. Should my answer simply put:
Domain = a
|Fa & ¬Gaa| = F, and |Fa & Gaa| = T? Or would I need to specify, for example, that the interpretation of the predicate '|G|' = \emptyset and the interpretation of '|F|' = a to give an example of an occasion where the formula is true?
Answer: No; you have to specify the domain of the interpretation and the "menaing" assigned to the predicates.
E.g.

as domain, the set $\mathbb N$ of natural numbers;
$F(x)$ interpreted as $x=0$;
$G(x,y)$ interpeted as $x > y$.

With this interpretation, the formula :

$∃x(Fx ∧ ¬∃yGxy)$

means :

"there is a number $x$ such that $x=0$ and for all numbers $y : \ x \le y$"

which is true in $\mathbb N$ exactly when $x=0$.


How to setup Topshelf with WhenCustomCommandReceived?

Question: I'm using Topshelf to create a windows service (ServiceClass)  and i'm thinking of sending custom commands using WhenCustomCommandReceived.
HostFactory.Run(x =>
{
    x.EnablePauseAndContinue();
    x.Service<ServiceClass>(s =>
    {
        s.ConstructUsing(name => new ServiceClass(path));
        s.WhenStarted(tc => tc.Start());
        s.WhenStopped(tc => tc.Stop());
        s.WhenPaused(tc => tc.Pause());
        s.WhenContinued(tc => tc.Resume());
        s.WhenCustomCommandReceived(tc => tc.ExecuteCustomCommand());
    });
    x.RunAsLocalSystem();
    x.SetDescription("Service Name");
    x.SetDisplayName("Service Name");
    x.SetServiceName("ServiceName");
    x.StartAutomatically();
});

However, i'm getting an error on the WhenCustomCommandReceived line:
Delegate 'Action< ServiceClass, HostControl, int>' does not take 1 arguments
The signature is
ServiceConfigurator<ServiceClass>.WhenCustomCommandReceived(Action<ServiceClass, HostControl, int> customCommandReceived)

I already have methods for start, stop, pause in my ServiceClass: public void Start(), etc.  Can anyone point me in the right direction on how to setup the Action?  Thanks!
Answer: So, as you can see in the signature of the method, the Action takes three parameters, not just one. This means that you need to set it up like this:
s.WhenCustomCommandReceived((tc,hc,command) => tc.ExecuteCustomCommand());

The interesting parameter in this case would be command which is of type int. This is the command number that is sent to the service.
You might want to change the signature of the ExecuteCustomCommand method to accept such command like this:
s.WhenCustomCommandReceived((tc,hc,command) => tc.ExecuteCustomCommand(command));

And in the ServiceClass:
public void ExecuteCustomCommand(int command)
{
    //Handle command
}

This allows you to act differently based on the command that you receive.
To test sending a command to the service (from another C# project) you can use the following code:
ServiceController sc = new ServiceController("ServiceName"); //ServiceName is the name of the windows service
sc.ExecuteCommand(255); //Send command number 255

According to this MSDN reference, the command value must be between 128 and 256.
Be sure to reference the System.ServiceProcess assembly in your test project.
Comment: Thanks Yacoub!  Solved my problem.  Learned something new today.
Comment: is it possible with topshelf to send command from cmd.exe? like > myservice.exe 128?