Histogram of my cam in real time

Question: Im trying to show the histogram in real time from grayscale of my webcam, the problem is that histogram is not being update and my cam stop until i close histogram's window. How can i fix this? I want to show the grayscale img from my webcam and its histogram in same time, is possible do that?
import numpy as np
import cv2
from matplotlib import pyplot as plt

cap = cv2.VideoCapture(0)

while(True):
    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    cv2.imshow('Janela', frame)
    cv2.imshow('Outra', gray)
    plt.hist(gray.ravel(), 256, [0, 256])
    plt.show()

    if (cv2.waitKey(1) & 0xFF == 27):
        break

cap.release()
cv2.destroyAllWindows()

Comment: `plt.show()` opens a windows that blocks unlike `cv2.imshow` which requires `cv2.waitkey` to block. So showing the histogram using a different function is your best option.
Answer: I have been working for a while in the same task. After some time, I have a piece of code that works very good. It displays the camera image in a window and a histogram in other. Since I'm interested in finding colors, I'm working with the "hue" channel of each frame.
# import the necessary packages
import cv2
import numpy as np

#Create window to display image
cv2.namedWindow('colorhist', cv2.WINDOW_AUTOSIZE)

#Set hist parameters
hist_height = 64
hist_width = 256
nbins = 32
bin_width = hist_width/nbins

camera_id = 0 # type fo webcam [0 built-in | 1 external]
cameraWidth = 320
cameraHeight = 240

if camera_id == 0:
   cameraId = "PC webcam"
elif camera_id == 1:
   cameraId = "External webcam"

camera = cv2.VideoCapture(camera_id)

# set camera image to 320 x 240 pixels
camera.set(3,cameraWidth)
camera.set(4,cameraHeight)

cameraInfo = "Image size (%d,%d)" % (camera.get(3),camera.get(4))

# initialize mask matrix
mask = np.zeros((cameraHeight,cameraWidth),  np.uint8)

# draw a circle in mask matrix
cv2.circle(mask,(cameraWidth/2,cameraHeight/2), 50, 255, -1)

#Create an empty image for the histogram
h = np.zeros((hist_height,hist_width))

#Create array for the bins
bins = np.arange(nbins,dtype=np.int32).reshape(nbins,1)

while True:
   # grab the current frame
   (grabbed, frame) = camera.read()

   if not grabbed:
      "Camera could not be started."
      break

   hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
   #Calculate and normalise the histogram
   hist_hue = cv2.calcHist([hsv],[0],mask,[nbins],[0,256])
   cv2.normalize(hist_hue,hist_hue,hist_height,cv2.NORM_MINMAX)
   hist=np.int32(np.around(hist_hue))
   pts = np.column_stack((bins,hist))

   #Loop through each bin and plot the rectangle in white
   for x,y in enumerate(hist):
      cv2.rectangle(h,(x*bin_width,y),(x*bin_width + bin_width-1,hist_height),(255),-1)

   #Flip upside down
   h=np.flipud(h)

   #Show the histogram
   cv2.imshow('Color Histogram',h)
   h = np.zeros((hist_height,hist_width))

   frame = cv2.bitwise_and(frame,frame,mask = mask)
   cv2.putText(frame, cameraInfo, (10, 20),
    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

   cv2.imshow(cameraId, frame)
   key = cv2.waitKey(1) & 0xFF

   # if the `q` key is pressed, break from the loop
   if key == ord("q"):
      break

camera.release()
cv2.destroyAllWindows()

The code is based on the code from JohnLinux post (How do I plot a 32-Bin Histogram for a Grayscale Image in Python using OpenCV) and other lines of code come from what I have learnt from Adrian Rosenbrock's site https://www.pyimagesearch.com/.
Comment: Exactly this i was trying to do! I just had to cast some variables to int, because was giving error type.  `` cv2.circle(mask,(cameraWidth/2,cameraHeight/2), 50, 255, -1)
TypeError: integer argument expected, got float´´
Answer: I see there is very little feedback on this topic. Some solutions look overwhelmingly complex. My brother needed just a simple version of such programme for his studies, so I decided to write it as simple as it can be (11 lines of code - enjoy:))
import cv2 as cv
from matplotlib import pyplot as plt

vid = cv.VideoCapture(0, cv.CAP_DSHOW)
while True:
    ret, img = vid.read()
    cv.imshow("img", img)
    plt.hist(img.ravel(), 256, [0, 256])
    plt.draw()
    plt.pause(0.1)
    plt.clf()
cv.destroyAllWindows()

Answer: another solution for the picamera doing stream + histogram
from picamera import PiCamera
from picamera.array import PiRGBArray
import cv2
import matplotlib.pyplot as plt

camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 40
rawCapture = PiRGBArray(camera, size=(640, 480))

for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
    bgr_arr = frame.array
    gray_arr = cv2.cvtColor(bgr_arr, cv2.COLOR_BGR2GRAY)
    cv2.imshow("Preview", gray_arr)
    key = cv2.waitKey(1) & 0xFF
    rawCapture.truncate(0)
    plt.hist(gray_arr.ravel(), 256, [0, 256], histtype='stepfilled')
    plt.title('Grayscale histogram')
    plt.xlabel('Grayscale value')
    plt.ylabel('Pixel count')
    plt.plot()
    plt.pause(0.1)
    plt.clf()

    if key == ord("q"):
        cv2.destroyAllWindows()
        break

Histogram of my cam in real time


Correct way to call animation through and IF statement (A-Frame animation mixer)?

Question: I am currently working with the animation mixer in A-Frame and am attempting to make a condition occur when a specific animation is playing (e.g. Animation 'B' out of A, B, C).
I'm unfamiliar with Javascript but I've made several attempts:
<code> if(character.getAttribute == character.getAttribute("animation-mixer", {clip: "B"}){
 //  Insert action here
 }

 if(character == character.setAttribute("animation-mixer", {clip: "B"})){
 //  Insert action here
 };

 if(character.getAttribute("animation-mixer") == {clip: "B"}){
 //  Insert action here
 };
</code>
Answer: To retrieve component data You should just use getAttribute(componentName):
<code>// grabbing the component data object
character.getAttribute("animation-mixer")

// same + accessing the clip property
character.getAttribute("animation-mixer").clip

// checking if the property equals "B"
if (character.getAttribute("animation-mixer").clip === "B") {
   console.log("Its B!")
}
</code>

setAttribute() will change the animation:
<code>setAttribute("animation-mixer", "clip", "A")
// or character.setAttribute("animation-mixer", {clip: "B"})
character.getAttribute("animation-mixer").clip // A
</code>


Parsing json in JQuery Ajax Call From PHP

Question: I have seen a bunch of different ways to call a php script via ajax that returns json but i cant seem to get any working  I was curious if someone could help me.
Here is my PHP script:
//database connection stuff....
$query = "SELECT * FROM  weather limit 10";
$result = mysql_query($query) or die("SQL Error 1: " . mysql_error());
// get data and store in a json array
while ($row = mysql_fetch_array($result, MYSQL_ASSOC)) {
    $orders[] = array(
        'zip' => $row['zip'],
        'temp' => $row['temp'],
        'time' => $row['time']
      );
}

echo json_encode($orders);

Next here is my ajax call:
$.ajax({
            url : 'getweather.php',
            type : 'post',
            dataType: 'json',
            success: function(data){
                for (var x = 0; x < data.length; x++) {

                    alert(data['zip']);

                }

            }
        });

    });

With this i just get 10 alerts that say undefined.  Can someone please help me figure out what i am doing wrong?
Thanks in advance for the help!
Craig
Comment: You are not using your x. Try `data[x]['zip']`
Answer: Your data is an array of arrays, you need to index the outer array before the inner array:
alert(data[x]['zip'])

Comment: i knew it'd be something simple. Thanks!


Matrix calculus with row vectors

Question: The rules for matrix calculus I find assume column vectors. Are the rules different for row vectors? (I am having a hard time finding them)
I used them when deriving a formula for backpropagation.
\begin{gather*}
Y\ =\ XW\ +\ B\\
X=\begin{bmatrix}
x_{0} & x_{1} & x_{2}
\end{bmatrix} ,\ Y=\begin{bmatrix}
y_{0} & y_{1}
\end{bmatrix} ,\ W=\begin{bmatrix}
w_{00} & w_{01}\\
w_{10} & w_{11}\\
w_{20} & w_{21}
\end{bmatrix} ,\ B=\begin{bmatrix}
b_{0} & b_{1}
\end{bmatrix}
\end{gather*}
\begin{gather*}
\left(\frac{\partial L}{\partial W}\right)^{T} =\begin{bmatrix}
\frac{\partial L}{\partial w_{00}} & \frac{\partial L}{\partial w_{00}}\\
\frac{\partial L}{\partial w_{10}} & \frac{\partial L}{\partial w_{11}}\\
\frac{\partial L}{\partial w_{20}} & \frac{\partial L}{\partial w_{21}}
\end{bmatrix} =\begin{bmatrix}
\color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{00}} & \color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{01}}\\
\color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{10}} & \color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{11}}\\
\color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{20}} & \color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{21}}
\end{bmatrix}\\
\\
Focus\ on\ one\ term:\\
y_{0} \ =\ w_{00} x_{0} +w_{10} x_{1} +w_{20} x_{2} \ +b_{0}\\
y_{1} \ =\ w_{01} x_{0} +w_{11} x_{1} +w_{21} x_{2} +b_{1}\\
\\
\frac{\partial Y}{\partial w_{00}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{00}}\\
\frac{\partial y_{1}}{\partial w_{00}}
\end{bmatrix} =\begin{bmatrix}
x_{0}\\
0
\end{bmatrix}\\
\color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial w_{00}} =\ \begin{bmatrix}
\color{red}{\frac{\partial L}{\partial y_{0}}} & \color{red}{\frac{\partial L}{\partial y_{1}}}
\end{bmatrix}\begin{bmatrix}
x_{0}\\
0
\end{bmatrix} =\color{red}{\frac{\partial L}{\partial y_{0}}} \ x_{0} \ +\ \color{red}{\frac{\partial L}{\partial y_{1}}} *\ 0\ =\color{red}{\frac{\partial L}{\partial y_{0}}} \ x_{0}\\
\\
\frac{\partial Y}{\partial w_{10}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{10}}\\
\frac{\partial y_{1}}{\partial w_{10}}
\end{bmatrix} \ =\begin{bmatrix}
x_{1}\\
0
\end{bmatrix} ,\ \frac{\partial Y}{\partial w_{01}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{01}}\\
\frac{\partial y_{1}}{\partial w_{01}}
\end{bmatrix} \ =\begin{bmatrix}
0\\
x_{0}
\end{bmatrix} ,\ \frac{\partial Y}{\partial w_{11}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{11}}\\
\frac{\partial y_{1}}{\partial w_{11}}
\end{bmatrix} \ =\begin{bmatrix}
0\\
x_{1}
\end{bmatrix} ,\\
\frac{\partial Y}{\partial w_{20}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{20}}\\
\frac{\partial y_{1}}{\partial w_{20}}
\end{bmatrix} \ =\begin{bmatrix}
x_{2}\\
0
\end{bmatrix} ,\ \frac{\partial Y}{\partial w_{21}} =\ \begin{bmatrix}
\frac{\partial y_{0}}{\partial w_{21}}\\
\frac{\partial y_{1}}{\partial w_{21}}
\end{bmatrix} \ =\begin{bmatrix}
0\\
x_{2}
\end{bmatrix}\\
\\
Finally:\\
\left(\frac{\partial L}{\partial W}\right)^{T} =\begin{bmatrix}
\frac{\partial L}{\partial y_{0}} \ x_{0} & \frac{\partial L}{\partial y_{1}} \ x_{0}\\
\frac{\partial L}{\partial y_{0}} \ x_{1} & \frac{\partial L}{\partial y_{1}} \ x_{1}\\
\frac{\partial L}{\partial y_{0}} \ x_{2} & \frac{\partial L}{\partial y_{1}} \ x_{2}
\end{bmatrix} =\begin{bmatrix}
x_{0}\\
x_{1}\\
x_{2}
\end{bmatrix}\begin{bmatrix}
\frac{\partial L}{\partial y_{0}} & \frac{\partial L}{\partial y_{1}}
\end{bmatrix} =\ X^{T}\color{red}{\frac{\partial L}{\partial Y}}
\end{gather*}
However, I am not sure if the final result has the correct shape.
\begin{gather*}
  \frac{\partial L}{\partial X} =\ \color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial X}\\
  \frac{\partial Y}{\partial X} =\begin{bmatrix}
  \frac{\partial y_{0}}{\partial x_{0}} & \frac{\partial y_{0}}{\partial x_{1}} & \frac{\partial y_{0}}{\partial x_{2}}\\
  \frac{\partial y_{1}}{\partial x_{0}} & \frac{\partial y_{1}}{\partial x_{1}} & \frac{\partial y_{1}}{\partial x_{2}}
  \end{bmatrix} =\begin{bmatrix}
  w_{00} & w_{10} & w_{20}\\
  w_{01} & w_{11} & w_{21}
  \end{bmatrix} =W^{T}\\
  \frac{\partial L}{\partial X} =\ \color{red}{\frac{\partial L}{\partial Y}} W^{T} ,\color{red}{\frac{\partial L}{\partial Y} =}\color{red}{\begin{bmatrix}
  \color{red}{\frac{\partial L}{\partial y_{0}}} & \color{red}{\frac{\partial L}{\partial y_{1}}}
  \end{bmatrix}}\\
  \\
  If\ the\ rules\ were\ simply\ reversed:\\
  \color{red}{\frac{\partial L}{\partial Y}}\color{red}{=}\color{red}{\begin{bmatrix}
  \color{red}{\frac{\partial L}{\partial y_{0}}}\\
  \color{red}{\frac{\partial L}{\partial y_{1}}}
  \end{bmatrix}} ,\frac{\partial Y}{\partial X} =\begin{bmatrix}
  \frac{\partial y_{0}}{\partial x_{0}} & \frac{\partial y_{1}}{\partial x_{0}}\\
  \frac{\partial y_{0}}{\partial x_{1}} & \frac{\partial y_{1}}{\partial x_{1}}\\
  \frac{\partial y_{0}}{\partial x_{2}} & \frac{\partial y_{1}}{\partial x_{2}}
  \end{bmatrix}\\
  Then\ the\ dimensions\ for\ \color{red}{\frac{\partial L}{\partial Y}}\frac{\partial Y}{\partial X} \ won't\ match
  \end{gather*}
Comment: Almost anything being defined or described that use column vectors you can take the transpose of both sides of such equations to have things described in terms of row vectors instead.  For instance, I prefer using column vectors for markov chains and having $Av_t = v_{t+1}$ with $v_i$ as a column vector but you could just as easily had phrased this as $w_{t+1}=w_tB$ with row vectors where here $w_i=v_i^T$ and $B=A^T$
Comment: Yes, I agree with @JMoravitz, I believe you can transpose everything on the equations and they would be equivalent. Just make sure you're multiplying the matrix and the vector in the correct order (left vs right). Check for a given row to make sure.
Comment: @Tom What confuses me is that the rule for dL/dW (scalar and 3 by 2 matrix) will be the same whether I am using row or column vectors (right?) and I arrived at the shape 2 by 3 (which is correct since W is in the denominator?) even though I used rules for column vectors in the derivation
Answer: $
\def\o{{\tt1}}
\def\BR#1{\Big(#1\Big)}
\def\LR#1{\left(#1\right)}
\def\op#1{\operatorname{#1}}
\def\trace#1{\op{Tr}\LR{#1}}
\def\frob#1{\left\| #1 \right\|_F}
\def\qiq{\quad\implies\quad}
\def\p{\partial}  \def\grad#1#2{\frac{\p #1}{\p #2}}
\def\m#1{\left[\begin{array}{r}#1\end{array}\right]}
\def\c#1{\color{red}{#1}}
\def\gradLR#1#2{\LR{\grad{#1}{#2}}}
$Assume that the gradient $\LR{G=\grad LY}$ is known and use it to find the other gradients.
Calculate the differential of the loss function
$$\eqalign{
dL &= G:dY \\
  &= G:\BR{dX\:W + X\:dW} \\
  &= \LR{GW^T}:dX + \LR{X^TG}:dW \\
}$$
Then hold $W$ constant to obtain the gradient with respect to $X,\,$
and vice versa
$$\eqalign{
\grad LX &= GW^T \;\qquad\; \grad LW &= X^TG \qquad \\
\\
}$$

In the above, a colon is used to denote the Frobenius product.
It has the following properties
$$\eqalign{
A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; \trace{A^TB} \\
A:A &= \frob{A}^2 \qquad \{ {\rm Frobenius\;norm} \}\\
A:B &= B:A \;=\; B^T:A^T \\
C:\LR{AB} &= \LR{CB^T}:A \;=\; \LR{A^TC}:B \\
}$$
Note that the above derivation is quite general. It does not matter if $\,\{B,X,Y\}\,$ are row vectors or column vectors. They could even be matrices.
Comment: @BPDev Consider the initial known gradient $\left(G=\frac{\partial L}{\partial Y}\right),$ where $Y$ is a row vector. The _Layout Convention_ determines whether $G$ is a row vector or a column vector. For me, the best convention is the one that makes $G$ the **same shape** as $Y$ because that's the one which allows me to write $$dL = G:dY$$ without introducing unnecessary transpose operations.
Comment: Based on [this](https://en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-matrix) rule, shouldn't dL/dW (W: 3 x 2) have shape 2 x 3? (X_T * G: (3 x 1) x  (1 x 2))
Comment: @BPDev I use the opposite _Layout Convention_ than the one used in that rule. These issues are addressed in the section immediately following your link.
Comment: Ah thank you. To confirm, the layout conventions do not depend on how we choose to represent vectors correct? (For numerator layout, dY/dL is a column vector, but Y doesn't have to be a column vector despite the phrasing "Numerator layout, i.e. lay out according to y and xT.")

Matrix calculus with row vectors


Some files from RPM have bogus permissions

Question: I'm building RPMs with fpm. One of the goals is to set the user and group of the installed files, so I'm using the --rpmuser and --rpmgroup flags.
It's working for the most part, however one of the directories is not receiving the desired user/group. I've run fpm with the -e flag to inspect the Spec File. All the files and directories are marked beneath the %files directive that should set the desired user and group - adminuser,admingroup.
%files
%defattr(-,adminuser,admingroup,-)

# Reject config files already listed or parent directories, then prefix files
# with "/", then make sure paths with spaces are quoted. I hate rpm so much.
/etc/admin-services/admin.properties
/usr/share/admin-app/static/admin-console/index.html
/usr/share/admin-app/static/admin-console/console-env.js
/usr/share/admin-app/static/admin-console/css/styles.css
/usr/share/admin-app/webapps/admin-services.war

After installation, all the files belong to adminuser,admingroup except the /usr/share/admin-app/static directory (and everything beneath it), they all belong to root,root.
I don't think this is fpm's fault, the Spec File looks good. I believe this is an issue with rpmbuild under the hood. Any idea what could be going on?
I've read through the documentation on Spec Files, and I don't see any other directives that could be affecting the /usr/share/admin-app/static directory.
Comment: Those comments in there are from the _fpm_ author btw, hilarious!
Comment: I've managed to cope via an `--after-install` flag to fpm. Therein I set the perms on the `/usr/share/admin-app/static` directory recursively. Would still like to know why the `--rpmuser` and `--rpmgroup` flags are failing though.
Answer: You should use the --directories option. From fpm --help:
--directories DIRECTORIES     Recursively mark a directory as being owned by the package

Without it, the ownership is only set for files, directories are omitted and will belong to root:root.
If you add --directories /usr/share/admin-app/static and use fpm -e to view the spec file, you'll see it adds the %dir directive which explicitly sets the permissions as you'd expect them:
%dir %attr(775, adminuser, admingroup) /usr/share/admin-app/static

And other entires for subdirectories beneath that path, if any.
Also, if you have multiple directories in your rpm's root (like I did), you have to use a --directories option for each.
This is a bit confusing, but let's thank rpm for that, not fpm's fault.


Como dar stop em uma função através do console?

Question: Gostaria de stopar um fadeIn() durante sua execução de modo que a tela fique exatamente como estava no momento que eu parei.
O ideal seria algo genérico, que funcione para fadeIn, slideUp, slideDown, setTimeout etc.
É possivel fazer isso utilizando o console do navegador? Como seria?
OBS: Só para informação, quero fazer isso para poder pegar o css do item no momento da transição, pois é um componente e ele muda as classes e consequentemente o css...
Answer: $( "seu-elemento" ).stop();

Esse método serve para parar animações em geral.
.stop() | API jQuery
Exemplo retirado do jQuery:

// Start animation
$( "#go" ).click(function() {
  $( ".block" ).animate({ left: "+=100px" }, 2000 );
});

// Stop animation when button is clicked
$( "#stop" ).click(function() {
  $( ".block" ).stop();
});

// Start animation in the opposite direction
$( "#back" ).click(function() {
  $( ".block" ).animate({ left: "-=100px" }, 2000 );
});
<!doctype html>

  <meta charset="utf-8">
  <title>stop demo</title>

<button id="go">Go</button>
<button id="stop">STOP!</button>
<button id="back">Back</button>


Is the convex combination realizing the convex envelope unique?

Question: Let  $F:[0,\infty) \to [0,\infty)$ be a continuous function satisfying $F(1)=0$, which is strictly increasing on $[1,\infty)$, and strictly decreasing on $[0,1]$. Suppose also that $F|_{(1-\epsilon,1+\epsilon)}$ is convex for some $\epsilon>0$. Suppose that $F$ is not affine on any subinterval.
Let $\hat F(x) = \sup \{ h(x) \mid \text{$h$ is convex on $[0, \infty)$}, h \le F \} \, $ be the convex envelope of $F$. Let $c\in (0,1)$, and suppose that $\hat F(c) < F(c)$.

Question: Let $x,y \in [0,\infty)$ and $\lambda \in [0,1]$ satisfy $c = \lambda \, x + (1-\lambda)\, y$ and $\hat F(c) = \lambda \, F(x) + (1-\lambda) \, F(y)$. Are such $x,y$ unique?

(Here is an argument for the existence of such $x$ and $y$, under slightly different conditions).
We always have
$
\hat F(c) \le  \lambda \, \hat  F(x) + (1-\lambda) \, \hat F(y) \le \lambda \, F(x) + (1-\lambda) \, F(y),
$
so $\hat F(c) = \lambda \, F(x) + (1-\lambda) \, F(y)$ if and only if $\hat  F(x)=F(x), \hat  F(y)=F(y)$, and $\hat F$ is affine on  $[x,y]$.
Comment: No, they are not unique. The function can "bump" on, for example, the line $y+x=1$ multiple times
Answer: Take $c = 1/2$,
$$x_1=1/2 -a,\quad y_1=1/2 +a,\\ x_2=1/2 -b, \quad y_2=1/2 +b,$$
where $a<b<1/2$. You can find a function $F$ satisfying all the hypotheses such that $\hat F(c)=1-c< F(c)$, and
$$ \hat F(x) = F(x)=1-x,\quad x = x_1,x_2,y_1,y_2$$
Comment: Thanks, that's a very nice idea!


Performance is notoriously slow

Question: This feels like a repeat post but I can't find an actual answer. I just installed Drupal 8.4.4 on an Azure App Service (S3 Standard 4core 7GB RAM / PHP 7.0.6) and Azure Database for MySQL (Standard 100 computes / v22.214.171.124). I've been told the host isn't the issue.
Out of the box, the admin side is incredibly slow, like 4-5 seconds per click. I enabled page caching, and CSS/JS are aggregating. Dog slow. I worked through this and created my content types, added a couple of extensions, and enabled BigPipe (reported to help but didn't).
What am I missing? How do I improve performance before I ask my content authors to start loading up content?
Status report from Drupal. (I have since fixed the reported trusted host settings error)
Performance of the page in Chrome. Looks like 2.8s waiting on the page
I created a test.php file, and queried mysql directly, the key_value table, and dumped all of that to the page, about 576 rows, and it returned instantly. Appears to be Drupal specifically?
Devel bar on public page 182 DB queries
Devel bar on admin page 233 DB queries
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/76096/discussion-on-question-by-david-lozzi-performance-is-notoriously-slow).
Comment: Can you test the site in another environment?
Answer: It appears Azure's docker container services, which App Services leverages, is the cause of the problem.
Two big pains with Azure's app service (as explained by Azure support):
First, the docker containers are spread out across multiple drives, and PHP/Drupal's nature in using file caching causes it to run slower than it should.
Second, setting PHP settings to setup xdebug for instance, were lost when the service was restart. A "feature" in Azure, the container since it's a PHP image, everything outside of the home directory gets reset. This could be fixed by using an empty container, but then I'd probably still have the first issue.
My solution was to spin up a VM in Azure and manage it there. Runs MUCH faster, more reasonably now.
THANK YOU ALL
Answer: Odds are you have some code that is doing an external http request. If MySQL is fast, php isn't that slow; leaves something like an http request. If you don't know where to look, a cachegrind via xdebug is how to pinpoint the slow part.
Comment: using devel it shows no HTTP calls
Answer: One possibility is that Drupal caching will attempt to make use of APCu caching if it is available. Generally not a bad thing since it's fast.  However, sometime the host system has this available but it is set to a very small memory size.  This can cause problems.
You might try adding the following to your settings file to force Drupal to ignore this:
<code>  $settings['cache']['default'] = 'cache.backend.database';

  # Force common chainedfast bin to use database.
    $settings['cache']['bins']['discovery'] = 'cache.backend.database';
    $settings['cache']['bins']['bootstrap'] = 'cache.backend.database';
    $settings['cache']['bins']['render'] = 'cache.backend.database';
    $settings['cache']['bins']['data'] = 'cache.backend.database';
    $settings['cache']['bins']['config'] = 'cache.backend.database';
    $settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.database';
</code>
If you have memcache available, you can change the database part to memcache.
Another thought is to check your opcache size (in php setting via status report).  It might be too small and your PhP code is having to be parsed into op codes to run.  My servers are using 96MB for D8.
Finally, can you change your PhP version to 7.1?  I know that 8.5 has some requirements for it and will not install via composer with 7.0.
Comment: I added the settings, didn't seem to help. Looking at opcache size, I'm not sure what setting I should be looking at, `opcache.max_file_size` which is 0, and `opcache.file_cache` is set to no value...
Comment: This saved my bacon on a dev setup running Drupal 7 in a docker container - forcing DB caching took page requests from ~90 secs to <1.  Nice one!
Answer: There's some sort sub-optimal LAMP/Drupal configuration that's occurring here. As the case with most these types of problems, you run through the common checklist of Drupal slow stack performance problems (e.g. DB caching, app server caching, web server config, etc.) to see if you're hitting one of the common instances that can occur.
Past that, you need to deep dive into all performance profiling tools for your stack to find out where in the web/app/DB request pipeline the bottleneck is occuring. From comment groking (you last mention trying a fresh Drupal install), it sounds like this might be a new stack setup so you might want to consider a freshly provisioned stack & Drupal install on Azure following the practices recommended by MS so you can find a known-to-be-reliable setup on your hosting provider.
That's about the whole scope that you can address from the Drupal perspective since this is more a question about a host environment than it is about the CMS application itself.
Comment: sadly that Msft article is 4 years old, and doesnt match the current Azure landscape or Drupal version... but i pulled what i could out of it
Comment: I noticed as such, I took a quick look at what MS has on offer and that was the most authoritative source. Whatever the case, one would think that MS would have a default stack setup that would play nice with any LAMP-based app.

Performance is notoriously slow


In SQLite3, is there a way to not allow a duplicate field when another field is the same value?

Question: This is very hard to explain which is probably why I can't find an answer. For example, I have one table of "recipe", "ingredient". Ingredients would obviously be reused for different recipes so I can't use UNIQUE, but I want the command to refuse any attempts to insert into the database of a specific recipe where the ingredient already exists in that recipe.
For example, a record: ingredient = "apple", recipe = "applePie"; don't allow the insertion of another "apple" with the field recipe = "applePie" (in my actual application, other fields besides this may exist, they may be different on both records but they both have the same ingredient and recipe, so the second attempt to insert a value should be refused unless the ingredient does not exist).
Sorry if this is incredibly basic, I thought there must be a way to natively handle this without needing my program to select and compare within the code. I don't know how well I explained everything, please let me know if you need clarity.
Thank you!
Comment: "This is very hard to explain" -- Then a [example], which is required in every question here anyway, would certainly help a lot.
Comment: Sorry, someone managed to answer it anyway though!
Answer: In the table's CREATE statement define a UNIQUE constraint for the combination of the columns ingredient and recipe:
CREATE TABLE tablename (
  recipe TEXT,
  ingredient TEXT,
  UNIQUE(recipe, ingredient)
)

Now, if you try to insert a row with the same recipe and ingredient as an existing row, you will get an error and the new row will not be inserted.


Drupal 7 - Anyway to make Facebook Picture take place of User Picture if not available

Question: I am creating a view of users that displays a list of Users with Picture and Name. This drupal install allows facebook users to sign in and automatically have an account created. Their account is also added to this jCarousel view.
The problem I am facing is that when a user signs up from Facebook, but does not upload an image directly to drupal, their facebook picture does not show on the list. Anywhere else on the site, however, their facebook pictures shows with the profile.
On the views field, I have:

User:Picture
User:Name

Is there anyway to have a logic in the view where I can request the Facebook Picture if User:Picture is not available?
Thank you in advance,
Answer: You can use the Views PHP module and use PHP to create whatever logic you want for this image output.

This module allows the admin account (user 1) to add fields, filters
  and sorts to views which use PHP code. It also supports additional
  accounts when the core PHP Filter module is enabled.
This is the successor of Views Custom Field for both D6 and D7. The D6
  version supports both Views 2 and Views 3.
Note
While this module allows you to directly use PHP inside views
  which may be useful for quick and easy solutions, it is highly
  advisable to use regular handlers and plugins when available (or even
  to create one yourself). Take note that filtering and sorting a view
  using PHP always has a considerable perfomance impact.

So in your view you can have:

3 fields, user profile picture, facebook picture and the php field used to display a picture.
for the user profile picture, make it a default image display and exclude it from the output.
for the facebook picture make it a default image display and exclude it from render output.
in the php field you simply want to check if a file exists for user profile picture and then drupal_render() or return the profile or facebook picture Views will have already formatted for you using the other 2 fields.
be sure the 2 image fields are higher up in the field list of the View than the PHP field, so the PHP field can make use of those values in code.

EDIT
An alternative that is slightly less magical and tied to Views is to use Display Suite and add a Custom Field in display suite for a User. This custom field does basically all the logic between the 2 fields and it's literally a pseudo-field for Users. Views can then use the view_mode for Users that you've provided by DS.


Informing an ajax call that the session has timed-out or user needs to re-authenticate

Question: I have nailed down a challenge for me which was the ability to log and continue/Inform a user about server exceptions when the request is ajax. Using the OnException in the base controller and looking for the request being ajax I can return a customized "...uhoh" view to be rendered.
Now, I am battling the not authenticated portion of stuff. I am struggling with this modified code I was thinking of using in place of the [Authorize] attribute in my base classes.
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
    public class AjaxAuthorizeAttribute : AuthorizeAttribute
    {
        protected override void HandleUnauthorizedRequest(AuthorizationContext filterContext)
        {
            if (filterContext.HttpContext.Request.IsAjaxRequest())
            {
                filterContext.Result = new JsonResult{Data = new{NotAuthenticated=1}, JsonRequestBehavior = JsonRequestBehavior.AllowGet};
            }
            else
            {
                base.HandleUnauthorizedRequest(filterContext);
            }
        }
    }

Currently, there are a few areas where this could return NotAuthenticatedas data to the Ajax calls in the client. Am I going about this the right way? Just located the main flows that end in an Ajax.
if(returnData.NotAuthenticated){
    if (that.onNotAuthenticated != null)
       that.onNotAuthenticated();
    else
      _loadView(viewElement, "Request requires authorization. Please login again.");
} else
      _loadView(viewElement, returnData);
}

However, I am using the Kendo complete library and possibly others. Is there a better "Global" way to handle session timeouts or auth redirects when triggered via an Ajax call?
Please help!
Comment: One advice on asking your questions, keep it simple and short with only required information. I just read your last paragraph :)
Answer: I think Global Ajax Event Handlers meets your needs. For example following code sets a global error handler which will redirect user to login page if her session is timeout.
$(document).ajaxError(function (event, jqxhr, settings, exception) {
    if (jqxhr.status == 401 || jqxhr.statusText == "Unauthorized") {
        window.location.href = '/login';
    }
});

Comment: Thanks, do you by chance know off the top of your head if this replaces the local handlers or runs in conjunction before or after?
Comment: @lrb: You're welcome. AFAIK, It doesn't replace any _local handlers_, it runs in conjunction  with them. I have used it in many of our web apps.


Optimization problem, not sure how to proceed

Question: So I'm a bit confused by this optimization word problem.  I would be able to solve it I think given number values for the speeds but I'm uncertain how to get an exact answer when you don't know the difference between $v_1$ and $v_2$.  Maybe I'm just not sure how to approach the question.

You stand adjacent to point $A$ at the edge of a circular lake of
  radius $r$.  Want to get to $B$ diametrically opposite $A$. You can
  swim straight, run around the shore, or swim to a point $C$ somewhere
  in between and run the rest of the way to $B$.
Assume running speed $v_1$, swimming speed $v_2$, with $v_2 < v_1$.
Comment: Swimming straight across requires a time given by the diameter of the lake divided by the swimming speed.  Walking all the way will require a time given by half the circumference divided by the walking speed.  The question is whether it takes less than _either_ of these times to swim along the chord of a circle from A to a point on the shore , C , (the length of which you will need to find) divided by swimming speed _plus_ the time to walk the remaining distance along the circumference of the lake at walking speed.  You will need some geometry of circles to find the necessary lengths.
Answer: This is an example of what I regard as a "sneaky" optimization problem, because it turns out that the optimal choice is not given by the "critical point" in the interval.

We have the choices of: (1) swimming along the diameter $ \ AB \ $ ; (2) running along half the circumference of the circle from $ \ A \ $ to $ \ B \ $ ; or (3) choosing a point $ \ C \ $ on the shore to which to swim from $ \ A \ $ , and then to run along the shore from $ \ C \ $ to $ \ B \ $ .
BirdKiller1989 describes a good approach to setting up the analysis for finding the time required, starting from a point $ \ A \ $ to reach the "far side" of the circular lake of radius $ \ R \ $ at point $ \ B \ $ .  A convenient variable to use in our work on is the "central angle" $ \ \angle \ AOC \ $ , which has the measure $ \ \theta \ $ in the interval $ \ 0 \ \le \ \theta \ \le \ \pi \ $ . It will be useful to know the length of the chord $ \ AC \ $ of the circle, which we can calculate from, for instance, the Law of Cosines (or we could just look it up), as
$$ \ s \ = \ \sqrt{2R^2 \ - \ 2R^2 \ \cos \ \theta} \ = \ \sqrt{2} \ R \ \sqrt{1 \ - \  \cos \ \theta} \ \ . $$
We shall also want the measure of the other central angle, $ \ \angle \ COB \ $ , which is, in radians, $ \ \pi \ - \ \theta \ $ .  The distance run along the shore from $ \ C \ $ to  $ \ B \ $ is then $ \ d \ = \ R \ ( \ \pi \ - \ \theta \ ) \ $ .
We have two unspecified quantities still to deal with, which are the swimming speed $ \ v_2 \ $ and the running speed $ \ v_1 \ $ .  It will make comparisons easier if we write $ \ v_1 \ = \ k \cdot v_2 \ , \ k \ > \ 1 \ $ .
Choice (1) then requires a time of $ \ T_1 \ = \ \frac{2R}{v_2} \ $ , while choice (2) requires $ \ T_2 \ = \ \frac{\pi R}{v_1} \ = \ \frac{\pi R}{k v_2} \ $ .  We shall want to keep these results in mind.
Choice (3), the "mixed" solution of swimming and running, will need a total time of
$$ \ T_3 \ = \ \frac{s}{v_2} \ + \ \frac{d}{v_1} \ = \ \frac{\sqrt{2} \ R \ \sqrt{1 \ - \  \cos \ \theta}}{v_2} \ + \ \frac{R \ ( \ \pi \ - \ \theta \ )}{kv_2}   $$
$$ = \ \frac{R}{v_2} \left[ \ \sqrt{2} \  (1 \ - \  \cos \ \theta)^{1/2} \ + \ \frac{( \ \pi \ - \ \theta \ )}{k} \ \right] \ \  .   $$
It is this function that we shall wish to examine for critical points on the interval $ \ 0 \ < \ \theta \ < \ \pi \  $ , with the possibility that the result will depend in some important way on the value of $ \ k \ $ .  Upon differentiating this function with respect to $ \ \theta \ $ , we obtain
$$ \ \frac{d \ T_3}{d\theta} \ = \ \frac{R}{v_2} \left[ \ \sqrt{2} \ \cdot \ \frac{1}{2} \ (1 \ - \  \cos \ \theta)^{-1/2} \ \cdot \ \sin \ \theta \ - \ \frac{1}{k} \ \right] \ \ ; $$
we can make this a little easier to work with by applying a "conjugate factor" to write
$$ \frac{\sin \ \theta}{\sqrt{1 \ - \  \cos \ \theta}} \ \cdot \ \frac{\sqrt{1 \ + \  \cos \ \theta}}{\sqrt{1 \ + \  \cos \ \theta}} \ = \ \frac{\sin \ \theta \ \sqrt{1 \ + \  \cos \ \theta}}{\sqrt{1 \ - \  \cos^2  \theta}} \ = \ \sqrt{1 \ + \  \cos \ \theta} \ \ , $$
(which is equal to the original ratio on the interval $ \ 0 \ < \ \theta \ < \ \pi \  $ ) , leading to
$$ \ \frac{d \ T_3}{d\theta} \ = \ \frac{R}{v_2} \left[ \   \frac{\sqrt{2}}{2} \ (1 \ + \  \cos \ \theta)^{1/2} \  - \ \frac{1}{k} \ \right] \ \ . $$
Now, it would seem, we want to set this derivative equal to zero and solve for the optimal value of $ \ \theta \ $ .  However, we won't bother doing that:  as we discover by taking the next derivative (or by graphing $ \ T_3 \ $ for various values of $ \ k \ $ ) ,
$$ \ \frac{d^2 T_3}{d\theta^2} \ = \ \frac{R}{v_2}  \  \cdot \ \frac{\sqrt{2}}{2}  \ (1 \ + \  \cos \ \theta)^{-1/2} \ \cdot \ ( - \sin \ \theta ) \ = \ -\frac{R \ \sqrt{2}}{2v_2}  \  \frac{\sin \ \theta}{\sqrt{1 \ + \  \cos \ \theta}} \ \ . $$
Since both the numerator and the denominator in the trigonometric ratio are positive on the interval  $ \ 0 \ < \ \theta \ < \ \pi \  $ , the second derivative is negative everywhere on the interval, meaning that the critical point for the function $ \ T_3 (\theta) \ $ is a maximum.  Moreover, there is no dependence of the second derivative upon $ \ k \ $ ; this tells us then that choice (3) is never the proper choice for passing from $ \ A \ $ to $ \ B \ $ in the shortest possible time.
This is a situation where what we learn in first-semester calculus as the "Extreme Value Theorem" becomes useful:  a continuous function on a closed interval has an absolute maximum value and an absolute minimum value.  Since the critical point in the open interval is not minimal, that absolute minimum must lie at one of the endpoints, either $ \ \theta \ = \ \pi \ $ (all swimming) or $ \ \theta \ = \ 0 \ $ (all running).
So here is where we recall our times for choices (1) and (2) ; by comparing them, we have
$$ \frac{T_1}{T_2} \ = \ \frac{2R \ / \ v_2}{\pi R \ / \ k v_2}  \ = \ \frac{2k}{\pi} \ \ . $$
It takes equally long to run or swim all the way ( $ \ \frac{T_1}{T_2} \ = \ 1 \ $ ) for $ \ k \ = \ \frac{\pi}{2} \ \approx \ 1.57 $ ; if $ \ k \ < \ \frac{\pi}{2} \ $ , it will take less time to swim the diameter of the lake; otherwise, it's quicker to run along the semi-circle.  Since the "typical person" has a (sustained) running speed-to-swimming speed ratio of $ \ k \ > \ 4 \ $ , it would take less time for most people to run around the lake.
Answer: Since the lake is circular, we can use angles to determine the position of the person and the points. Let point A be at $0 ^{\circ}$ and point B be at $180 ^{\circ}$
Note that the segment between point A and point C in the circle is a chord, and its length can easily be found and proven as well.
I would approach this problem by creating a time function based on the angle between point A, center of the lake, and some point C at the edge of a lake. This function measures the total time for the person to reach from point A to the midpoint of the lake.
There would be two components to this function, the time it took for the person to swim and the time it took for the person to run. These two components are also based on the angle of the function. There is a relationship to these two components based on the angle: the longer the person swims, the shorter amount s/he has to run around the lake to the midpoint.
After you generate a time function that reflects the two components and how they're related, it's simply taking a matter of the function's derivative to find the absolute minima of the time.

The time it takes for the person to swim and get to shore at some point C would be something like $(r*2*sin(\theta/2))/v_2$. The numerator is a well known length of the chord between point A and C in the circle. Notice that if the person decided not to swim at all, then point C = point A and $\theta = 0$ and this component $= 0$ as well.

The time it takes for the person to then run from point C to point B would then be: $2*r *(\pi - \theta)/v_1$. Notice that if the person decided not to run at all, then point C = point B and $\theta = \pi$ and this  component $=0$ as well.

Therefore, the function that determines the time for the person to go from point A to point B, with the possibility of point C in between, would be something like $t(\theta) =  (2r*sin(\theta/2))/v_2 + (2r * (\pi - \theta)/v_1)$

I say "something like" because I didn't critique this equation much, but the idea behind this is correct.


Disable request logging to console in flask

Question: I tried everything in here: Disable console messages in Flask server
but the only thing I was able to do it disable the *starting server thing.
I found what seems like its functions in the werkzeug module in the Serving.py:
def log_request(self, code='-', size='-'):
    msg = self.requestline
    code = str(code)

    if termcolor:
        color = termcolor.colored

        if code[0] == '1':    # 1xx - Informational
            msg = color(msg, attrs=['bold'])
        elif code[0] == '2':    # 2xx - Success
            msg = color(msg, color='white')
        elif code == '304':   # 304 - Resource Not Modified
            msg = color(msg, color='cyan')
        elif code[0] == '3':  # 3xx - Redirection
            msg = color(msg, color='green')
        elif code == '404':   # 404 - Resource Not Found
            msg = color(msg, color='yellow')
        elif code[0] == '4':  # 4xx - Client Error
            msg = color(msg, color='red', attrs=['bold'])
        else:                 # 5xx, or any other response
            msg = color(msg, color='magenta', attrs=['bold'])

    self.log('info', '"%s" %s %s', msg, code, size)

def log(self, type, message, *args):
    _log(type, '%s - - [%s] %s\n' % (self.address_string(),
                                     self.log_date_time_string(),
                                     message % args))

But even when putting pass in all these functions instead of the code, it printed them to the console. I'm really clueless. Does anyone have a working solution for this?
Comment: Why do you need to do this?
Comment: I'm trying to monitor site activity using the console in real time, and it just spams it. Even if disabling it is stupid, why can't one disable it?
Comment: Well, if you were to host on a proper production server you wouldn't get any logs. It is, after all, a development server
Comment: _But even when putting pass [...] it printed them to the console_ How do you launch your flask app? Emptying those 2 func should totally disable the `127.0.0.1 - - [01/Dec/2018 00:00:00] "POST / HTTP/1.1" 200 -` messages (even if it's not a correct way to do so, you should disable the `logging.getLogger('werkzeug')` logger), but your WSGI server or a web-server still may be emitting those messages.
Comment: As mentioned in the original post, I have tried to simply disable the logger but with no success. How I launch the app:

    `app = Flask(__name__)
    mysql = MySQL(app)
    socketio = SocketIO(app)
    if __name__ == '__main__':
        socketio.run(app , debug=debug, host=host, port=port)`
Comment: You mentioned you got no success disabling the logger, can you share what you're still seeing after you disabled the logger?
Comment: 127.0.0.1 - - [01/Dec/2018 00:00:00] "POST / HTTP/1.1" 200 - messages
Answer: import flask.cli
flask.cli.show_server_banner = lambda *args: None

import logging
logging.getLogger("werkzeug").disabled = True


Solving natural log equations

Question: The equation $\ln(|y+1|)= x-2$ where you solve for $y$, I am just unsure of how the absolute value plays into this. I am assuming that I would convert to exponential form to get $|y+1|=e^{x-2}$ and then I am stuck. Sorry if this is an easy question and I'm just missing something obvious. :/
Comment: You're in the right track. Now that you've got to $|y+1| = e^{x-2}$, consider the two possible values of $|y+1|$ and solve for $y$ in each case. You'll end up with two possible solutions in the end. Hint: the definition of $|a|$: $|a| = a$ when $a \ge 0$ and $|a| = -a$ when $a < 0$, for any real number $a$.
Answer: Notice, $$\ln|y+1|=x-2$$ $$|y+1|=e^{x-2}$$ Since, $e^{x-2}>0 \ \forall\ x\in R$ $$ y+1=\pm e^{x-2}$$
$$\bbox[5px, border:2px solid #C0A000]{y=\pm e^{x-2}-1}$$
Comment: $y = -1 - e^{x-2}$ is also a solution.
Comment: Hah... better now. I'll take away my down vote now.
Comment: yes,, you are absolutely right
Comment: so the whole inequality thing is not needed?

Solving natural log equations


What is a "thin" svd?

Question: I've seen lots of mention of this in the implementation of SVDs in various programming environments.  What does it actually mean?
Comment: Is the definition given in [Wikipedia](http://en.wikipedia.org/wiki/Singular_value_decomposition#Reduced_SVDs) sufficient for your purposes?
Comment: I suppose it would be.
Answer: Let the SVD of an $m \times n$ matrix be $A=U \Sigma V^T$. Further, suppose it has rank $r$. Then, $A$ can be written as
$$ A = \sum_{i=1}^r \sigma_i u_i v_i^T + \sum_{i=r+1}^{\min(n,m)}0 \cdot u_i v_i^T$$.
The "thin" SVD is just the first part where the "fat" (?) SVD is the entire expression. In other words, the remaining parts can be discarded. Therefore, if we know the matrix is of rank $r$, we only need to find those $r$ terms.

What is a "thin" svd?


How to get the node from the url referer?

Question: I have a custom entity named Application that has an Entity Reference field named Product.
When I am on an Apllication page and click on a Product to see it, I would like to know that I come from the Application page and know its ID.
This would allow me to display information from the Application page on the Product page.
But how to do it ?
Comment: There might be a new module which can do that: https://www.drupal.org/project/referer_to_entity_reference
Comment: The information will need to be displayed only when one comes from application page or anyone who visits product will see the associated application info ?
Comment: Only when someone comes from the application page to the product page.
Comment: Did my answer help you? maybe you can consider checking it as accepted.
Answer: This method works for me:
<code>use Symfony\Component\HttpFoundation\Request;
use Drupal\Core\Url;
// Getting the referer.
$request = \Drupal::request();
$referer = $request->headers->get('referer');
// Getting the base url.
$base_url = Request::createFromGlobals()->getSchemeAndHttpHost();
// Getting the alias or the relative path.
$alias = substr($referer, strlen($base_url));
// Getting the node.
$params = Url::fromUri("internal:" . $alias)->getRouteParameters();
$entity_type = key($params);
$node = \Drupal::entityTypeManager()->getStorage($entity_type)->load($params[$entity_type]);

// To use dpm you need the devel module.
dpm($node->id());
dpm($node->title->value);
</code>
Answer: <code>$previousUrl = \Drupal::request()->server->get('HTTP_REFERER');
$fake_request = Request::create($previousUrl);
$url_object = \Drupal::service('path.validator')->getUrlIfValid($fake_request->getRequestUri());
$route_name = $url_object->getRouteName();
$url = Url::fromRoute($route_name, $url_object->getRouteParameters());
$previousPath = $url->getInternalPath();
</code>


Definition of a bundle associated to a character

Question: In page 31 of the book "Heat kernels and Dirac operators" we find the following paragraph

The bundle of densities $| \wedge|$ is very closely related to the bundle of volume forms $\wedge^n T^*M;$ the first corresponds to the character $| det(A)|^{-1}$ of $GL(n)$ and the second to the character $det(A)^{-1}$.

What is the definition of a bundle which associated to a character of $GL(n)$ ?
Answer: Let $P\to M$ be a principal bundle with structure group $G$.
To any vector space $V$ and representation $\rho \colon G \to GL(V)$, one can define the associated vector bundle $P\times_{\rho}V\to M$, which is a vector bundle over $M$ with fibers isomorphic to $V$.
It is defined as the quotient of $P\times V$ by the diagonal (right-)action of $G$ defined by $(p,v)\cdot g = (p\cdot g, \rho(g^{-1})v)$.
Now, consider the principal frame bundle $F(TM)\to M$.
It is a principal bundle with structure group $GL_n(\Bbb R)$.
Consider the two representations
$$
\rho \colon A \in GL_n(\Bbb R) \longmapsto \det A^{-1} \in \Bbb R^* = GL(\Bbb R),\\
|\rho| \colon A \in GL_n(\Bbb R) \longmapsto |\det A^{-1}| \in \Bbb R^* = GL(\Bbb R).
$$
As one dimensional representations, they are usually called characters.
Then one can show that
$$
\Lambda^n(T^*M) \simeq F(TM)\times_{\rho}\Bbb R \quad \text{and} \quad |\Lambda^n|(T^*M) \simeq F(TM)\times_{|\rho|}\Bbb R.
$$
This is what is meant by "they are the vector bundles associated with the characters $\rho$ and $|\rho|$".


SwiftUI - Generating Picker options from Multi-Level Struct (created from JSON)

Question: I'm struggling to populate Picker on SwiftUI from Multi-Level Struct that is generated from a JSON data source. The struct has the following format
struct DataStruct : Codable {
    let name: String
    let group: [groupData]
}

struct groupData: Codable, Hashable {
    let id, name : String
    let point : [pointData]

}

struct pointData : Codable, Hashable {
    let id : String
    let name : String
    let enumOptions: [String: String]
}

After loading the data from JSON, I get the following (simplified, real data has multiple Groups with multiple pointData groups, with each different Enum(erations).
DataStruct(name: "Top Level", group: [JSON.groupData(id: "1", name: "Group View", point: [JSON.pointData(id: "1", name: "Enumerated Point 1", enumOptions: ["1": "Enum2", "2": "Enum3", "0": "Enum1"])])])

Now I would like use the "Enumerated Point 1" data to create a picker for this data point. If this was 'hard-coded' this would be something like.
struct PickerView: View {
    @State var pickerSelection : String = "2"

    var body: some View {
        Picker("Enumerations", selection: $pickerSelection) {
            Text("Enum1").tag("0")
            Text("Enum2").tag("1")
            Text("Enum3").tag("2")
        }
    }
}

But I'm struggling to create Picker for 'Enumerated Point 1" as it is not an array but a dictionary? On the SwiftUI I have NavigationView on the top level with links to each "groups" of data (groupData), and then links to each "points" (pointData) that then have navigation link to the point.
The code snippet below illustrates how I drill in from top level to point level passing groupID and pointID as reference from one view to an other.
struct PickerView: View {
    @State var pickerSelection : String = "2"
    var groupID : String = "1" // Passed from Group Navigation Link
    var pointID : String = "1" // Passed from Points Navigation Link

    var body: some View {

        VStack{

            ForEach(dataStruct.group, id: \.self) { group in
                if group.id == groupID {

                    ForEach(group.point, id: \.self) { point in
                        if point.id == pointID {
                            Text(point.name) // prints "Enumerated Point 1"

                            // Now here I would like to create the Picker from enumerations
                            // *** CODE HERE ***
                        }

                    }
                }
            }

        }
    }
}

Now I have tried to create Picker Enumerations using a further ForEach loop but without luck. Any ideas ?
Comment: For the dictionary, you can create an array of values by `Array(enumOptions.values)`. Assuming the values are unique, you can then use them as your different choices. Using an enum in a picker is convenient if you know all of your choices at coding time, but if your choices are determined at runtime, they won't work.
Answer: The code below works. Each point element of the struct has array of pointEnumerations that are looped through using ForEach and then using Text("") the enumerations are created.
           ForEach(group.point, id: \.self) { point in
                if point.id == pointID {

                    VStack{
                        Text(point.name)
                        VStack {
                            Text("Selected option \(pickerSelection)")

                            Picker("Enum", selection: $pickerSelection) {
                                ForEach(0..<point.pointEnumerations.count, content : { index in
                                    Text(point.pointEnumerations[index])
                                })
                                    .pickerStyle(.wheel)
                                    .frame(width: 230)
                                    .padding(.leading, 35)
                            }
                        }
                    }
                }
            }


Task Scheduler Cannot Apply My Changes - Adding a User with Permissions

Question: When I login to the server using a domain account with administrator privileges and create a task in the Task Scheduler, I am unable to successfully save the Task once I select the following options: "Run whether the user is logged on or not" and "run with the highest priveleges" I am prompted to enter "Enter user account information for running this task." When I do, I get the following message:
Task Scheduler cannot apply your changes.
The user account is unknown, the password is incorrect,
or the account does not have permission to modify the task.

When I check Log on as Batch Job Properties (found this from the Help documentation):
This policy is accessible by opening the Control Panel,
Administrative Tools, and then Local Security Policy.
In the Local Security Policy window, click Local Policy,
User Rights Assignment, and then Logon as batch job.

Everything is grayed out, so I can't add a user. How can I add a user or get the task to save?
Answer: Sounds like the setting is under the control of a Group Policy. You'll need to add the account in an existing GPO that applies to the server, or if that's too general then you could create a new GPO that just applies to that server via security filtering.
Comment: I'm going through my GPOs and I cannot figure out which one could possibly be affecting the Task Scheduler. Any other suggestions?
Comment: Use RSOP.msc to tell you which policy is affecting those settings
Comment: Could I gain some insight on where exactly I should be looking within ROPS.msc? Thank you.
Comment: When you run RSOP.msc on the server that is affected, you'll get an interface that looks very much like the Group Policy editor. Drill down to Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> User Rights Assignment. In here you'll see a column at the top called 'source GPO'. Look there to see which Group Policy is setting the parts that you want to change.
Answer: I had the same issue.  In my case, it was because I had swapped to a new domain server and the user "guid" had changed.  I had to change the task to my username and enter my password, then go back in and change back to the other username that was originally erroring out.


IntelliJ IDEA, maven project does not automatically import dependencies

Question: I imported a maven project, but idea does not automatically import dependencies.
But I've checked the box for "Import Maven projects automatically".
I tried to delete .idea  and .iml and re-import the project.
About idea:
IntelliJ IDEA 2018.1.1 (Ultimate Edition)
Build #IU-181.4445.78, built on April 10, 2018
Licensed to **
Subscription is active until October 14, 2018
JRE: 1.8.0_152-release-1136-b27 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.13.4

The structure of my project in idea:

Update problem!
I found something wrong with my pom.xml,If I change into other pom.xml, I can import the dependency automatically. But I don't know where the problem is.
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.dai</groupId>
    <artifactId>micro</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <properties>
        <spring-boot.version>2.0.2.RELEASE</spring-boot.version>
        <commons-lang3.version>3.1</commons-lang3.version>
        <postgresql.version>9.4.1212</postgresql.version>
        <commons-codec.version>1.10</commons-codec.version>
        <httpclient.version>4.5.2</httpclient.version>
        <maven-compiler-plugin.version>3.1</maven-compiler-plugin.version>
        <java-jwt.version>3.2.0</java-jwt.version>
        <wsdl4j.version>1.6.3</wsdl4j.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.fasterxml.jackson.datatype</groupId>
                <artifactId>jackson-datatype-jsr310</artifactId>
                <version>2.9.0</version>
            </dependency>
            <dependency>
                <groupId>com.auth0</groupId>
                <artifactId>java-jwt</artifactId>
                <version>${java-jwt.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.commons</groupId>
                <artifactId>commons-lang3</artifactId>
                <version>${commons-lang3.version}</version>
            </dependency>
            <dependency>
                <groupId>org.postgresql</groupId>
                <artifactId>postgresql</artifactId>
                <version>${postgresql.version}</version>
                <classifier/>
            </dependency>
            <dependency>
                <groupId>commons-codec</groupId>
                <artifactId>commons-codec</artifactId>
                <version>${commons-codec.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.httpcomponents</groupId>
                <artifactId>httpclient</artifactId>
                <version>${httpclient.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-thymeleaf</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web-services</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>

            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <version>${spring-boot.version}</version>
                <scope>test</scope>
            </dependency>

            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-data-jpa</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-data-redis</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-aop</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-amqp</artifactId>
                <version>${spring-boot.version}</version>
            </dependency>
            <dependency>
                <groupId>wsdl4j</groupId>
                <artifactId>wsdl4j</artifactId>
                <version>${wsdl4j.version}</version>
            </dependency>

        </dependencies>

    </dependencyManagement>
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>${maven-compiler-plugin.version}</version>
                    <configuration>
                        <source>1.8</source>
                        <target>1.8</target>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <version>${spring-boot.version}</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>repackage</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>jaxb2-maven-plugin</artifactId>
                    <version>1.6</version>
                    <executions>
                        <execution>
                            <id>xjc</id>
                            <goals>
                                <goal>xjc</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <schemaDirectory>${project.basedir}/src/main/resources/</schemaDirectory>
                        <outputDirectory>${project.basedir}/src/main/java</outputDirectory>
                        <clearOutputDir>false</clearOutputDir>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build> </project>

Answer: First, make sure to use the latest Intellij Idea version. Current latest  version is 2018.1.4. You could use the Jetbrains Toolbox to update automatically.
Second, your pom.xml should be under ~/dev/micro-service and not ~/dev/micro-service/src - read Introduction to the Standard Directory Layout in order to understand how a maven project should look like.
Comment: My *pom.xml* is in the *~/dev/micro-service*
Comment: Sorry, It looks like it's under `~/dev/micro../src`. Did you try to update? What Maven version do you use?
Comment: I have solved it, because my POM file has some problems. If I change into other POM files, I can import the dependency automatically.
Answer: Your packaging define is pom, and you have just dependencyManagement not dependency section. These defines is used for parent pom. You may check maven for information about them.
To solve:
Change packaging to jar and remove the dependencyManagement while keep the dependency should let you import your dependencies.
Comment: Yes, that's the problem


Show upstream Cloudflare server IP after enabled nginx / set_real_ip_from

Question: I've followed cloudflare doc [1] that enabled set_real_ip_from in order to show original client's IP
My log format is
 log_format  main  '$remote_addr $http_cf_connecting_ip $http_x_forwarded_for
Assume client real IP is X, and cloudflare server is Y
Before enabling
the log is Y X X
After enabling
the log is X X X
Questions

Is this normal?
After enabled, the CF server IP info is lost, how to log them?

[1] https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs-Logging-visitor-IP-addresses-with-mod-cloudflare-
Answer: When you use the realip module to substitute the client IP address for the downstream proxy IP address (in this case Cloudflare), the original proxy IP address that contacted you is stored in the variable $realip_remote_addr and the remote port in $realip_remote_port. You can use these variables in your custom log_format.


SQLite code optimization

Question: I am a hobbyist programmer and writing a reporting tool to export values from an SQLite database to Excel.
The Excel part is written and working, the data I am retrieving from SQLite is creating a block in the program and taking several minutes to process.
I have rewritten the code out using generic values to help illustrate the processes. The initial populateList module is taking a negligible amount of time, but I have included it below as that is the providing the data for the doStuff module. The populateList currently retrieves approximately 500 distinct records.
I need the program to iterate through all the values retrieved by the populateList and do several counts. It is then populating another list valuesCount with the counted values.
I tried to improve the speed by looping through the list without closing the SQLite connection, but the improvement wasn't enough. Is there a more efficient way to retrieve this information from the databse?
public list<string>populateList()
{
List<string>values = new list<string>();
using (SQLiteConnection con = new SQLiteConnection(Passer.connstr))
{
    con.Open();
    string distinctValues = "SELECT DISTINCT \"value list\" FROM valueTable order by \"value list\" ";
    using (SQLiteCommand cmd = new SQLiteCommand(distinctValues, con))
    {
        SQLiteDataReader sqReader;
        sqReader = cmd.ExecuteReader();
        while (sqReader.Read())
        {
            values.Add(sqReader["value list"].ToString());
        }
    }
}
return values;
}

public void doStuff()
{
bool blanks = false;
string singleValue = string.Empty
string query = string.Empty;

List<string> getInitialValues = populateList();
list<string> valuesCount = new list<string>();
    using (SQLiteConnection con = new SQLiteConnection(Passer.connstr))
    {
        con.Open();
        for(int i = 0; i < getInitialValues.Count; i++)
        {
            blanks = false;
            singleValue = getInitialValues[i];
            if(singlevalue == "")
            {
                singleValue = \"\";
                blanks = true;
            }
            for (int x = 0; x < 6; x++)
            {
                string statement = string.Empty;
                switch(x)
                {
                    case 0:
                        statement = "SELECT COUNT(*) from valueTable where \"column1\" = ";
                        break;
                    case 1:
                        statement = "SELECT COUNT(*) from valueTable where \"column2\" = \"condition 1\" and \"column1\" = ";
                        break;
                    case 2:
                        statement = "SELECT COUNT(*) from valueTable where \"column3\" = \"condition 3\" and \"column1\" = ";
                        break;
                    case 3:
                        statement = "SELECT COUNT(*) from valueTable where \"column4\" = \"condition 4\" and \"column1\" = ";
                        break;
                    case 4:
                        statement = "SELECT COUNT(*) from valueTable where \"column5\" = \"condition 5\" and \"column1\" = ";
                        break;
                    case 5:
                        statement = "SELECT COUNT(*) from valueTable where \"column6\" = \"condition 6\" and \"column1\" = ";
                        break;
                }
                if (blanks == true)
                {
                    query = System.String.Format("{0}{1}", statement, singleValue);
                }
                else
                {
                    query = System.string.format("{0}\"{1}\"", statement, singleValue);
                }
                using (SQLiteCommand cmd = new SQLiteCommand(query, con))
                {
                    string countValues = cmd.ExecuteScalar().ToString();
                    valuesCount.Add(countValues);
                }
            }
        }
    }
}

Answer: Consider writing it as a single SQL query.
You are performing many queries when it very much looks like you simply need to perform a 'conditional count' on the columns. The SQL would be along the lines of
select
  val,
  col1 = sum(case when col1 = 'cond1' then 1 end)
from valtbl
group by val

you wouldn't even need the first method to get the list of distinct values.
Alternatively as the table seems reasonably small select what you need into a list of 'rows' and use Linq to Objects to do the counts.
Answer: You are querying the database multiple times to get the same information.
I would suggest not to do any db calls on dostuff method, rather use a single query for fetch the records form valuetable.
then you can do the count operations on the list itself.
For example get
"SELECT 'valueCol' , 'col1', 'col2' from valueTable"

would be your only query and you would store it in a list(say valueList).
Then on C# side you can use
 //not actual code just a sample idea
    var distinctValues = valueList.select(v => w.valueCol).Distinct()
    var count = 0;
    switch(case):
        case 0:
            count += valueList.where(v => v.col1 == condition).Count();
                break; //and so on...

Comment: Thank you SadiRubaiyet, i have changed the code around as you suggested and there is a significant performance boost.

SQLite code optimization


Playing a steady beat consistently on drums

Question: I've been self-learning to play drums over the last month and my metronome has been the most useful tool in helping me learn to play things that feel difficult to me. I managed to learn how to play consistently evenly spaced triplets by listening to how the metronome landed in relation to the triplets and making sure the 2nd beat lands evenly spaced between 2nd and 3rd beats (much better than the sloppy mess of rushing then dragging I used to do).
With the metronome, I've gotten pretty good at understanding how to fine-tune my speed if I start dragging or rushing. But I find it difficult to not drag or rush within 10 measures. I also find I can't keep a steady beat if I remove the metronome. Counting helps, but sometimes my foot pulls my counting with it (or I'm adjusting my counting without noticing to match my foot). I notice when I do lose the beat, I am tense or there's some kind of moment where I feel unsure of the beat (even though I pick it up again very quickly)
Currently I'm focusing on

Counting, playing a single note (e.g. kick), and keeping the metronome on at the same time
Keeping my counting in line with the metronome even if my foot goes off
Matching my foot with the counting, not the other way around
Using a consistent counting pattern (e.g. if I match the first count beat the measure number and cycle back to 1 after 4 measures, then I never count to 5 or cycle back after 3 measures)

Is this a good strategy? Is there something else I should keep in mind?
I've looked at some similar questions

Can't feel meter when playing - Describes practicing in a similar way to what I'm already doing, but for a slightly different problem (OP could not feel the start of measures)
What are important techniques to improve solid timing on drums - Answers here talk about establishing a personal playing style. I'm not concerned with my personal playing style, I'm concerned with learning how to play consistently
Keeping in time - Describes how practicing counting will eventually make things click, and how practicing trouble spots with the metronome can help make them easier.

I'm more interested in learning how to not momentarily lose the beat, and whether the strategy I've outlined will help me achieve that with enough practice.
Comment: I noticed with practice I got better at sort of resting in the beat. It's a different sensation than trying to hold the beat in place. It's somewhat like balancing on one foot: when you get super neurotic about whether you're balanced and keep on trying to adjust, you end up falling over. However when you rest still once you've found the balance point and do *nothing* then you end up staying there. I think this is one of the big problems I run into both in music and out of music. I worry a lot about whether I'm doing something exactly right and end up tripping myself up.
Comment: How's it going a year later?
Comment: I've also moved from drums to piano now that I have access to one at home, but the same principles apply there too.
Answer: I am not a drummer, but I think losing the beat is something every musician comes across at one point or another.
What helped me in this situation is to not only count the full beats, but counting the subdivisions too.
When I only count like 1 2 3 4 there is so much space between the beats and thats where I stumble out of the rythm.
But if I count the subdivisions like 1 and 2 and 3 and 4 and. The space becomes smaller. And if I go for 1 e and a 2 e and a 3 e and a 4 e and a its even better.
I think this is because when you start to gain speed on beat one your next reference point would be beat 2 where you will already be so much off the beat, that you can feel it which will make you feel unsure.
But if you count the subdivisions, your next reference point would be much earlier. At this point you will be not so much off the beat like wihtout subdivisions, which will allow you to adjust the rythm before you get totally off. This will happen unconsciously.
I dont know if the following makes sense, but another thing that helped me was not to think about the rythm as a static click which goes from 1 to 2 and so on like a clock, but more like a wave which is waving from 1 and goes up to the e the and and  the a to the 2 and than falling down again through e, and, a until it reaches its minimum at 2 and so on. With this image in my head, trying to feel the beat felt much more naturally and less mechanic to me.
Comment: For your last paragraph, try mentally picturing a pendulum. Although the speed it travels at, at any given length, varies from 0 to max as it swings!
Answer: Your context determines the advice - this is why it is important to work with a person (i.e. a teacher). Had you been playing for 10 years instead of 1 month, the advice would be completely different. (this is just a bit of philosophising :)
Since you are fairly early in your drumming "career", I would recommend something that you have yourself identified. Notice that you say that your foot kicks you out of time? You would benefit from organizing your practice into several sections. I.e. warmup, technique, rhythm, tunes, just having fun. Work through some regular and structured material (i.e. Stick Control) so that you go through all the combinations. This all seems like woodoo magic in the beginning but patient regular (and slow) practice brings many benefits over time.
What I want to say is that, in addition to working on Time, you will need to work on Technique. Practically, it is possible to do it at the same time, but I would recommend that initially you focus on ONE thing only. If you are doing technique, just focus on your limb movement (hands/feet) and don't worry about time. And vice versa. Over time (pun intended), you will be able to focus on time only if you can execute the technique effortlessly. For example, my left foot was making me drag the tempo while playing jazz, simply because I did not have enough control over it.
When you are (finally) practicing time, there are several techniques you can do. Basically, you should always have control over time, even if your technique is lacking, or your fill is shorter/longer, etc. You always need to be aware of the time. That's why metronome helps - it's like another musician you're playing with. But you still need to develop this internal sense of time. Playing with others will help you lock in with another and therefore with the metronome or your own internal clock.
Some do counting, or simply hearing the metronome clicks internally. Some do singing or just hearing the melody internally. Some play with music, it provides context. Then it is possible to play around and listen how different intervals sound within the musical context (i.e. playing triplets, 16-th notes, fills, etc.).
Having a metronome or another time source helps to build the internal clock. It will, at a minimum, serve as an indicator for what you need to work on.
There are various possibilities for using the metronome. I.e. it does not need to beep on the quarter note but can be "placed" on any of the 16-th (or triplet) notes within the quarter note. For example you could have the metronome beep on the last 16-th note within a quarter note. You should also increase the time between the beeps. Have the metronome beep on every second quarter note (1 and 3, or 2 and 4), then once per bar, once per 4 bars, etc. Try hitting the quarter notes on tempo 40 or less. As has been suggested, you can hear the subdivisions internally (i.e. hearing the 16- or 32-notes while playing only quarter notes in tempo 30).
The possibilities are almost limitless. Finally, here's something that really blew my mind. The way to "accept" the metronome, in a sense, and not fight it. It is a video series by Robert, the creator of the Bounce Metronome.


Does there exist a function in $f \in L^\infty (\mathbb{R})$ such that $\operatorname{dist} (f, BC(\mathbb{R})) > 0$?

Question: In the title $BC (\mathbb{R})$ refers to the bounded continuous functions on $\mathbb{R}$.
Clearly $f$ must be discontinuous, and I also believe that $f$ should at least be discontinuous on a set of positive measure. An idea I had which I couldn't flesh out was that if we use some sort of indicator function which varies between $0$ and $1$ on some sets of $\mathbb{R}$ such that the "best possible" approximation in the sense of the $L^\infty$ norm would be the constant function $g(x) = \frac{1}{2}$.
Any ideas?
Comment: $BC(\Bbb R)$ is a closed subspace of $L^\infty$.
Comment: @DavidC.Ullrich ...and since $BC (\mathbb{R})$ is a proper subset of $L^\infty (\mathbb{R})$ then there must exist some $f$ in the complement of $BC (\mathbb{R})$ such that $d(f, BC (\mathbb{R})) > 0$ since otherwise it would be in the closure of it, which would imply in it.
Comment: Have you tried to prove that such function does not exist?
Comment: No I have not, I will give it a try, but I strongly believe such a function exists.
Answer: For fun, a specific example: Let $f= \chi_{[0,\infty)}.$ Suppose $g\in BC$ and $\|f-g\|_\infty <1/2.$ Then $\sup_{[0,\infty)} |f-g| < 1/2.$ Therefore $g>1/2$ on $[0,\infty).$ Similarly, $g<1/2$ on $(-\infty,0).$ But no continuous function can satisfy both inequalities, by the intermediate value theorem. This is a contradiction, so we have $\|f-g\|_\infty \ge 1/2.$ Thus $d(f,BC) \ge 1/2.$ (And in fact $d(f,BC) = 1/2,$ since the constant fuction $1/2$ belongs to $BC,$ and $d(f,1/2) = 1/2.$)
Comment: The comment by David C. Ullrich reduced the problem to an elementary one of topology, but I will give you the answer for a nice construction! Thank you!


How to derive the growth rate rules from the log growth equation?

Question: These are approximations, which, I guess, only apply for small growth rates $g$ ($g$ is a function that denotes the change in its argument).
$$\begin{align}
g(xy) &= g(x) + g(y) \\
g(x/y) &= g(x) − g(y) \\
g(x^\alpha) &= \alpha g(x)
\end{align}$$
And there are also these rules
$$\dfrac{\Delta(x*y)}{xy} = \dfrac{\Delta x}x + \dfrac{\Delta y}y$$
and
$$\dfrac{\Delta(x^\alpha)}{x^\alpha} = z*\dfrac{\Delta x}x$$
I guess $g(x)$ equals $\dfrac{\Delta x}x$? So these rules are actually the same as the above ones.
However, I don't see how this can be derived from a log growth equation (which one?). I know a natural log function of a percentage change (e.g. $1.02$) approximates a percentage change that represents growth rates very close to zero (e.g. $0.02$), but ...
Thank you
babi
Comment: This document helped me a bit, but didn't answer my question in the end: http://lhendricks.org/econ520/growth/growth_algebra_sl.pdf
Answer: Let $ \dot x $ be the time derivative of $x(t)$.  That is $$ \dot x  = \frac{d}{dt} x(t) $$
We define the growth rate of $x$ as $$ g = \frac{\dot x}{  x(t)} $$
To see this "roughly" in a discrete example, let $x(t_0) = 100 $ and suppose at the next instant $x(t_1) = 110$.  Then g = .1 or 10%.
Lets try to compute your formula using our definition:
\begin{align} g(xy) &= \frac{ \dot {(xy)} }{xy} \\ &= \frac{ \dot x y + x \dot y } {xy}\\ & = \frac{\dot x y}{xy} + \frac{x \dot y}{xy} \\ &= \frac{\dot x}{x}  + \frac{\dot y}{y} \\ &= g(x) + g(y) \end{align} Where the second inequality came from applying the product rule to $\dot {(xy)}  $
Also note that by definiton $$ g = \dot {log (x)} = \frac{\dot x}{x} $$  That is to say our growth rate is just the time derivative of the logged variable.
We can use this definition to do another of your questions
$$ g(x^{\alpha}) =  \log {\dot { (x^{\alpha})}} = \frac{\alpha x^{\alpha - 1}\dot x}{x^{\alpha}} = \frac{\alpha \dot x}{x} = \alpha g(x)$$
Comment: Thanks a lot! Do these rules only apply for small growth rates or any growth rate? Like 250%? When you write log(x) with a dot, the dot applies to both the log and the x, right?
Answer: Not explicitly an answer to your question, but perhaps helpful.
I took a quick look at the link. I think it's trying to explain exponential growth without using calculus. The other questions you've asked suggest that you are learning calculus now. I think you'd be better off trying to understand this economics from scratch using calculus. If $f$ is growing at a constant rate (thinking of the independent variable as time $t$) the
$$
f(t) = f(0)e^{rt};
$$
the growth rate is $r$. Its units are $1/\text{time}$.
Suppose population is growing at $2\%$ per year you can use $r=0.02$ in that formula if that rate is interpreted as "continuous compounding".  If the $2\%$ is the increase in one year then in the formula you'd need
$$
r = \ln(1.02) = 0.0198
$$
which is (as you understand) not much less than $0.02$.
Then you could write the function as
$$
f(t) = f(0)e^{0.0198t}
$$
or
$$
f(t) =   f(0)(1.02)^t .
$$
The first is classier mathematics, the second captures the idea of $2\%$ annual growth" somewhat better.
Comment: Yep, you are right to link it with exponential growth, but indeed, it doesn't directly answer the question.


Error "column c.CreatedOn does not exist..." in PostgreSQL logs during code first context initialization using the Devart dotConnect provider

Question: I'm getting the following error in my PostgresSQL logs whenever my context is initialized:
<code>2014-06-03 09:51:25 PDT ERROR:  column c.CreatedOn does not exist at character 10
2014-06-03 09:51:25 PDT STATEMENT:  SELECT

    c."CreatedOn"

    FROM "__MigrationHistory" AS c

    LIMIT 1
</code>
Everything seems to work okay for the most part, but I'd like to better understand the error and make it go away if possible.
Is this an error with the dotConnect provider or am I expected to add this custom column to my __MigrationHistory table?
Answer: The "column c.CreatedOn does not exist" exception is expected for this scenario. This is the internal logic of Entity Framework.
The CreatedOn column is used only in EF 4.3 and below. However, the internal check for the table "__MigrationHistory" and, accordingly, for the "CreatedOn" column still exists and, consequently, in the later EF versions this gives an error. However, the implementation of this request does not stop the application and does not affect the result.
Comment: You might start seeing this if you *uncheck* the option Tools -> Options -> Debugging -> General -> Enable Just My Code.


How to convert a NameValueCollection to IEnumerable> FormValues?

Question: Can someone explain how I can convert the data in this NameValueCollection to be able to call the function below?
string ResponseHash = CalculateHash(Request.Form); // What to do here? Request.Form is a NameValueCollection

private string CalculateHash(IEnumerable<KeyValuePair<string, StringValues>> FormValues)
{

}

Comment: Do you know how to loop through a collection? Create instances of KeyValuePairs? Add those to some IEnumerable? Which part of that are you stuck on?
Answer: You could use such an extension method:
public static class NameValueCollectionExtensions{
    public static IEnumerable<KeyValuePair<string, StringValues>> AsEnumerable(this NameValueCollection query)
    {
        return query?.Cast<string>().Select((s, ix) => new KeyValuePair<string, StringValues>(s, query[ix]))
            ?? Enumerable.Empty<KeyValuePair<string, StringValues>>();
    }
}

Now you should be able to use it in this way:
string ResponseHash = CalculateHash(Request.Form.AsEnumerable());

Additional info: You can use Cast<string>  on NameValueCollection since it derives from NameObjectCollectionBase which in turn has a method GetEnumerator that returns the keys as objects(so you need the Cast). Cast<string> returns IEnumerable<string>, the keys.
Comment: @bmadtiger: added that information to my answer
Comment: Worked! Thank you so much!!
Comment: Helps to realise that `.Cast()` returns an `IEnumerable` populated with the keys
Answer: Something like:
var nvc = new NameValueCollection();
nvc.Add("key1", "value 1");
nvc.Add("key2", "value 2");
nvc.Add("key3", "value 3");
nvc.Add("key4", "value 4");

var list = nvc.AllKeys.Select(k => new KeyValuePair<string, string>(k, nvc[k]));

https://dotnetfiddle.net/QRbqmj
Might work


How do you test your Perl module to check if Makefile.PL declares all dependencies?

Question: I would like to write a t/00-check-deps.t module to find all dependencies in MyModule.pm and make sure they exist in Makefile.PL before release.
This way when I do make test before distributing to CPAN I will know that nothing was forgotten prior to publishing.  I've looked at the ExtUtils suite, but I've not seen anything obvious that already solves this.  It seems like a common issue people would want to solve.
How would you do this?
Comment: Tool/library recommendations are off-topic *ach[Module::ScanDeps](https://metacpan.org/pod/Module::ScanDeps)?oo*
Comment: @ikegami, I'm _not_ asking for a tool/library requirement, I'm asking _how it should be done_!  But thanks for the library hint :)
Comment: So you're saying you're expecting us to write said tool? Also off-topic (too broad). SO is not a code-writing service.
Comment: @ikegami, *sigh*, no not wanting you to write the tool... just looking for how it might be done.  Maybe its off topic, but it seems like there is somewhere in the grey area that might be on-topic.  shrug.
Comment: Well, it has to be one or other, or a mix of both. You either need to use a pre-existing solution or create a new one. There's no other way. And both are off-topic for the reasons explained
Answer: Here is how I would do it.  Thanks @ikegami for the scandeps hint:
find lib -name '*.pm' | xargs scandeps.pl -R | \
  perl -MJSON -le '
    undef $/;
    %d=eval(<STDIN>);
    $j=JSON::from_json(`cat MYMETA.json`);
    foreach (keys(%d)) {
      warn "Missing: $_ => $d{$_}\n" if !defined($j->{prereqs}{runtime}{requires}{$_})
    }
    '

prints:
Missing: Carp => 1.42
Missing: PDL::Constants => 0.02
Missing: Exporter => 5.72
Missing: constant => 1.33
Missing: PDL => 2.080
Missing: PDL::LinearAlgebra => 0.35
Missing: PDL::Ops => undef

Comment: Not quite.  Consider the situation if you have `Foo => 1.0` and there's a dependency on `Foo => 2.0`


What are different time-series types?

Question: I am having a time-series classification problem. I tried to using movingaverage and firstdifferences as two models to perform my classification. However, the results I am getting are pretty bad.
Therefore, I am thinking whether there are any other types of time-series data that I can use instead (i.e. instead of movingaverageandfirstdifferences`). If so please suggest me.
I am happy to provide more details if needed.
Comment: I do not think these are classification models. How did you classify time series with them?
Comment: @RichardHardy I use LSTMs to classify my timeseries :)
Comment: Why do you need moving averages or first differences then? Or are they a part of LSTM and you are wondering about the specification of LSTM? (Probably a stupid question since I am not all that familiar with LSTM.)
Answer: Both of the "models" that you listed are particular types of ARIMA models where one series exists and one wishes to use the history of the series to generate a forecast. Examples of this are here Forecasting/predicting total sum of donations (following GLM with poisson family and log link) and here Help me about using ARIMA forecasting rainfall  .
If you wish you can post one of your data sets (must be equally spaced in time) and I will try and help further.  The methodology underlying model identification is briefly introduced here. How to interpret PACF?    .

What are different time-series types?


How do I get pinch harmonics during solos?

Question: Could someone tell me how to get the pinch harmonics sound during solos? Specifically on the "Wanted Dead or Alive" solo by Bon Jovi?
Is it an amp setting? Is it something I do with my left or right hand? Is it possible with a cheap practice amp and no effects  pedal?
Comment: Maybe a bit of a heavy handed edit, but I made your question a little more general -- about how to get feedback during solos as opposed just one specific song.
Comment: Sorry, I'm kinda new here. Hope I didn't mess up too badly... :P
Comment: The question is about feedback and you accept an answer about...pinch harmonics? The question and the answer now no longer match.
Comment: @Ian, Samuel clearly used the wrong name for the sound he was hearing. Your edit to the question made the question more clear, but seems to have obscured the original intent of the question and caused the mismatch between Q&A to be bigger than is really the case. I would suggest an edit to the effect of, "I think it's feedback, but I could be wrong. What's the technique called if it's something else?"
Comment: Incidentally, that's the most common complaint about editing a question. What if you change the question so that it is "better" but it's not longer the question the OP was really asking? Seems to be what's happened here based on the accepted answer.
Comment: @yossarin: it was impossible to tell that until *after* he accepted an answer that was outside the scope of the question he asked. I think, in this case, a post-acceptance edit is what needs to happen because someone searching for help getting feedback will find this, and not see an answer that's relevant as the accepted answer. Also: my edit didn't change the question, just removed the "Wanted Dead or Alive" from the title. It still stands that he asked a question and accepted an answer that has nothing to do with the question.
Comment: @yossarin: [lets take this to meta and discuss what to do there](http://meta.guitars.stackexchange.com/questions/80/when-the-accepted-answer-has-nothing-to-do-with-the-question-asked).
Comment: This Was A Bad Edit - The guy was talking about shirts and you changed it to a pants questions...
Answer: I think you're talking about pinch harmonics, there are a quite few in that solo. The way it was explained to me was that you touch the string with the thumb of your picking hand right after the pick hits the string. It's easier on string bends and on the neck pickup.
Here's a good lesson: http://www.youtube.com/watch?v=5I5O8P-r5Rk
Comment: There's also a good question on the technique on the site already: http://music.stackexchange.com/questions/1394/how-to-do-metal-harmonics-pinch-etc
Answer: Feedback starts with volume. Lots and lots of volume. You need to create a physical loop between your guitar and your amplifier: sound goes out of guitar, in to amp, out of speakers and the vibrating waves vibrate your guitar and it's strings which in turn goes back in to your amp and out of the speakers and...well...you get the point.
So start by turning that amp up.
Controlling the feedback is the next thing and the rub is: how you control it to produce a specific note is going to depend on the guitar and amp and the environment. It can change. But there are some basic techniques you can employ in every scenario. First, you'll want to mute all but one string. You can use a combination of right-hand palm mutes and left-hand string mutes to achieve this. But you want to make sure that just one string can vibrate freely. That'll let you home in on that long, singing, held note you're after.
Next you need to find the right string and fret position where good resonance starts to happen. This is going to differ for every guitar. I'd start by trying to play the note around the 9th fret area of the neck, and on one of the strings in the A-D-G-B range rather than the low E or high E string. With all the other strings muted, pick the note (lets say an E on 9th fret, A string) and hold it. Try pointing your guitar so the pickups are aimed right at that loud amp, and then move the guitar around until you can feel the body vibrate, stronger and stronger. That's the feedback loop amplifying as it moves through the closed system and the you get constructive interference in the vibrations in the body and the string.
It's going to take some practice, but eventually you should be able to pick any note and find the right way to angle your guitar at your speakers to have the feedback for that specific frequency build nicely.
I recommend hearing protection and sympathetic neighbours.
Comment: You don't need lots of volume or a high sound pressure level. Set the amp to a reasonable volume where the cabinet is shaking a little as you're playing, hit a chord, maybe an open A, and rest the headstock against the cabinet. The physical connection between the speaker shaking the cabinet, and the headstock shaking the neck will go a long way toward holding the notes. Usually one of the notes will start to feed back. Increase the volume a bit if it doesn't. Once it consistently holds a nice rumble you can start messing with other notes and learning to control it.
Comment: @Samuel: so you're going to have a hard time achieving this if you're not using speakers -- if you're just running your guitar through headphones. You need to be "moving air" with speakers, lots of air, to make feedback happen. But do use the headphones to lower the SPL of that blaring amplifier!
Comment: @Samuel Andrew SPL = [Sound Pressure Level](http://en.m.wikipedia.org/wiki/Sound_pressure).
Comment: Thanks, I'll try and work on that. And I've got earphones, so HOPEFULLY I won't blow out my eardrums...
Comment: Right, I thought about that right after I posted. And, sorry to sound stupid, but what do you mean by SPL?
Comment: @the Tin Man good call with resting the headstock on your cab.
Comment: @Ian C. "good call...", well, it's just something we used to do, and I've seen lots of pros do it. Even with a Deluxe-sized amp you can have fun that way if it's loud enough to be shaking. You can do it with a floor monitor too. :-)
Answer: Pinch Harmonics - it is a tricky picking method that involves  using the side of your pick, an extra subtle twist as you are picking, all coupled with a little more force behind your attack.  There is no getting around the fact that you will need to try different angles as you are picking in order to see where the string reacts to the angle of attack, as well as moving the area of picking horizontally on the string itself (closer/further away from the bridge).
Angle of Attack
Slightly heavier attack at the moment of impact between pick and string
Sweet spot on string measured from the bridge
Slight bending or aggressive vibrato while picking
I would listen to "Tush" - the second solo, and try to emulate the finesse between how a picking attack and left hand string bending work in unison to produce the little squeals of delight.
Answer: Enygma is absolutely right on this! there is no feedback whatsoever on that solo! But it is packed with pinch harmonics! You can play these even on clean settings (even though they sound better with gain). The video however explains it pretty well...(Would have commented if I could...). One good advice though, To do these you should hold your pick so only the really end of the tip is visible. And make sure you pick at the right spot(Needs to be a harmonic there)
Answer: I heard of the technique as "squealers". They're a big part of the Roy Buchanan style, but also a metal thing.
In essence, you're hitting a harmonic, like you would at the 5th or 7th fret, except you're using your picking thumb as the node instead of your fretting fingers. This does mean you'll have to use a thumb-and-index-finger pick-holding technique instead of the three-finger technique. Sorry about that. Hold the pick so very little of the pick is sticking out.
You can do it dead clean, or even with an acoustic, but the issue with harmonics is that the further you get from the fundamental, the less energy the harmonic gets, so you need to do things like string bending and using distortion or overdrive to get the note to stand out more.
This is very dependent on where you're hitting the harmonic, and the sweet spot is very dependent on what your fretting hand is doing, but in general, you should be able to get two harmonics that really go, one being closer to the bridge and the other closer to the neck pickup, but there should be some harmonic most anywhere above the pickguard.