C++ ftell Segmentation Fault

Question: I am trying to read data from a binary file. I have been given a struct with all of the variables to extract the data. I am using the fread function, and I need to get the total size of the file as one of freads inputs. I am using the ftell function to find the size, but I get a segmentation fault when I call the ftell function.
Here is my code:
#include<iostream>
#include<fstream>
#include<stdio.h>
using namespace std;

struct header{
  int so_on;
  int so_forth;
  int etc;
};

int main(){
  FILE *fin;
  header cvvHeader;
  fin=fopen("file.binaryextension","r");
  int headersize= sizeof(header);
  header* ptr=&cvvHeader;
  long filesize =ftell (fin); //Segmentation fault happens here
  fread(ptr,headersize,filesize,fin);
}

Thanks for your help
Comment: I can´t see something wrong with ftell, but some other errors: Open with "rb" instead of "r"; parameter 2+3 of fread make no sense, fclose is missing, endianess and struct order/packing are not considered. @ftell-thing: Is this exactly the (a) code which crashes or some minimized example?
Comment: check if (fin != null) first?
Comment: besides the problems already mentioned: what do you expect to return ftell() on a freshly opened file?
Answer: Your following code is creating problem, You need to check whether fin is valid or NULL.
fin=fopen("file.binaryextension","r");
//....code
long filesize =ftell (fin); //Segmentation fault happens here

// fin should be non-null pointer when file is successfully opened.
if(fin) {
  long filesize =ftell (fin);
  }

Comment: Found my problem. fin was null because I had the file misspelled. Thanks!
Answer: Several problems:

In order to open file file.binaryextension for reading, you need to have it in the correct path:
If you are running your program directly (e.g., by clicking it, or by calling it from a command-line), then it should be in the same path as the executable file.
If you are running your program from Visual Studio, then the path is usually where the vcproj file is located. Alternatively, you can change the path itself within the project settings.
In order to obtain the size of the file:
Call fseek(fin,0,SEEK_END).
Call long filesize = ftell(fin).
Call fseek(fin,0,SEEK_SET).
In order to read the header data from the file:
Call fread(ptr,headersize,1,fin).
In order to read the remaining data from the file:
Call char* data = (char*)malloc(filesize-headersize).
Call fread(data,filesize-headersize,1,fin).
Call free(data) when you finish using it.

C++ ftell Segmentation Fault


My blank bitmap does not appear transparent on Android phone, but does on Android emulator

Question: On the emulator, newBitmap (see below) is transparent. I update newBitmap with data from another bitmap. Updated area appears whenever I call to draw newBitmap.
On the phone, I see a black screen. I set the alpha of newBitmap to 0 and it showed defaultbg. How can I achieve the same result that I get on the emulator?
background and background2 are bitmaps:
defaultbg = background;
newBitmap = Bitmap.createBitmap(background2.getWidth(), background2.getHeight(), background2.getConfig());

I update newBitmap like this:
int [] pixels = new int[width * height];
background2.getPixels(pixels, 0, width, cell.getX(), cell.getY(), width, height);
newBitmap.setPixels(pixels, 0, width, cell.getX(), cell.getY(), width, height);

I draw them like this:
canvas.drawBitmap(defaultbg, offset, 0, null);
canvas.drawBitmap(newBitmap, offset, 0, null);

Comment: Can you check background2.getPixel(x,y), to see it contains correct alpha value for certain location, and similarly newBitmap.getPixel(x,y) for same location after setting its content from background2. On phone that is, to make sure alpha values are handled properly.
Answer: create xml file in drawable with this source
< ?xml version="1.0" encoding="utf-8"?>
< shape xmlns:android="http://schemas.android.com/apk/res/android">
<solid android:color="#00000000" />

< /shape>
then give xml file name in your layout or wherever
android:background="@drawable/kbb_border"


Add HTML Canvas Offset Text Outline

Question: How can I add an offset outline to a text in an HTML canvas like in this example:

here's what I got so far:
var canvas = /** @type {HTMLCanvasElement} */ (
    document.getElementById("myCanvas")
);
var ctx = canvas.getContext("2d");

// Function to draw text with an offset outline
function drawTextWithOutline(
    text,
    x,
    y,
    textColor,
    outlineColor,
    outlineWidth
) {
    // Draw the outline
    const font = "80px Dancing Script";
    ctx.font = font;
    ctx.strokeStyle = outlineColor;
    ctx.lineWidth = outlineWidth;
    ctx.lineJoin = "round";
    ctx.strokeText(text, x, y);

    // Draw the text
    ctx.fillStyle = textColor;
    ctx.fillText(text, x, y);
}

// Call the function with desired parameters
drawTextWithOutline("Hello, World!", 50, 100, "red", "blue", 20);

how do i remove the fill and retain only the outline like in the example?
Comment: Draw the outline twice. One smaller than the other (less line width). For the smaller one, draw it white or whatever color to hollow it out (to match the background, if desired). Then draw the text as you have in red.
Answer: So basically what I did is when you draw the outline, draw another outline that matches the background color and has a smaller outline width.

var canvas = /** @type {HTMLCanvasElement} */ (
    document.getElementById("myCanvas")
);
var ctx = canvas.getContext("2d");

// Function to draw text with an offset outline
function drawTextWithOutline(
    text,
    x,
    y,
    textColor,
    outlineColor,
    outlineWidth
) {
    // Draw the outline
    const font = "40px Dancing Script";
    ctx.font = font;
    ctx.strokeStyle = outlineColor;
    ctx.lineWidth = outlineWidth;
    ctx.lineJoin = "round";
    ctx.strokeText(text, x, y);
    ctx.lineWidth = outlineWidth - 5;
    ctx.strokeStyle = 'white';
    ctx.strokeText(text, x, y);

    // Draw the text
    ctx.fillStyle = textColor;
    ctx.fillText(text, x, y);
}

// Call the function with desired parameters
drawTextWithOutline("Hello, World!", 50, 100, "red", "blue",20);
canvas{
  width: 100%;
  height: 100%;
}
<canvas id = 'myCanvas'></canvas>
Comment: OP said s/he tried this already and was unsatisfied with the result. (for some reason the comment was deleted). It's exactly what I suggested he do. Yours looks pretty good! ... But OP probably wants it to be exactly as the source.

Add HTML Canvas Offset Text Outline


Need help with integral related to Mellin transform

Question: I need help solving the following integral:
$$I  = \frac{1}{2 \pi i} \int_{c-i\infty}^{c+i\infty} \mathrm{d}p \hspace{2pt} m^{d-2p} \Gamma(-p)\Gamma(p-\frac{5}{2})A(p)$$
where $A(p)$ is an analytic function of $p$ everywhere. In an old thread there was a reference to the book "Assymptotics and Mellin-Barnes Integrals." On page 96 of that reference (in section 3.3.4 - Gamma Function Integrals) there is a potentially helpful formula:
\begin{equation}\frac{1}{2 \pi i} \int_{c-i\infty}^{c+i\infty} \mathrm{d}s \hspace{2pt} \Gamma(s+a)\Gamma(b-s)z^{-s} = z^a(1+z)^{-a-b}\Gamma(a+b) \hspace{1in} (1)\end{equation}
which is an application of Parseval's formula
$$\int_0^\infty f(x)g(x)\mathrm{d}x =  \frac{1}{2 \pi i} \int_{c-i\infty}^{c+i\infty} F(1-s)G(s)ds$$
where F and G are the Mellin transforms. My $m^2$ translates to $z$ nicely. However, $(1)$ is only valid for $Re(a+b)>0$, so $b=0$, $a=-5/2$ wont work. Also, it doesn't say how do to deal with an extra function ($A(p)$), although I think I could solve that with some extra use of Parseval's formula (considering the product of Gamma's as one function, I'd just need a function that gives two Gamma's as its Mellin transform). Any ideas?
Answer: The condition $Re(a+b)$ for equation (1) ensures the convergence of the integral if it is performed along a line with $-a<c<b$. To get the value of the integral with $a=-5/2$, it is necessary to follow another integration path that goes around some poles of the gamma functions. The path you have to select depends on $A$. For instance with $A(p)=1$, you will find in the table of inverse Mellin transforms given in Erdélyi, Magnus, Oberhettinger, Tricomi, Table of integrals transforms, the equation that you need, number 7.3(16) (chapter 7). For $0<c<1/2$, the formula should be used with $\alpha=-5/2$, $\beta=0$, $h=3$ and $k=1$ and the inverse Mellin transform provides the identity:
$$ \frac1{2\pi\mathrm{i}} \int_{c-\mathrm{i}\infty}^{c+\mathrm{i}\infty}
   \Gamma(-s)\Gamma\left(s-\frac 52\right)z^{-s}\,\mathrm{d}s=\\
   \Gamma\left(-\frac52\right)z^{-5/2}(1+z)^{5/2}-\Gamma\left(-\frac52\right)z^{-5/2}+\Gamma\left(-\frac32\right)z^{-3/2}-\frac12\Gamma\left(-\frac12\right)z^{-1/2}-\Gamma\left(-\frac52\right).$$
The integers $h$ and $k$ have to be chosen such that $0<a+h+c<1$, $0<b+k-c<1$. This because the poles of the gamma functions are the negative integers: $h$ and $k$ count how many poles you have to go around for $\Gamma(s-5/2)$ and $\Gamma(-s)$ respectively.
So to compute $I$ in the case $A(p)\neq 1$, you need to know which domain you want to use, it will give you the poles you have to take into account.


Asymptotic expansion about branch point

Question: If we have an analytic function but which has a branch point (yes technically it's only analytic in open sets which are disjoint with a branch cut but that's besides the point) is there a way to expand the function around the branch point? In particular there's a function that comes up in thermal field theory, which is the one-loop thermal contribution to the effective potential (if anyone knows any QFT)
$f(y) = \int_0^\infty x^2\log(1-e^{-\sqrt{x^2+y^2}})=-\frac{\pi^2}{90}+\frac{y^2}{24}-\frac{y^3}{12\pi}-cy^4(\log(y)+d)+O(y^6)$
where c,d are some constants. A derivation of the above expansion for small $y$ can be found in a paper by Dolan and Jackiw called Symmetry Behaviour at Finite temperature.
They use techniques from dimensional regularisation but I was wonder if there was any formalism in complex analysis which allowed one to expand sufficiently nice complex functions around a branch point in terms of elementary ones.
Comment: I don't understand the above expansion.  When $y=0$, the integral should look like $$\int_0^{\infty} dx \, x^2 e^{-x} = 2!$$  How do you have $-\pi^2/90$ as the leading term?
Comment: @RonGordon Sorry I mistyped the integral. It is now correct.
Answer: This is not a full answer but I've discovered that there is a type of series called a Puiseux series which allows functions with branch points in the series. Most sources state that a Puiseux series a power series where each term in the series looks like $x^i$ where $i$ can be a rational number whose denominator is bounded by some integer $n$; this basically means $i=k/n$ for some fixed $n$ and any integer $k$. However, WolframMathworld also allowed logarithms in the expansion which may be multiply nested, e.g. $\ln\ln x$. This seems to be contrary to most other sources however.
I've not discovered so far how to calculate the Puiseux series. If one precludes logarithms from being in the series, if $f(x)$ admits a Puiseux expansion at zero then $f(x^n)$ will be analytic (since $n$ bounds the denominators of the exponents). However they seem to have more relevance in polynomial ring theory and algebraic geometry than complex analysis.

Asymptotic expansion about branch point


Probability proportional to size

Question: Consider the problem of drawing a sample of size $2$ from a finite population of size $20$ . The sampling is done with replacement using probability proportional to size scheme  . The normed size measures $p_1 , p_2 ,  \ldots, p_{20}$ . are given by
$ p_i = \frac{1}{40}  ,  i  =1 ,\ldots,10  $
$p_i = \frac{3}{40} ,  i  =11 ,\ldots,20                      $
   , find the expected no. of distinct units drawn
attempt  :
let $X$ denote the no. of distinct units drawn then $X$ denotes the value
$0 , 1 , 2$ .
Now how to proceed to caculate probabilities , please provide some approach for this question or any alternative suggestion
Answer: Guide:
There are only two possibility, either they are identical or they are not.
For the probability of being identical, compute$$\sum_{i=1}^{20}P(X_2=i|X_1=i)P(X_1=i)=\sum_{i=1}^{20}P(X_2=i)P(X_1=i)$$
You might like to consider the sum for $i$ from $1$ to $10$ and the sum from $11$ to $20$ separately.
Comment: If you can compute the probability, you can get the expected value right?
Comment: we need expected no. of distinct units , not identical ,

Probability proportional to size


JQuery attr() is not function, how to turn it in an object?

Question: I have an error with attr() in jQuery - it is not a function. I understand the problem, my element is not in the DOM and I should turn it into an object. However I don't see how to do it.

// Works
$("#progress_bar").text(iTempsMax);

// Doesn't works
$("#progress_bar").attr("aria-valuenow").val(iTempsMax);

How could I turn into object for the last JS line?
Answer:
I understand the problem, my element is not in the DOM and I should turn it into an object

This is not the problem.
The issue is because attr() returns a string, not a jQuery object, so you can't chain val() off it. You need to separate the calls to the same jQuery object.
let $el = $("#progress_bar");
let valuenow = $el.attr("aria-valuenow");
$el.val(iTempsMax); // set the value

Alternatively, set the val() first, then get the attribute, although this is potentially obfuscates the val() call.
let valuenow = $("#progress_bar").val(iTempsMax).attr("aria-valuenow");

Using this pattern is a matter of personal preference over code clarity or brevity:
On the other hand if you are attempting to only set the value of the aria-valuenow attribute (and not the value attribute of the element itself) then you need to omit the val() call and pass the second argument to attr(), like this:
$("#progress_bar").attr("aria-valuenow", iTempsMax);

Comment: @Wimanicesir `val()` and `attr(key, value)` are not the same thing
Comment: I know, look in comments of my solution. I just suppose he wants to change the value of his attribute and tried it this way.
Answer: Are you trying to change the value of your attribute? The attr() function accepts a second argument to set this:
$("#progress_bar").attr("aria-valuenow", iTempsMax)

https://api.jquery.com/attr/
Comment: No, I mean he wants to set the value of that attribute to the given variable
Comment: `aria-valuenow` is not the same attribute as `value`.
Comment: Ah, I see what you mean.


Use induction to solve the recurrence relation..!!

Question: I don't know how to start this problem. I just need someone to show me the first couple steps
of doing the induction for this relation. c is a constant.
$T(n) = T(n - 4) + c\cdot n^{1/4}$
Thanks for the help..!!
Answer: It looks like this splits into 4 completely independent sequences. Each of them only depends on its own initialization (first term).
In this sense, this is a recurrence realtion of first order.
For induction, you first need a proposed solution, otherwise you don't have anything to prove. The solution for each subsequence is simply the sum $T(4n+m)=T(m)+c\sum_{k=0}^n (4k+m)^{1/4}$ where $m=0,1,2,4$ is the subsequence you are observing.
The sum itself doesn't look like something you could simplify.
Answer: It would appear that $$T(4n)=c\sum_{k=0}^{{n}}{(4k)^{{1\over 4}}}$$ for $n$ divisible by 4, and similarly for other types of $n$ (there are four cases to consider). Assume this formula is true for some n, and show (using the recurrence formula) that it will also be true for the next $n+1$. There isn't really a base case to worry about, as you didn't specify and initial conditions.


Stop Android for killing my Activity after 30 minutes from background

Question: I am creating an application where there is an activity which contains a timer running when ever user comes to that screen. If user leaves that screen by pressing home button without stopping the timer then the user is notified by the notification and then if the user again come back to the app from any means then the same screen should appear.
It is working fine if the user comes back to the screen before 30 minutes. Because after 30 minutes android kills my activity from background and my application starts from beginning.
I want to stop android for killing my activity from background. Please help if any body has any idea...
Thanks.
How can i link an Activity to a service? So that when Android OS tries to destroy my activity then it won't let it do because my activity is linked with the running service.
Comment: Can you use a Service, Broadcast Receiver, or just pass session data to the app via the Notications intent?
Comment: Are you saying that restart activity with the data saved in notification for the recreation?
Comment: Yes, if you are just showing some text/numbers, you can store them in the intent (please note, afaik, you're limited to about 1MB of data).
Answer: Use Services to doing background work. In this case Activity should be like a simple UI facade linked to Service.
Comment: I am using a Service to send notification but can you please elaborate how can i link my activity with my service?
Comment: How can i link an Activity to a service? So that when Android OS tries to destroy my activity then it won't let it do because my activity is linked with the running service.


Converting Scala case class to PySpark schema

Question: Given a simple Scala case class like this:
package com.foo.storage.schema
case class Person(name: String, age: Int)

it's possible to create a Spark schema from a case class as follows:
import org.apache.spark.sql._
import com.foo.storage.schema.Person

val schema = Encoders.product[Person].schema

I wonder if it's possible to access the schema from a case class in Python/PySpark. I would hope to do something like this [Python]:
jvm = sc._jvm
py4j_class = jvm.com.foo.storage.schema.Person
jvm.org.apache.spark.sql.Encoders.product(py4j_class)

This throws an error com.foo.storage.schema.Person._get_object_id does not exist in the JVM. The Encoders.product is a generic in Scala, and I'm not entirely sure how to specify the type using Py4J. Is there a way to use the case class to create a PySpark schema?
Answer: I've found there's no clean / easy way to do this using generics, also not as a pure Scala function. What I ended up doing is making a companion object for the case class that can fetch the schema.
Solution
package com.foo.storage.schema
case class Person(name: String, age: Int)
object Person {
  def getSchema = Encoders.product[Person].schema
}

This function can be called from Py4J, but will return a JavaObject. It can be converted with a helper function like this:
from pyspark.sql.types import StructType
import json
def java_schema_to_python(j_schema):
  json_schema = json.loads(ddl.json())
  return StructType.fromJson(json_schema)

Finally, we can extract our schema:
j_schema = jvm.com.foo.storage.Person.getSchema()
java_schema_to_python(j_schema)

Alternative solution
I found there is one more way to do this, but I like the first one better. You can make a generic function that infers the type of the argument in Scala, and uses that to infer the type:
object SchemaConverter {
  def getSchemaFromType[T <: Product: TypeTag](obj: T): StructType = {
     Encoders.product[T].schema
  }
}

Which can be called like this:
val schema = SchemaConverter.getSchemaFromType(Person("Joe", 42))

I didn't like this method since it requires you to create a dummy instance of the case class. Haven't tested it, but I think the function above could be called using Py4J too.


Highlight Field Errors with an Ajax Form

Question: I'm using a number of Ajax form, and while the I can get the error validation to work, and present the error message using the following:
  if (null !== form_get_errors()):
    return array(
      '#type' => 'ajax',
      '#commands' => array(
        ajax_command_remove('.alert-dismissable'),
        ajax_command_prepend('#myErrorDiv', theme('status_messages')),
      ),
    );
  endif;

However, I am unsure how to highlight the specific fields (usually done by adding a 'error' class to the form item.)
What the best way to achieve this, or is there a better way to handle the form errors that does not take over the normal error highlighting and message display?
Answer: Not as hard as I thought, but for anyone else trying to solve the same issue, you need to get all the relevant fields from form_get_errors() and add the error class to the same - note I also clear the previous errors:
if (null !== form_get_errors()):
  $commands[] = ajax_command_invoke('#myFormDiv .form-item.error', 'removeClass', array('error'));
  $commands[] = ajax_command_remove('.alert-dismissable');
  $commands[] = ajax_command_prepend('#myErrorDiv', theme('status_messages'));
  foreach (form_get_errors() as $field => $message) :
    $commands[] = ajax_command_invoke('.form-item-'.$field, 'addClass', array('error'));
  endforeach;
  return array(
    '#type' => 'ajax',
    '#commands' => $commands,
  );
endif;

Note that your $field will be dependent in what you've set in your validation function, so you might need to do some funky things to make sure '.form-item-'.$field is finding the right div.


Unable to connect to Azure DB using JDBC with ActiveDirectoryServicePrincipal

Question: I'm trying to connect to an Azure DB via JDBC using authentication mode ActiveDirectoryServicePrincipal.
Connection string: jdbc:sqlserver://xxx.database.windows.net:1433;Database=abcd;Authentication=ActiveDirectoryServicePrincipal;
Used Java components:

mssql-jdbc-11.1.1.jre8-preview.jar
msal4j-1.11.3.jar
oauth2-oidc-sdk-9.22.1.jar
json-smart-2.4.7.jar

When trying to connect, the attempt fails with the following error:
2022-04-28 15:23:50 INFO  - Connecting to Source Database jdbc:sqlserver://xxx.database.windows.net:1433;Database=abcd;Authentication=ActiveDirectoryServicePrincipal; ...
2022-04-28 15:23:50 INFO  - Connecting to Source JDBC data source jdbc:sqlserver://xxx.database.windows.net:1433;Database=abcd;Authentication=ActiveDirectoryServicePrincipal; ...
2022-04-28 15:23:51 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:51 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:51 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:51 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:51 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:51 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:52 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:52 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:53 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:53 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:54 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:54 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:55 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:55 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:56 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:56 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:57 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:57 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:23:59 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:23:59 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:00 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:24:00 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:01 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:24:01 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:02 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:24:02 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:03 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:24:03 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:04 DEBUG - SkipCache set to false. Attempting cache lookup
2022-04-28 15:24:04 DEBUG - Cache lookup failed: Token not found in the cache
2022-04-28 15:24:04 ERROR - Error establishing database connection to Source DB (URL: jdbc:sqlserver://xxx.database.windows.net:1433;Database=abcd;Authentication=ActiveDirectoryServicePrincipal;)
com.microsoft.sqlserver.jdbc.SQLServerException: Failed to authenticate the user yyy in Active Directory (Authentication=ActiveDirectoryServicePrincipal). com/nimbusds/common/contenttype/ContentType
    at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getCorrectedException(SQLServerMSAL4JUtils.java:240)
    at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getSqlFedAuthTokenPrincipal(SQLServerMSAL4JUtils.java:97)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.getFedAuthToken(SQLServerConnection.java:5440)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.onFedAuthInfo(SQLServerConnection.java:5388)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.processFedAuthInfo(SQLServerConnection.java:5275)
    at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onFedAuthInfo(tdsparser.java:305)
    at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:128)
    at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:37)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:6262)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:4880)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:90)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:4818)
    at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7601)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3885)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3331)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2923)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2763)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1657)
    at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1063)
    at com.synabi.dqnconnect2jdbc.util.DriverShim.connect(DriverShim.java:19)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at com.synabi.dqnconnect2jdbc.Importer.run(Importer.java:484)
    at com.synabi.dqnconnect.App.main(App.java:621)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: com/nimbusds/common/contenttype/ContentType
    at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getCorrectedException(SQLServerMSAL4JUtils.java:238)
    ... 23 more
Caused by: java.lang.RuntimeException: com/nimbusds/common/contenttype/ContentType
    at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getCorrectedException(SQLServerMSAL4JUtils.java:230)
    ... 23 more

However, when trying to connect using authentication mode SqlPassword, it's working without a problem (different user, of course, but with the same rights). Additionally, the username and password of the Service Principal user works as well when trying to connect using Powershell.
So there seems to be a problem with the JDBC components. Does anyone have any idea how to solve this?
Edit (for clarification): I've read the official documentation (https://learn.microsoft.com/en-us/sql/connect/jdbc/connecting-using-azure-active-directory-authentication) and followed the described steps to set up the connection.
Comment: @PaulJackson, Yes, I was able to solve it, but didn't post it as an answer because I can't explain in detail what solved it in the end. I was just trying to connect to the database using DataGrip, and DataGrip downloaded a bunch of required jar files automatically. I just tried using the exact same jars, and it finally worked. So it definitely must have been a missing jar, but I don't know which one.
Comment: I'm having the exact same problem except I'm using `authentication=ActiveDirectoryPassword`. I can connect with a different user using `SqlPassword` but get this same ContentType exception and stack trace, right down to the line numbers. Did you ever resolve it?
Answer: I got a very similar error and in my case it was a missing jar.
Caused by: java.lang.RuntimeException: com/nimbusds/common/contenttype/ContentType

is SQLServerMSAL4JUtils.getCorrectedException's way of swallowing the true cause of the exception:
java.lang.NoClassDefFoundError: com/nimbusds/common/contenttype/ContentType
    at com.nimbusds.oauth2.sdk.http.HTTPMessage.setContentType(HTTPMessage.java:98)
    at com.nimbusds.oauth2.sdk.http.HTTPRequest.setContentType(HTTPRequest.java:73)
    at com.microsoft.aad.msal4j.TokenRequestExecutor.createOauthHttpRequest(TokenRequestExecutor.java:41)
    at com.microsoft.aad.msal4j.TokenRequestExecutor.executeTokenRequest(TokenRequestExecutor.java:31)
    at com.microsoft.aad.msal4j.AbstractClientApplicationBase.acquireTokenCommon(AbstractClientApplicationBase.java:128)
    at com.microsoft.aad.msal4j.AcquireTokenByAuthorizationGrantSupplier.execute(AcquireTokenByAuthorizationGrantSupplier.java:63)
    at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:71)
    at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:20)
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: com.nimbusds.common.contenttype.ContentType
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 12 more

In this case the class can be found in: content-type-2.2.jar
which is a dependency of oauth2-oidc-sdk.
Answer: Are you using simple Java or Spring framework for making the Microsoft SQL connection? In place of the JDBC string, set parameters in the com.microsoft.sqlserver.jdbc.SQLServerDataSource object.
Comment: I was using simple Java. The problem turned out to be missing dependent jars.


Box shadow not rounding

Question: So i'm working on this website -
http://www.roseandlilyprinting.com/
If you go to the port-folio section you would see that the box shadow is not rounding to suite the border radius making the box shadow disrupt the feel
Here's the CSS Snippet the #navBarPort is what has the box shadows along with #allImages
Thanks for your help

#navBarPort {
    background-color: #CCCCCC;
    display: block;
    height: 80px;
    margin: 55px auto;
    position: relative;
    text-align:center;
    font-size: 1.2em;
    border-radius: 10px 10px 10px 10px;
    box-shadow: 1px 1px 1px #999,
    2px 2px 2px #999,
    3px 3px 3px #999,
    4px 4px 4px #999,
    5px 5px 5px #999,
    6px 6px 6px #999,
    7px 6px 7px #999,
    8px 6px 8px #999,
    9px 6px 9px #999;
}

#navBarPort a{
    color: #FFFFFF;
    display: inline-block;
    height: 39px;
    line-height: 37px;
    padding: 0 15px;
    text-shadow:1px 1px 1px #315218;
    text-decoration: none;
}

#navBarPort a:hover{
    text-decoration:none;
    color: #27A5BA;
}

#navBarPort a.active{
    color: #27A5BA;
}

/*----------------------------
 Content area
-----------------------------*/
#images {
    height: 460px;
    display: inline-block;
    text-align: center;
    padding-left: 20px;
    overflow: hidden;

}

#allImages{
    display:block;
    height: 490px;
    margin:0 auto;
    padding: 10px;
    border-radius: 10px 10px 10px 10px;
    box-shadow: 10px -10px 5px #999;
    -moz-box-shadow: 10px -10px 5px #999;
    -webkit-box-shadow: 10px -10px 5px #999;
    -o-box-shadow: 10px -10px 5px #999;
    background-color: #CCCCCC;
}

#allImages li{

    float: left;
    height: 96px;
    list-style: none outside none;
    margin: 6px;
    position: relative;
    width: 115px;
    overflow: hidden;
    -moz-box-shadow: 0 0 5px #000;
    -webkit-box-shadow: 0 0 5px #000;
    box-shadow: 0 0 5px #000;
}

#allImages ul li img{
    cursor: pointer;
    width: 100%;
    height: 100%;
}

#allImages ul{
    overflow:auto;
    display: inline-block;
    text-align: center;
}

#allImages ul.hidden{
    display:none;
}
Comment: Visited the link...after 30 second the page hadn't loaded...so I gave up.
Comment: Please change this: border-radius: 10px 10px 10px 10px; to this: border-radius: 10px; This makes it more readable and removes redundant code.
Comment: @Chrillewoodz _"Omit things in your code whenever you can."_ But not in the sense of code-golfing, right? I'd reword it as "Make code as readable as possible and remove as much redundancy as possible".
Comment: @Xufox Very true, I've re-phrased my comment accordingly. Cheerios.
Comment: @Chrillewoodz actually I was doing it that way to try an fix it by changing values
Answer: Your box shadows are cut off because of the overflow: hidden property on the div element with class flex-viewport.
Note that box shadows don't necessarily stay within an element's bounding box, and thus can be considered overflow.
Note that removing that property will interfere with the flex slider, but it's possible to find a workaround (for example, setting overflow: hidden on a parent element).
Comment: Can u help me find which parent element? I'm a bit new to css3
Comment: Your best bet is to use "inspect element" and try moving the property to various parents. see which one works out the best and needs the least modification. :)
Comment: I figured it out bro thanks I see where the overflow u were talking about

Box shadow not rounding


Compute $\|x\|=\max_{1 \leq i_1 \leq \dots \leq i_r \leq n} |x_{i_1}| + \cdots + |x_{i_r}|$ by optimization?

Question: Let $1 \leq r \leq n$ be an integer. Define $\|x\|=\max_{1 \leq i_1 \leq \dots \leq i_r \leq n} |x_{i_1}| + \cdots + |x_{i_r}|$ as a norm on $\mathbb{R}^n$.
My question is why $\|x\|$ can be computed as the optimal value of the following problem
$$
\max_y \quad x^Ty \\
\text{subject to} \quad \|y\|_{\infty} \leq 1 \\
\quad \quad \quad \quad \|y\|_1 \leq r
$$
I'm very stuck on this question and would appreciate any help on it. Thanks!
Answer: To solve the original problem, we just have to sort the entries of $x$ and take the first $r$ elements with the largest elements and sum the magnitude up. Hence, we can just focus on the magnitude of $x$ and the optimization problem is equivalent to
$$\max_y \sum_i |x_i|y_i$$
subject to
$$0 \le y_i \le 1, i \in \{1,\ldots, n\}$$
$$\sum_i  y_i \le r$$
of which we know an optimal solution is attained at BFS, that is there is an optimal solution where there will be $n$ independent constraints that are active.
Suppose the last constraint is not active, that is sum of $y_i$ is less than $r$, notice that we can always make one more $y_i$ to take value $1$ and it remaisn feasible and the solution won't be worse. Hence, we can assume that the last constraint is active. If the last constraint is active, and suppose we let at least $n-1$ of the $y_i$ takes value $\{0,1\}$. Since the sum is an integer, each $y_i$ must be integer as well and hence the last $y_i$ will also take value from $\{0,1\}$, that is exactly $r$ components will take value $1$. In the original formulation, it means that there is an optimal solution with exactly $r$ of them will take value with magnitude $1$.
Comment: Suppose $x_i$ is negative, then in your original formulation, you would have chosen $y_i \le 0$. Overall we would want $x_i y_i \ge 0$ for the maximization problem. My formulation let $x_i$ take absolute value and let $y_i$ take nonnegative value. As for the BFS thing, it's a standard result from LP.
Comment: Thanks for your help! I have a few questions: 1.why can you assume that $y_i \geq 0$ and focus just on the $|x_i|$, will this be the same as the original optimization problem? 2. your statement about "we know an optimal solution is attained at BFS, that is there is an optimal solution where there will be $n$ independent constraints that are active." I'm not very familiar with this, but I guess it's a standard result from LP?
Comment: Thanks for the clarification!


Convolutional neural network architecture calculation question

Question: I'm attempting to understand the neural network architecture used in this paper: Visualizing and Understanding Convolutional Networks.
Here's an image of the network achitecture from the paper that is bothering me:

So in the first layer, a (224 x 224 x 3) image is convolved by 96 (7 x 7) kernels with a stride of 2. There is no mention of padding in the architecture description. Plugging these parameter values in the algorithm for the spatial size of the output volume from the Wikipedia article on CONVs, I have:

W = 224
K = 7
P = 0
S = 2
This gives (W - K + 2P) / S + 1 = 109.5  According to commenters from a StackOverflow question, we simply round down to 109 when the output of the formula is not an integer. I tested this out in Tensorflow and confirmed this. However, in this paper, the output size is not 109, but 110. So I guess they rounded up, but I don't understand why. This method of rounding up repeats itself throughout the network architecture.
Am I incorrectly applying this formula, were things done differently back in 2014 when the paper came out? Can anybody explain why they round up as opposed to down, and perhaps how I can do the same in Tensorflow?
Answer: You're correctly applying the formula (with a typo in +1 in your post), and it's typically rounded down in many libraries. Some lower level libraries provide an option to round down or up, but I didn't see this in tensorflow or pytorch.
The paper is from 2014, where there was no tensorflow nor pytorch publicly available. So, everything was a bit less standard (not saying they're right now, but comparatively).
For this reason, I believe the paper used their own implementation according to their understanding and assumptions. The formula given does not hold for subsequent layers as well, where 13 x 13 matrices are turned into 13 x 13 matrices with 3 x 3 conv. operations.
It's hard to understand what exactly they did, but except the pooling from 55 to 26, every dimension seems to hold with padding = 1. The rounding up trick does not work for 13x13 kernels.
Comment: Thank you for answering! I assume they completely changed the process for the 13 x 13 matrices, since the stride has changed from 2 to 1 and they seem to use what tensorflow calls "same" padding. All in all, I'm disappointed in the clarity of the paper, but glad I got an explanation about the formula. (I'll fix the typo)


Incorrect technique or signal chain noise?

Question: I am getting a "hiss" noise from my amp sim after I play a note. This is not present in low gain. But gets pretty audible on higher gain. My signal chain is a guitar, interface, Logic pro with a toneforge amp sim (the noise was present in other amp sims too). I tried changing the guitar, cable and the interface. But it did not change a thing. The noise gate is already at a high level. Increasing it anymore starts to clip my sound. I have attached a link to a clip i recorded with palm mutes. Could this be because of wrong right hand technique? If so, how can I improve? If not, how can i remove this noise?
Drive link to audio clip
Comment: I can't hear a problem with your sample. Sounds very clean to me.
Answer: That sounds like a fairly normal level of noise for a high-gain sound. That's the reason these amp sims have noise gates built into them - high-gain means you amplify all sorts of noise along with the signal, from sources both external (the power grid, radio, various electronic devices) and internal (thermal noise in the pickup and electronics).
As far as technique is concerned: of course you have to mute all other strings while playing muted rhythm stuff, but it doesn't sound like your hiss is caused by resonating strings, so you're probably doing the right things here.
Regarding the signal chain, there are things you can try:

Most importantly, use a humbucker instead of a single coil, but it sounds like you're doing that already.
Keep your guitar at a distance (more than 1m) from the computer and other electronic devices, put your cell phone in flight mode.
Check if the electronics cavity of your guitar is shielded. If not, and if you're willing to tinker with it, search on Youtube for "guitar shielding".
Try different cables.

If none of that helps, learn to live with it, and see if you can still even perceive it in a mix, with bass and vocals and cymbals and everything. You're playing high-gain guitar - it doesn't have to be squeaky clean.
Comment: OK. I was worried because this noise was not present when doing nothing. It comes up right after I pick. Will check the other parameters u specified.


Another round table problem involving 12 chairs and 5 persons

Question: I spent a couple of hours thinking about the following excercise:
Given a round table with 12 chairs, how many possibilites exist to place 5 persons at this chair. There should be at least one chair between each two persons. The chairs are not numbered and the possibilities only differing in the rotation of the table are equal.
I think there are the following three possible patterns of free chairs (0) and persons (1)
1 0 1 0 1 0 1 0 1 0 0 0
0 0 1 0 0 1 0 1 0 1 0 1
0 0 1 0 1 0 0 1 0 1 0 1
every other combination of persons and chairs could be shifted to one of these three.
So there is one last thing to do, the possibilities of the 5 persons sitting on 5 chairs, this leads to 5!.
So my answer is:
$
3 * 5! = 360
$
There is a bonus task, I should give a generall solution für n chairs and k persons. It would be great to get some advice on this bonus task, because I got stuck thinking about this.
Edit:
The persons are distinguishable.
Comment: You should clarify if the 5 persons are indistinguishable (seem that way) or distinguished.
Comment: you can distinguish between persons
Answer: Your method works, but here is an approach that will generalize better. (Everything I do here can be done for $k$ people sitting in $n$ chairs.)
Imagine each person sitting at the table rudely puts their feet on the chair to their left. This should always be possible if there's a gap between any two people. Conversely, if it is possible, then the gap is ensured.
So we can imagine that each person sitting at the table occupies two chairs in this way, and we are trying to arrange $5$ people and $2$ empty chairs around the circle.
To deal with rotations, the easiest thing to do is to fix the position of person 1. Then we are trying to arrange the remaining $4$ people and $2$ empty chairs in a sequence that can no longer be rotated, and this is easy: it's $\frac{6!}{2!} = 360$.
(There are $6$ people and empty chairs to rearrange in $6!$ ways, but we divide by the $2!$ ways to rearrange the empty chairs, which doesn't change the outcome.)


Tráfego de pacotes na rede

Question: Há muito tempo atrás, tive conhecimento de uma técnica que basicamente todo o dado que é enviado ou recebido pelo roteador, é "duplicado" e encaminhado para um endereço específico e o endereço remetente/destinatário original óbvio. Obviamente isso causa uma sobrecarga nesse endereço específico e isso é utilizado por maliciosos p/ acessar os dados mas também propositalmente para verificar a integridade dos pacotes. Minha dúvida é: Qual o nome dessa "técnica"?
Comment: Broadcast, Seria isso?
Comment: Sniffer é verficar todos os pacotes que estão passando na rede incluindo os que não são destinados a vc. Broadcast é quando vc envia um ou mais pacotes para todos os hosts na rede.
Comment: Não sei muito sobre redes mas o que eu buscava se chama Sniffer
Comment: Se for um switch você pode alcançar isto fazendo "Port mirroring", na qual todo o tráfego é encaminhado para uma porta especifica em que você tem um equipamento sniffando.
Answer: Talvez a técnica "man-in-the-middle" seja o que você procura, já que há um "sniffer pivot" entre o remetente e o destinatário do pacote, onde esse "pivot" é responsável por receber o pacote enviado, fazer o que for preciso com o mesmo e enviar para o destino. Caso precise de mais informações, pode ler aqui: https://www.owasp.org/index.php/Man-in-the-middle_attack

Tráfego de pacotes na rede


LAMP - file doesn't appear under tmp directory

Question: I am uploading an image using an HTML form.
When I get the full path to the image, it outputs something like this:
'/tmp/phpkIv1BY/10259944_770025219687938_1184503840380306483_n.jpg'
but when I go to the /tmp folder, the sub-folder phpkIv1BY doesn't even exist! What's going on here?
Comment: Maybe it is just going to your machine's tmp folder
Comment: How do I find out where it is?
Comment: Try checking if /tmp or /var/tmp folders have the sub-folder mentioned.
Comment: They don't, I've already checked.
Answer: Reason for this behaviour
All uploaded files are temporarily stored in the folder location as defined in
(In php.ini)
upload_tmp_dir =
(Read more about this: http://php.net/upload-tmp-dir)
These temporarily stored files are no longer exist after that php script execution(As mentioned in previous answer request life span) finished.
Do the following to view uploaded file while script executing.

upload.html

    <form action="upload_file.php" method="post"
    enctype="multipart/form-data">
    Filename:
    <input type="file" name="file" id="file">

    <input type="submit" name="submit" value="Submit">
    </form>

upload_file.php

<?php
if ($_FILES["file"]["error"] > 0) {
  echo "Error: " . $_FILES["file"]["error"] . "
";
} else {
        $fp=fopen("/tmp/write.log","w+");
        fputs($fp,"Original Name:".$_FILES["file"]["name"].";temporay Name:".$_FILES["file"]["tmp_name"]."\n");
        fclose($fp);
        sleep(50000);
}
?>

Upload file by upload.html
In another terminal apply tail in /tmp/write.log after uploading image(Because sleep time is too much so you able to find the image)
#tail -f /tmp/write.log
Copy that temporary file into any place and place the extension of original image file
For example my printed log line is
Original Name:digits.png;temporay Name:/tmp/phpOKlpoK
(Needed to do this with root privilages)
#cp /tmp/phpOKlpoK /home/sandeep/Desktop/file.png
#chown sandeep.sandeep /home/sandeep/Desktop/file.png
So this is the uploaded file that you.
(
Instead of doing all these steps for checking uploaded images you can use move_uploaded_file()
)
Answer: I can not find anywhere if PHP might create a temporary folder for temporary uploaded files. But what is certain is that uploaded files are kept temporary and as soon as the request is done, they are removed. So if you think you can find some uploaded file in the /tmp folder, think again.
If you would like some uploaded file to live longer than a request life span, then you need to move it somewhere safe using move_uploaded_file().
Comment: Once the file is submitted using a form, I try to process it using the Intervention Image library by passing the tmp url to the `Image::make()` method, but it acts as if it can't handle the file. If I move the file to a folder and the pass it on to the library, it works. How do I make it work with the tmp file? I wish to do it because I want to process the file as image first, and only then move it to the upload folder (in case it's a potentially dangerous file).
Comment: What exactly you want to do with the intervention library ?


How to properly duplicate a variable so that the value of the copy changes when the original variable changes it's value

Question: probably a silly question by I can't figure it out myself so here it goes.
I'm trying to duplicate a variable so that when the original variable changes it's value, the copy changes as well.
I've tried it this way but it's clearly not working:
<code>x = 10
y = x
x = 20
y
10
</code>
how do I make y to equal 20 only by changing x?
tnx
Answer: That depends on the type of the object, and whether the object is mutable or immutable. Since you values are integers, you can't. It would work if, for example, they were lists.


How does this follow from the theorem?[normed linear space]

Question: I have this theorem:

Let X and Y be normed linear spaces and let $T:X\rightarrow Y$ be a
  linear transformation. The following are equivalent:
a. T is uniformly continuous.
b. T is continuous.
c. T is continuous at 0.
d. there exists a positive real number k such that $\|T(x)\|\le k$
  whenever $x \in X$ and $\|x\|\le 1$.
e. there exists a positive real
  number k such that $\|T(x)\|\le k \|x\|$ for all $x \in X$.

It then says in the book that a consequence of this theorem is:

$\sup\{\|T(x)\|: \|x\| \le 1\}=\inf\{k: \|T(x)\| \le k\|x\|\ \ \forall
 x \in X\}$

However, I am only able to prove the inequality, not equality:
$\sup\{\|T(x)\|: \|x\| \le 1\}\le\inf\{k: \|T(x)\| \le k\|x\|\ \ \forall
 x \in X\}$
I get this inequality like this:
Let: $A=\{\|T(x)\|: \|x\| \le 1\}$, $B=\{k: \|T(x)\| \le k\|x\|\ \ \forall
 x \in X\}$.
Then if $\|T(x^*)\|$ is any element in A, and k is any element in B. Then $\|T(x^*)\|\le k\|x\|\le k$, since $\|x\|\le 1.$
So any element in A is smaller than any element in B.
Then for any a in A, we have that $a \le k$ for any k in B. So a is a lower bound for B, it must be smaller than the biggest lower bound for B, so $a \le \inf B$. But since this holds for any a in A, we must have that $\inf B$ is an upper bound for A, so it must be bigger than the smallest upper bound for A, hence $\sup A\le \inf B$.
Hence I am only able to prove that $\sup A\le \inf B$, but how do I get that $\sup A=\inf B$?
Answer: Let $\alpha = \sup(A), \beta = \inf(B)$, and $\epsilon > 0$, then $\beta-\epsilon \notin B$. Hence, $\exists x_0 \in X$ such that
$$
\|T(x_0)\| > (\beta-\epsilon)\|x_0\|
$$
But then $x_0\neq 0$, so with $y_0 := x_0/\|x_0\|$, we have $\|y_0\| \leq 1$ and thus
$$
\alpha \geq \|T(y_0)\| > \beta-\epsilon
$$
This is true for every $\epsilon > 0$, so $\alpha \geq \beta$
Comment: Very nice, thank you!:) May I ask how you came up with this idea?
Comment: When comparing a sup/inf with another quantity, it helps to add/subtract an $\epsilon$ to it, so that the defining property fails to hold. This gives you "room" to stick an element in between, which usually pays off.


Marginalising over standard deviation of normal to get the posterior on mean

Question: I am trying to understand the concepts of Bayesian data analysis by examples. I have managed to "do" something, and I would like some advise on where I have gone wrong.
My data is that I have 50 students who take a test and score in the range [0, 1.0], I refer to this as $y$. I would like to calculate the posterior distribution of their mean score $P(\mu|y)$ assuming that the scores are normally distributed with mean $\mu$ and standard deviation $\tau$.
Looking at other examples, I was able to do the following in Pymc:
import pymc as pm
import numpy as np
import matplotlib.pyplot as plt

# Generate fake data
data_scores = np.random.normal(0.5, 0.1, size=50)

mu = pm.Uniform("mu", lower=0, upper=1)
tau = pm.Uniform("tau", lower=0, upper=500)
scores = pm.Normal("score", mu=mu, tau=tau, observed=True, value=data_scores)
model = pm.Model([scores, mu, tau])
mcmc = pm.MCMC(model)
mcmc.sample(40000, 10000, 1)

mu_samples = mcmc.trace('mu')[:]

plt.hist(mu_samples, bins=30)
plt.show()

Which produces a nice histogram.
However, I am confused because I feel like I have calculated $P(\mu, \tau| y)$ and hence now need to marginalise of $\tau$ before considering the distribution of $\mu$.
Eventually I want to use this compare how to classes did on a test, but I am at a very basic level of understanding right now so any comments are appreciated.
Comment: Observation: the normal model seems somewhat strange compared to your verbal explanation of the test scores. If the test scores are restricted to $[0,1]$, then $y$ (not just the mean parameter $\mu$) should be restricted to that interval.
Comment: Sounds like you'd need to create your own step method in pymc, or perhaps you could use a Beta distribution instead of Normal. I'm not an expert on this, though.
Comment: Thanks for the observation and you are correct - I certainly am concerned that I could get a tail of the probability outside the range. May I ask what you would suggest?
Answer: Note that you do not 'calculate' $p(\mu,\tau \mid y)$. Instead, you samples from it (in the sense that MCMC methods sample from their target distributions).  If you sample from the joint distribution of $(\mu,\tau)$ and ignore the value of $\tau$, you have a sample from the marginal distribution of $\mu$ - this is essentially what the concepts of joint and marginal distributions mean. Therefore, your mu_samples can be interpreted as a sample from $p(\mu \mid y)$ and there is no need to do any special 'marginalization' to your samples.
However, analytically marginalizing over some variables, if possible, may improve the performance of an MCMC algorithm.


Get nid of nodes which contains FPP

Question: I have some fieldable panel pane and I can attach this FPP in diferent nodes with panelizer functionality.
How can I get nid of nodes which contains(attach) this FPP programmatically ?
I have fpid but how can I check in which nodes use this FPP ?
Answer: I'm looking for a similar solution, I work on a site with thousands of fieldable panel panes and need to find all the nodes that contain a certain fpid. The most promising lead I've found is: Find node programmatically by field value so if you've found a better solution (maybe using views) I'd like to hear about it.

Get nid of nodes which contains FPP


Dataflow Task.WhenAll causes A task was canceled Exception

Question: I am new to Dataflow, and I follow this walkthrough How to: Cancel a Dataflow Block.
I click add button first, and then click cancel, but I got exception about "A task was canceled Exception" after clicking cancel button. I fail to find any way to resolve this error.
Any help would be appreciated.
Update:
Code for demo:
    public partial class Form1 : Form
{
    CancellationTokenSource cancellationTokenSource;
    TransformBlock<WorkItem, WorkItem> startWork;
    ActionBlock<WorkItem> completeWork;
    ActionBlock<ToolStripProgressBar> incProgress;
    ActionBlock<ToolStripProgressBar> decProgress;
    TaskScheduler uiTaskScheduler;
    public Form1()
    {
        InitializeComponent();
        uiTaskScheduler = TaskScheduler.FromCurrentSynchronizationContext();
        Cancel.Enabled = false;
    }

    private void Add_Click(object sender, EventArgs e)
    {
        if (!Cancel.Enabled)
        {
            CreatePipeline();
            Cancel.Enabled = true;
        }

        for (int i = 0; i < 20; i++)
        {
            toolStripProgressBar1.Value++;
            startWork.Post(new WorkItem());
        }
    }

    private async void Cancel_Click(object sender, EventArgs e)
    {
        Add.Enabled = false;
        Cancel.Enabled = false;

        cancellationTokenSource.Cancel();

        try
        {
             await Task.WhenAll(
                completeWork.Completion,
                incProgress.Completion,
                decProgress.Completion);
        }
        catch (OperationCanceledException)
        {

            throw;
        }
        toolStripProgressBar4.Value += toolStripProgressBar1.Value;
        toolStripProgressBar4.Value += toolStripProgressBar2.Value;

        // Reset the progress bars that track the number of active work items.
        toolStripProgressBar1.Value = 0;
        toolStripProgressBar2.Value = 0;

        // Enable the Add Work Items button.
        Add.Enabled = true;
    }
    private void CreatePipeline()
    {
        cancellationTokenSource = new CancellationTokenSource();

        startWork = new TransformBlock<WorkItem, WorkItem>(workItem =>
        {
            workItem.DoWork(250, cancellationTokenSource.Token);
            decProgress.Post(toolStripProgressBar1);
            incProgress.Post(toolStripProgressBar2);
            return workItem;
        },
        new ExecutionDataflowBlockOptions
        {
            CancellationToken = cancellationTokenSource.Token
        });

        completeWork = new ActionBlock<WorkItem>(workItem =>
        {
            workItem.DoWork(1000, cancellationTokenSource.Token);
            decProgress.Post(toolStripProgressBar2);
            incProgress.Post(toolStripProgressBar3);
        },
        new ExecutionDataflowBlockOptions
        {
            CancellationToken = cancellationTokenSource.Token,
            MaxDegreeOfParallelism = 2
        });

        startWork.LinkTo(completeWork);

        startWork.Completion.ContinueWith(delegate { completeWork.Complete(); },cancellationTokenSource.Token);
        incProgress = new ActionBlock<ToolStripProgressBar>(progress =>
        {
            progress.Value++;
        },
        new ExecutionDataflowBlockOptions
        {
            CancellationToken = cancellationTokenSource.Token,
            TaskScheduler = uiTaskScheduler
        });

        decProgress = new ActionBlock<ToolStripProgressBar>(progress => progress.Value--,
            new ExecutionDataflowBlockOptions
            {
                CancellationToken = cancellationTokenSource.Token,
                TaskScheduler = uiTaskScheduler
            });

    }

    class WorkItem
    {
        public void DoWork(int milliseconds, CancellationToken cancellationToken)
        {
            if (cancellationToken.IsCancellationRequested == false)
            {
                Thread.Sleep(milliseconds);
            }
        }
    }
}

Comment: Well, do not rethrow the catched exception? - BTW you have an exception handling for that ;o)
Comment: Better post relevant code in question itself instead of posting a link to a huge tutorial.
Comment: Well if you cancel a task then you should not be surprised to get a TaskCanceledException. It just reflects what you have done
Comment: @SirRufo How to avoid this exception instead of exception handling this exception?
Comment: Unrelated to your issue but the `startWork.Completion.ContinueWith` is unnecessary. You just need to [Propagate Completion](https://stackoverflow.com/questions/33518074/how-do-i-signal-completion-of-my-dataflow). Also the `incProgress` and `decProgress` `ActionBlock`s aren't doing you much good, they could simply be [Progress](https://stackoverflow.com/questions/28430872/task-run-and-ui-progress-updates)
Comment: @JSteward Thanks, I am new to Dataflow, could you share us code related with Propagate and Progree?
Comment: any reason for vote down?
Answer: As @SirRufo pointed out, the solution to your question is simply don't re-throw the exception after you've caught it. But to highlight some of the other techniques you can use with dataflow as discussed in the comments I put together a small sample. I've tried to keep the spirit and intent of your original code intact. To that end; the original code didn't show how the flow would complete normally, as opposed to cancelled, so I left it out here as well.
using System;
using System.Data;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
using System.Windows.Forms;

namespace WindowsFormsApp1 {
    public partial class Form1 : Form {

        private CancellationTokenSource cancellationTokenSource;
        private TransformBlock<WorkItem, WorkItem> startWork;
        private ActionBlock<WorkItem> completeWork;
        private IProgress<int> progressBar1Value;
        private IProgress<int> progressBar2Value;

        public Form1() {
            InitializeComponent();
            btnCancel.Enabled = false;
        }

        private async void btnAdd_Click(object sender, EventArgs e) {
            if(!btnCancel.Enabled) {
                CreatePipeline();
                btnCancel.Enabled = true;
            }
            var data = Enumerable.Range(0, 20).Select(_ => new WorkItem());
            foreach(var workItem in data) {
                await startWork.SendAsync(workItem);
                progressBar1.Value++;
            }
        }

        private async void btnCancel_Click(object sender, EventArgs e) {
            btnAdd.Enabled = false;
            btnCancel.Enabled = false;

            cancellationTokenSource.Cancel();

            await completeWork.Completion.ContinueWith(tsk => this.Invoke(new Action(() => this.Text = "Flow Cancelled")),
                                                       TaskContinuationOptions.OnlyOnCanceled);

            progressBar4.Value += progressBar1.Value;
            progressBar4.Value += progressBar2.Value;

            // Reset the progress bars that track the number of active work items.
            progressBar1.Value = 0;
            progressBar2.Value = 0;

            // Enable the Add Work Items button.
            btnAdd.Enabled = true;
        }

        private void CreatePipeline() {
            cancellationTokenSource = new CancellationTokenSource();
            progressBar1Value = new Progress<int>(_ => progressBar1.Value++);
            progressBar2Value = new Progress<int>(_ => progressBar2.Value++);

            startWork = new TransformBlock<WorkItem, WorkItem>(async workItem => {
                await workItem.DoWork(250, cancellationTokenSource.Token);
                progressBar1Value.Report(0); //Value is ignored since the progressbar value is simply incremented
                progressBar2Value.Report(0); //Value is ignored since the progressbar value is simply incremented
                return workItem;
            },
            new ExecutionDataflowBlockOptions {
                CancellationToken = cancellationTokenSource.Token
            });

            completeWork = new ActionBlock<WorkItem>(async workItem => {
                await workItem.DoWork(1000, cancellationTokenSource.Token);
                progressBar1Value.Report(0); //Value is ignored since the progressbar value is simply incremented
                progressBar2Value.Report(0); //Value is ignored since the progressbar value is simply incremented
            },
            new ExecutionDataflowBlockOptions {
                CancellationToken = cancellationTokenSource.Token,
                MaxDegreeOfParallelism = 2
            });

            startWork.LinkTo(completeWork, new DataflowLinkOptions() { PropagateCompletion = true });
        }
    }

    public class WorkItem {
        public async Task DoWork(int milliseconds, CancellationToken cancellationToken) {
            if(cancellationToken.IsCancellationRequested == false) {
                await Task.Delay(milliseconds);
            }
        }
    }
}

Answer: After checking the code, I released that the tasks will be cancelled if I click Cancel.
await Task.WhenAll(
                completeWork.Completion,
                incProgress.Completion,
                decProgress.Completion);

But, above code Task.WhenAll need all of the tasks return complete status, then the "A task was canceled Exception" throw as expected if it returned cancelled instead of completed.
For a possible way to resolve this issue, we should return Task completed if we cancelled the task, and the code below works for me.
       await Task.WhenAll(
           completeWork.Completion.ContinueWith(task => cancelWork(task, "completeWork"), TaskContinuationOptions.OnlyOnCanceled),
           incProgress.Completion.ContinueWith(task => cancelWork(task, "incProgress"), TaskContinuationOptions.OnlyOnCanceled),
           decProgress.Completion.ContinueWith(task => cancelWork(task, "decProgress"), TaskContinuationOptions.OnlyOnCanceled));

Is it reasonable?


Tentando colocar as imagens uma ao lado da outra com o display; flex

Question:

 Moletom Branco

 Moletom Branco

 Moletom Branco

 Moletom Branco

 Moletom Branco

 Moletom Branco

css abaixo:
img{
    max-width: 10%;
    display: block;
}

\*css flexbox*\

.flex{
    display: flex;
    flex-wrap: wrap;
    max-width: 880px
    margin: 0 auto;
}

.flex > div {
    flex: 1 1 200px;
    margin: 10px;

Estou tentando alinhar as imagens lado a lado, ficando 3 imagens em cima e 3 embaixo como se fosse uma pagina de uma loja, exibindo produtos
Comment: Importante você [edit] sua pergunta e explicar de maneira objetiva e pontual a dificuldade encontrada, acompanhada de um [mcve] do problema e tentativa de solução. Para entender que tipo de pergunta serve para o site e, consequentemente, evitar fechamentos e negativações vale a pena ler [O que é o Stack Overflow](https://pt.meta.stackoverflow.com/q/7256/70) e o [Guia de sobrevivência (resumido) do Stack Overflow em Português](https://pt.meta.stackoverflow.com/q/8089/70).
Answer: Basicamente o erro o erro foi falta de atenção na hora de escrever o código o comentário não foi feito da maneira correta
Errado *css flexbox*\
Correto /* css flexbox */
E a falta de um ; depois do max-width: 880px evitou que o próximo estilo margin: 0 auto; carregasse
Ajustando isso seu código estará quase pronto, falta somente ajustar os cálculos o seu box que está agrupando os produtos tem 880px de largura e cada produto tem margin de 10px como vc quer 3 produtos por linha e esse margin é em todos os lado, devemos subtrair 60px de 880px e ai sim dividir por 3
Dessa forma 880-60=820/3=273px
Então ao invés de flex: 1 1 200px; o correto seria flex: 1 1 273px;
img{ max-width: 10%; display: block; }

/*css flexbox*/

.flex{ display: flex; flex-wrap: wrap; max-width: 880px; margin: 0 auto; }

.flex > div { flex: 1 1 200px; margin: 10px; }

Lembrando que é uma boa prática "dar nomes" a suas div's através das classes
Answer: A melhor forma de fazer isso é você separando as imagens de 3 em 3 dentro de divs, como o demonstrado abaixo:

.flex {
  display: flex;
  justify-content: center;
  align-items: center;
  flex-direction: column
}

.row1, .row2 {
  display: flex;
  margin: 15px 5px
}

                 Moletom Branco

                 Moletom Branco

                 Moletom Branco

                 Moletom Branco

                 Moletom Branco

                 Moletom Branco

Como no exemplo é só você centralizar tudo com o justify-content e align-items e setar o flex-direction para coluna que fica do jeito esperado!


Azure asp.net - why the webjob %temp% folder is not the same as the web app %temp% folder?

Question: I have a webjob that writes some data on the %temp% folder, which maps to d:\local\temp.
When i use the console tool on the azure portal, the data is there, on d:\local\temp.
When i try to access the files on the website (asp.net) the files are not there. But %temp% also maps to d:\local\temp.
Why?
Is there a path that maps to the same location?
The data I write on the local %temp% is a cache of a data stored on azure storage blob. I can re-create the cache if the data on %temp% is lost.
Thanks.
Answer: The main web app and the webjob run in 2 different sandboxed environments on the same VM. They have access to the same D:\home, but different D:\local.
Keep in mind that D:\home and D:\local are virtualized paths. They don't really exist on the filesystem. That's why you can run as many sites as you want on the same VM and each will have their own D:\home and D:\local.
If you want them to share something through the filesystem, you'll have to drop it under D:\home


Preimage, surjection, and injection for f: Z x Z -> Q, f(m, n) = m/(|n| + 1)

Question: Let $f\colon \mathbb Z \times \mathbb Z \to \mathbb Q$, $\ f(m, n) = \frac m{|n| + 1}.$
I need to define the preimage of $f^{-1}(\{0\})$. The way I understand that means that I need to find all the values that would give $f(m, n)$ the value $0$. I'm not sure if this is correct though. If it is, then there would be infinite values, so long as the numerator and denominator are equal.
The I need to determine whether or not $f$ is an injection and a surjection and prove why.
Would it be correct to say that $f$ is not an injection because for example, if $m = 2$ and $n = 2$, then $f(m, n) = 2/3$. If $m = 2$ and $n = -2$, then $f(m, n) = 2/3$. So $f(m, n)$ can have the same value with two different inputs, which means it is not an injection.
As for being a surjection - would it  be correct to say that it is a surjection because the function can have any value depending on m and n. This seems like a very feeble "proof", but I'm not sure how to go about it.
Answer: Yes, $f^{-1}(\{0\})$ consists of all ordered pairs $(m,n)$ such that $f(m,n)=0$. Setting
$$\frac{m}{|n|+1}=0
$$
the solutions are all $(m,n)$ such that $m=0$ (no restrictions on $n$).
You injection argument is correct.
For surjection, it helps to write out the whole statement with quantifiers in their correct place and with all elements properly named: for any $\frac{k}{l} \in \mathbb{Q}$ there exists $(m,n) \in \mathbb{Z} \times \mathbb{Z}$ such that $f(m,n)=\frac{k}{l}$. Put another way, given two integers $k$ and $l$, with $l \ne 0$, find two integers $m,n$ such that
$$\frac{m}{|n|+1} = \frac{k}{l}
$$
Can you take it from here?
Comment: Yes, that's correct about restrictions for $n$. The symbols "$f : \mathbb{Z} \times \mathbb{Z} \to \mathbb{Q}$" are a mathematical sentence whose English translation is "$f$ is a function whose domain is $\mathbb{Z} \times \mathbb{Z}$ and whose range is $\mathbb{Q}$".
Comment: The only restriction for n would be that it has to be an element of Z? I'm not entirely sure if that's what the f: Z x Z -> Q implies.

I'm not really sure how to continue from the surjection part. m = k and |n| = l - 1 ? I'm not sure how to prove anything with this.
Comment: As for the surjection argument, the proof should be in two cases. In the case that $l$ is positive you are correct, your formulas $m=k$ and $|n|=l-1$ look good, and then all you have to do is to say what $n$ itself is, and the proof is complete in that case. But in the case that $l$ is negative the formulas are incorrect and you'll have to do something else.
Comment: If n is negative then n = -l + 1?
Comment: That doesn't look right. First I would rewrite the rational number $\frac{k}{l}$ in the form $\frac{-k}{-l}$...
Comment: I'm not sure I understand why...


Floyd Rose in Standard Tuning with .011 gauge strings

Question: I have always played with medium to heavy gauge strings in standard tuning. I got used to that feel while in school for Jazz Studies and have stuck with it ever since.
I'm interested in learning some Satriani tunes that make use of the Floyd Rose. My question is:
What will happen if I have a Floyd Rose-style guitar setup with .011s or .012s at Standard Tuning? Are those systems designed in a way that is going to make it play badly with that amount of tension?
Answer: Any vibrato is set up as a balance between the strings and the springs. So once that balance is achieved, it shouldn't make much if any difference whether they're .008s or .013s. Obviously the bendability will be different, but that's a different issue. You may also need to check action and intonation.
Answer: Moving from light gauge strings to heavier gauge stings on a Floyd Rose style bridge will cause you problems; unless you take care to set the bridge up properly, or have a shop or someone set it up for you.
As Tim said in his answer, once the guitar is set up properly, there should not be much of a difference, in terms of action at least. The string tension will certainly be greater. Floyd Rose style bridges usually use 3-5 springs (though I have seen one with two springs and .009 gauge strings) to tension the strings to the proper level. If you are doing this change yourself, you will probably want to take a look at the bridge cavity on the back of the guitar.
You should find a claw apparatus that is affixed to the guitar body with wood screws. If that claw is already close to the body, then you don't have much room to apply extra tension the springs, and you may need to get an extra spring or two in there to make up for the difference. Pulling the old strings off, and then finding you can't tension the bridge properly due to a lack of springs is not a fun time.
Getting the springs/strings set up is a careful balancing game, and it is usually a lot of back and forth to get it all dialed in right. You will want to undo the locking plates at the nut so you can restring and start to tune the guitar, and watch what happens to the bridge as you start to tune the strings to the desired pitches. Keep an eye on the bridge, and if the back of the bridge starts to raise up past where it used to be, you will want to back off on the tension of the strings, and tighten the screws attached to that claw. Go back to the strings and tune up. A bit of back and forth like that is always required when setting up a Floyd Rose bridge with strings of a different tension. I own two guitars with Floyd Rose bridges, and I've been through that dance more times than I can remember.
When you get the springs/strings set up to the right tension, try to handle any intonation changes afterward. You will also want to pay attention to the neck, in case the truss rod needs any adjustment to put the action back to where you expect it to be. Moving up to a heavier gauge of strings may call for tightening the truss rod a little bit, which you always want to be careful with. Replacing a broken truss rod is not trivial.
The thing I really like about a Floyd Rose bridge is that once it's all dialed in and locked in place, they tend to be rock-solid. I can pick one up after a half year of it sitting there, and it's still be properly in tune. Worth the effort!


Why is a random variable on a sigma algebra not necessarily measurable on a sub sigma algebra?

Question: I have a question regarding the following bolded claim made on Wikipedia
Consider the following:

$(\Omega, \mathcal{F}, P)$ is a probability space.
$X: \Omega \rightarrow \mathbb{R}^{n}$ is a random variable on that probability space with finite expectation.
$\mathcal{H} \subseteq \mathcal{F}$ is a sub-\sigma-algebra of $\mathcal{F}$.

Since $\mathcal{H}$ is a sub $\sigma$-algebra of $\mathcal{F}$, the function $X: \Omega \rightarrow \mathbb{R}^{n}$ is usually not $\mathcal{H}$-measurable, thus the existence of the integrals of the form $\left.\int_{H} X d P\right|_{\mathcal{H}}$, where $H \in \mathcal{H}$ and $\left.P\right|_{\mathcal{H}}$ is the restriction of $P$ to $\mathcal{H}$, cannot be stated in general. However, the local averages $\int_{H} X d P$ can be recovered in $\left(\Omega, \mathcal{H},\left.P\right|_{\mathcal{H}}\right)$ with the help of the conditional expectation. A conditional expectation of $X$ given $\mathcal{H}$, denoted as $\mathrm{E}(X \mid \mathcal{H})$, is any $\mathcal{H}$ measurable function $\Omega \rightarrow \mathbb{R}^{n}$ which satisfies:
$$
\int_{H} \mathrm{E}(X \mid \mathcal{H}) \mathrm{d} P=\int_{H} X \mathrm{~d} P
$$
Why is a random variable on a sigma algebra not necessarily measurable on a sub sigma algebra? If a sub-sigma algebra is a collection of subsets from the original sigma algebra which forms a sigma algebra in its own right, why wouldn't this enable any random variable to continue to be measurable on some sub-sigma algebra?
Comment: Consider the case $\mathcal{H} = \{\emptyset , \Omega \}$. Which functions are measurable here?
Comment: @JoseAvilez Would it only be only bernoulli random variables?
Comment: No. Only constants would be measurable.
Comment: @JoseAvilez So since this is a sub-sigma algebra of every sigma algbera on Omega,  this is a counter example?
Comment: It's an example that exhibits why an $\mathcal{F}$-random variable need not be measurable with respect to $\mathcal{H}$. If $X$ is any non-constant random variable that is $\mathcal{F}$-measurable, it cannot be $\mathcal{H}$-measurable.
Comment: @JoseAvilez So we can still integrate the random variable $X$ on subsets in $\mathcal{H}$ but $X$ just might not be $\mathcal{H}$  measurable? What is the benefit of having a random variable which is $\mathcal{H}$-measurable if the expectations on these subsets are already defined and the integrals between $X$ and the conditional expectation of $X$ on $H$ are the same?
Comment: You may not integrate $X$ when viewed as a function from $(\Omega , \mathcal{H})$ because it is not measurable. This is the entire point of conditional expectations: to find a random variable which you *can* integrate which is $\mathcal{H}$-measurable.
Comment: @JoseAvilez Then how is this integral $\int_{H} \mathrm{E}(X \mid \mathcal{H}) \mathrm{d} P=\int_{H} X \mathrm{~d} P$ defined if $f$ is not $\mathcal{H}$ measurable?
Comment: The integral on the left treats $E(X | \mathcal{H})$ as an $\mathcal{H}$-random variable. The integral on the right treats $X$ as a $\mathcal{F}$-random variable.
Comment: @JoseAvilez I see does the expected value of the random variable $E(X|\mathcal{H})$ differ from $X$ on any subsets? It seems like these would end up being the same since we are only considering a sub sigma algebra of $\mathcal{F}$ on the LHS.
Comment: Yes. Take the following example. Let $\Omega = [0,1]$ with Lebesgue measure and $\mathcal{F}$ the Borel $\sigma$-algebra of $[0,1]$. Now set $\mathcal{G} = \{ \emptyset , \Omega , [0,0.5), [0.5,1] \}$. Let $X(\omega) = \omega$. Can you compute $E(X|\mathcal{G})$?
Comment: @JoseAvilez Well, I know what the lebesgue integral works out to be on the subsets of $\mathcal{G}$, its just the riemann integral of $x$ with bounds being the endpoints of the sets.  So I need some $\mathcal{G}$-measurable function. But what would the measure be on $\mathcal{G}$? Would I continue to use the lebesgue-measure restricted to subsets of $\mathcal{G}$? There must be some formula to commpute this because otherwise, I would have to conjecture the right function to use
Comment: $E(X | \mathcal{G})$ doesn't have a measure, it's a function. The measure of the sets in $\mathcal{G}$ is their usual Lebesgue measure.
Comment: @JoseAvilez I'm thinking $\mathcal{G}$ would be some sort of step function. On the $[0,1/2)$ the function is $1$ and on $[1/2,1]$ the function is $3/4$? Then the integrals would work out to be the same on these intervals while $\mathcal{G}$ is still measurable.
Comment: Almost! This sequence of comments was getting a bit long, so I've posted an answer.
Answer: The sequence of comments above was getting a bit long, so I'll convert it into an answer.
As noted in @William's answer, making the domain's $\sigma$-algebra smaller while keeping the co-domain's $\sigma$-algebra fixed makes it harder for a function to be measurable.
As an example, if we take any non-constant random variable $X$ on a $\sigma$-algebra $\mathcal{F}$, and then set $\mathcal{G} = \{\emptyset, \Omega\} \subset \mathcal{F}$, then we note that $X$ cannot be measurable, as only constants are $\mathcal{G}$-measurable.
As an example for computing a conditional expectation, set $\Omega = [0,1]$, $\mathcal{F} = \mathcal{B}[0,1]$, and $P = \lambda$ (Lebesgue measure). We now set $\mathcal{G} = \{\emptyset , [0,1], [0,0.5), [0.5, 1]\}$ and relativise $\lambda$ to $\mathcal{G}$ (i.e. we assign Lebesgue measure to the sets in $\mathcal{G}$). Consider the random variable $X : \Omega \to \mathbb{R}$ given by $X(\omega ) = \omega$. This is $\mathcal{F}$-measurable, but it is not $\mathcal{G}$-measurable.
Notice that $\int_{[0,0.5)} X dP = \frac{1}{8}$ and $\int_{[0.5, 1]} X dP = \frac{3}{8}$. We now wish to find a $\mathcal{G}$-measurable function that integrates to these same values. For a function to be $\mathcal{G}$-measurable, it must be constant on $[0,0.5)$ and $[0.5,1]$. Thus, we may set $$Y(\omega) = \begin{cases}
\frac{1}{4} & 0 \leq \omega < 0.5 \\
\frac{3}{4} & 0.5 \leq \omega \leq 1
\end{cases}$$
Notice that $Y$ is $\mathcal{G}$-measurable and $\int_{[0,0.5)} Y dP = \frac{1}{8}$ and $\int_{[0.5, 1]} Y dP = \frac{3}{8}$. Thus $E(X | \mathcal{G}) = Y$.
Comment: Thank you for your many comments and very instructive guidance! This helps so much for preparing for my seminar course which uses conditional expectation as assumed knowledge
Comment: @SamKirkiles Pleasure! Good luck on your seminar course.
Answer: A function $f : (X_1,\Sigma_1,\mu_1) \to (X_2,\Sigma_2,\mu_2)$ is $(\Sigma_1,\Sigma_2)$-measurable if for all $M \in \Sigma_2$, it holds that $f^{-1}(M) \in \Sigma_1$.
If $S \subset \Sigma_1$, it may hold that $f^{-1}(M)$ is not in it, which is the technical statement of the bolded text.
As a general example, suppose that $f$ is $(\Sigma_1,\Sigma_2)$-measurable, and $\varnothing \neq M \subsetneq X_2$ so that $M \in \Sigma_2$. If $S$ is any $\sigma$-subalgebra of $\Sigma_1$ which does not contain $f^{-1}(S)$, then $f$ is not $(S,\Sigma_2)$-measurable.
As a concrete example, if the trivial $\sigma$-algebra $\{\varnothing,X_1\}$ is $T$, then any nonconstant which is $(\Sigma_1,\Sigma_2)$-measurable ($\Sigma_1 \neq T$) is not $(T,\Sigma_2)$-measurable.

A high-level takeaway of this fact is that although the codomain's $\sigma$-algebra plays a vital role in measurability, it is the domain's $\sigma$-algebra that can "make it hard" to be measurable. Indeed, see that for a fixed codomain with a fixed $\sigma$-algebra, the fewer sets in the domain's $\sigma$-algebra, the harder it is for a function between those spaces to be measurable with respect to those $\sigma$-algebras.
Comment: If someone were to say to compute the conditional expected value with respect to some set, would this mean to produce a random variable? Or to produce the expected value of that random variable?


How to solve ERROR Executor - Exception in task 0.0 in stage 20.0 (TID 20)?

Question: I know similar question has been answered briefly, but i could not add my personal additional doubt there due to lack of minimum reputation...hence i am asking it here
I want to process Twitter data using Apache Spark + Kafka. I created a pattern for this. But when I run it, I get the following error. I searched a lot of places about this error, but I couldn't get the solution I wanted, or it didn't work. The last time I ran Spark with a smaller memory space, thinking memory was insufficient, but I still get the same error. This is my code that I received this error:
from kafka import KafkaConsumer
from pyspark.streaming import StreamingContext
import json
import pandas as pd
from pyspark import SparkConf,SparkContext
from pyspark.streaming.kafka import KafkaUtils

#cd /opt/hadoop-3.2.0-7/hadoop/spark      $sudo ./bin/spark-submit --packages org.apache.spark:spark-streaming-kafka-0-8_2.11:2.3.0  /opt/twitterConsumer.py

conf = SparkConf()
conf.setAppName("BDA-Twitter-Spark-Kafka")
sc = SparkContext(conf=conf)
sc.setLogLevel("ERROR")
ssc = StreamingContext(sc,1)

KafkaStream = KafkaUtils.createStream(ssc, "localhost:2181",'tks',{"xmas":1})   # directKafkaStream = KafkaUtils.createDirectStream(ssc, [topic], {"metadata.broker.list": brokers})
KafkaStream.pprint()

print("HERE1")

ssc.start()
ssc.awaitTermination()

ans my error is:
    ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.AbstractMethodError
    at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
    at org.apache.spark.streaming.kafka.KafkaReceiver.initializeLogIfNecessary(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
    at org.apache.spark.streaming.kafka.KafkaReceiver.log(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
    at org.apache.spark.streaming.kafka.KafkaReceiver.logInfo(KafkaInputDStream.scala:68)
    at org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:90)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
19/12/29 09:57:49 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
19/12/29 09:57:49 ERROR ReceiverTracker: Receiver has been stopped. Try to restart it.
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost, executor driver): java.lang.AbstractMethodError
    at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
    at org.apache.spark.streaming.kafka.KafkaReceiver.initializeLogIfNecessary(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
    at org.apache.spark.streaming.kafka.KafkaReceiver.log(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
    at org.apache.spark.streaming.kafka.KafkaReceiver.logInfo(KafkaInputDStream.scala:68)
    at org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:90)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
    at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
    at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
    at scala.Option.foreach(Option.scala:257)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
Caused by: java.lang.AbstractMethodError
    at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
    at org.apache.spark.streaming.kafka.KafkaReceiver.initializeLogIfNecessary(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
    at org.apache.spark.streaming.kafka.KafkaReceiver.log(KafkaInputDStream.scala:68)
    at org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
    at org.apache.spark.streaming.kafka.KafkaReceiver.logInfo(KafkaInputDStream.scala:68)
    at org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:90)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:149)
    at org.apache.spark.streaming.receiver.ReceiverSupervisor.start(ReceiverSupervisor.scala:131)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:601)
    at org.apache.spark.streaming.scheduler.ReceiverTracker$ReceiverTrackerEndpoint$$anonfun$9.apply(ReceiverTracker.scala:591)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.SparkContext$$anonfun$37.apply(SparkContext.scala:2212)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:121)
    at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

HOW TO MATCH VERSIONs OF ALL THE REQUIRED TOOLS HERE ?
Comment: 1) what version of Java and Scala are installed? 2) Have you tried using Structured Streaming instead? 3) Do you need Spark? Kafka Connect works from Twitter and you can remove `from kafka import KafkaConsumer` since that's just the plain Kafka Python library
Comment: 1. versions are as follows: Hadoop 3.2.0
kafka_2.12-2.3.0.jar
spark 2.3.1
java openjdk version "11.0.5"
2. actually i wanted to do real time analysis.. on the data streaming. 3. i am making this project for learning purpose...learning kafka was my semester assignment....i need to learn spark as well.. so i thought to use them both in one.. @cricket_007
Answer: The error you see likely comes from version mismatches
Hadoop and Spark require Java 8
You're using "Kafka with Scala 2.12" (Maven: kafka_2.12), so your packages must also use Scala 2.12 (Maven: spark-xyz_2.12), and also must match your Spark version (2.3.1). Your command shows you've pulled the Kafka streaming packages for Scala 2.11 for Spark 2.3.0. Also note that the Spark Streaming packages are deprecated, and you should be using spark-sql-kafka instead, Structured Streaming.
You can still do real time analysis without Spark & Hadoop
Comment: Intrigued by the last line. I was wondering over the past week that whether Spark is really needed? Or are we good with just Kafka? We intend to build a real time video stream processing pipeline which would scale to hundreds of cameras. Any comments?
Comment: @ParasJain You still need a database, for example Couchbase, Druid or Clickhouse for large TimeSeries aggregates


How come Belgium and even Switzerland are atop of Holland on FifaRankings for WorldCup 2014?

Question: Holland

reached finals at WorldCup2010;
qualified to Euro2012;
won the qualifying group for WorldCup2014 with only one draw.

Belgium

missed WorldCup2010
missed Euro2012
won the qualifying group for WorldCup2014 with two draws.

Switzerland

3rd place in group at WorldCup2010
missed Euro2012
won the qualifying group for WorldCup2014 with three draws.

Still the rankings are:
<code>1    Spain          1513
2    Germany        1311
3    Argentina      1266
4    Colombia       1178
5    Belgium        1175
6    Uruguay        1164
7    Switzerland    1138
8    Netherlands    1136
</code>
Comment: Related link: [Are Switzerland really better than Italy? FIFA's insane ranking system explained](http://eurosport.yahoo.com/blogs/pitchside-europe/switzerland-really-better-italy-fifa-insane-ranking-system-144405289.html)
Comment: I don't know the answer, but is strength of opposition taken into account in calculating rankings? If Belgium and Switzerland were in stronger qualifying groups that could explain it.
Comment: If you follow that rule, Uruguay is not even in WC yet.. and if it reach it (almost sure, seeing that it won 5-0 it first leg), it will be head of group... That is more strange...
Comment: Here's how the FIFA ranking works - http://sports.stackexchange.com/questions/2904/how-does-fifa-rank-national-teams/2909#2909
Comment: [Are Switzerland really better than Italy? FIFA's insane ranking system explained](http://www.eurosport.co.uk/football/are-switzerland-really-better-than-italy-fifas-insane-ranking-system-explained_sto4717005/story.shtml) new working link.
Answer: One key thing to keep in mind is that the FIFA ranking does not indicate how strong a team is but how well they have done.  As the ranking procedure indicates, it only uses results of past games to give a score. Citing results from 2010 is also slightly misleading since those results are divided by 5 (20%), where results from 2013 are worth significantly more. Netherlands played a lot of games that were outside of the three bullet points you've mentioned. Those friendlies made all the difference.  FIFA shows all the games Netherlands have played here.
You will see draws to Portugal, Estonia, Germany, Japan, Colombia, Italy.  Some of these were before the draw took place and ended up costing them a seed.
That being said...it was very close and could have changed even in the last 2 games of qualifying.
Answer: this link show you how the points are calculated. I don't remember how well did Netherlands did in the Euro2012, But you have to take in consideration how difficult stage group in the qualy round was, and how it is meassured according to the table. Also, take in mind, that not all the teams plays the same amount of games. So maybe Netherlands didn't play too much lately???


Calculate surface normal of each equilateral triangle in a tetrahedron

Question: How can I calculate the surface normal of each equilateral triangle that makes up a tetrahedron?
These are the xyz coordinates of the tetrahedron
(+1, +1, +1)
(−1, −1, +1)
(−1, +1, −1)
(+1, −1, −1)
Answer: Sketch-hint:

If those are the vertices, say $a,b,c,d$, then how should you find the edge vectors? That is, the displacement vectors between vertices that represent the edges' directions?
When you have two edge vectors of one of the equilateral triangle faces, how can you compute the normal vector of the face? The two vectors both lie on the face's 2D plane, so they're both perpendicular to normal, so...

It might be the case you need to make them all outward-pointing, in which case you have to keep track of the edge vectors' directions relative to each other; it might help to draw a picture, and keep in mind a certain rule for the type of calculations you'll be doing... (Also, you might want to normalize the answers, if you want unit normals.)
Answer: There are slicker ways of doing this, but...
Each side is determined by 3 points. Ignore one of the points and you're left with points which determine one of the four possible triangular sides. Then by taking differences of points you'll obtain vectors parallel to that side. Finally cross product two such vectors and you'll get a normal (then normalize).
For example: (Ignore point number 4)
$(1,1,1) - (-1,-1,1) = (2,2,0)$ and
$(1,1,1) - (-1,1,-1) = (2,0,2)$. Take the
cross product and get $(4,-4,-4)$ finally normalize and get
                      $(1/\sqrt{3})(1,-1,-1)$
Now repeat for other sides.
(if they need to be outward pointing, a quick sketch will help determine if you need to multiply by -1)