Parallel proccessing in R doParallel foreach save data

Question: Progress has been made on getting the parallel processing part working but saving the vector with the fetch distances is not working properly. The error I get is
df_Test_Fetch <- data.frame(x_lake_length)
Error in data.frame(x_lake_length) : object 'x_lake_length' not found
write.table(df_Test_Fetch,file="C:/tempTest_Fetch.csv",row.names=TRUE,col.names=TRUE, sep=",")
Error in is.data.frame(x) : object 'df_Test_Fetch' not found

I have tried altering the code below so that the foreach step is output to x_lake_length.  But that did not output the vector as I hoped.  How can I get the actually results to be saved to a csv file.  I am running a windows 8 computer with R x64 3.3.0.
Thanks you in advance
Jen
Here is the full code.
 # make sure there is no prexisting data
rm(x_lake_length)

# Libraries ---------------------------------------------------------------
if (!require("pacman")) install.packages("pacman")
pacman::p_load(lakemorpho,rgdal,maptools,sp,doParallel,foreach,
               doParallel)

# HPC ---------------------------------------------------------------------
cores_2_use <- detectCores() - 2
cl          <- makeCluster(cores_2_use, useXDR = F)
clusterSetRNGStream(cl, 9956)
registerDoParallel(cl, cores_2_use)

# Data --------------------------------------------------------------------

ogrDrivers()

dsn <- system.file("vectors", package = "rgdal")[1]
# the line below is commented out but when I run the script on my data the line below is what I use instead of the one above
# then making the name changes as needed
# dsn<-setwd("J:\\Elodea\\ByHUC6\\")
ogrListLayers(dsn)
ogrInfo(dsn=dsn, layer="trin_inca_pl03")
owd <- getwd()
setwd(dsn)
ogrInfo(dsn="trin_inca_pl03.shp", layer="trin_inca_pl03")
setwd(owd)
x <- readOGR(dsn=dsn, layer="trin_inca_pl03")
summary(x)

# Analysis ----------------------------------------------------------------
myfun <- function(x,i){tmp<-lakeMorphoClass(x[i,],NULL,NULL,NULL)
x_lake_length<-vector("numeric",length = nrow(x))
x_lake_length[i]<-lakeMaxLength(tmp,200)
print(i)
Sys.sleep(0.1)}

foreach(i = 1:nrow(x),.combine=cbind,.packages=c("lakemorpho","rgdal"))  %dopar% (
  myfun(x,i)
)
options(digits=10)
df_Test_Fetch <- data.frame(x_lake_length)
write.table(df_Test_Fetch,file="C:/temp/Test_Fetch.csv",row.names=TRUE,col.names=TRUE, sep=",")
print(proc.time())

Answer: I think this is what you want, though without understanding the subject matter I can't be 100% sure.
What I did was add a return() to your parallelized function and assigned the value of that returned object to x_lake_length when you call the foreach. But I'm only guessing that that's what you were trying to do, so please correct me if I'm wrong.
# make sure there is no prexisting data
rm(x_lake_length)

# Libraries ---------------------------------------------------------------
if (!require("pacman")) install.packages("pacman")
pacman::p_load(lakemorpho,rgdal,maptools,sp,doParallel,foreach,
               doParallel)

# HPC ---------------------------------------------------------------------
cores_2_use <- detectCores() - 2
cl          <- makeCluster(cores_2_use, useXDR = F)
clusterSetRNGStream(cl, 9956)
registerDoParallel(cl, cores_2_use)

# Data --------------------------------------------------------------------

ogrDrivers()

dsn <- system.file("vectors", package = "rgdal")[1]
# the line below is commented out but when I run the script on my data the line below is what I use instead of the one above
# then making the name changes as needed
# dsn<-setwd("J:\\Elodea\\ByHUC6\\")
ogrListLayers(dsn)
ogrInfo(dsn=dsn, layer="trin_inca_pl03")
owd <- getwd()
setwd(dsn)
ogrInfo(dsn="trin_inca_pl03.shp", layer="trin_inca_pl03")
setwd(owd)
x <- readOGR(dsn=dsn, layer="trin_inca_pl03")
summary(x)

# Analysis ----------------------------------------------------------------
myfun <- function(x,i){tmp<-lakeMorphoClass(x[i,],NULL,NULL,NULL)
                      x_lake_length<-vector("numeric",length = nrow(x))
                      x_lake_length[i]<-lakeMaxLength(tmp,200)
                      print(i)
                      Sys.sleep(0.1)
                      return(x_lake_length)
}

x_lake_length <- foreach(i = 1:nrow(x),.combine=cbind,.packages=c("lakemorpho","rgdal"))  %dopar% (
  myfun(x,i)
)

options(digits=10)
df_Test_Fetch <- data.frame(x_lake_length)
write.table(df_Test_Fetch,file="C:/temp/Test_Fetch.csv",row.names=TRUE,col.names=TRUE, sep=",")
print(proc.time())


How to call a value at a specific index of a list in python?

Question: I am trying to call a value at a specific index from a list. For an example index,
at index[2]. I can't seem how to figure how to do this. I can only get the value at the last index.
# number of arrivals per second
_lambda = 0.00923579

# number of AC arriving
_num_events = 10

_event_num = []
_inter_event_times = []
_event_times = []
_event_time = 0

for i in range(_num_events):
    _event_num.append(i)
    n = random.random()

    _inter_event_time = -math.log(1.0 - n)/_lambda # gap between successive AC in seconds
    _inter_event_times.append(_inter_event_time) # list of gaps between successive AC in seconds

    _event_time = _event_time + _inter_event_time # arrival time of AC
    _event_times.append(_event_time) # list of arrival times of AC

    print(str(i) +',' + str(_inter_event_time) + ',' + str(_event_time))

Comment: Can you be more specific of what list you're trying to access, what code you have used to attempt to access that item, what the error is...?
Comment: `nameoflist[2].methodname()`? This is *really* basic; suggest you run through a tutorial from scratch to fill glaring gaps in knowledge.
Comment: I don't understand it. What `index`? What `index[2]` ?
Answer: Please provide some more specific information on what variable in your example you want to retrieve this value. If you want to do it in loop in your last line just use:
print(str(i) +',' + str(_inter_event_times[i]) + ',' + str(_event_times[i]))
You can allways use this syntax: array[index]


Problems implementing equation for 2d elastic collision

Question: So i am trying to make a 2d ball physics simulator. Before I try to jump ahead and implement visuals i am testing how the physics are going to work, and I am stuck on implementing this equation. This is where i have gotten up to so far:
import math

b1m = 50  # ball 1 mass
b2m = 40  # ball 2 mass

b1vx = 35  # ball 1 velocity x
b1vy = 10  # ball 1 velocity y
b2vx = -15  # ball 2 velocity x
b2vy = -40  # ball 2 velocity y

b1v = math.sqrt(abs(b1vx) + abs(b1vy))  # ball 1 velocity | calculates overall velocity/speed
b2v = math.sqrt(abs(b2vx) + abs(b2vy))  # ball 2 velocity | calculates overall velocity/speed

b1ma = math.degrees(math.atan2(b1vy, b1vx)) % 360  # ball 1 movement angle | calculates the angle at
which the ball is travelling
b2ma = math.degrees(math.atan2(b2vy, b2vx)) % 360  # ball 2 movement angle | calculates the angle at
which the ball is travelling

ca = 31.5  # contact angle

b1vx = (b1v * math.cos(b1ma - ca) * (b1m - b2m) + 2 * b2m * math.cos(b2ma - ca)) / (b1m + b2m) *
math.cos(ca) + b1v * math.sin(b1ma - ca) * math.cos(ca + (math.pi/2))
b1vy = (b1v * math.cos(b1ma - ca) * (b1m - b2m) + 2 * b2m * math.cos(b2ma - ca)) / (b1m + b2m) *
math.sin(ca) + b1v * math.sin(b1ma - ca) * math.sin(ca + (math.pi/2))

print(b1vx)  # print to check if correct values | correct value should be -17.11
print(b1vy)  # print to check if correct values | correct value should be 21.94
#  according to http://www.sciencecalculators.org/mechanics/collisions/

I am not sure whats going wrong but its producing the wrong results for the variables I have asigned. I tried converting the angles to radians, as I know thats python default, I tried editing the equation slightly, seeing if the order of operation had been messed up. But still no results. I'm basing the values needed off of this website, which shows a 2d ball simulation based off of user given variables. Any help is greatly appreciated!
Answer: Thanks for the interesting question, I liked working on it.
There are 3 problems in your code:

The total velocity of both balls depends on the square root of the sum of the squared components (Pythagorean theorem). The individual components were not squared in your code
In your formula to calculate the resulting velocities, a component is missing in the numerator: b2v (in 2* m2 * v2 * cos(θ2−φ))
math.cos and math.sin bith take angular values in radians as input.

By fixing all three I was able to reproduce the expected values. However, while typing this answer I noticed that the velocity in y direction had the wrong sign. The same was true for the y component of the second ball. I checked the equations again, but was unable to spot another mistake. However, if you look at the animation of the website you linked you can see that the ball is moving towards the lower left of the animation, so x and y seem to be negative. I played with different input values and the movement direction of the ball never fitted to its presented y velocity. Unless there is some obvious sign inversion I am missing, this might be an error on that website.
Here's the code:
import math

b1m = 50  # ball 1 mass
b2m = 40  # ball 2 mass

b1vx = 35  # ball 1 velocity x
b1vy = 10  # ball 1 velocity y
b2vx = -15  # ball 2 velocity x
b2vy = -40  # ball 2 velocity y

#1 square root of squared components
b1v = math.sqrt(b1vx**2 + b1vy**2)  # ball 1 velocity | calculates overall velocity/speed
b2v = math.sqrt(b2vx**2 + b2vy**2)  # ball 2 velocity | calculates overall velocity/speed

# ball 1 movement angle | calculates the angle at which the ball is travelling
b1ma = math.atan2(b1vy, b1vx)
# ball 2 movement angle | calculates the angle at which the ball is travelling
b2ma = math.atan2(b2vy, b2vx)

ca = math.radians(31.5)  # contact angle

#3 component missing in second component of numerator of fraction (b2v)
b1vx = ((b1v * math.cos(b1ma - ca) * (b1m - b2m) + 2 * b2m * b2v * math.cos(b2ma - ca)) / (b1m + b2m)) * math.cos(ca) + b1v * math.sin(b1ma - ca) * math.cos(ca + (math.pi/2))
b1vy = ((b1v * math.cos(b1ma - ca) * (b1m - b2m) + 2 * b2m * b2v * math.cos(b2ma - ca)) / (b1m + b2m)) * math.sin(ca) + b1v * math.sin(b1ma - ca) * math.sin(ca + (math.pi/2))

b2vx = ((b2v * math.cos(b2ma - ca) * (b2m - b1m) + 2 * b1m * b1v * math.cos(b1ma - ca)) / (b2m + b1m)) * math.cos(ca) + b2v * math.sin(b2ma - ca) * math.cos(ca + (math.pi/2))
b2vy = ((b2v * math.cos(b2ma - ca) * (b2m - b1m) + 2 * b1m * b1v * math.cos(b1ma - ca)) / (b2m + b1m)) * math.sin(ca) + b2v * math.sin(b2ma - ca) * math.sin(ca + (math.pi/2))

print(b1vx)  # print to check if correct values | correct value should be -17.11
print(b1vy)  # print to check if correct values | correct value should be 21.94

print(b2vx)
print(b2vy)

#  according to http://www.sciencecalculators.org/mechanics/collisions/

Comment: Thanks so much for the help. I can finally implement it. I believe, the final problem you were talking about may be because the coordiantes system on the website i linked, starts (0,0) from bottom left. But either way, I have put it in my code, and it works perfectly thanks to you.


A question about conic sets of functionals.

Question: The problem is the following.
Let $(X,\|\cdot\|)$ be a normed space and let $C \subseteq X$ be a closed convex set with nonempty interior.
Let be $x\in C$. I define to be a "normal cone to $C$ in $x$" the set:
$N_C(x):=\{ f \in X^\ast : \langle f,y-x\rangle \leq 0, \forall y \in C \}$
I would like to prove that if $x \in \partial C$, then $N_C(x)$ has at least a nonzero functional.
I have no idea about where to start... I've tried something with the geometric forms of Hahn Banach for separating the convex sets, but probably is not the right way to follow.
Any help is appreciated!
Answer: I found the answer myself after I started the bounty and I lost the reputation.
Now I'm feeling stupid!
Well, if the interior $int(C)$ is convex, open (by definition of interior), and nonempty, and $x_0$ is a point such that $x_0 \notin int(C)$, then by the geometric form of Hahn Banach it's possible to find a functional $f \in X^\ast$ such that $f(y)<\alpha \leq f(x_0)$ for every $y \in int(C)$.
Then $f(y-x)=\langle f,x-y\rangle < 0$ for every $y \in int(C) \neq \emptyset$. Then $f \neq 0$.
The fact that the bounty is not refunded if you find the answer by yourself may discourage some people in trying to find the answer after the bounty is placed. Do you agree?
Comment: here is +15 rep.


Why entity autoloading doesn't work?

Question: I'm trying to set up entity autoloading using hook_menu like this:
    $menu['e-shop/produkt/detail/%commerce_datasheet'] = array(
    'title' => t('Products'),
    'page_callback' => 'commerce_products_datasheet_detail',
    'page_arguments' => array(3),
    'access callback' => TRUE,
     );

function commerce_datasheet_load($datasheet_id) {
  return entity_load_single('commerce_datasheet',$datasheet_id);
}

function commerce_products_datasheet_detail($entity, $view_mode='full') {
    dpm($entity);
}

dpm() function doesn't return anything, so I assume, that $entity is empty.
I am using latest versions on Drupal 7 and Entity API module.
Comment: It's `page callback` and `page arguments`, no underscores
Comment: @Clive: Thanks, you're right, of course. I still wasn't able to solve my problem, but at least this is corrected.
Comment: Oh, you probably need to return something from the page callback too to invoke the theme handler. Your menu/load stuff looks fine to me, that should be working
Comment: @Clive: Yes, you're right. Details are in the answer, if you're interested in my mistake...
Answer: It was my mistake. I was using another module, which provided page callback with higher priority. For the next time - it's advisable to install devel module and go to devel/menu/item?path=<uri> where used page callback can be clearly seen.

Why entity autoloading doesn't work?


SQL View/Query to return sum by current month

Question: Sorry for such a newbie question but I am that a newbie.
SELECT     SUM([Sales Total]) AS Sales, Salesperson, Date
FROM         dbo.[Sales by Salesperson]
GROUP BY Date, Salesperson

Trying to return only the SUM of Sales by Salesperson by current month.
Example:
Jaime Smith   | 1,000,000
John Doe      | 1,200,000

I have the data pulling for the Salesperson but don't know how to show only the current month.
Comment: Your DB is SQL Server? Just want to confirm?
Answer: Try this...
SELECT SUM([Sales Total]) AS Sales,
    Salesperson,
    YEAR(Date) [Year],
    MONTH(Date) [Month]
FROM dbo.[Sales by Salesperson]
WHERE YEAR(Date) = YEAR(GETDATE()) AND MONTH(Date) = MONTH(GETDATE()) /*To filter by month*/
GROUP BY Salesperson, YEAR(Date), MONTH(Date)

Answer: try this
SELECT SUM([Sales Total]) AS Sales, Salesperson, Date
FROM dbo.[Sales by Salesperson]
where year(date) = year(getdate())
and month(date) = month(getdate())
GROUP BY Date, Salesperson

Comment: Grouping by date will divide the result by the day part too and so will not work.
Comment: Oh Yes, you are right. should be group by year(date), month(date)


In mathematics, what is meant by "Mean Reversion"?

Question: I am trying to understand the concept of "Mean Reversion" from a mathematical perspective. For example, suppose we are given a generic Stochastic Process - how do we know if this process is "mean reverting" or not?
Part 1: As an example, consider the following Stochastic Processes:

ARIMA(p, d, q) Process:

$$X_t = c + \sum_{i=1}^{p} \phi_i X_{t-i} + \sum_{i=1}^{q} \theta_i \epsilon_{t-i} + \epsilon_t$$

p (order of the autoregressive part)
d (degree of first differencing involved)
q (order of the moving average part).
$\epsilon_t$ is white noise with zero mean and constant variance.

Brownian Motion:
A Weiner Process $W_t$ is a stochastic process that satisfies the following properties - a Browian Motion is the cumulative sums of the Weiner Process:

$W_0 = 0$
The increments are independent and normally distributed: $W_t - W_s \sim N(0, t-s)$ for $0 \leq s < t$.

Ornstein-Uhlenbeck Process:
The Ornstein-Uhlenbeck process is a stochastic process that satisfies the following stochastic differential equation:

$$dX_t = \theta(\mu - X_t)dt + \sigma dW_t$$

$\theta > 0$ is the rate of reversion to the mean
$\mu$ is the mean value, $\sigma$ is the standard deviation
$W_t$ is a Wiener process.

If I just look at the three equations, I would not be able to know which of these processes are mean reverting and which of them are not.
Part 2: Using the R programming language, we can plot these a single trajectory for these 3 processes:
library(ggplot2)

n <- 1000

# ARIMA(1,0,0) Process
set.seed(123)
arima_process <- arima.sim(n, model=list(ar=0.6))

# Brownian Motion
set.seed(123)
brownian_motion <- cumsum(rnorm(n))

# Ornstein-Uhlenbeck Process
set.seed(123)
theta <- 1; mu <- 0; sigma <- 1
ou_process <- numeric(n)
for (t in 2:n) {
  ou_process[t] <- ou_process[t-1] + theta*(mu - ou_process[t-1]) + sigma*rnorm(1)
}

df <- data.frame(Time = 1:n,
                 ARIMA = arima_process,
                 Brownian = brownian_motion,
                 OU = ou_process)

ggplot(df, aes(Time)) +
  geom_line(aes(y = ARIMA, color = "ARIMA")) +
  geom_line(aes(y = Brownian, color = "Brownian Motion")) +
  geom_line(aes(y = OU, color = "Ornstein-Uhlenbeck")) +
  labs(x = "Time", y = "Value", color = "Process") +
  theme_minimal()

Clearly, it looks like the Ornstein-Uhlenbeck Process and the ARIMA Process "hover" around the mean whereas the Brownian Motion does not. Does this mean the Brownian Motion is not mean reverting?
I also plotted multiple simulations for each process:
df_arima <- data.frame(Time = rep(1:n, n_simulations), Value = numeric(n * n_simulations))
df_brownian <- data.frame(Time = rep(1:n, n_simulations), Value = numeric(n * n_simulations))
df_ou <- data.frame(Time = rep(1:n, n_simulations), Value = numeric(n * n_simulations))

set.seed(123)
for (i in 1:n_simulations) {
    # ARIMA(1,0,0) Process
    arima_process <- arima.sim(n, model=list(ar=0.6))
    df_arima$Value[(i-1)*n + 1:n*i] <- arima_process

    # Brownian Motion
    brownian_motion <- cumsum(rnorm(n))
    df_brownian$Value[(i-1)*n + 1:n*i] <- brownian_motion

    # Ornstein-Uhlenbeck Process
    theta <- 1; mu <- 0; sigma <- 1
    ou_process <- numeric(n)
    for (t in 2:n) {
        ou_process[t] <- ou_process[t-1] + theta*(mu - ou_process[t-1]) + sigma*rnorm(1)
    }
    df_ou$Value[(i-1)*n + 1:n*i] <- ou_process
}

# Plots
p_arima <- ggplot(df_arima, aes(Time, Value)) +
    geom_line(alpha = 0.1, color = "blue") +
    labs(x = "Time", y = "Value", title = "(Multiple) ARIMA(1,0,0) Processes") +
    theme_minimal()

p_brownian <- ggplot(df_brownian, aes(Time, Value)) +
    geom_line(alpha = 0.1, color = "red") +
    labs(x = "Time", y = "Value", title = "(Multiple) Brownian Motions") +
    theme_minimal()

p_ou <- ggplot(df_ou, aes(Time, Value)) +
    geom_line(alpha = 0.1, color = "green") +
    labs(x = "Time", y = "Value", title = "(Multiple) Ornstein-Uhlenbeck Processes") +
    theme_minimal()

I might be hallucinating, but I can see some faint "bands" in the Brownian Motion simulations which might be illustrating the same concept of non-mean reversion?
My Question: From a strict mathematical perspective, just by looking at the mathematical equation for a stochastic process - is it possible to determine if the process is mean-reverting or non-mean reverting?
Thanks!

Note: Does "mean reversion" have anything to do with "memoryless property" or "stationarity"?
Comment: I think the bands are a graphical artifact. Are they still there if you zoom in? As for telling if a process is mean reverting at a glance, you can at least check that the coefficient of the $X_t$ term of $dX_t$ is negative. This is prevents X from getting too big or too negative.
Answer:
Note: Does "mean reversion" have anything to do with "memoryless property" or "stationarity"?

Mean reversion refers to the concept that given some observation of a random variable, the next event is more likely to be closer to the mean that this one was. In other words, given an extreme event, the next event is more likely to be less extreme. So, kind of to the memoryless property, in some sense, but generally not really. For stationarity yes! If it's variance or mean is not constant over time, then the memoryless property won't really happen in terms of values and not relative portions.
Since stationarity demands that we have a constant mean and variance over time, and random variable/stoachastic process/time series that obeys stationarity also obeys the mean reversion principle. They are pretty closely tied together, and they may be synonymous assuming finite variance and mean, but I'm unsure.
You ask why Brownian motion does not have  mean reversion. We can see this very easily in the definition, where:
$$W_t - W_s \thicksim \mathcal{N}(0, t-s)$$
The variance of our Brownian motion compared to the start is increasing over time. Mean reversion works because "as you move away from the mean, the proportion of the distribution that lies closer to the mean than you do increases continuously."  but this doesn't happen when our variance is changing at a linear rate. Less amount of the distribution lies closer to the mean than it did $1$ weiner process ago (heuristically speaking), so it doesn't make sense that we should be more likely to fall within the same bounds.
Answer: Mean reversion has nothing to do with the memoryless property but, if you prefer my definition of a mean-reverting process, then it is very related to stationarity,
In the hedge fund world, people use the term "mean reverting" strategy which means that the strategy waits for the return ( or price ) of the stock to "come back" from its "unusual state". So, interpret that as you wish but, it basically means "Is there a long term mean that the process will eventually return to no matter how far process goes away from that long term mean ?".
So, by the definition above (which is my definition and not necessarily everyone's), any stochastic process which has a long term ( and constant ) mean, ( along with a constant variance ) could be could be considered a mean reverting process.
In your question, you mentioned the OU and Brownian Motion (BW) and the AR(1).
The AR(1) should exhibit mean reversion ( to zero ) because the long term mean is zero and the process is stationary.
BW cannot be mean-reverting because it doesn't have a stationary mean.
In fact, the process itself doesn't have a long term mean. If you simulate
a BW and it looks like it's mean reverting, then that's purely random
and not a characteristic of BW.
OU on the other hand is mean reverting. You may be able to see why it's mean reverting better in the discrete version of it which is an AR(1) with certain parameters. I'll see if I can find something and include it here.
So, to try to answer your question, if you have an equation for a stochastic process and the process is stationary, one can technically categorize that process as mean reverting. OU is a specific mean reverting process ( continuous also ) where the mean that the process reverts to is specified in the model along with the speed of reversion. This is very convenient but one can achieve the same type of behavior using the discrete AR(1) counterpart of the OU.
ADDENDUM: 12-28-2023
#===============================================================
Below is a thread where the discrete AR(1) is derived from the OU.
I didn't look at it closely and it seems to use a different parameterization than you used, but I just wanted to make the point that the AR(1) can do the same thing that the OU does in continuous time, as long as it's stationary. OU looks nicer though because the parameters have a very clear interpretation.
How to approximate an AR(1) with an Ornstein-Uhlenbeck Process
Comment: thank you so much for your answer. Reading your comment :"BW cannot be mean-reverting because it doesn't have a stationary mean. In fact, the process itself doesn't have a long term mean." .Just by looking at the equation for a Brownian Motion - how can I know that the mean of the Brownian Motion is non-stationary? Is there some analysis I can do to see this? Thank you so much!
Comment: Hi: Take an increment of Brownian motion so, say $(X_{t_{1}} - X_0)$. The variance of this increment is $\sigma^2 \times t_{1}$. Now, suppose we increase the same increment so consider say $(X_{t_{2}} - X_0)$ where $t_{2} > t_{1}$.  The variance of this increment is $\sigma^2 \times t_{2}$. So, as the increment gets bigger and bigger , the variance gets larger.and larger  There's probably a more mathematical way of saying this but, as the variance increases without bound, the mean becomes non-existent. The process, when viewed as one long increment,  is not stationary.
Comment: @stats_noob : I just read what Robert Murray wrote below and I think we are kind of saying the same thing as far as why BM can't be stationary.  Particular overlap is near the end of his statement.
Comment: "BW cannot be mean-reverting because it doesn't have a stationary mean". What do you mean it doesn't have a 'stationary mean'? The mean of Brownian motion is zero for all $t$, so it certainly exists.
Comment: Hi Jose: brownian motion has a mean (E(W_t|F_t) = 0  but an long term equilibrium doesn't exist. By this I mean that, once the process goes away from zero, there is no tendency for the overall long term mean to go back towards zero.  So, I shouldn't have said the mean doesn't exist but rather than brownian motion does not have an equilibrium. Maybe that's due to the increasing variance as time passes.
Comment: Jose: You are saying that E(W_t) = 0 which is true. But, if one takes a brownian motion increment ( say t-s ) and then allows that increment to get longer and longer and longer so that $t$ continues to increase,  then what's the E(W_t) as the increment increases with $t$ ?
Comment: For example, suppose one has the increment (t - s) and assume that $W_s = 0$ but during the increment, it happens that at time say k where $s < k < t$  we are given that $W_k = 1$.  Then, now what is E(W_t) for t > k ? It's W_k. So, the process just takes on whatever value it takes on with no tendency to a long term equibrium, whether zero or some other value. Formally, I don't know what this is called but it's some form of non-stationarity for sure.
Comment: Jose: In case you're interested,  here's a nice document on brownian motion. She defines it two different ways and then says that they are equivalent. I have the impression that  Definition 2 is more the way you think of it and Definition 1 is definitely more the way I think of it. https://cims.nyu.edu/~holmes/teaching/asa19/handout_Lecture6_2019.pdf  She says that the two ways are equivalent. The thing for me is that the " scaled random walk" nature of brownian motion gets lost and, for me, it's the random walk nature of BM that causes the non-stationarity.
Comment: Formally, yes, the non-stationarity is due to the variance increasing with $t$. But the unit root process that is kind of hidden by the fact that $\Delta t \rightarrow 0$ is ( for me ) really what causes the non-stationarity of BM.
Comment: @markleeds thank you for your comments. I think I now understand what you mean. For mean reversion you want, informally, that $E(X_t \, | \, F_s)$ to be closer to some "long-run" mean $\mu$ than $X_s$. Indeed, for Brownian motion you don't have this because $E(B_t \, | \, F_s) = B_s$, which is not closer to a long-run mean.
Comment: EXACTLY. I like to think of the BM process ( informally  because it's not correct) as  $W_{t+\Delta_t} = W_{t} + \epsilon_{t+\Delta_t}$ where $\Delta_t$ is really, really, really, teeeny.  And we all know that unit root processes don't have a true mean. All the best and thanks for discussion.
Answer: Basically, it means that over the long term, experimental probabilities revert to their theoretical "mean" probability or expected value.  For example, let's say a basketball player is a 75% free-throw shooter.  One game, he makes 72% of his free-throws, the next game he makes 87% of his free-throws, and the third game he makes 77% of his free-throws.  The 3-game average is 78.67%.  Assuming he stays at his same skill level, the more games he plays, the closer his average over all the games will approach 75%, regardless of whether he has a "hot streak" or a "cold streak".  Same thing with recorded weather measurements in a given climate, etc.


How to compare two coefficent estimates in a multilevel model?

Question: How can I compare two coefficient estimates from a lme4 or nlme model?
In HLM7 (software I'm more familiar with) such comparisons are evaluated with
$\chi^2$ tests. I can't find out what function in lme4, nlme or just base can do the same.
An imaginary example I'm working with, is a modified sleepstudy dataset - with two score variables added at level 2. The model being tested is this:
Level1:
$Reaction = \beta_0 + r$
Level2:
$\beta_0 = \gamma_{00} + \gamma_{01}(score1) + \gamma_{02}(score2) + u_{0j}$
I'm looking for a $\chi^2$ comparing both level 2 coefficients.
Here is my modified dataset:
library(lme4)

set.seed(123)
score1=NULL
score2=NULL
for(i in 1:18) {
      x1 = rep(sample(1:5, size = 1), 10)
      score1 = c(score1, x1)
      x2 = rep(sample(1:5, size = 1), 10)
      score2 = c(score2, x2)
}

sleepstudy$score1 <- score1
sleepstudy$score2 <- score2

model <-lmer(Reaction ~ score1 + score2 + (1|Subject), data=sleepstudy)

P.S. if this kind of questions should be posted on SO please let me know so that I move it there
Comment: you're basically looking to compute a **contrast** here.  You can do it by hand or possibly via one of many packages (`emmeans`, `contrast`, `multcomp`) - although I agree that  a description plus worked example here would be good (when I have a chance to get around to it)
Comment: Please don't cross-post to `firstname.lastname@example.com` and here, unless you've waited for a while (say a day or two) for an answer without receiving a response - although thank you for including the link in your cross-posting.  Where would you rather have your answer?
Comment: I'm sorry for that. I won't do it again - as for the answer here is just fine. Where ever is easier for you. Thank you
Comment: Ben - I initially thought that I'm good to go with your hint on contrasts. Problem is - from what I read in `?help` for given packages I don't see any scenario where two continuous predictors, actually their coefficient estimates are compared. I might be missing something basic here...
Answer: With suggestions given by @Ben Bolker on computing contrasts I found what I was looking for, although in an other package.
In order to compare two fixed effects one can use the linearHypothesis function from car package:
library(car)
linearHypothesis(model, "score1=score2")
Linear hypothesis test

Hypothesis:
score1 - score2 = 0

Model 1: restricted model
Model 2: Reaction ~ score1 + score2 + (1 | Subject)

  Df  Chisq Pr(>Chisq)
1
2  1 2.8187    0.09317 .
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1


CSS background with image and gradient, how to write in multiple lines ?

Question: I'd like to express the following CSS property definition broken out to each  individual properties.
This is the one-liner property definition which does work :
div{
  background: url('packages/select2/select2.png') no-repeat 100% -22px, linear-gradient(to bottom, #fff 85%, #eee 99%) 0 0;
}

I tried this but it doesn't work:
div{
    background: linear-gradient(to bottom, #fff 85%, #eee 99%) 0 0;
    background-image: url('packages/select2/select2.png');
    background-repeat: no-repeat 100% -22px;
}

Answer: Edit:
Actually, it's not possible to layer things the way you want to by defining the individual properties. Since you're trying to position and repeat layers differently, the only way to do it is with a single-line background definition. Otherwise you'd just override previous property definitions with later ones. It would work if you had a single background-position property for all layers, but not otherwise.
Either use a single-line background definition with commas between the layers, or break the background into multiple elements and layer them in the DOM.
Comment: I tried your solution, but the background image is in the wrong position.
Comment: You're right, because what you want is impossible hehe. See my edit.
Comment: Turns out I don't need the linear-gradient,  so if I break out these properties it seems to work : `background-image: url('packages/select2/select2.png')
    background-repeat: no-repeat
    background-position: 100% -22px`
Comment: @looshi yep it's perfectly valid to define properties individually when you have a single layer.
Comment: Sure it's possible...just comma separate them.
Comment: @jcd What do you mean by 'layer' ?
Comment: Your original one-liner property definition has two layers, the image and the gradient.


Case sensitive Directory.Exists / File.Exists

Question: Is there a way to have a case sensitive Directory.Exists / File.Existssince
Directory.Exists(folderPath)

and
Directory.Exists(folderPath.ToLower())

both return true?
Most of the time it doesn't matter but I'm using a macro which seems not to work if the path doesn't match cases 100%.
Comment: I know, that's why I'm asking...
Comment: MSDN clearly noted this: "The path parameter is not case-sensitive.", see http://msdn.microsoft.com/en-us/library/system.io.directory.exists.aspx
Comment: I know this is 10 years later and maybe the document has changed, but it specifically says "The case-sensitivity of the path parameter corresponds to that of the file system on which the code is running. For example, it's case-insensitive on NTFS (the default Windows file system) and case-sensitive on Linux file systems." meaning it can be case sensitive.
Answer: Since Directory.Exists uses FindFirstFile which is not case-sensitive, no. But you can PInvoke FindFirstFileEx with an additionalFlags parameter set to FIND_FIRST_EX_CASE_SENSITIVE
Comment: Sorry, might be a dump question, but is this working for C#? All your references relate to C++ *confused*
Comment: Yes it should do, you can use P/Invoke to call out to Win32 from C#, read up on P/Invoke and then check out the link to pinvoke.net
Comment: +1. @theknut, PInvoke link gives you detail on how to make it callable in C#/other .Net languages.
Comment: I tried it and it doesn't really work for my string. It is D:\Dir1\dir2\Dir3 (physical dir on drive is D:\Dir1\Dir2\Dir3). When I try FindFirstFile it will only return me Dir3. FindFirstFileEx(bla, FINDEX_INFO_LEVELS.FindExInfoBasic, out findData, FINDEX_SEARCH_OPS.FindExSearchLimitToDirectories, IntPtr.Zero, FIND_FIRST_EX_CASE_SENSITIVE);
Comment: Does [this](http://stackoverflow.com/questions/2239872/how-to-get-list-of-folders-in-this-folder/2241811) help? If not post your code and will try to figure it out together.
Comment: This doesn't help either. It wil still return Dir3 and FindNextFile returns false. The code I used is the exact one before your comment and the path is D:\Dir1\dir2\Dir3 to be compared with D:\Dir1\Dir2\Dir3.
Comment: What does your `bla` equal to?
Comment: `bla = "D:\Dir1\Dir2\Dir3"` but the directory on my drive is `"D:\Dir1\dir2\Dir3"`. This is why I need to compare them on cases.
Comment: In [example](http://stackoverflow.com/questions/2239872/how-to-get-list-of-folders-in-this-folder/2241811)
**FindFirstFileEx** is used to get _list of folders in this folder_. In your case if your `lpFileName="D:\\"` It should return "Dir1","dir2" and "Dir3". Please confirm if that's the case
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/28808/discussion-between-jacob-seleznev-and-theknut)
Answer: Try this function:
public static bool FileExistsCaseSensitive(string filename)
{
    string name = Path.GetDirectoryName(filename);

    return name != null
           && Array.Exists(Directory.GetFiles(name), s => s == Path.GetFullPath(filename));
}

Update:
As stated in comments, this only check cases in filename, not in the path. This is because GetFullPath method doesn't return the Windows original path with original cases, but a copy of the path from the parameter.
Ex:
GetFullPath("c:\TEST\file.txt") -> "c:\TEST\file.txt"
GetFullPath("c:\test\file.txt") -> "c:\test\file.txt"

All methods I tried work the same way: Fileinfo, DirectoryInfo.
Here is a solution using a kernel32.dll method:
    [DllImport("kernel32.dll", CharSet = CharSet.Auto)]
    public static extern int GetLongPathName(
        string path,
        StringBuilder longPath,
        int longPathLength
        );

    ///
    /// Return true if file exists. Non case sensitive by default.
    ///
    /// <param name="filename"></param>
    /// <param name="caseSensitive"></param>
    /// <returns></returns>
    public static bool FileExists(string filename, bool caseSensitive = false)
    {
        if (!File.Exists(filename))
        {
            return false;
        }

        if (!caseSensitive)
        {
            return true;
        }

        //check case
        StringBuilder longPath = new StringBuilder(255);
        GetLongPathName(Path.GetFullPath(filename), longPath, longPath.Capacity);

        string realPath = Path.GetDirectoryName(longPath.ToString());
        return Array.Exists(Directory.GetFiles(realPath), s => s == filename);
    }

Comment: It works here: If you have 'c:\temp\TEST.txt', FileExistsCaseSensitive(@"c:\temp\test.txt") will return false
Comment: This is working for me. (tested on windows platform)
Comment: Problem with this is, it's only case sensitive for the _file name_, not for the whole path. Example: Testing the method with `c:\tEst\test.txt` and `c:\test\test.txt`, while the real path on Windows is: `c:\Test\test.txt`, both return `true`. I except both to return `false` (case-sensitivity for the whole path).
Comment: Right. I only use the filename part. Anyway, I updated the answer with a fully working solution.
Answer: Try these 2 simpler options that do not need to use PInvoke and return a nullable Boolean (bool?). I am not a subject expert so I do know if this is the most efficient code but it works for me.
Simply pass in a path and if the result is null (HasValue = false) no match is found, if the result is false there is an exact match, otherwise if true there is a match with a difference case.
The methods GetFiles, GetDirectories and GetDrives all return the exact case as saved on your file system so you can use a case sensitive compare method.
NB: for the case where the path is an exact drive (e.g. @"C:\") I have to use a slightly different approach.
using System.IO;
class MyFolderFileHelper {
    public static bool? FileExistsWithDifferentCase(string fileName)
    {
        bool? result = null;
        if (File.Exists(fileName))
        {
            result = false;
            string directory = Path.GetDirectoryName(fileName);
            string fileTitle = Path.GetFileName(fileName);
            string[] files = Directory.GetFiles(directory, fileTitle);
            if (String.Compare(files[0], fileName, false) != 0)
                result = true;
        }
        return result;
    }

    public static bool? DirectoryExistsWithDifferentCase(string directoryName)
    {
        bool? result = null;
        if (Directory.Exists(directoryName))
        {
            result = false;
            directoryName = directoryName.TrimEnd(Path.DirectorySeparatorChar);

            int lastPathSeparatorIndex = directoryName.LastIndexOf(Path.DirectorySeparatorChar);
            if (lastPathSeparatorIndex >= 0)
            {
                string baseDirectory = directoryName.Substring(lastPathSeparatorIndex + 1);
                string parentDirectory = directoryName.Substring(0, lastPathSeparatorIndex);

                string[] directories = Directory.GetDirectories(parentDirectory, baseDirectory);
                if (String.Compare(directories[0], directoryName, false) != 0)
                    result = true;
            }
            else
            {
                //if directory is a drive
                directoryName += Path.DirectorySeparatorChar.ToString();
                DriveInfo[] drives = DriveInfo.GetDrives();
                foreach(DriveInfo driveInfo in drives)
                {
                    if (String.Compare(driveInfo.Name, directoryName, true) == 0)
                    {
                        if (String.Compare(driveInfo.Name, directoryName, false) != 0)
                            result = true;
                        break;
                    }
                }

            }
        }
        return result;
    }
}

Answer: Based on the solution of this question, I wrote the code below which is case sensitive for the whole path except the Windows Drive letter:
 static void Main(string[] args)
    {
        string file1 = @"D:\tESt\Test.txt";
        string file2 = @"d:\Test\test.txt";
        string file3 = @"d:\test\notexists.txt";

        bool exists1 = Case_Sensitive_File_Exists(file1);
        bool exists2 = Case_Sensitive_File_Exists(file2);
        bool exists3 = Case_Sensitive_File_Exists(file3);

        Console.WriteLine("\n\nPress any key...");
        Console.ReadKey();
    }

   static bool Case_Sensitive_File_Exists(string filepath)
   {
        string physicalPath = GetWindowsPhysicalPath(filepath);
        if (physicalPath == null) return false;
        if (filepath != physicalPath) return false;
        else return true;
   }

I copied the code for GetWindowsPhysicalPath(string path) from the question
  [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
    static extern uint GetLongPathName(string ShortPath, StringBuilder sb, int buffer);

    [DllImport("kernel32.dll")]
    static extern uint GetShortPathName(string longpath, StringBuilder sb, int buffer);

    protected static string GetWindowsPhysicalPath(string path)
    {
        StringBuilder builder = new StringBuilder(255);

        // names with long extension can cause the short name to be actually larger than
        // the long name.
        GetShortPathName(path, builder, builder.Capacity);

        path = builder.ToString();

        uint result = GetLongPathName(path, builder, builder.Capacity);

        if (result > 0 && result < builder.Capacity)
        {
            //Success retrieved long file name
            builder[0] = char.ToLower(builder[0]);
            return builder.ToString(0, (int)result);
        }

        if (result > 0)
        {
            //Need more capacity in the buffer
            //specified in the result variable
            builder = new StringBuilder((int)result);
            result = GetLongPathName(path, builder, builder.Capacity);
            builder[0] = char.ToLower(builder[0]);
            return builder.ToString(0, (int)result);
        }

        return null;
    }

Note the only problem I found with this function is, the drive letter seems to be always in lowercase. Example: The physical path on Windows is: D:\Test\test.txt, the GetWindowsPhysicalPath(string path)function returns d:\Test\test.txt
Comment: Id recommend making a field for `[ThreadStatic]StringBuilder _builder=new StringBuilder(255);` and thereby avoid adding an allocation to each call. Actually.. eliminating allocations would be a bit bigger change than just that... hm.
Answer: Here's a relatively simple way to check if a directory actually exists as named. I needed this because I have an option to rename a folder, and it checks that there won't be a clash first. So, for example, if I want to rename the folder "cars" to "Cars".
It's pretty straight-foward really. If the system reports that the folder exists, then I just call GetDirectories on the parent folder, and passing the directory name as the wildcard (thus returning exactly 1 result). Just a simple comparison gives me the answer.
static public bool DirExistsMatchCase(string path)
{
    // If it definitely doesn't return false
    if (!Directory.Exists(path)) return false;

    // Figure out if the case (of the final part) is the same
    string thisDir = Path.GetFileName(path);
    string actualDir = Path.GetFileName(Directory.GetDirectories(Path.GetDirectoryName(path), thisDir)[0]);
    return thisDir == actualDir;
}

Answer: If the (relative or absolute) path of your file is:
string AssetPath = "...";

The following ensures that the file both exists and has the correct casing:
if(File.Exists(AssetPath) && Path.GetFullPath(AssetPath) == Directory.GetFiles(Path.GetDirectoryName(Path.GetFullPath(AssetPath)), Path.GetFileName(Path.GetFullPath(AssetPath))).Single())
{
}

Enjoy!


String bending on acoustic

Question: Any tips on bending notes on an acoustic? Its really hard to do and my string loses it sound really fast.
Should I even try it?
Bending 1 semitone is easy, after that its quite hard...
edit: By losing it sound I mean that I can't hear the sound after a short while
Comment: Nylon or steel stringed?
Comment: I suppose it would be nasty to suggest using a fretless instrument (signed, a cellist) :-)
Answer: Bending strings on a steel stringed acoustic guitar can be done to good effect. It works on a nylon stringed too, but is less common.
You can try a lighter string gauge to make it easier. Especially steel stringed guitars are usually stringed with pretty brutal gauges. All major string manufacturers have lighter gauge acoustic strings. For nylon stringed guitar there's less tensions to choose from, but you can get lighter strings there too.
It's like everything else, practice makes perfect. There are special devices for training the strength of the fingers, I'm using this one. Don't know if it makes a big difference or not, but it doesn't seem to hurt. With the right job, you can improve your strength on working time.
Comment: +1 on lighter string gauge to make bending (and playing) easier. Check [this question and its answers](http://music.stackexchange.com/questions/24309/what-type-of-strings-to-use-for-easier-bending-in-fast-solos) for more about strings and bending (it's about electric guitar, but the same rules apply more or less). Try a set of .010" (extra light) or .011" (light) if you play steel. Nylon in general is much easier to bend, but not as common.
Comment: Acoustic guitars also like electric guitar strings, which have more choice of gauges, and are often slightly cheaper.
Comment: For clarity: Nylon stringed acoustic guitars do *not* like electric guitar strings, as they will break.
Answer: Since it hasn't been mentioned (my apologies if I missed it), I'd like to point out that you can be very effective bending only a 1/2 musical step. Wider bends (like a whole step bend) are also easier near the middle of the string span, rather than down where you are playing open chords. So look for places within the scales of the chords you are playing where 1/2 step fits, and you may be surprised. how effective it is. For faking wider bends, you can also momentarily slide up one or two half steps, and quickly bend another 1/2 step from there. Its not really "faking" though... its just a technique. There's no such thing as 'cheating" here. :-)
Answer: Another "trick" you can do is tune down the guitar (I recommend 1 or 1/2 tone, more may require neck adjustments) and use, if you want, a capo; the strings will have less tension and will be easier to bend. You can also experiment alternative tunings, which are usually softer than standard.
However, difficult bendings are only a matter of exercise, keep on practicing and you will be able to do it in any gauge or tuning.
Answer: I would change string gauges and see what you think. If sou do decide to do this I would recommend a slight tightening on the truss rod.
Comment: You mean loosening right? Lighter strings > easier bending > less tension > loosen truss rod.
Answer: If you're using metal (steel) strings, acoustic strings are generally thicker than electric guitar strings. For that reason, string bending is more common with electric guitarists.
The gauge of the string makes a difference in that lighter strings are physically easier to bend (less metal to move about) and are more responsive in that you don't have to bend them so far to get the pitch to rise.
You could try lighter gauge strings but the reason they're thicker on acoustic guitars is that it's the string that generates the sound volume, so the more metal you're moving about, the more sound you'll make. You're likely to find that lighter gauge strings are quieter.
An alternative is to slide up the frets instead. For example if you were going to bend a G up to an A, try just playing the G note and slide the finger along the frets up to an A. You'll get the sound of the note moving up one semitone (a fret) at a time but yif you're playing it quickly, it's not too noticable.
It's a bit fiddly at first but once you get used to it, it can be quite effective.
If you're using nylon strings, you'll find an odd effect comes into play which is that they're generally a lot less responsive to being bent than steel strings are. Again the gauge of the strings  plays a part but here the 'sliding' technique might prove a better option.
And yes. . it's hard to convincingly bend acoustic guitar strings !
You are not alone there

String bending on acoustic


promoting of your own content

Question: So far I haven't see any guidelines on how we want to deal with promoting your own content. Some questions that come to mind are:

Any etiquette we should follow?
Is it OK to link to your own blog to provide background information
to an answer?
Is it OK to mention your own extensions?
Should there be a distinction between freely available and commercial    extensions?
Comment: http://magento.stackexchange.com/faq#promotion
Comment: Personally I feel it's a bit shady to be writing blog posts as a response to a question and posting that as the source of your solution.
Answer: Straight from the FAQ; this is network policy:

May I promote products or websites I am affiliated with here?
Be careful, because the community frowns on overt self-promotion and tends to vote it down and flag it as spam. Post good, relevant answers, and if some (but not all) happen to be about your product or website, so be it. However, you must disclose your affiliation in your answers.
If a huge percentage of your posts include a mention of your product or website, you're probably here for the wrong reasons. Our advertising rates are quite reasonable; contact our ad sales team for details. We also offer free community promotion ads for open source projects and non-profit organizations.
Comment: I used the word promote on purpose to get some additional views into this discussion. One thing that is very common with Magento is that you need to post the contents of a lot of files to outline a solution. In my view this site is not the best tool for sharing lots of code so it would be easier to link to an online repository or blog with download optios. This site could then be used to discuss the merits of the linked content. But we as community might also decide that copying everything to here is the way to go.
Answer: I can't give a definitive answer but I imagine that as long as the content promoted is a legitimate and useful reference towards the answer then I see no reason why not.
I don't think a single link answer is appropriate - it needs at least a sentence of why you're providing the link.
If I happen to have written a blog post on the exact subject the poster has asked about then it would make sense for me to link it up rather than copy & paste all the content into my answer.
Likewise if an extension link is a suitable answer to a 'how do I do this?' type question then it should be allowed - as long as there's a line of text with it such as:
"This free extension does what you're asking for: http://...."
That's my opinion anyway :)
Comment: I would have agreed with you, but with many a slapped wrist on SO and SF - with the comment "...Link rot...". Thereafter, I soon started to include quotes or entire re-formatted documents of my own work in answers. I argued about duplication of the internet - but that's not a key concern over here. Retention of information is.
Comment: ...indeed. There are a lot of valuable resources that may fail to exist if a prominent member of the community were hit by a bus.
Answer: The first rule of promoting your own work in answers is to first, answer the question. If, links or references to other stuff aside, the answer is incomplete or unhelpful, it shouldn't be posted - and if it appears to be just a thin excuse to promote your own work or send folks elsewhere, then it's spam.
The second rule: be clear about your connection to things you link to. If you wrote it, say so. If you host it, say that. If your wife's brother plays golf with the guy who wrote it, you probably don't need to say that - but if that's the reason you're posting it, you might want to re-think that.
And of course, don't ever just use the site as a way to promote something, even if you're diligent about following all the rules while doing so. This just looks shady.
The FAQ spells all of this out pretty clearly.
Comment: If the solution to the problem stated *is* a project on Github, then the purpose of an answer is to explain *how* it can be used to solve the problem.
Comment: Some good advice in here. What I am hoping to get out of this discussion is if we want the Magento stackoverflow community to be inclusive of other Magento community efforts - sometimes it is plain easier to link to a github repository for example - than what seems to be the stackoverflow favored approach of copying the whole solution onto here.

promoting of your own content


Como gerenciar Sessão com Session em aplicações C# para desktop e não web?

Question: Eu não consigo encontrar um exemplo de uso do objeto Session para aplicações desktop, todos os que eu encontro são para aplicações feitas em C# para a web com asp, etc.
Alguém sabe como verificar se um usuário está autenticado no programa ao abrir o mesmo?
Eu sei que é possível fazer isso com o objeto Session, porém só encontro exemplos pra Web e não consigo implementar uma verificação de sessão.
Comment: Já solucionou seu problema?
Answer: Não é possível trabalhar com Session da mesma forma que você trabalha na Web.
Recomendo criar uma classe estática com propriedades estáticas(get e set). Essas propriedades vão ficar disponíveis enquanto o programa estiver "rodando".
Exemplo abaixo.
Sua classe que gerencia as sessões pode ficar assim:
<code>public static class SessaoApp
{

    public static int UsuarioId { get; set; }

    public static string UsuarioNome { get; set; }

    public static string Login { get; set; }

    public static string Email { get; set; }
}
</code>
Na sua implementação ficaria
<code>       seuTextBoxEmail = SessaoApp.Email;
       seuTextBoxLogin = SessaoApp.Login;
       seuTextBoxNome = SessaoApp.UsuarioNome;
       seuTextBoxUsuarioId = SessaoApp.UsuarioId;
</code>


import dataframe from dict without index

Question: I'm trying to generate a dataframe from a dict but I want to set the names of the columns. So, I Want a dataframe with index = [1,2,3,4], Column 1 = [keys of the dict], Column 2 = [values of the dict]
I put what I was trying to do just in case, but it works for only one column
a = pd.DataFrame.from_dict(stdic, orient='index', columns=['Col 1'])

Input dictionary is:
{key1: value1, key2: value2, ...}

Expected dataframe:
index Column 1    Column 2
1         key1      value1
2         key2      value2

Comment: can you post the sample input and expected output
Answer: What about:
d = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
df = pd.DataFrame({'col1': list(d.values()), 'col2': list(d.keys())})
print(df)

outputs
   col1 col2
0     1    a
1     2    b
2     3    c
3     4    d

Comment: If you really wants to use `from_dict`, use `pd.DataFrame.from_dict(d, orient='index').reset_index()`, but you have to rename the columns later.
Comment: your answer should use `pd.DataFrame.from_dict`. read the question carefully
Comment: @deadshot "generate a dataframe from a dict" != "use the from_dict method". The only reference to "from_dict" in the question is the code OP used.


Instalação de uma aplicação WinForms C# no cliente

Question: Desenvolvi uma aplicação em WinForms C# em 03 camadas com Banco de Dados Access.MDB:
Agora as questões para que o Aplicativo possa ser instalado no cliente e funcionar corretamente.

Os arquivos que mando para o cliente são os seguintes:
1.1 - Executável winforms: Ex: MinhAplicação.exe
1.2 - DLL MODELS: Modelsapp.dll
1.3 - DLL DAL:    Dalapp.dll
1.4 - DLL BLL:    BLLlsapp.dll
1.5 - BD Access:  database.mdb

Mesmo o cliente tendo o Office instalado (no caso 2010) o C# me obriga a instalar o OLEDB 12.0 no cliente.
Pergunta: Há alguma maneira do Aplicativo C# reconhecer o Office do Cliente sem eu ter que instalar o OLEDB 12.0
Meus relatórios feitos com ReportViwer não rodam no cliente, a não ser que eu instale o Visual Studio no cliente.
Já tentei mandar o reportviwer.exe, o run-time do Visual Studio. Mas o C# pede o CRL TYPES DO SQL previamente instalado. Isso é complicado para o cliente.
Pergunta: Quais os arquivos e procedimentos necessários para que o meu aplicativo instale e rode no cliente?
Comment: Algum problema para usar o instalador que vem com o Visual Studio?
Comment: quando você gerar o executavel, irá tudo pro projeto.
Answer: Sobre o report viewer. Procure no site da Microsoft por "Pacote do Microsoft Report Viewer yyyy Redistributable". Ele só vai instalar os binários necessários no cliente para ele gerar o relatório.
Link para versão 2010: Pacote do Microsoft Report Viewer 2010 Redistributable
Answer: Sobre o Access existe o Access Runtime a versão tem que ser a mesma do office que você criou o banco de dados ex: se você criou o MDB no access 2007 baixa o runtime 2007.. Nesse link tem todas as versões:
https://www.microsoft.com/pt-br/search/result.aspx?q=Access+Runtime&form=MSHOME
Agora sobre o cliente ja possuir o Office na instalação do mesmo teria de marcar uma opção que seria algo assim: Suporte a Aplicações do .NET se não marcarem essa opção da erro já tive muito problema com isso, Só o Office com access em si aplicação não reconhece. porém o Runtime resolve esse problema.
Uma Recomendação: Tente começar a Usar o MySql ele é simples de se instalar e de usar. Ate mesmo mais simples do que o Access em determinados casos.
Comment: Embora este link possa responder à pergunta, é melhor incluir as partes essenciais da resposta aqui e fornecer o link para referência.  As respostas apenas por link podem ser invalidadas se a página com o link for alterada. - [From Review](/review/low-quality-posts/89095)
Comment: Obrigado Diego. o Link foi para fornecer ajuda ou facilitar a busca que no caso seria o Access Runtime. Grado da Dica.


Freelists with concurrent allocators

Question: Freelists are a common way to speed up allocation by reusing existing memory that was already allocated.  Is there a way to use free-lists in a concurrent allocator, without incurring the overhead of a lock for every allocation (which would neutralize the intended performance gain of the freelist)?
Answer: Use a lock-free linked list.
Comment: Is that actually a performance win over using a lock, it's my understanding that lock-free datastructures still usually have approximately the same speed as a locked variant (but loses the potential for deadlocks, etc.)?
Comment: If there is little contention, a lockfree implementation should outperform using a lock (read-modify-write instead of a lock-read-modify-write-unlock), especially if using the lock incurs an OS call. If there is heavy contention, the lock free implementation may actually have worse performance (due to having to back off and retry repeatedly) but in that kind of situation you're not actually gaining anything by having multiple threads with a locked implementation either since if linked list mutation is the majority of your work, the majority of your work would becomes serialised anyway.
Answer: You could have thread-specific free list chunks.
Basically, there is some system which populates the free lists (e.g. a garbage collector). Then each thread could have its own free list chunk, containing a small number of entries. Locking would be used to allocate a new chunk. With chunks with 30 entries, you would lock only once every 30 allocations. Conversely, with thread-specific chunks you may have to run the GC sooner, because the shared list could become empty even if some of the thread-specific chunks still have some free entries.

Freelists with concurrent allocators


Where to put a code that interacts with DB and should run once on the project startup in django 2.2 or higher?

Question: I'm trying to write some code that checks on something, if exists then cool and if not creates it.
The problem is, where to put this code? Django doesn't recommend putting anything that deals with DB at an app's ready() function - this is what I'm currently doing but it gives me an error on debugging and give an error too on the beginning but works later on-.
Also, what I have found seems outdated and a bit hacky like putting it at urls.py or wsgi.py.
I feel there must be a better way to do it.
Thank you all.
Comment: Does this answer your question? [How to define default data for Django Models?](https://stackoverflow.com/questions/39739439/how-to-define-default-data-for-django-models)
Answer: Typically DB interactions within Django apps happen within the views.py file. I would suggest writing a view that uses the get_or_create model method.
Here is the documentation:
https://docs.djangoproject.com/en/3.2/ref/models/querysets/#get-or-create


Handle transaction when invoking SQL stored procedure in side a C# loop

Question: I'm working on a system which is already developed by some other developers.
And in that system they have invoked a stored procedure which use to insert records in side a loop in C# without using User define table types.
And what I need to do is to add a transaction to this scenario. But the problem is I have no idea about the place to have the transaction.
I would like to know whether have it in the C# code warping up the loop or inside the stored procedure.
Answer: You can only have it inside the C# loop. A transaction started inside the procedure must commit before the procedures exits. SQL Server checks the @@TRANCOUNT before and after running a procedure and if the results don't match an exception is raised. So is impossible for a procedure to start a transaction that has to be committed by caller.
The easiest think to do is to wrap your C# code in a transaction scope:
using(var scope =  new TransactionScope(
      TransactionScopeOption.Required,
      new TransactionOptions() {
         IsolationLevel = IsolationLevel.ReadCommitted
      })) {
   // Do the work here
   scope.Complete();
}

Note that passing the IsolationLevel = IsolationLevel.ReadCommitted is critical.
Answer: In this case Transaction needs to begin in the C# code, you have following options:

Use the Transaction from the Connection object:
using (var conn = new SqlConnection(connString))
{
conn.Open();
using (IDbTransaction tran = conn.BeginTransaction())
{
    try
    {
        // transactional code...
        using (SqlCommand cmd = conn.CreateCommand())
        {
            // Loop the Insert operation here
            cmd.CommandText = "INSERT Operation";
            cmd.Transaction = tran as SqlTransaction;
            cmd.ExecuteNonQuery();
        }
        tran.Commit();
    }
    catch(Exception ex)
    {
        tran.Rollback();
        throw;
    }
}
}

Use Implicit or Explicit Ambient transaction
Use TransactionScope (Implicit) Ambient transaction (On Exception it's an automatic rollback)
using (var scope =  new TransactionScope(
     TransactionScopeOption.Required,
     new TransactionOptions() {
     IsolationLevel = IsolationLevel.ReadCommitted
   }))
{
using (var conn = new SqlConnection(connString))
{
   conn.Open();

    using (SqlCommand cmd = conn.CreateCommand())
    {
        // Loop the Insert operation here
        cmd.CommandText = "INSERT Operation";
        cmd.ExecuteNonQuery();
    }
}
 scope.Complete();
}

Use CommittableTransaction (Explicit) Ambient transaction
var tran = new CommittableTransaction();
using (var conn = new SqlConnection(connString))
{
 conn.Open();
try
{
   conn.EnlistTransaction(tran);

using (SqlCommand cmd = conn.CreateCommand())
{
    // Loop the Insert operation here
    cmd.CommandText = "INSERT Operation";
    cmd.ExecuteNonQuery();
}
  tran.Commit();
}
catch(Exception ex)
 {
    tran.Rollback();
    throw;
 }
}

For Ambient transactions, please remember that if you open more than one connection, then it will get promoted from local to global transaction, thus requiring MSDTC and that has a performance impact.
Check the following link:
http://www.codeproject.com/Articles/690136/All-About-TransactionScope
Answer: In SQL SERVER
BEGIN TRY
BEGIN TRAN

-- DO YOUR INSERT /UPDATE /DELETE HERE

COMMIT TRAN T1
END TRY
BEGIN CATCH
-- // ROLL BACK IF ERROR OCCURS

ROLLBACK TRAN T1
END CATCH

In C#
using(TransactionScope tran = new TransactionScope()) {

     // DO YOUR INSERT /UPDATE /DELETE HERE

    tran.Complete();
}

Comment: Both (Sql Server, C#) would be same only when all inserts / DML take place in same transaction context, which is not the case here since the inserts are multiple round trip. For both to be same it needs to be using TVP


All variables invisible to twig files

Question: I created a theme whose minimal.theme file contains the following lines (among others).
/**
 * Implements hook_preprocess_HOOK() for node templates.
 */
function minimal_preprocess_node(&$variables) {
  // Declare the test variable and set its value.
  $variables['test_variable'] = 'Hello, this is a test variable!';
}

The following is the content of the template file.
{#
  This is a custom Twig template for the front page (homepage).
  It will override the default page template for the front page.
#}
<!DOCTYPE html>

Welcome to Our Website

{{ test_variable }}
 {# not showing up #}

The template file does not show the value for test_variable?
What else do I need to do to make test_variable usable in the template file?
Comment: Preprocess node hook will make variables available in node templates. You are in the page template. Use preprocess page hook.
Answer: Variables that are set in hook_preprocess_node() are only accessible in node templates. You are trying to access test_variable in a page template and that is the reason why that template file does not render any value for test_variable.
If you want to add the same variable to all the template files, you should implement hook_preprocess().
/**
 * Implements hook_preprocess().
 */
function minimal_preprocess(&$variables, $hook, $info) {
  // Declare the test variable and set its value.
  $variables['test_variable'] = 'Hello, this is a test variable!';
}

You could also implement hook_preprocess() when the same code is used for more than one template file, but not all the template files. In that case, you can check the value of $hook.
/**
 * Implements hook_preprocess().
 */
function minimal_preprocess(&$variables, $hook, $info) {
  if ($hook == 'page' || $hook == 'node') {
    // The variable is accessible only from page and node templates.
    $variables['test_variable'] = 'Hello, this is a test variable!';
  }
}

All variables invisible to twig files


Multiple Node Variants with Different Paths

Question: I have my node content type set up using Page Manager. I have different variants for this content type that pass different arguments to my view.  Is there any way to have these variants have different URLs? For example:
<code>my/node/path
editors/my/node/path
retailers/my/node/path
my/node/path/anotheridentifier
</code>
The content will be different so I don't see an issue with SEO.
Answer: Try creating aliases that point to the same path (your page manager page). To create aliases go to Configuration > Search and metadata > URL aliases
Comment: If I create multiple aliases to a node it always redirects to the main one so I don't think this will work.
Comment: Unless I am misunderstanding how page_manager works, I thought that the point of creating variants is that they are all invoked using the same path, and the page_manager module chooses which specific variant to display based on the Selection Rules you define when you create each variant. Is this not the case with your situation?
Comment: See https://www.drupal.org/node/1583366
Comment: Yes you are correct.  I don't think it is possible what I'm trying to do.


RequireJS - custom js-library not working

Question: I'm trying to include a custom js-library/script in my RequireJS-config but it doesnt seem to work. So I hope someone can help me out. I'm using RequireJS in combination with Backbone and Handlebars, so just to mention it...
In my Require config I have:
require.config({

paths: {

    jquery: 'lib/jquery/jquery',
    backbone: 'lib/backbone/backbone',

    // Templating.
    handlebars: 'lib/handlebars/handlebars',

    // Plugins.
    jqueryEffects: 'lib/jquery/jquery.effects',
    ... //some more libraries

},

shim: {
    backbone: {
        deps: ['jquery', 'lodash','jqueryEffects'],
        exports: 'Backbone'
    },
    lodash: {
        exports: '_'
    },
    handlebars: {
        exports: 'Handlebars'
    },
    jqueryEffects : {
        deps: ['jquery']
    }
  }
});

The jquery.effects.js is a simple script i created my own to handle special click events or run animations etc. When i start to run my backbone app, the console tells me that the script is loaded. So now on one of my Views, I have rendered a HTML file which contains an anchor with a class which serves as an identifier, which after clicking it should trigger something...BUT, nothing happens, so I tried to make an alert in the jquery.effects.js-file:
$(function() {
 alert($(".videoname").lenght);
});

This gave me the response undefined. Does anyone maybe have an idea? The same goes when I add more libraries, console says they are loaded, but nothing happens... ?!?!?!??
Answer: try this (length not lenght) :
(function() {
    alert($(".videoname").length);
})();

That's means that your script is working ;)
Comment: Ha you were right! I misspelled... Now I get the value `0`, and after I clean the cache, it says `$ not defined` (that is a general problem of mine)
Comment: Can you add the jquery.effects.js full code to your question ?
Comment: Or you can use it as a require.js module :
define(['jquery'], function($) {
    alert($(".videoname").length);
});


Detecting for structural changes in a time series near the end

Question: My interest is to check for structural changes in a time series. I know the time point which I wish to check for structural break. This point happens to be near the end of the series. Also I am doing a univariate modeling on the series (so no other regressors other that it's own lags).
Following is what I understood after browsing through the net:

There are many tests to check for Structural breaks: Chow test, sup-Wald test, sup-LM test, sup-LR (Andrews test), etc.
The classical chow test can handle near the end problem but requires linearity of the model, exogenous regressors and normality of errors. Since auto.arima in R gives a (1,0,0)(1,0,0) model, two of these three requirements are not met.
Some websites suggest to use Andrews test as this test can handle non-linearity and own-lagged regressors (Q1: would be great if someone can confirm this). Further, this test allows for automatic detection of break points. Q2: Can it detect only one breakpoint or many?
Then I discovered that this test may not be good as it assumes large number of observations before and after the breakpoint.
Latest information that I have is that Andrews developed another test in 2003, which can handle all these problems.

Q3: Is there a R package which has this latest test?
Q4: Can either of sup-Wald test or sup-LM test work for me?
Q5: Can someone give a brief idea of the tests above, particularly their assumptions, applicability and possibly a source to study theory behind them?
Answer: "I know the time point which I wish to check for structural break."
A structural break can be due to
1) an observation that is significantly different from the expected thus a pulse indicator would be approriate ....to identify this pulse follow http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html
2) a sequence of recent errant observations can be suggested by the need for a level/step shift using the above reference .
3) a sequence of recent errant observations can be suggested by a change in the model form testable by the 1960 CHOW test https://de.wikipedia.org/wiki/Chow-Test
 for constant PARAMETERS across groups. This is different from the "typically available Chow test for a level shift in the expected value" which is dealt with 2)
4) a sequence of errant observations can be suggested by detecting deterministic change points in the model error variance using the same reference
Precisely which structural break are you after ?
Finally you say you know the break point thus this is de jure  . Often time there is a de facto break point which can differ from the presumed one requiring it to be found via analytics from the aforementioned reference.
Comment: I didn't say I know the breakpoint. I am saying there's a time point that I want to check for structural break. What you are referring to seems similar to AO and LS type outlier detection . But I think structural change covers more possibilities that what outliers cover. My interest as mentioned is know more about the tests. Chow test shouldn't be applicable as I said it requires linear models with exogenous regressors only.
Comment: When you specify the breakpoint that is implicitly assuming knowledge or a "possible truth". We agree on that. The search for AO and LS is conducting one kind of "structural break" while the tests for constant parameters and constant error process are also conducting "structural break tests".  These two tests do not require an ARIMA component to be nul but are fully applicable to a general SARMAX model https://autobox.com/pdfs/SARMAX.pdf
Comment: By "these two tests" are you referring to AO and LS tests? I am aware about outlier detection in time series. In fact I did use the algorithm from Chen and Liu (1993) to detect an LS type outlier. My interest here, as also mentioned above, is to know about structural break tests near the for time series.
Comment: The two tests are 1) do the parameters (collectively) duffer from one one region to another ... and 2) does the model error variance change from one region to another
Comment: Right. Can you give the names of these tests and confirm whether these tests would work for breaks near the end of sample? Please refer to [this] (https://onlinelibrary.wiley.com/doi/abs/10.1111/1468-0262.00466) for details.
Comment: no ,, the reference does not have these tests referred to


Como usar re.split() para pegar somente as palavras de um texto, ignorando números e sinais de pontuação

Question: É-me dada a seguinte frase:
texto = """
Em 2020 observamos, e catalogamos (com fotografias), os barcos que chegaram ao Porto!
Até breve.
"""
for p in gerarPalavras(texto):
 print(p)

Criei o seguinte código para separar as palavras e colocar cada uma numa linha diferente:
import re

def gerarPalavras(texto):
    fraseFinal=[]
    frase = re.split("[ ,2020().!]" ,texto)

    for i in frase:
        if i != ' ':
            fraseFinal.append(i)

    return fraseFinal

O output desejável é que apareça uma palavra por linha, livrando-me da pontuação e números.
No meu output isto acontece, no entanto, inadvertidamente estou a criar espaços sem saber como:
Em

observamos

e
catalogamos

com
fotografias

os
barcos
que
chegaram
ao
Porto

Até
breve

Answer: Em regex, os colchetes definem uma classe de caracteres (ou seja, o que tem dentro deles é uma lista dos caracteres possíveis). Portanto [ ,2020().!] significa "um espaço, ou uma vírgula, ou o dígito 2, ou o dígito 0, ou o dígito 2 (de novo, então é redundante tê-lo aqui novamente), etc...". O detalhe é que toda esta expressão corresponde a apenas um único caractere (qualquer um que esteja indicado dentro dos colchetes).
Isso quer dizer que qualquer um desses caracteres é tratado como um separador. Se tiver só um espaço, ele separa e cria um elemento no resultado final. Se depois tiver um 2, é feito outra separação, depois, se tiver um ponto, outra separação, e assim por diante.
Então o resultado do split está bem diferente do que você precisa, pois no caso de ter 2020, ele vê que tem um 2 e depois um zero, e faz o split entre eles (e como entre eles não tem nada, a lista fica cheia de strings vazias). Ou seja, se você fizer isso:
print(re.split("[ ,2020().!]", texto))

Verá que a lista resultante é:
['\nEm', '', '', '', '', '', 'observamos', '', 'e', 'catalogamos', '', 'com', 'fotografias', '', '', 'os', 'barcos', 'que', 'chegaram', 'ao', 'Porto', '', '\nAté', 'breve', '\n']

E como na sua função você só elimina os espaços (as strings que são ' ', que é diferente de uma string vazia, como os vários '' vistos acima), então sua lista tem várias strings vazias, além de quebras de linha (o \n que você vê em alguns lugares acima). Por isso a saída acaba tendo tantas linhas em branco.

Se a ideia é pegar somente palavras (considerando que "palavra" é uma sequência de letras), pode fazer assim:
import re

def gerarPalavras(texto):
    return [ s for s in re.split(r'[\W\d]+', texto) if s != '' ]

texto = """
Em 2020 observamos, e catalogamos (com fotografias), os barcos que chegaram ao Porto!
Até breve.
"""
for p in gerarPalavras(texto):
    print(p)

A saída é:
Em
observamos
e
catalogamos
com
fotografias
os
barcos
que
chegaram
ao
Porto
Até
breve

A expressão [\W\d]+ pega uma ou mais ocorrências (indicado pelo quantificador +) de [\W\d]. O atalho \W é "qualquer caractere não-alfanumérico" (ou seja, qualquer coisa que não seja letra, dígito ou _) e o \d indica "um dígito". Assim, o split faz a quebra em qualquer sequência de caracteres que não sejam letras. Com isso ele já elimina também as quebras de linha.
Só tive que eliminar algumas strings vazias que são colocadas no início e fim.

Match em vez de split
Outra opção é fazer o contrário: em vez de dizer o que você não quer (caracteres que não são letras) e fazer o split, você pode dizer o que quer (ou seja, eu digo o que quero que a regex encontre em cada match) e usar findall:
def gerarPalavras(texto):
    return re.findall(r'[^\W\d]+', texto)

Agora eu uso uma classe de caracteres negados: o [^ indica que o que está nos colchetes são coisas que eu não quero. No caso, é \W (qualquer coisa que não seja alfanumérico) ou \d (um dígito). Se eu não pego esses caracteres, tudo que sobra são as letras. O resultado é o mesmo do código anterior, afinal, split e match são dois lados da mesma moeda: no split eu digo o que não quero, e no match/find/search eu digo o que quero.

Se os textos se restringem ao nosso alfabeto, outra opção é:
re.findall('[a-záéíóúãõâêîôûç]+', texto, re.I)

Assim, eu coloco todas as letras possíveis, e a flag re.I diz para considerar tanto maiúsculas quanto minúsculas.

Claro que aqui usamos uma definição de "palavra" mais simples, já que não leva em conta palavras compostas (como "beija-flor" e "olho-d'água"), mas se quiser complicar mais, pode dar uma olhada aqui, aqui e aqui.
Comment: Muito cool! (como habitualmente...)
Answer: Para processamento de lingua, é habitual começar pela tokenização (dividir o texto
nos seus elementos básicos: palavras, pontuações, etc) -- problema mais abrangente que o
pedido.
No exemplo abaixo segue uma versão simplificada do tokenizador que normalmente uso, para
apresentar o modo re.X que permite exp.reg. de mais fácies de ler, ajustar e documentar.
import re;

texto = """Em 2020 observamos, e catalogamos (com fotografias), os barcos ... ao Porto!
Até breve."""

print(re.findall(r'''
     \b\w[\w\-.]*\w\b      # palavras: 2020 barcos  ver-se dir-se-ia  file.txt
   | \w                    #
   | \.\.\.                # ...
   | [,.:;?!()[\]]         # pontuação
   | \S
       ''',texto,re.X))

Como era de esperar, a saída é:
['Em', '2020', 'observamos', ',', 'e', 'catalogamos', '(', 'com', 'fotografias', ')',
',', 'os', 'barcos', '...', 'ao', 'Porto', '!', 'Até', 'breve', '.']


How to map abstract JPA base entity?

Question: Rather simple case, yet I'm having issues finding a working mapping. Example stripped down to the very basic problem.
Two entities: Player and Coach. Both have a date of birth.
Database
Table PL // Player
NUMERIC plid AUTOINCREMENT

Table CO // Coach
NUMERIC coid AUTOINCREMENT

Table BASE
NUMERIC enid - playerId or coachId
CHAR    entp - 'PL' for Player, 'CO' for Coach
DATE    bdt

Java
Abstract BaseClass {
    Date dateOfBirth;
}

@Entity
@Table(name="PL")
Player extends BaseClass {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name="plid")
    Long playerId;
}

@Entity
@Table(name="CO")
Coach extends BaseClass {
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name="coid")
    Long coachId;
}

How can we map the BaseEntity class to persist Player.dateOfBirth in table BASE as 1, 'PL', 1970-01-01?
Comment: not my business, but why do you need CO and PL? You could just get rid of these tables and use only BASE
Comment: Did I not say it's a stipped down example?
Comment: Yes, but still looks weird for me :-)
Comment: But supposing CO and PL are huge classes with very little in common, I'd still prefer to reuse the autoincrement in the base class and use the flag to differentiate one subclass from another
Comment: This has nothing to do with the question but if you're so sure of your idea, create these classes, generate id in the base class table and then try to override the id attribute, so that enid gets saved in plid and coid. Good luck.
Comment: I think @csn approach down below is the one I'd use. In fact that's exactly what I am doing in my current project :-) I have an abstract base class that uses @Inheritance(strategy=InheritanceType.JOINED) and the concrete classes under it has what is specific to each one. You don't have to keep different columns for the specific subclasses IDs. The ID is unique anyway.
Answer: You need to use InheritanceType.JOINED which allows you to keep common data in base table and subclasses can have data specific to their needs.
In your example, 'entp' can act as discriminator column. You should annotate your classes as follows:
    @Entity
    @Inheritance(strategy=InheritanceType.JOINED)
    @DiscriminatorColumn(name="ENT_TYPE")
    @Table(name="BASE")
    public abstract class BaseClass {
          @Id
          Long id;
          @Column(name = "ENT_TYPE")
          String entp;
          ...
    }

    @Entity
    @DiscriminatorValue("PL")
    @Table(name="PL")
    public class Player extends BaseClass {

    }

    @Entity
    @DiscriminatorValue("CO")
    @Table(name="CO")
    public class Coach extends BaseClass {

    }

Comment: Looks promising. I have lots of things going on in my code, so need a day to get to try this.

How to map abstract JPA base entity?


Maximum Likelihoood Criterion vs Maximum A Posteriori

Question: This problem was taken from a free online Computational Neuroscience class {1}.

Suppose we are diagnosing a very rare illness, which happens only once in 100 million people on average. Luckily, we have a test for this illness but it is not perfectly accurate. If somebody has the disease, it will report positive 99% of the time. If somebody does not have the disease, it will report positive 2% of the time.
Suppose a patient walks in and tests positive for the disease.
Using the maximum likelihood (ML) criterion, would we diagnose them positive?
What if we used the maximum a posteriori (MAP) criterion?

I'm curious about the actual maths behind the decision outcomes. The correct answers are Yes and No, respectively, but I'm unsure of how to analytically derive those solutions. I would love to see some of the work texted up.

References:

{1} https://compneuromooc.wikispaces.com/HW3 and Coursera Computational Neuroscience by Adrienne Fairhall, Rajesh P. N. Rao
Comment: Just a side note attacking the question posed by the class: the ML and MAP answer should be *identical*. What the questioner is considering to be the prior (proportion of population with disease) can easily just be joined directly into the likelihood function itself. There is no rule banning hierarchical models from maximum likelihood estimation.
Comment: This question was asked here previously.  Were you the OP.  If so what was the disposition?
Answer: If we let $d=\mathrm{disease}$, $\sim\!d=\text{not disease}$, and similarly for $t=\mathrm{test}$, then I believe the two approaches are comparing the conditional probabilities:
\begin{array}{}
\text{MLE}:& \text{positive} &\implies \Pr\big[\,t\mid\,d\,\big]>\Pr\big[\,\phantom{\sim}\,\,\,t\mid\,\sim\!d\,\big] \\
\text{MAP}:& \text{positive} &\implies \Pr\big[\,d\mid\,t\,\big]>\Pr\big[\,\sim\!d\mid\phantom{\sim}t\,\,\big] \\
\end{array}
The problem statement gives you the prior probability, $\Pr\!\big[\,d\,\big]$ (a.k.a. base rate), along with the true and false positive rates, $\Pr\!\big[\,t\mid\,d\,\big]$ and $\Pr\!\big[\,t\mid\sim\!d\,\big]$.
To compute the MAP diagnosis you need the posterior probabilities, which can be computed using Bayes' theorem* and the law of total probability. (*The "Examples" section here includes several problems directly analogous to yours.)
Finally, note that the example serves as a demonstration of the dangers of "base rate neglect". This is because the MLE diagnosis only considers the likelihood ratio, while the MAP diagnosis also incorporates the base rate.

Note: As pointed out by Mark Stone, in the real world self selection bias should be considered (i.e., the base rate among patients seeking a test will generally differ from the population base rate). The "textbook" solution is more relevant to screening.
Answer: I strongly disagree in the real world with the answer provided in the question and the answers which have already been provided by @GeoMatt22 and @user99889 (I don't blame them as they are showing how to get the answer in the question).  In the fake world of a class problem by a professor who is not being realistic, this is probably the correct answer in the sense of that's what the professor wants the student to give as the answer. In the real world, it can be off by orders of magnitude. The answer provided in the question exhibits a very common fallacy among  people who are sophisticated enough to try to use Bayes' theorem, but not sophisticated enough to think things through properly.
An implicit, but possibly very invalid assumption is being made that the population of patients who walk in and get tested for the disease has the same distribution of the very rare illness occurrence as the population of "all" people for whom the occurrence is 1 in 100 million.  It may be that only patients exhibiting symptoms, or having a family history, or having already received certain other test results are the ones who are getting tested for the very rare illness.  Or at least that such people are more likely than people in the whole population, to be  the ones getting tested.  Now the prior distribution of having the very rare illness is not 1 in 100 million, it might be orders of magnitude higher.  Then of course there's the matter of dependence ("correlation") between test results among different tests given a sequential sequence of different tests.
Comment: (+1) Yes, in the real world [self selection bias](https://en.wikipedia.org/wiki/Self-selection_bias) should be considered. The "textbook" solution is I believe more relevant to [screening](https://en.wikipedia.org/wiki/Screening_(medicine)).
Comment: @user99889 But that's not what the online course content creator was doing.
Comment: GeoMatt22 is right for screening false positives can be a real problem. That is why screening tests are often repeated or if the subject tests positive on the first test a more sensitive test is provided to confirm the diagnoses.


Bivariate Gamma Probability Function

Question: This might be a very basic question, but my textbook doesn't provide a satisfactory formula for how to calculate the probability for the Gamma function. Any help is appreciated.
$f(t,r) =$ {$\frac{1}{8}te^{\frac{-1}{2}(t+r)}$} for $t,r >0$ and 0 elsewhere
I'm asked to find the probability that $t,r >2$ and after a long integration by parts I was able to get that $P(t,r>2) = \frac{2}{e^2}$
Similarly, I found that $P(t,r>1) = \frac{3}{2e}$ and
$P(t,r>3)= \frac{5}{2e^3}$
There appears to be a clear pattern, is this normal? And is there a formula that allows you to get this answer with the long integration by parts?
Answer: You just integrate the pdf, then you wil find the pattern.
\begin{align*}
P(T>a,R>b)&=\int_a^\infty\int_b^\infty \frac{1}{8}te^{\frac{-1}{2}(t+r)}dtdr\\&=\frac{1}{8}\int_a^\infty\int_b^\infty te^{-\frac{1}{2}t}*e^{-\frac{1}{2}r}dtdr\\&=\frac{1}{8}\int_a^\infty te^{-\frac{1}{2}t}\int_b^\infty e^{-\frac{1}{2}r} drdt\\&=\frac{1}{8}\int_a^\infty te^{-\frac{1}{2}t} (-2)e^{-\frac{r}{2}}|_b^{\infty}dt\\&=\frac{e^{-\frac{b}{2}}}{4}\int_a^\infty te^{-\frac{1}{2}t}dt\\&\text{(integration by parts)}\\&=-\frac{e^{-\frac{b}{2}}}{2}\int_a^\infty tde^{-\frac{t}{2}}\\&=-\frac{e^{-\frac{b}{2}}}{2}\left [ te^{-\frac{t}{2}}|_a^{\infty}-\int_a^\infty e^{-\frac{t}{2}}dt \right ]\\&=\frac{a}{2}e^{-\frac{a+b}{2}}+e^{-\frac{a+b}{2}}
\end{align*}
When $a=b=1$
$$P(T,R>1)=\frac{1}{2}e^{-1}+e^{-1}=\frac{3}{2e}$$
When  $a=b=2$
$$P(T,R>2)=e^{-2}+e^{-2}=\frac{2}{e^2}$$
When  $a=b=3$
$$P(T,R>3)=\frac{3}{2}e^{-3}+e^{-3}=\frac{5}{2e^3}$$
You can see there are nothing speical just
$$P(T>a,R>b)=\frac{a}{2}e^{-\frac{a+b}{2}}+e^{-\frac{a+b}{2}}=(\frac{a}{2}+1)e^{-\frac{a+b}{2}}$$
Comment: Welcome to this site, there are a lot of people here can help.

Bivariate Gamma Probability Function


Is it a good ball or inbound violation?

Question: In basketball, let's say a player is inbounding the ball, tosses the basketball at the referee who's inside the playing court, and the same player takes the ball back to action afterwards.
It is a good ball or violation?
Answer: This is covered by NBA Rules Rule 10, Section III

A thrower-in shall not (1) carry the ball onto the court; (2) fail to release the ball within 5
seconds; (3) touch it on the court before it has touched another player; (4) leave the designated
throw-in spot which is one step to his left or right; (5) throw the ball so that it enters the
basket before touching anyone on the court; (6) step on the court over the boundary line before
the ball is released; (7) throw the ball out-of-bounds without it being touched by a player
in the game; (8) exit the playing surface to gain an advantage on a throw-in; (9) hand the ball
to a player on the court.

PENALTY: Loss of ball. The ball is awarded to the opposing team at the original spot of
the throw-in.

The referee is not a player and does not satisfy the prohibition for clause (3).  This is a violation and the opposing team gets the ball.
FIBA Rules are very similar here, and the corresponding section is 17.3.1.

Is it a good ball or inbound violation?


multiple webbrowser control, separate cookies?

Question: I notice when I visit a certain page with the webbrowser-control I am logged in as the same user as I am in IE. I want to visit the same page as and not as the logged in user. How do I do this? I would hate to repeatedly log in and out.
-edit- I no longer need this. But I'll leave this question up.
Answer: Make sure you clear out the current session, the most likely place the logged in state will be stored.


VB.NET Lambda expression instead of iterator

Question: I am fairly certain that this could be rewritten as a Lambda expression, but every attempt fails miserably.  I know, C# Lambda reads cleaner, but I'm stuck with VB.NET.  Here is the code - could someone point me in the right direction?  Thanks!
For Each e As EventToMonitor In Events
    If e.TypeID = 1 Then
        If ("," & e.Values).Contains("," & b.ChoiceID & ",") Then
            Notify(cacheValues, e, "Event notification (button press)", "Button pressed: " & b.Text & " on screen: " & b.GroupBox.Text & Environment.NewLine &
                       "User: " & cacheValues.CurrentUserName & Environment.NewLine & _
                       "Pressed at: " & Date.Now.ToShortDateString & " " & Date.Now.ToShortTimeString)
        End If
    End If
Next

Comment: What do you want the lambda *for* in this case? the current code is fine as-is. That said, I'd use a LINQ expression to filter (instead of `If`) and transform (instead of concatenation inside the `Notify` call), and then iterate over the result of that, calling `Notify(cacheValues, value)` for each, where `value` is the iteration variable.
Answer: Something like this should work if you want to convert the whole thing to lambda:
Events.Where(Function(e) e.TypeID = 1 AndAlso ("," & e.Values).Contains("," & b.ChoiceID & ",")) _
      .ToList() _
      .ForEach(Sub(e) Notify(cacheValues, e, "Event notification (button press)", "Button pressed: " & b.Text & " on screen: " & b.GroupBox.Text & Environment.NewLine & _
              "User: " & cacheValues.CurrentUserName & Environment.NewLine & _
              "Pressed at: " & Date.Now.ToShortDateString & " " & Date.Now.ToShortTimeString))

Honestly, though, I usually prefer to just use them to filter down results or build a collection:
Dim eventsList = Events.Where(Function(e) e.TypeID = 1 AndAlso ("," & e.Values).Contains("," & b.ChoiceID & ","))

For Each e As EventToMonitor In eventsList
    Notify(cacheValues, e, "Event notification (button press)", "Button pressed: " & b.Text & " on screen: " & b.GroupBox.Text & Environment.NewLine &
               "User: " & cacheValues.CurrentUserName & Environment.NewLine & _
               "Pressed at: " & Date.Now.ToShortDateString & " " & Date.Now.ToShortTimeString)
Next

Comment: Agreed, the latter example is the route I would go too.  It's more clear that way.


Riddle: how did Sossinsky code this left trefoil knot?

Question: In the notes at the end of his book Knots, Mathematics with a Twist (2002), Sossinsky leaves a small riddle: how can a knot be recognized by a computer? He gives the example of the left trefoil knot that its computer recognizes as 1+-2--3+-1--2+-3--1.

Could you explain the mechanism with which this node was coded?
PS: I'm looking for the solution to this puzzle but I thought it would be fun to share it, the way to code looks more basic than entering a Seifert matrix.
Source: Knots, Mathematics with a Twist (2002), p122.
Comment: This looks like a variation on [Gauss Codes](https://en.wikipedia.org/wiki/Gauss_notation).  But I am not quickly sure what the extra pluses and minuses are for.
Comment: https://en.wikipedia.org/wiki/Dowker%E2%80%93Thistlethwaite_notation
Answer: You mention Seifert matrices, but note that different knots can have the same Seifert matrix. The code that Sossinsky is mentioning is different because it's enough information to recover an entire knot diagram uniquely.
The code is a type of Gauss code. There are many ways you can write these down or encode them (there ends up being some information redundancy), but here's the core idea: label each of the crossings (in this case by 1, 2, and 3), give the knot an orientation, then follow around the knot and record, for each crossing you visit, (a) the crossing label, (b) whether you are going over or under, and (c) whether it is a right-handed or left-handed crossing.  The information in (b) & (c) can be recorded in a couple different ways.
Here's an example for how we could write down a Gauss code, using special symbols to record the (b) & (c) data:

Usually we don't put the 1 at the end -- I guess Sossinsky wants to emphasize it's a closed loop.
I have to admit I'm not sure what convention Sossinsky is using. It seems like he's using "-" for right-handed crossings, which would usually be "+". In any case, if you record "over" as "+" and "right-handed crossing" as "-", then that code could be "1+-2--3+-1--2+-3--1".
If you're interested, here's a post I wrote about different ways of representing knots, including Gauss codes.
Comment: I just realized that I put an illustration of the right trefoil knot, not the left trefoil knot. The convention that Sossinsky uses is indeed the usual one.

If you edit your answer, I will edit the image to put in the correct drawing.

Thank you very much for your explanation and for your link, I was really interested.