What is the differences between Int and Integer in Scala?

Question: I was working with a variable that I had declared as an Integer and discovered that > is not a member of Integer. Here's a simple example:
scala> i
warning: there were deprecation warnings; re-run with -deprecation for details
res28: Integer = 3

scala> i > 3
<console>:6: error: value > is not a member of Integer
       i > 3
         ^

Compare that to an Int:
scala> j
res30: Int = 3

scala> j > 3
res31: Boolean = false

What are the differences between Integer and Int? I see the deprecation warning but it's unclear to me why it was deprecated and, given that it has been, why it doesn't have a > method.
Comment: Thanks, all, for the useful answers. You all answer my question, so I'll let the highest rated response rise to the top and mark that as the answer. In the meantime, I've rated all your responses up.
Answer: "What are the differences between Integer and Int?"
Integer is just an alias for java.lang.Integer.  Int is the Scala integer with the extra capabilities.
Looking in Predef.scala you can see this the alias:
 /** @deprecated use java.lang.Integer instead */
  @deprecated type Integer = java.lang.Integer

However, there is an implicit conversion from Int to java.lang.Integer if you need it, meaning that you can use Int in methods that take an Integer.
As to why it is deprecated, I can only presume it was to avoid any confusion over which kind of integer you were working with.
Answer: Integer gets imported from java.lang.Integer and is only for compatibility with Java. Since it is a Java class, of course it can't have a method called "<".
EDIT: You can mitigate this problem by declaring an implicit conversion from Integer to Int.
 implicit def toInt(in:Integer) = in.intValue()

You'll still get deprecation warning though.
Answer: I think the problem you're seeing has has to do boxing/unboxing of value types and the use of the Java class Integer.
I think the answer is here: Boxing and unboxing in Scala.  There is no implict unboxing in Scala.  You've defined i as the Java class Integer but in the i > 3, the 3 is being treated and an int.
Comment: the link is broken now.
Answer: Integer is a Java class, java.lang.Integer. It's different from Java's primitive type int, which is not a class. It can't have < defined, because Java does not allow operators to be defined for classes.
Now, you might wonder why such a type exist at all? Well, primitive types cannot be passed as references, so you can't pass an int to a method expecting java.lang.Object, equivalent to Scala's AnyRef, for example. To do that, you put that int inside an Integer object, and then pass the Integer.
Answer: Integer gets imported from java.lang.Integer and is only for compatibility with Java. Since it is a Java class, of course it can't have a method called "<".
EDIT: You can mitigate this problem by declaring an implicit conversion from Integer to Int.


Fetch issue using OpenWeatherMap

Question: I've tried all sorts of ways to get weather data from open weather
if I paste this into the browser I get data
http://api.openweathermap.org/data/2.5/weather?q=London&appid=6e93b3d15872f914c6929fed9ea71e9a
but if I use it with fetch or Axios or various other methods that work for me with other API's I get nothing back. Can anyone tell me what I am doing wrong?
Comment: thanks everyone - it seems it was not having https that was the issue (I was working with repl.it)
Comment: Yes, it works fine.
Answer: Use https instead of http. You will get data.
Answer: This is the correct format:
https://api.openweathermap.org/data/2.5/weather?q="+cityName+"&units=metric&apikey="+key
change http to https
Answer: Try this
const response = await fetch("http://api.openweathermap.org/data/2.5/weather?q=London&appid=6e93b3d15872f914c6929fed9ea71e9a");

const data =  await response.json();

console.log(data);

Also wrap you code inside try/catch to catch any possible errors.
You can also use promises instead of async/await -
fetch("http://api.openweathermap.org/data/2.5/weather?q=London&appid=6e93b3d15872f914c6929fed9ea71e9a")
.then(response => response.json())
.then(data => console.log(data))
.catch(error => {
  console.log(error);
});

Also it will be better if you provide solutions that you have already tried in the question.
Comment: thanks - it was just not having https in the URL - I was trying the same code you've provided. Does just http work for you? I am using the REPL online IDE so that might be a factor
Comment: I tried in the browser console, worked fine without https.
Answer: index.html

    <title>Document</title>

Fetch issue using OpenWeatherMap


Estimating the break point in a broken stick / piecewise linear model with random effects in R [code and output included]

Question: Can someone please tell me how to have R estimate the break point in a piecewise linear model (as a fixed or random parameter), when I also need to estimate other random effects?
I've included a toy example below that fits a hockey stick / broken stick regression with random slope variances and a random y-intercept variance for a break point of 4. I want to estimate the break point instead of specifying it. It could be a random effect (preferable) or a fixed effect.
library(lme4)
str(sleepstudy)

#Basis functions
bp = 4
b1 <- function(x, bp) ifelse(x < bp, bp - x, 0)
b2 <- function(x, bp) ifelse(x < bp, 0, x - bp)

#Mixed effects model with break point = 4
(mod <- lmer(Reaction ~ b1(Days, bp) + b2(Days, bp) + (b1(Days, bp) + b2(Days, bp) | Subject), data = sleepstudy))

#Plot with break point = 4
xyplot(
        Reaction ~ Days | Subject, sleepstudy, aspect = "xy",
        layout = c(6,3), type = c("g", "p", "r"),
        xlab = "Days of sleep deprivation",
        ylab = "Average reaction time (ms)",
        panel = function(x,y) {
        panel.points(x,y)
        panel.lmline(x,y)
        pred <- predict(lm(y ~ b1(x, bp) + b2(x, bp)), newdata = data.frame(x = 0:9))
            panel.lines(0:9, pred, lwd=1, lty=2, col="red")
        }
    )

Output:
Linear mixed model fit by REML
Formula: Reaction ~ b1(Days, bp) + b2(Days, bp) + (b1(Days, bp) + b2(Days, bp) | Subject)
   Data: sleepstudy
  AIC  BIC logLik deviance REMLdev
 1751 1783 -865.6     1744    1731
Random effects:
 Groups   Name         Variance Std.Dev. Corr
 Subject  (Intercept)  1709.489 41.3460
          b1(Days, bp)   90.238  9.4994  -0.797
          b2(Days, bp)   59.348  7.7038   0.118 -0.008
 Residual               563.030 23.7283
Number of obs: 180, groups: Subject, 18

Fixed effects:
             Estimate Std. Error t value
(Intercept)   289.725     10.350  27.994
b1(Days, bp)   -8.781      2.721  -3.227
b2(Days, bp)   11.710      2.184   5.362

Correlation of Fixed Effects:
            (Intr) b1(D,b
b1(Days,bp) -0.761
b2(Days,bp) -0.054  0.181

Comment: Any way to make bp a random effect?
Comment: [This](http://www.tandfonline.com/doi/abs/10.1081/BIP-100102501?journalCode=lbps20) is a paper that proposes a mixed effects MARS. As @lockedoff mentioned, I don't see any implementations of the same in any package.
Answer: Another approach would be to wrap the call to lmer in a function that is passed the breakpoint as a parameter, then minimize the deviance of the fitted model conditional upon the breakpoint using optimize.  This maximizes the profile log likelihood for the breakpoint, and, in general (i.e., not just for this problem) if the function interior to the wrapper (lmer in this case) finds maximum likelihood estimates conditional upon the parameter passed to it, the whole procedure finds the joint maximum likelihood estimates for all the parameters.
library(lme4)
str(sleepstudy)

#Basis functions
bp = 4
b1 <- function(x, bp) ifelse(x < bp, bp - x, 0)
b2 <- function(x, bp) ifelse(x < bp, 0, x - bp)

#Wrapper for Mixed effects model with variable break point
foo <- function(bp)
{
  mod <- lmer(Reaction ~ b1(Days, bp) + b2(Days, bp) + (b1(Days, bp) + b2(Days, bp) | Subject), data = sleepstudy)
  deviance(mod)
}

search.range <- c(min(sleepstudy$Days)+0.5,max(sleepstudy$Days)-0.5)
foo.opt <- optimize(foo, interval = search.range)
bp <- foo.opt$minimum
bp
[1] 6.071932
mod <- lmer(Reaction ~ b1(Days, bp) + b2(Days, bp) + (b1(Days, bp) + b2(Days, bp) | Subject), data = sleepstudy)

To get a confidence interval for the breakpoint, you could use the profile likelihood.  Add, e.g., qchisq(0.95,1) to the minimum deviance (for a 95% confidence interval) then search for points where foo(x) is equal to the calculated value:
foo.root <- function(bp, tgt)
{
  foo(bp) - tgt
}
tgt <- foo.opt$objective + qchisq(0.95,1)
lb95 <- uniroot(foo.root, lower=search.range[1], upper=bp, tgt=tgt)
ub95 <- uniroot(foo.root, lower=bp, upper=search.range[2], tgt=tgt)
lb95$root
[1] 5.754051
ub95$root
[1] 6.923529

Somewhat asymmetric, but not bad precision for this toy problem.  An alternative would be to bootstrap the estimation procedure, if you have enough data to make the bootstrap reliable.
Comment: I had some optimization problems and wide CIs when inverting the profile likelihood with my real data, but got narrower bootstrap CIs in my implementation. Were you envisioning a nonparametric bootstrap with sampling with replacement on subjects' data vectors? I.e., for the sleepstudy data, this would entail sampling with replacement from the 18 (subject) vectors of 10 data points, without doing any resampling within a subject's data vector.
Comment: Yes, I was envisioning a nonparametric bootstrap as you describe, but partially that's because I don't know much about advanced bootstrap techniques that may (or may not) be applicable.  The profile likelihood-based CIs and bootstrap are both asymptotically accurate, but it could well be that the bootstrap is significantly better for your sample.
Comment: Thank you -- that was very helpful. Is this technique called a two-stage estimation procedure, or does it have a standard name that I could refer to / look up?
Comment: It's maximum likelihood, or would be if lmer maximized the likelihood (I think the default is actually REML, you need to pass a parameter REML=FALSE to lmer to get ML estimates).  just estimated in a nested manner rather than all at once.  I've added some clarification at the front of the answer.
Answer: The solution proposed by jbowman is very good, just adding a few theoretical remarks:

Given the discontinuity of the indicator function used, the profile-likelihood might be highly erratic, with multiple local minima, so usual optimizers might not work. The usual solution for such "threshold models" is to use instead the more cumbersome grid search, evaluating the deviance at each possible realized breakpoint/threshold days (and not at values in between, as done in the code). See code at bottom.
Within this non-standard model, where the breakpoint is estimated, the deviance does usually not have the standard distribution. More complicated procedures are usually used. See the reference to Hansen (2000) below.
The bootstrap is neither always consistent in this regard, see Yu (forthcoming) below.
Finally, it is not clear to me why you are transforming the data by re-centering around the Days (i.e., bp - x instead of just x). I see two issues:

With this procedure, you create artificial days such as 6.1 days, 4.1 etc. I am not sure how to interpret the result of 6.07 for example, since you only observed values for day 6 and day 7 ? (in a standard breakpoint model, any value of the threshold between 6 and 7 should give you same coef/deviance)
b1 and b2 have the opposite meaning, since for b1 days are decreasing, while increasing for b2? So the informal test of no breakpoint is b1 != - b2

Standard references for this are:

Standard OLS: Hansen (2000) Sample Splitting and Threshold Estimation, Econometrica, Vol. 68, No. 3. (May, 2000), pp. 575-603.
More exotic models: Lee, Seo, Shin (2011) Testing for threshold effects in regression models, Journal of the American Statistical Association (Theory and Methods) (2011), 106, 220-231
Ping Yu (forthcoming) The Bootstrap in Threshold Regression", Econometric Theory.

Code:
# Using grid search over existing values:
search.grid <- sort(unique(subset(sleepstudy, Days > search.range[1] &
Days<search.range[2], "Days", drop=TRUE)))

res <- unlist(lapply(as.list(search.grid), foo))

plot(search.grid, res, type="l")
bp_grid <- search.grid[which.min(res)]

Answer: You could try a MARS model.  However, I'm not sure how to specify random effects.
earth(Reaction~Days+Subject, sleepstudy)
Comment: Thanks -- I browsed through the package documentation but it did not seem to support random effects.


Accessing Apple Earbud Clicker Controls in C

Question: I wanted to find out if there was any way to have applications access and listen to the Apple Earbud's Clicker controls on the newer headphones, much like applications can listen to the IR Receiver for customized Apple Remote functionality. I know that iTunes has this ability (more precisely iTunes Helper) because when I double-click the play/pause clicker iTunes opens. I have looked into trying to access the clicker from HID/IOServices but currently am unable to find an answer, if anyone is able to shed light on this subject it would be much appreciated.
Answer: I suspect you could hook into this by making your app play audio (silence perhaps?) and then receive the events as "control events" for your "audio player."  You might look at the AV Foundation Framework as a place to start.
Comment: This seems like a good start, I will see if I can't get this working and post the progress here...


PHP build for Jenkins failing with 'Cannot run program "phploc"'

Question: I'm getting the following error when attempting to run my build script.
/var/lib/jenkins/jobs/IPS (trunk)/workspace/build.xml:62: Execute failed: java.io.IOException:
Cannot run program "phploc": java.io.IOException: error=2, No such file or directory

I tried switching to the jenkins user and was able to run the script without a hitch.  I also checked the $PATH and tried reinstalling.  No luck!

CentOS 5/6
PHP 5.4 (Zend Server)
Jenkins 1.477
PHP QA components (most recent)
Answer: I finally figured this out... checking the $PATH variable is not helpful in this context.  The various shell startup files are not executed when running a Jenkins job, so the $PATH variable was not being properly populated.  This can be remedied by following the three steps below:

Click Manage Jenkins
Click Configure System
Add new environment variable where name is PATH and value is $PATH:/usr/local/zend/bin.
Comment: If you have a path in Jenkins home directory make sure you don't use `~` but the absolute path instead.


Looking for software RAID-style FTP solution

Question: This software (TAHOE-LAFS) allows you to setup an FTP server that will encrypt and split your data, putting it redundantly on a bunch of remote servers. It is highly customizable so that you can adjust the amount of redundancy that you need and that you can easily add additional severs as nodes to the pool of space available.
I'm looking for something very similar, but instead of servers being nodes, I want normal hard drives to be the nodes. In other words, I want a software RAID (even if it is behind an FTP server) that will allow me to add more drives, adjust the redundancy, etc... without having to rely on standard RAD10, RAID5, RAID6, etc...
The purpose of this system would be to allow me to build a cheap file server that I can expand over time. I want to be able to mix and match hard drive sizes and I do not want to buy a special card for it. I don't want to rely on the Windows software RAID.
Is there anything out there?
Answer: You can run more than one Tahoe node on a machine. If your server has five drives, run five Tahoe nodes on it, each configured to use one drive.
Answer: Buy a DROBO (I don't own any of their stock, btw) and attach it via FireWire. Expandable up to 16 TB, any size drives in any combo. It will auto resize and it's got redundancy built in you can lose any single drive and not lose data.
Comment: Yes, this is similar to what I want... but I want to roll my own.
Comment: Unless it gives you competitive advantage to roll your own. I'd recommend not re-inventing the wheel. The DORBO will probably give you the best performance and reliability anyway. Just my opinion. Best of luck.
Answer: You mentioned encryption so I would think that this is sensitive data. Be careful regarding FTP as it is an insecure method of file transfer: namely the authentication is in clear-text. You should also look into implementing a SCP or SFTP over SSH solution too.
One idea would be to setup Truecrypt for encryption on the host server.
Answer: FreeNAS has software RAID, including Raid-Z.
OpenFiler also has software RAID, including Raid-5.
You can for example pick your 3 (or more) drives, make them array members and then join them together as a software raid volume.
Since these are NAS solutions, you can create various types of file shares and more.
Later you can add more drives to an array, or even create additional separate arrays and join them together to expand your volume.
It would be great if Tahoe-LAFS could be used as a volume in a FreeNAS!


onCreate flow continues after finish()

Question: I would like to finish an activity from inside the onCreate method. When I call finish(), onDestroy() is not immediately called, the code keeps flowing past finish(). onDestroy() isn't called until after the onCreate() closing brace.
Per the onCreate() description at developer.android.com/reference.

You can call finish() from within this function, in which case
  onDestroy() will be immediately called without any of the rest of the
  activity lifecycle (onStart(), onResume(), onPause(), etc) executing.

Reason I ask is: I would like to check data from the Bundle passed to onCreate(). Of course I have control of what is passed to onCreate, but I still think it should be checked at the point of delivery.
My code contains class A, which starts Activity B. I believe the last two "outside of if clause" tags, shouldn't be called because the finish method in the if statement should have destroyed the activity.  It has nothing to do with the if clause because the tag line after the second finish() call is still also read.
My Code:
Class A
@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    // goToBButton: when pressed sends message to class B.
    Button goToBButton = (Button)this.findViewById(R.id.go_to__b_btn);
    goToBButton.setOnClickListener(new OnClickListener() {
        @Override
        public void onClick (View v) {
            Log.i(TAG,"A Class: goToBButton, onClick");
            Intent i = new Intent(A.this, B.class);
            startActivityForResult(i,REQ_TO_B);
        }
    });
} // end onCreate

My Code ClassB
    public class B extends Activity{

private static final String TAG = "tag";

@Override
   public void onCreate(Bundle savedInstanceState) {
  super.onCreate(savedInstanceState);
  setContentView(R.layout.layoutb);

  // set as true, should always print Tag: one line before first finish"
  if (true)  {

    Log.i(TAG,"B Class: one line before 1st finish");
    finish();
  }

  // shouldn't get here after first finish
  Log.i(TAG,"B Class: outside of if clause, before second finish");
  finish();
  // shouldn't get here after second finish
  Log.i(TAG,"B Class: outside of if clause, after finish");
   } // end onCreate

@Override
public void onStart () {
    super.onStart();
    Log.i(TAG,"B Class: onStart");
}

@Override
public void onRestart() {
    super.onRestart();
    Log.i(TAG,"B Class: onRestart");
}

@Override
public void onResume () {
    super.onResume();
    Log.i(TAG,"B Class: onResume");
}

@Override
public void onPause () {
    super.onPause();
    Log.i(TAG,"B Class: onPause");
}

@Override
public void onStop () {
    super.onStop();
    Log.i(TAG,"B Class: onStop");
}

@Override
public void onDestroy () {
    super.onDestroy();
    Log.i(TAG,"B Class: onDestroy");
}

 } // end B Class

Here are the results of my tags:

11-26 15:53:40.456: INFO/tag(699): A Class: goToBButton, onClick
11-26 15:53:40.636: INFO/tag(699): A Class: onPause
11-26 15:53:40.865: INFO/tag(699): B Class: one line before 1st finish
11-26 15:53:40.896: INFO/tag(699): B Class: outside of if clause,
  before second finish
11-26 15:53:40.917: INFO/tag(699): B Class: outside of if clause,
  after finish
11-26 15:53:41.035: INFO/tag(699): A Class: onResume
11-26 15:53:41.165: INFO/tag(699): B Class: onDestroy
Comment: "immediately" was a very poor word choice in the docs...
Answer: I'm guessing that it is because finish() doesn't cause the onCreate method to return. You could try simply adding
finish();
return;

Or use an if else
@Override
public void onCreate(Bundle savedInstanceState) {
  super.onCreate(savedInstanceState);
  setContentView(R.layout.layoutb);
  if(good data){
      //do stuff
  }else{
      finish();
  }
}

Comment: First suggestion of "simply adding return" worked! After it gets to return, ClassA on resume is called, then Class B onDestroy is called.
Comment: Simply adding return; shows errors thereafter for unreachable code. To bypass this use the return in `if`
Answer: It seems like finish() does not work until onCreate() return control to system. Please refer to this post: about finish() in android. You have to consider this issue if you don't want any of your code to be executed after calling finish.
Hope it helps.

onCreate flow continues after finish()


Android Emulator Error missing WinHvPlatform.dll

Question: today, suddendly, all my Android Emulators (on Win10 / IntelliJ IDEA),
started complaining about a missing library.
When i launch any emulator, during loading, i read on the console log:
Emulator: Could not load library 'WinHvPlatform.dll'
then, the emulator starts and seems to run OK.
But... does anyone have an idea what it could be the cause ?
What is that library ?
Comment: Same thing for me: PS D:\Android\sdk\emulator> ./emulator -avd Nexus_5X_API_22
Could not load library 'WinHvPlatform.dll'.
D:\Android\sdk\emulator/emulator-x86.exe: invalid option -- '-enable-whpx'
Comment: WinHvPlatform.dll is probably the Hyper-V platform library.
Comment: I've got the same issue. Happened all of a sudden, and I'm having trouble installing my app to the emulator. Been working fine previously and no significant updates installed to cause this
Answer: Hyper-V should be disabled for the Android emulators, and use Intel HAXM instead (if your processor supports it). Also make sure you are using the updated Android SDK 27 tools and emulators, they can be updated as explained in this article.
Answer: It's because Microsoft has worked with Google to allow using Hyper-V instead of Intel HAXM as hypervisor. See these articles:

https://blogs.msdn.microsoft.com/visualstudio/2018/05/08/hyper-v-android-emulator-support/
https://learn.microsoft.com/en-us/xamarin/android/get-started/installation/android-emulator/hardware-acceleration

The Windows Hypervisor Platform was introduced in the Windows 10 April
  2018 Update and enables third-party virtualization stacks to utilize
  the Windows Hypervisor for hardware acceleration. If you are using
  Hyper-V, this stack replaces Intel HAXM as the hypervisor for the
  Android emulator.

If you run the ANDROID_SDK\emulator\emulator.exe -avd YOURAVD -verbose you might get this output:
emulator: Checking whether Windows Hypervisor Platform (WHPX) is available.
emulator: Could not load library WinHvPlatform.dll
emulator: WHPX is either not available or not installed.
emulator: CPU Acceleration: DISABLED
emulator: CPU Acceleration status: HAXM is not installed on this machine
emulator: ERROR: x86_64 emulation currently requires hardware acceleration!

You could follow the instructions in the above posts to turn off Windows Hypervisor for Google's Android emulator in %USERPROFILE%\.android\advancedFeatures.ini by setting:
WindowsHypervisorPlatform=off


Intuition for the standard error of the difference of sample means

Question: I read in Wilcox, 2003 p. 247 that the standard error of the difference between two sample means is (assuming the normality and homoskedasticity assumptions):
$\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$
Rather than simply adding the two sample standard errors as in:
$\frac{\sigma_1}{\sqrt{n_1}} + \frac{\sigma_2}{\sqrt{n_2}}$
What is the intuition behind taking the square of the sum of the two variances divided into their respective sample size, rather than the sum of the standard errors?
Comment: Perhaps the most famous--and likely the earliest--completely intuitive demonstration of this relation was obtained by Galton with his [quincunx](http://www1.kcn.ne.jp/~h-uchii/quinc.html).  (Don't visit Wikipedia or any of the top hits on this term, because they explain only its most trivial uses: the link I gave is the first I could find that reproduces one of Galton's illustrations and hints at the revelations this machine makes possible.)
Comment: The intuition is one of *cancellation of independent errors.*  It took scientists rather a long time--approximately 150 years from the beginning of probability theory in the mid 17th century to the work of Gauss and Laplace around 1800 as applied to combining astronomical observations--to realize that this actually happens with measurements! Because such cancellation is at the heart of the Central Limit Theorem, [intuitions for that](http://stats.stackexchange.com/questions/3734) are relevant here, too.
Comment: If you want standard errors, your second formula shouldn't have squared terms, and it's not the standard error of the distributions but the standard error of the sample means you're talking about there.
Comment: Yes sample means, not distribution means. Fixed the question.
Comment: You still have the first error I mentioned. I will fix it for you.
Comment: I have made the thing you called a sum of standard errors into a sum of standard errors.
Answer: You seem to be thinking that $\sqrt{\text{Var}(\bar X-\bar Y)} = \sqrt{\text{Var}(\bar X)} + \sqrt{\text{Var}(\bar Y)}$.
This is not the case for independent variables.
For $X,Y$ independent, $\text{Var}(\bar X-\bar Y) = \text{Var}(\bar X) + \text{Var}(\bar Y)$
Further,
$\text{Var}(\bar X) = \text{Var}(\frac{1}{n}\sum_iX_i) = \frac{1}{n^2}\text{Var}(\sum_iX_i)= \frac{1}{n^2}\sum_i\text{Var}(X_i)= \frac{1}{n^2}\cdot n\cdot\sigma^2_1= \sigma^2_1/n$
(if the $X_i$ are independent of each other).
http://en.wikipedia.org/wiki/Variance#Basic_properties
In summary: the correct term:

$\color{red}{(1)}$ has $\sigma^2/n$ terms because we're looking at averages and that's the variance of an average of independent random variables;
$\color{red}{(2)}$ has a $+$ because the two samples are independent, so their variances (of the averages) add; and
$\color{red}{(3)}$ has a square root because we want the standard deviation of the distribution of the difference in sample means (the standard error of the difference in means). The part under the bar of the square root is the variance of the difference (the square of the standard error). Taking square roots of squared standard errors gives us standard errors.
The reason why we don't just add standard errors is standard errors don't add - the standard error of the difference in means is NOT the sum of the standard errors of the sample means for independent samples - the sum will always be too large. The variances do add, though, so we can use that to work out the standard errors.

Here's some intuition about why it's not standard deviations that add, rather than variances.
To make things a little simpler, just consider adding random variables.
If $Z = X+Y$, why is $\sigma_Z < \sigma_X+\sigma_Y$?
Imagine $Y = kX$ (for $k\neq 0$); that is, $X$ and $Y$ are perfectly linearly dependent. That is, they always 'move together' in the same direction and in proportion.
Then $Z = (k+1)X$ - which is simply a rescaling. Clearly $\sigma_Z = (k+1)\sigma_X = \sigma_X+\sigma_Y$.
That is, when $X$ and $Y$ are perfectly positively linearly dependent, always moving up or down together, standard deviations add.
When they don't always move up or down together, sometimes they move opposite directions. That means that their movements partly 'cancel out', yielding a smaller standard deviation than the direct sum.
Comment: *Every* part of the correct formula is covered by my discussion above. Can you more clearly explain what you don't get and I can point to where it's already covered by my answer. I can try expanding my explanation further.
Comment: The intuition I can't capture is when you say "...the sum will always be too large". I need to read through @whuber links, if you can elaborate a bit more it would help. Thank you.
Comment: Do you see that $\sqrt{a^2+b^2}\leq a+b$ (with $a, b\geq0$), and you only have equality if one of them *is* $0$? (This is just that the hypotenuse of a right angled triangle is smaller than the sum of the other two sides)
Comment: Looked at another way, consider the implied variance. If $a+b$ is the standard error of the difference, the implied variance is $a^2+b^2+2ab$, which for $a,b>0$ is bigger than the sum of the variances. If you accept that $\text{Var}(X-Y)=\text{Var}(X)+\text{Var}(Y)$ then if you say "add the standard errors", clearly this results in an additional positive term in the variance that shouldn't be there. i.e. making it 'too large'
Comment: ok, I see all the numbers, although the intuition probably lies in geometry, rather than statistics. I get that adding variances would overestimate the standard error, because of the extra $2ab$ term, but why squaring the sum of the variances would give the "right" estimate? Yes, this is really a geometry question.
Comment: Sorry to be dense for so long about what you're after; I will add some more detail to my answer.
Comment: I'm ok with the variance algebra, what I am asking is why the square root goes to the sum of the two variances divided into the sample sizes. Notice that in the second (wrong) formula the variance is already squared, that is we use the standard deviation.
Comment: I have gone into more detail. If that doesn't suffice, please indicate clearly where the problem lies.
Answer: Algebraic intuition
The standard error of the mean for $n$ independent observations is $\frac{\sigma}{\sqrt{n}}$ where $\sigma$ is the standard deviation.
So if we have two independent samples we have the standard errors for the means of group 1 and group 2.
$$\sigma_{\mu_1}=\frac{\sigma_1}{\sqrt{n_1}}$$
$$\sigma_{\mu_2}=\frac{\sigma_2}{\sqrt{n_2}}$$
If we square these values we get the variance of the mean:
$$\sigma^2_{\mu_1}=\frac{\sigma^2_1}{n_1}$$
$$\sigma^2_{\mu_2}=\frac{\sigma^2_2}{n_2}$$
The variance of the sum or difference of two independent random variables
is the sum of the two variances. Thus,
$$\sigma^2_{\mu_1 - \mu_2} =\sigma^2_{\mu_1} + \sigma^2_{\mu_2} =  \frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2} $$
So if we want the standard error of the difference we take the square root of the variance:
$$\sigma_{\mu_1 - \mu_2} =\sqrt{\sigma^2_{\mu_1} + \sigma^2_{\mu_2}} =  \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}} $$
So I imagine this is intuitive if the component steps are intuitive. In particular it helps if you find intuitive the idea that the variance of the sum of independent variables is the sum of the variances of the component variables.
Fuzzy Intuition
In terms of more general intuition, if $n_1 = n_2$ and $\sigma=\sigma_1=\sigma_2$ then the standard error of the difference between means will be $\sqrt{2}\sigma_\mu\approx 1.4\times \sigma_\mu$. It makes sense that this value of approximately 1.4 is greater than 1 (i.e., the variance of a variable after adding a constant; i.e., equivalent to one sample t-test) and less than 2 (i.e., the standard deviation of the sum of two perfectly correlated variables (with equal variance) and the standard error implied by the formula you mention: $\frac{\sigma_1}{\sqrt{n_1}} + \frac{\sigma_2}{\sqrt{n_2}}$).
Answer: You don't square the sum of the variances, you take the square root of the sum of the variances. You do this for the same reason that the standard deviation is the square root of the variance: It make the units the same as the original ones, rather than squared units.
Although we often lose sight of it while doing statistics, the square of a measure involve squaring the measure as well as the number of units. For example, the square of 2 meters is not 4 meters, it is 4 meters squared, more commonly called 4 square meters. The same thing happens with other units that we aren't used to thinking of in this way: e.g. if you are measuring IQ, the square of an IQ is not an IQ of 10,000; it is a squared IQ of 10,000.
You divide by the sample size as a scaling technique. Variances (tend to) go up with sample size; you divide by $n$ to deal with that.


Why Is It Important To Simulate Data From A Statistical Model?

Question: Within Statistics, I have seen the following applications of simulation:

Bayesian Sampling: When the Posterior Distribution can not be analytically integrated, we use MCMC algorithms (e.g. Metropolis-Hastings) to "simulate" points from the Posterior Distribution
Model Misspecification: Suppose I create a statistical model where I assume that the response variable has a certain distribution (e.g. Exponential Distribution) - I can simulate data from different distributions (e.g. Gamma Distribution) and see how "robust" my model is to conditions outside of what the model was created under

However, I keep hearing that it can be important to simulate data from the same model itself - for example, I fit a specific regression model on a specific dataset, I might need to simulate data from this regression model itself. I am trying to understand why this might be important and useful.
As an example, suppose there is a school with 100 students. I assume that the heights of these students follow a Normal Distribution. I then measure the height of all students, calculate the average height (e.g. "a") and the standard deviation (e.g. "b"). I now have a Normal Distribution(a,b).
If I simulate many samples from this Normal(a,b) - the average value of all these simulations should equal to "a". The way I see it, simulating numbers from this Normal(a,b) has not really provided me with any new information .
This is where my confusion is:

I can understand why its important to simulate data from a different model to test the robustness of your own model. As an example, if I design a bridge that I believe is well suited to design weights of "x KG" and windspeeds of "y KM/h" - I might be interested in seeing how this bridge behaves under different conditions, such as with weights of " x + z KG" and windspeeds of "y + w KM/h".
But why is it important to simulate data from the same regression model that you just fit - how can this result any new information that the you can't analytically infer from the model itself?

To reiterate my confusion one more time -  I can simulate 1000 random samples from a Normal(a,b) and find out that the average value of these samples is close to "a".... but I could have just taken the Expected value of Normal(a,b) and determined that the average of many samples from this distribution should be close to "a". Thus, what is the importance of simulating from the same model itself?
Thanks!
Note:

Additional Reference:
"When conducting simulation studies to evaluate the performance of new and existing statistical methods for analyzing survival data, one is required to simulate event times under a known data generating model. Similarly, one may need to simulate event times for the purpose of power calculations when designing new studies." (https://www.jstatsoft.org/article/view/v097i03)
However, when reading this quotation - it is not immediately clear to me why simulating data from the same model is required for goals such as  "power calculations"
Comment: You start your story with explaining why sampling is useful, and then following that you add a twist where you say that it is unclear why sampling is useful. But *why* you think this is unclear does not become clear.
Comment: "When conducting simulation studies to evaluate the performance of new and existing statistical methods for analyzing survival data, one is required to simulate event times under a known data generating model. Similarly, one may need to simulate event times for the purpose of power calculations when designing new studies."
Comment: If I understand the reference correctly, it doesn't actually say that you should simulate from the assumed model when doing data analysis. Rather it looks like this is meant for methodological research; if you don't have (exact) theory, you need simulations to compare estimators and make recommendations. (Of course this doesn't make it invalid to do it also in practical situations, but I don't think this is what they have in mind.) It's correct of course that this is another use of simulated data, from the model of interest.
Comment: *"The way I see it, simulating numbers from this Normal(a,b) has not really provided me with any new information."* It hasn't provided ***you*** with any new information, because you already knew that the distribution was normal. But if you're feeding the simulated data into a model that doesn't know that, then this model will view this as new information.
Comment: Additional Reference: https://www.jstatsoft.org/article/view/v097i03
Comment: This question sounds so familiar. Did you try to look up previous question before posting? Maybe it is how you generated this question (possibly with the aid of AI), if so, then please add them to your question because this duplication of basic questions is not very beneficial for the database. (Unless the goal is to create duplicates and keep on reinventing wheels, maybe the process is more important than the actual wheel, one day that wheel might turn into a hoverboard)
Comment: *"I keep hearing that it can be important to simulate data from the same model itself"* where are you hearing this?
Comment: @SextusEmpiricus "This question sounds so familiar. " It sounds familiar because the OP had another question on this topic 3-4 days ago that got closed. (That attempt had a Bayesian flavor.) So the OP rewrote the question and tried again, with even more generic content.
Answer: As you've noted, simulating from the posterior is common in Bayesian modelling. It is also common to simulate from the prior, which is often called prior predictive sampling or prior predictive checking. The purpose of doing these simulations is to check that the model predicts scientifically justifiable outcomes even before it is updated with data. For example, I'm going to seriously doubt my prior is justifiable if it predicts that moose run at the speed of light with some non-negligible probability.
You can read more about prior predictive checking in Bayesian Workflow, or see some code in the pymc docs and stan docs.
Comment: @ Galen: Thank you for your answer! I have heard about "predictive prior checking" -but I did not mention it in my question as I was trying to keep it concise.
Comment: Spoken like someone who has never seen a moose run out in front of a car!
Comment: @Obie2.0 You should adjust your priors then b/c I actually have! :P
Answer: I should start by saying that most people in most data analysis situations will not simulate data from the assumed model, so the majority view seems to be that you can well get away without doing it. "I keep hearing" isn't exactly an authoritative reference. ;-)
However, a major use is connected to the issue of potential model misspecification. Simulating data from the model and using visualisation techniques in particular, you can see whether and in what sense your real data set is similar or deviates systematically from data sets simulated from the model.
Otherwise, very often the theory behind the methods that you are using is not exact (standard mean estimation under Normal assumption being an exception), but rather asymptotic (sample size going to infinity) or in other ways approximate. Simulating from the model then allows you to explore the simulated variation/uncertainty and compare it with (approximative) theory. An example is variable selection in regression in a stepwise manner using p-values (I don't recommend to do that!) - there is no (exact) theory that nails down how it will behave in general, and simulating from the model is a very good way to see what the problem with it is, and how reliable (or not) it is.
Comment: +1 Your second paragraph is a more general form of my answer.
Comment: @ Christian Hennig: Thank you for your answer! I added a reference (please see update). Thank you!
Answer: Resources
An answer that is quite a bit more practical is cost. If you are going to invest time and money into a research project that tax dollars are going to contribute to, it might be helpful to know that your modeling is actually accurate to a degree. If I get awarded 500k USD to conduct research on the survival rate of germ species in hot climates, I better have an idea of what I'm doing before I throw away every cent.
Assumption Checking
Part of it can also be for checking your own assumptions about statistical modeling. Perhaps I think that any sample size, so long as it has strong effects, is "good enough" to conduct a study. I can simulate truly random data to emulate what should be known about sample sizes.
Personal Interest
Simulating data is also just fun. I know quite little about simulation, but every small nugget I learn makes me fascinated by what info I derive from it.
Answer: Simulating from the same model that you use to analyze the data is useful for understanding the model and its limitations. To some extent it's a sanity check that the model does not have some serious issues, because a model that does not work very well when its assumptions are met is problematic. You may say:"But how could this ever be the case?" And the answer includes, but is not limited to:

You have messed up and something you assumed/derived/implemented is just not right.
If you simulate a huge amount of data, you should hope to get something close to the parameter values you simulated with back. If you don't there's a hint the first bullet might apply.
Parameter identifiability: I.e. the data you plan to collect/the experiment you do does not inform you at all - or at least not very well - about the parameter you are interested in. Example: randomized experiment with a placebo group and a single dose of a drug (let's say 50 mg), the outcome is wonderfully normally distributed and you fit a non-linear sigmoid Emax model assuming that $Y_i \sim \beta_0 + \beta_1 \text{dose}^h / (\text{dose}^h + \text{ED50}^h) + \epsilon_i$ with i.i.d. $\epsilon_i \sim N(0, \sigma^2)$. All assumption for your model are met, it is even the true data generating model, but with just one active dose the parameters are not identified. This may sound like a very specific situation (and it is), it's just pointing out that you may occasionally be in that kind of situation (and not realize it, e.g. you specify multiple levels of variability that cannot be distinguished).
Poor small sample performance (where "small sample" could occasionally be pretty large) e.g. due to using asymptotics. Example: Logistic regression fit using maximum likelihood with asymptotic standard errors (vs. exact logistic regression or similar) in case of 0 (or 1) events out of 100 (group 1) vs. 5 events out of 100 (group 2). You might not realize this is an issue, until you evaluate systematically what your model/its implementation does in these situations.
Some particular issues keep occurring that cause issues in defining what the maximum likelihood estimate would be (like complete separation in logistic regression).
Numeric optimization, convergence or sampling problems (that might be particularly common in some data situations).
Other numerical problems (e.g. covariates that are on very different scales cause issues, finite difference approximation makes a MCMC sampler not work when proper auto-differentiation would have worked etc.)
Answer: Parametric bootstrapping relies on simulating data from an estimated model.  If it is difficult to work out confidence intervals analytically, then simulating datasets from your estimated model can give you an estimate of the sampling distribution of your estimates.
Answer:
If I simulate many samples from this Normal(a,b) - the average value of all these simulations should equal to "a". The way I see it, simulating numbers from this Normal(a,b) has not really provided me with any new information.

In this example, simulation is not important. The distribution of Normal(a,b) is well known and can be estimated with several computational techniques.
Simulation is important when the distribution is not so easily computed.

However, I keep hearing that it can be important to simulate data from the same model itself

It is not clear where you got this from. It might relate to situations where the model is not so clearly computed as your normal distribution example.


Strict local extremum without $f'$ "changing signs"

Question: Let $f:\mathbb{R}\to \mathbb{R}$. Is it possible that $f$ has the following properties:

$f$ is differentiable in a neighborhood of $a\in \mathbb{R}$
$a$ is a strict local minimum
There is no neighborhood $(a-\delta, a+\delta)$ of $a$ such that
$f'<0$ in $(a-\delta, a)$ and $f'>0$ in $(a,a+\delta)$?

It is definitely possible if we require only that $a$ be a (non-strict) local minimum; the function $x^2\sin^2(1/x)$ exhibits such behavior at zero.
It is definitely impossible if $f$ is a polynomial, since we can write $f(x) = (x-a)^mg(x) + f(a)$, where $g(a)\neq 0$, and $f(x)-f(a)$ behaves locally like $(x-a)^m$. (This also shows that if $f$ is a polynomial, $f$ has a local extremum at $a$ iff $f'$ has a zero of odd order at $a$.)
I believe it is impossible if $f'$ is continuous at $a$, since then I claim that $f$ has a local extremum iff $f'$ has a neighborhood as in property (3).

If you're interested, the way I came up with this question is: I was showing that if $f=x^n + a_{n-1}x^{n-1} + \dotsb + a_1x + a_0$ has number of local maxima equal to $k_1$, and number of local minima equal to $k_2$, then $k_1=k_2$ if $n$ is odd, and $k_1 + 1=k_2$ if $n$ is even. I wondered if a similar property held for non-polynomial functions if we require that they be strict local maxima and minima.
Comment: See Example 4 [here](http://books.google.com/books?id=cDAMh5n4lkkC&pg=PA36&source=gbs_toc_r&cad=3#v=onepage&q&f=false) for an example with $f'$ satisfying the continuity condition. If you can't access it, the function is $f(x)=x^4\bigl(2+\sin(1/x)\bigr)$, $x\ne0$; $f(0)=0$.
Answer: How about
$$f(x)=\begin{cases}x^2(2+\sin(1/x)) & x\ne 0 \\ 0 & x=0 \end{cases}$$
Differentiable everywhere, global minimum at $x=0$, derivative both positive and negative in any interval with $0$ as an endpoint.
(Edit: Or just see David Mitra's comment which is even better)


Find the area bounded by the curve $y^2=4ax$ and $x=a$

Question: Find the area bounded by the curve $y^2=4ax$ and $x=a$
My attempt:
Solving equations $y^2=4ax$ and $x=a$ we get $y=\pm 2a$
And the area $=\int y. dx$
Is this correct?
Answer: If you swich both axsis you can figure out you can do like this:
$$Area = 4a^2-2\int _0^{2a}{y^2\over 4a}dy = 4a^2-2{y^3\over 12a}\Big|_0^{2a}$$ $$= 4a^2- 2{8a^3\over 12a}= 4a^2 -4a^2/3 =8a^2/3$$
Comment: the answer in my book is $\dfrac {8a^2}{3}$??
Comment: how did you get the limits $0$ and $a$?
Comment: Just reflect everything across the line $y=x$.
Answer: Solving $y^2=4ax$ and $y=6a$ we get
$x=9a$ so our integral will be $2\int_{0}^{9a}\sqrt{4ax}dx$
Answer: I always graph things to see what the region looks like. Using Desmos for $a=1$ this looks like the following.

We want the area between the red curve and the blue line.
There are many methods. The most straight-forward is to find the integral
$$
\int_{0}^{a} \sqrt{4ax} \,\,\mathrm{d}x
$$
which will give you the top half of the region, and then double it to get the whole region. Thus
$$
\int_{0}^{a} \sqrt{4ax} \,\,\mathrm{d}x = 2\sqrt{a} \int_{0}^{a} x^{\frac{1}{2}} \,\,\mathrm{d}x = 2\sqrt{a} \cdot\Big[ \frac{2x^{3/2}}{3} \Big]_{x=0}^{x=a} = \frac{4}{3}\cdot a^{2}.
$$
This is the top half of the region, so the area of the whole region is
$$
\frac{8}{3}\cdot a^{2}.
$$


Normalization Factor Wrong? (Bug?)

Question: I'm new to PyMC and Bayesian stuff in general, so I started off with what I thought was a very simple toy problem.  I generated some normally-distributed noise with a given mean and standard deviation, and then played with PyMC and looked at the posterior distribution of possible mu/sigma.
It's possible that I just don't understand what I'm looking at, but shouldn't the standard deviation of the trace for mu converge on the standard error of the mean?
It's currently off my a factor of sqrt(2).
Simply:
Parent normal distribution with parameters mu0, sigma0.
Child population sampled from parent distribution, resulting in mu1, sigma1.
SE = sigma1/sqrt(N)
SD = sigma0/sqrt(N)
Shouldn't SE~SD~mcmc.trace['mu'].std() for sufficiently large N and # of iterations?  What I actually see is:
SE~SD~mcmc.trace['mu'].std()/sqrt(2)
This is using PyMC 2.2 and here is the code:
%pylab inline
import pymc as mc

true_noise_mu = 50
true_noise_sigma = 5
true_noise_datapoints = 2000

noise_field = np.random.normal(true_noise_mu, true_noise_sigma, true_noise_datapoints)

data_mean = noise_field.mean()
data_std = noise_field.std()

mu = mc.Normal( 'mu', data_mean, 1/(data_std/sqrt(n_datapoints))**2)

# just guessed at tau here, but it shouldn't impact the result (nor does it), #only the time to converge, yes?
sigma = mc.Normal( 'sigma', 1/data_std**2, 0.01)

observation = mc.Normal( "obs", mu, sigma, value=noise_field, observed=True)

mcmc = mc.MCMC(model)
mcmc.sample( 400000, 20000, 1 )

mu_samples = mcmc.trace('mu')[:]
sigma_samples = mcmc.trace('sigma')[:]

print "SEM estimated from sample population: %.3f" % (noise_field.std()/sqrt(true_noise_datapoints))
print "SEM calculated from parent population: %.3f " % (true_noise_sigma / sqrt(true_noise_datapoints))
print "SEM calculated from posterior distribution: %.3f" % (mu_samples.std())
print "Ratio of posterior: sample estimate: %.3f" % ((mu_samples.std())/(noise_field.std()/sqrt(true_noise_datapoints)))
print "1/sqrt(2): %.3f" % (1/sqrt(2))

Last Print:
SEM estimated from sample population: 0.677
SEM calculated from parent population: 0.671
SEM calculated from posterior distribution: 0.478
Ratio of posterior: sample estimate: 0.707
1/sqrt(2): 0.707

I'm probably just an idiot, but I would love to understand what I'm missing. =)
Comment: EDIT: actually, you are interested in the evaluation of `Uniform(..)`, so I would code it as `sigma=  Uniform("sigma", 0, 50); precision = sigma**(-2)` and use precision in the observation variable.
Comment: An quick detail is that you are using a `Normal` to model the standard deviation. This us a poor choice as the Normal can return negative values (which a standard deviation forbids, and running your code throws a `ZeroProbability: Stochastic obs's value is outside its support` error.) A better choice is a Uniform, i.e.`sigma = mc.Uniform("sigma",
0, 50)**(-2)`.
Answer: Try removing the sqrt(n_sata_points) from the variance of the mu prior. The results are that the ratio in question is (approximately) 1. This suggests the error is in that term.
Assume for the moment that the true std. deviation is known, call it $\sigma$. Also, define
$\hat{\mu}$ as the empirical mean. You have assigned the prior
$$ \mu \sim \text{Normal}\left( \hat{\mu}, \frac{ \sigma^2}{N} \right)$$
to the unknown $\mu$.
The likelihood is Normal (Normally distributed data) as well. Hence, the posterior will also be Normal (by conjugancy, see here). The standard deviation of the posterior Normal, according to the wiki page, is:
$$\left( \frac{1}{ \frac{ \sigma^2 }{N} } + \frac{ N }{\sigma^2} \right)^{-1}$$
$$ = \frac{\sigma^2}{2N}$$
which I believe is where your sqrt 2 is coming from. (of course, this still assumes we know $\sigma$, but for large $N$ our estimate $\hat{\sigma} \approx \sigma$)

There is something I don't like about using $\frac{\sigma^2}{N}$ as the prior variance. It feels like you are double-counting your sample size (hence the additional factor of 2 cropping up).
I would suggest using $\hat{\sigma}$ or even better is a less subjective constant like 100 or something.

Normalization Factor Wrong? (Bug?)


How to force a menu parent link to be non-clickable?

Question: I want the parent link in the menu to not be clickable. As a path is mandatory, I have created a blank node, and in my module I substitute the link by a "#" char.
I used custom code in module because I have a multidomain/multilanguage site, and a mixture of Drupal and database own tables content for the links. Many solutions for custom menus could not be used as they were not compatible with Domain Access or there was some problem in customizing.
The code I use is:
<code>$a = menu_tree_output( menu_tree_page_data("main-menu") );
if(in_array($value["#href"], array("node/123", "node/456"))) $nolink = true;
</code>
I wonder if there is a workaround to this problem, as the typical behavior for a parent menu is to be non-clickable, not to load another page, which would be, IMHO, less typical.
Answer: You can take look at Special menu items.
User can create a new menu item and place either "nolink" or "separator" to the Path field, without quotes.
Nolink item will be rendered similar to a normal menu link item but there will be no link just the title.
Here is another module you can check: Content Menu
You can create a new menu item as

URL – With the traditional menu item form
Dummy – As a dummy menu item (with simple dummy link target)
New NODETYPE – directly create a new content for the new item


How are the number of laps in an F1 race determined?

Question: The number of laps in a F1 race seem to differ wildly. Obviously the length of each lap varies from circuit to circuit, but how are the number of laps for each race decided?
Additionally, is this number chosen by the owners of the race track, or something decided by the FIA/ teams?
Answer: The number of laps are decided by taking a distance of 305 km and dividing it by the length of a lap (rounded up), which varies from racetrack to racetrack. (1)

quoting the official regulations (from 2011):

5.3
  The distance of all races, from the start signal referred to in Article 38.9 to the chequered flag, shall be
  equal to the least number of complete laps which exceed a distance of 305 km (Monaco 260km). However,
  should two hours elapse before the scheduled race distance is completed, the leader will be shown the
  chequered flag when he crosses the control line (the Line) at the end of the lap during which the two hour
  period ended. However, should the race be suspended (see Article 41) the length of the suspension will be
  added to this period.
Comment: I think this answer actually covers the how perfectly - simple mathematics. Why wasn't asked, but is probablt a combination of identifying a useful length to interest drivers, advertisers, management etc.
Comment: The mathematical formula is therefore 305 / lap length in km (rounded up).  This also means that the longest races in terms of time are those where the average speed is slowest.


Hidden state models vs. stateless models for time series regression

Question: This is a quite generic question: assume I want to build a model to predict the next observation based on the previous $N$ observations ($N$ can be a parameter to optimize experimentally). So we basically have a sliding window of input features to predict the next observation.
I can use a Hidden Markov Model approach, i.e. Baum-Welch to estimate a model, then Viterbi to predict a current state based on the last $N$ observations, then predict the most likely next state based on the current state, and then predict the next observation using the most likely next state and the HMM parameters (or variants such as find the predictive distribution of the next observation).
Or I can use a much simpler approach, using a stateless model (which can get as input the previous $N$ observations), e.g. SVM, linear regression, splines, regression trees, nearest neighbors, etc. Such models are based on minimizing some prediction error over the training set and are therefore, conceptually, much simpler than a hidden state based model.
Can someone share her/his experience in dealing with such a modelling choice? What would speak in favour of the HMM and what in favour of a regression approach? Intuitively one should take the simpler model possible to avoid over-fitting; this speaks in favour of a stateless approach...We also have to consider that both approaches get the same input data for training (I think this implies that if we do not incorporate additional domain knowledge in the modelling of a hidden state model, e.g. fix certain states and transition probabilities, there is no reason of why a hidden state model should perform better). At the end one can of course play with both approaches and see what performs better on a validation set, but some heuristics based on practical experience might also be helpful...
Note: for me it is important to predict only certain events; I prefer a model which predicts few "interesting/rare" events well, rather than a model which predicts "average/frequent" events but the interesting ones not so well . Perhaps this has an implication for the modelling choice.
Thanks.
Comment: Perhaps [this earlier question](http://stats.stackexchange.com/questions/78287/what-are-disadvantages-of-state-space-models-and-kalman-filter-for-time-series-m) is one half of the question presented here.
Comment: Can you clarify why you believe regression models are necessarily *stateless*? Dynamic linear regression models (in which previous values of the predictand are included on the right-hand side of the model equation) would seem very much to be *state-conditioned*. But perhaps I am missing something.
Comment: thanks for reading the question. I would say it is a bit a question of semantics, I also give an example of regression models which include the n-past observation values on the right hand side of the model, such a model is of course dynamic. However, I was more referring to the concept of a hidden/latent variable for which usually EM techniques are used to find the model vs models for which we do not have such hidden states (i.e. the states are observable, they are the observations). From a practical and pragmatic perspective, is it possible to tell what works better and when?
Comment: I missed the fact that you refer to past values of the prediction as inputs.Are such models the equivalent of a hidden state model (in principle they would just include more than N observation, replacing the equation for the past predictions)?For me the question is more if we observe the state and model it or if we infer the state given an assumption of the model. I am more interested in the practical aspect however, not the mathematical one. I.e. is it possible to tell under which conditions the one or the other approach works better?(I think no theorem can provide an answer to this question)
Answer: In short, I think they are working in different learning paradigm.
State-space model (hidden state model) and other stateless model you mentioned are going to discover the underlying relationship of your time series in different learning paradigm: (1) maximum-likelihood estimation, (2) Bayes' inference, (3) empirical risk minimization.
In state-space model,
Let $x_t$ as the hidden state, $y_t$ as the observables, $t>0$ (assume there is no control)
You assume the following relationship for the model:
$P(x_0)$ as a prior
$P(x_t | x_{t-1})$ for $t \geq 1$ as how your state change (in HMM, it is a transition matrix)
$P(y_t | x_t)$ for $t \geq 1$ as how you observe (in HMM, it could be normal distributions that conditioned on $x_t$)
and $y_t$ only depends on $x_t$.
When you use Baum-Welch to estimate the parameters, you are in fact looking for a maximum-likelihood estimate of the HMM.
If you use Kalman filter, you are solving a special case of Bayesian filter problem (which is in fact an application of Bayes' theorem on update step):
Prediction step:
$\displaystyle P(x_t|y_{1:t-1}) = \int P(x_t|x_{t-1})P(x_{t-1}|y_{1:t-1}) \, dx_{t-1}$
Update step:
$\displaystyle P(x_t|y_{1:t}) = \frac{P(y_t|x_t)P(x_t|y_{1:t-1})}{\int P(y_t|x_t)P(x_t|y_{1:t-1}) \, dx_t}$
In Kalman filter, since we assume the noise statistic is Gaussian and the relationship of $P(x_t|x_{t-1})$ and $P(y_t|x_t)$ are linear. Therefore you can write $P(x_t|y_{1:t-1})$ and $P(x_t|y_{1:t})$ simply as the $x_t$ (mean + variance is sufficient for normal distribution) and the algorithm works as matrix formulas.
On the other hand, for other stateless model you mentioned, like SVM, splines, regression trees, nearest neighbors. They are trying to discover the underlying relationship of $(\{y_0,y_1,...,y_{t-1}\}, y_t)$ by empirical risk minimization.
For maximum-likelihood estimation, you need to parametrize the underlying probability distribution first (like HMM, you have the transition matrix, the observable are $(\mu_j,\sigma_j)$ for some $j$)
For application of Bayes' theorem, you need to have "correct" a priori $P(A)$ first in the sense that $P(A) \neq 0$. If $P(A)=0$, then any inference results in $0$ since $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$.
For empirical risk minimization, universal consistency is guaranteed for any underlying probability distribution if the VC dimension of the learning rule is not growing too fast as the number of available data $n \to \infty$


Tomcat Denying access to resource file in exploded war directory

Question: We have a WAR application deployed on Tomcat 7.0.65. This application uses a library that has a bunch of polygon.txt files in it's resource directory. Whenever the library is invoked it throws a file permission exception on the jar file in the exploded war directory.
here is the error:
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/var/lib/tomcat7/webapps/**/WEB-INF/lib/table-service-1.37.jar" "read")
    at org.apache.catalina.loader.WebappClassLoaderBase.getResourceAsStream(WebappClassLoaderBase.java:1659) ~[catalina-7.0.65.jar:7.0.65]
    at com.**.PropertyGeometryDAO.initialize(PropertyGeometryDAO.java:47) ~[table-service-1.37.jar:?]

Thrown at this line in the code:
try (InputStream stream = getClass().getClassLoader().getResourceAsStream(fileName))

Here is the catalina.policy file defining the permissions:
permission java.io.FilePermission "/usr/share/tomcat7/webapps/*", "read";

We also have separate WAR's deployed on the same Tomcat using the same library that are working just fine.
Comment: How about specifiying read,wite acess both?
 https://tomcat.apache.org/tomcat-7.0-doc/security-manager-howto.html#Tomcat_Custom_Permissions
Answer: The exception is for
"/var/lib/tomcat7/webapps/**/WEB-INF/lib/table-service-1.37.jar"

but your permission is for
permission java.io.FilePermission "/usr/share/tomcat7/webapps/*", "read";

Comment: I should have included, there is a link from /usr/share/tomcat7/webapps -> /var/lib/tomcat7/webapps
Comment: You may need the readLink FilePermission then https://docs.oracle.com/javase/7/docs/api/java/io/FilePermission.html


To prove: $\bigl(1 + \sin(A+B)\bigr)\bigl(1-\sin(A-B)\bigr) = (\cos A - \sin B)^2 $

Question: How could one show that
$$ \bigl(1 + \sin(A+B)\bigr)\bigl(1-\sin(A-B)\bigr) = (\cos A - \sin B)^2
$$
I have tried the below approach:
\begin{align*}
LHS & = \bigl(1 + \sin(A+B)\bigr)\bigl(1-\sin(A-B)\bigr)\\
& =  1-\sin(A-B)+\sin(A+B)-\sin(A+B)\sin(A-B)\\
& = 1 - \sin{A}\cos{B} + \sin{B}\cos{A} + \sin{A}\cos{B} + \sin{B}\cos{A}\\
& \qquad - (\sin{A})^2 + (\sin{B})^2\\
& = 1 + 2\sin{B}\cos{A} - 1 + (\cos{A})^2 + (\sin{B})^2\\
& = 2\sin{B}\cos{A} + (\cos{A})^2 + (\sin{B})^2\\
& = (\cos{A} + \sin{B})^2
\end{align*}
Comment: Please edit your question to show your attempt since that helps users detect any errors you may have made.  This [tutorial](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference?page=1&tab=votes#tab-top) explains how to typeset mathematics on this site.
Comment: After expanding, did you try using [these](https://mathworld.wolfram.com/TrigonometricAdditionFormulas.html)?
Comment: first I tried to expand the LHS but couldn't move forward.
Comment: i am getting (cosA+sinB)^2 after using the above formula and simplifying
Comment: I think you are correct. There must be a typo in the problem statement.
Comment: The case $A=B$ suggests we should either change the LHSs' sines' signs, or the $-$ sign to $+$ on the RHS.
Answer: \begin{aligned}
\text { LHS }=&(1+\sin (A+B))(1-\sin (A-B)) \\
=& 1-\sin (A-B)+\sin (A+B)-\sin (A+B) \sin (A-B) \\
=& 1- \sin A \cos B + \cos A \sin B + \sin A \cos B + \cos A \sin B -\sin (A+B) \sin (A-B)\\
=& 1 + 2\cos A \sin B - \sin (A+B) \sin (A-B) \\
=& 1 + 2\cos A \sin B - \frac{1}{2} (cos(A+B-A+B) - cos(A+B+A-B)) \\
=& 1 + 2\cos A \sin B - \frac{1}{2} (cos(2B) - cos(2A)) \\
=& 1 + 2\cos A \sin B - \frac{1}{2} [(1 - 2 \sin^{2} B )-(2 \cos^{2} A - 1)] \\
=& 1 + 2\cos A \sin B - \frac{1}{2} (1 - 2 \sin^{2} B ) +\frac{1}{2} (2 \cos^{2} A - 1) \\
=&  2\cos A \sin B + \sin^{2} B + \cos^{2} A\\
=& \cos^{2} A - 2\cos A \sin B + \sin^{2} B \\
=&(\cos A + \sin B)^{2}
\end{aligned}
Maybe the exercise is really wrong.
I am attaching a link where you can see the formulas used. (https://doza.pro/art/math/geometry/en/trig-formulas)
I hope I helped you.
Answer: $$(1+\sin A\cos B+\cos A\sin B)(1-\sin A\cos B+\cos A\sin B)$$
$$=(1+\cos A\sin B)^2-(\sin A\cos B)^2$$
$$=1+2\cos A\sin B+\cos^2A\sin^2B-(1-\cos^2A)(1-\sin^2B)$$
$$=\cdots$$
$$=(1+\cos A\sin B)^2$$


Maior valor comparando linhas

Question: Estou usando SQL Server 2008 R2 e possuo a seguinte tabela:
ID  NUMERO      NOME   MODELO
--- ----------- ------ -------
1   12          A      777
2   23          A      777
3   05          A      777
4   45          B      999
5   65          C      555
6   30          B      999
7   67          B      999
8   80          C      555
9   51          C      555
10  03          C      555

Gostaria que minha query retornasse a seguinte tabela:
ID  NUMERO      NOME   MODELO
--- ----------- ------ -------
2   23          A      777
7   67          B      999
8   80          C      555

Ou seja, quero que retorne os registros com os maiores valores da coluna NUMERO entre as linhas, independente de quantos registros existir.
Comment: `group by` por modelo, com `subselect` pegando o `max(numero)` ... :)
Comment: Se estiver certa convém colocar la o visto.
Comment: Obrigado, deu certo!
Answer: Creio que seria algo parecido com isso
SELECT A.* FROM Tabela A INNER JOIN (
SELECT MAX(numero) as numero, nome FROM Tabela GROUP BY nome
) B ON A.nome = B.nome AND A.numero = B.numero ORDER BY nome

Abraços
Comment: Isso aí. Eu só substituiria o `nome` por algo com mais garantia de unicidade, como `ID`.
Comment: Eu também, porém eu escrevi em conformidade ao exemplo dele.
Comment: Obrigado!!! Deu certo!
Answer: Outra solução:
-- código #1
with tabela2 as (
SELECT *,
       Seq= row_number() over (partition by NOME order by NUMERO desc)
  from tabela
)
SELECT ID, NUMERO, NOME, MODELO
  from tabela2
  where Seq = 1;

Maior valor comparando linhas


What is the meaning of these letters above the soprano part in Monteverdi's SV 332?

Question: Consider following score of Monteverdi's Sì dolce è 'l tormento for soprano and continuo (SV 332) from IMSLP. What is the meaning of the capital letters above the soprano part? They do not seem to correspond to chord names.
Comment: @ PiedPiper: it can't be this. a) what chords? b) later in the sheet there are O and other letters.
Comment: @PiedPiper  maybe?  and "E" over a pair of D notes seems odd, as does a "D" over an A and E note pair.
Comment: Any chance they indicate organ stops?  I don't know what organ music indicators look like
Comment: Very interesting. I couldn't find any source about this notation.
Answer: The letters correspond to particular tablature of the 5-sting guitar popular at the time. Here's one tablature translation from:
http://www.patriciadixon.net/guitar-lit-html/italianbaroque.htm
Comment: It's also worth noting that the 5-string guitar would have been tuned E-B-G-D-A like the top five strings of the modern guitar, and the diagram is inverted compared to modern tablature notation.
Comment: Now that you've identified the answer perhaps you should delete all of the earlier speculation or at least relegate it to a footnote.  Also, if the letters are editorial, they are the work of a 17th century editor, as you can see by looking at the first link in the IMSLP page.


Are there any good online resources on how to build drums?

Question: I'm really keen to have a go at crafting my own drum. Not a djembe or ethnic instrument but standard hooped kit drum. I particularly like the idea of something along the lines of Outlaw Drums and would love to reproduce something akin to that.
Does anyone know any good woodturning / woodworking sites or resources that offer useful information on how I might attack the problem of creating a drum?
Comment: The [Musical Instrument Design & Construction](http://area51.stackexchange.com/proposals/65027/musical-instrument-design-and-construction) proposal on Area 51 would be suitable for this kind of question
Comment: NB: I GUESS THIS BOUNTY GOES A LITTLE BEYOND THE SCOPE OF THE ORIGINAL QUESTION. Valid answers to the OP's question need not fulfil the bounty requirements...
Answer: I am not a drummer and I'm not affiliated with these folks, so if you down-vote, please tell me why in the comments so that I can improve my answers to this forum.
Here's a site that tells you how to build drum shells either ply or stave.  You'll have to have some woodworking chops however: http://pdgood.us/drumshed/buildmethods.html

A ply drum shell is the method most commonly used by the big drum
companies. You build a mold (usually a female) that is to the size of
the outside diameter. Order thin sheets of wood known as plies, glue
them and place them inside the mold. Apply pressure. When dry,
release the mold. Advanced builders will make their own plies from
very thin sheets of laminate. The plies a beginner should build from
will have several layers of laminate even though they are quite thin.
A stave drum shell is the method most commonly used by custom builders.
Built similarly to a barrel, the wood pieces are vertical with
bevelled edges arranged to form a circle. The staves are glued and
clamped. When the glue is dry the outside and inside are lathed to
achieve a smooth, round shell.

I assume you don't want to do metal working, so you can pick up the hardware parts in any of a number of places, including: http://www.drummaker.com/
It seems like your best bet may be to buy the following DVD. It concentrates on doing more with less and has some great reviews: http://guerrilladrummaking.com/
What sold me on the program was this:

If you are interested in purchasing GDM Digital, you must go through our
  FREE Training Series first, to make sure that GDM is the right fit for you.
  Once you're through the training, we will then provide you with information
  on how to purchase your GDM Digital account.
Comment: +1. I was about to reference the same website. I didn't found any other correct resources while googling and pdgood.us methods are pretty detailled even if there are not really tutorials. You need a certain experience in woodworking to understand these, and how a percussion instrument works. If you start your project Phil, you should create a website/blog in a tutorial way, it would be awesome ;)
Comment: @JCPedroza, but the question asked specifically for links and resources...?  Does the OP need to edit the question?
Comment: Yeah I just noticed how poorly the question is formulated. It asks for resources, you gave one. Nothing to downvote here, undid it and deleted my comment.
Comment: I got confused by the bounty, which asks something different than the answer. The bounty is how the question should have been formulated in the first place, and it might be the thing you want to answer to. And yes, this question would greatly benefit for an edit imo.
Comment: This is a good answer and a great start, thanks. Will see what other suggestions come up before marking an answer as accepted.
Comment: +1 I'm liking this answer too! But, I'm gonna see what else comes in before awarding the bounty...
Comment: @JoeBilly Yeah, that's definitely on my radar.
Answer: The Ghostnote online forum for drum builders is a great resource.
You can search for particular topics.   Some of the tutorials can only be accessed if you have a paid membership ($20 a year), but I've found lots of good info without it.
The pdgood.us site mentioned is also good, but if you search Ghostnote for particular topics of interest you will likely get more detail.  You can also post questions.
www.drumfactorydirect.com and www.drumfoundry.com are good sources of parts.  There have been reports of problems with the drummaker.com site.


OAuthException in Twitter with Flask, using OAuthlib.client

Question: I'm trying just to redirect a person if he/she logs int with Twitter.
So, in views I have:
    @twitter.tokengetter
    def get_twitter_token():
      if 'twitter_oauth' in flask.session:
      resp = flask.session['twitter_oauth']
    return resp['oauth_token'], resp['oauth_token_secret']

    @app.before_request
    def before_request():
      flask.g.user = None
      if 'twitter_oauth' in flask.session:
      flask.g.user = flask.session['twitter_oauth']

    @app.route('/login')
    def login():
      callback_url = flask.url_for('oauthorized', next=flask.request.args.get('next'))
    return twitter.authorize(callback=callback_url or flask.request.referrer or None)

    @app.route('/logout')
    def logout():
      flask.session.pop('twitter_oauth', None)
      return redirect(flask.url_for('index'))

    @app.route('/oauthorized')
    def oauthorized():
      resp = twitter.authorized_response()
      if resp is None:
        flask.flash('You denied the reques to sign in')
      else:
        flask.session['twitter_oauth'] = resp
      return flask.redirect(flask.url_for('success'))

I've already configured de TW_APP_ID and TW_APP_SECRET values in my configs.py, and when I execute, it just raises me a
       flask_oauthlib.client.OAuthException
       OAuthException: Failed to generate request token

Why is this for? What can I do?
Thanks
Comment: Sorry if there's a bad identation error. I checked in my code and everything is alright.
Answer: The Flask-OAuthlib examples were broken. They were fixed in https://github.com/lepture/flask-oauthlib/issues/90 .
You must handle the case isinstance(resp, OAuthException).
Check this commit to understand the changes made to the examples: https://github.com/PyBossa/pybossa/commit/26ec807e321da2f87e1ab44149f5b40ae04bf8bd
Comment: The relevant issue on GitLab: https://github.com/lepture/flask-oauthlib/issues/191


Unable to convert date variable in Snowflake Procedure

Question: I'm facing an error while passing date records into variable .
Want to convert the records into date format 'YYYY-MM-DD' instead of 'Feb 20 2022 HH:MM:SS' format.
Do we have any date conversion methods in javascript ?.
CREATE OR REPLACE PROCEDURE sp_ADDWORKDAYS_CUTOFF_PP(SRC_DB STRING, SRC_SCHEMA STRING)
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$

  var D_STARTDATE = '';
  var D_WORKDAYS = '';
  var seq_query = `SELECT "DELIV_CR_DT" AS D_STARTDATE,DERIVED_COL AS D_WORKDAYS  FROM `+SRC_DB+`.`+SRC_SCHEMA+`.v_v_testing`;
    var seq_stmt = snowflake.createStatement({ sqlText: seq_query});
    var seq_result = seq_stmt.execute();
    while (seq_result.next()){

    var STARTDATE = seq_result.getColumnValue(1);
    var WORKDAYS = seq_result.getColumnValue(2);
    var sub_call_query = `CALL `+SRC_DB+`.`+SRC_SCHEMA+`.sp_OTD_ADDWORKDAYS(`+STARTDATE+`,`+WORKDAYS+`,'01')`;
    var sub_call_stmt = snowflake.createStatement({ sqlText: sub_call_query ,binds: [STARTDATE.toISOString(),WORKDAYS] });
    var sub_call_result = sub_call_stmt.execute();
    sub_call_stmt.next();
    var return_value = sub_call_result.getColumnValue(1);
}
  $$
;

error:
SQL compilation error:
syntax error line 1 at position 52 unexpected 'Feb'.
At Statement.execute, line 14 position 40 (line 36)
Comment: Seems the error is coming from inner procedure call. Can you share what is in that proc - sp_OTD_ADDWORKDAYS?
Comment: A side topic... You have your `return_value` inside a loop. That will only return the value that's assigned the last time through the loop (last row in the result set). Also, it's not being returned so the output of the SP will be `NULL`.
Comment: Yea.. I removed the return statement from outside.. Inner Sp is having update statement and can see only one record is getting updated. May I know the issue with above Sp for not running in loops ?.
Answer: You can try to include something like this in your logic for date conversion -
Using Java-script procedure -
CREATE OR REPLACE PROCEDURE "GETJOBS"(SRC_DB STRING, SRC_SCHEMA STRING)
RETURNS varchar
LANGUAGE JAVASCRIPT
EXECUTE AS caller
AS
$$

  var D_STARTDATE = '';
  var D_WORKDAYS = '';
  var seq_query = `SELECT to_char(to_timestamp('Feb 20 2022 10:10:10','Mon DD YYYY HH24:MI:SS'), 'YYYY-MM-DD') AS D_STARTDATE`;
    var seq_stmt = snowflake.createStatement({ sqlText: seq_query});
    var seq_result = seq_stmt.execute();
    seq_result.next();
    return seq_result.getColumnValue(1);

$$
;

Procedure call -
call getjobs('a','b');

GETJOBS

2022-02-20

Using UDF -
create or replace function fn_getjobs(dt string)
returns string as
$$
select to_char(to_timestamp(dt,'Mon DD YYYY HH24:MI:SS'), 'YYYY-MM-DD') as return_dt
$$
;

Function call
select fn_getjobs('Feb 20 2022 10:10:10') as dt;

DT

2022-02-20


Flutter run JDK 17/SDK 34 migration issue 'flutter_native_image'

Question: i was expecting to migrate project App from JDK11/SDK33 to JDK 17/SDK 34 but fails everytime,
i have been for days tring to find the correct configuration..
My console error output:
FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':flutter_native_image'.
> Could not create an instance of type com.android.build.api.variant.impl.LibraryVariantBuilderImpl.
   > Namespace not specified. Specify a namespace in the module's build file. See https://d.android.com/r/tools/upgrade-assistant/set-namespace for information about setting the namespace.

     If you've specified the package attribute in the source AndroidManifest.xml, you can use the AGP Upgrade Assistant to migrate to the namespace value in the build file. Refer to https://d.android.com/r/tools/upgrade-assistant/agp-upgrade-assistant for general information about using the AGP Upgrade Assistant.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights

script: android/app/build.gradle
plugins {
    id "com.android.application"
    id "kotlin-android"
    id "dev.flutter.flutter-gradle-plugin"
}

def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader('UTF-8') { reader ->
        localProperties.load(reader)
    }
}

def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
    flutterVersionCode = '1'
}

def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
    flutterVersionName = '1.0'
}

def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file('key.properties')
if (keystorePropertiesFile.exists()) {
    keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}

android {
    namespace "com.ivc.lili"
    compileSdk 34
    ndkVersion "25.1.8937393"
//    ndkVersion flutter.ndkVersion

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_17
        targetCompatibility JavaVersion.VERSION_17
    }

    kotlinOptions {
        jvmTarget = '17'
    }

    sourceSets {
        main.java.srcDirs += 'src/main/kotlin'
    }

    defaultConfig {
        // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
        applicationId "com.ivc.lili"
        minSdkVersion 21
        targetSdkVersion 34
        versionCode flutterVersionCode.toInteger()
        versionName flutterVersionName
    }

    signingConfigs {
        release {
            keyAlias keystoreProperties['keyAlias']
            keyPassword keystoreProperties['keyPassword']
            storeFile file(keystoreProperties['storeFile'])
            storePassword keystoreProperties['storePassword']
        }
    }

    buildTypes {
        release {
            // TODO: Add your own signing config for the release build.
            // Signing with the debug keys for now, so `flutter run --release` works.
            signingConfig signingConfigs.debug
            //signingConfig signingConfigs.release
        }
    }
}

flutter {
    source '../..'
}

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
}

script: android/build.gradle
plugins {
    id "com.android.application"
    id "kotlin-android"
    id "dev.flutter.flutter-gradle-plugin"
}

def localProperties = new Properties()
def localPropertiesFile = rootProject.file('local.properties')
if (localPropertiesFile.exists()) {
    localPropertiesFile.withReader('UTF-8') { reader ->
        localProperties.load(reader)
    }
}

def flutterVersionCode = localProperties.getProperty('flutter.versionCode')
if (flutterVersionCode == null) {
    flutterVersionCode = '1'
}

def flutterVersionName = localProperties.getProperty('flutter.versionName')
if (flutterVersionName == null) {
    flutterVersionName = '1.0'
}

def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file('key.properties')
if (keystorePropertiesFile.exists()) {
    keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}

android {
    namespace "com.ivc.lili"
    compileSdk 34
    ndkVersion "25.1.8937393"
//    ndkVersion flutter.ndkVersion

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_17
        targetCompatibility JavaVersion.VERSION_17
    }

    kotlinOptions {
        jvmTarget = '17'
    }

    sourceSets {
        main.java.srcDirs += 'src/main/kotlin'
    }

    defaultConfig {
        // TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
        applicationId "com.ivc.lili"
        minSdkVersion 21
        targetSdkVersion 34
        versionCode flutterVersionCode.toInteger()
        versionName flutterVersionName
    }

    signingConfigs {
        release {
            keyAlias keystoreProperties['keyAlias']
            keyPassword keystoreProperties['keyPassword']
            storeFile file(keystoreProperties['storeFile'])
            storePassword keystoreProperties['storePassword']
        }
    }

    buildTypes {
        release {
            // TODO: Add your own signing config for the release build.
            // Signing with the debug keys for now, so `flutter run --release` works.
            signingConfig signingConfigs.debug
            //signingConfig signingConfigs.release
        }
    }
}

flutter {
    source '../..'
}

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
}

script: android/settings.gradle
pluginManagement {
    def flutterSdkPath = {
        def properties = new Properties()
        file("local.properties").withInputStream { properties.load(it) }
        def flutterSdkPath = properties.getProperty("flutter.sdk")
        assert flutterSdkPath != null, "flutter.sdk not set in local.properties"
        return flutterSdkPath
    }
    settings.ext.flutterSdkPath = flutterSdkPath()

    includeBuild("${settings.ext.flutterSdkPath}/packages/flutter_tools/gradle")
}

include ":app"

apply from: "${settings.ext.flutterSdkPath}/packages/flutter_tools/gradle/app_plugin_loader.gradle"

script: wrapper.properties
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.4-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

My Android Studio configuration with empty project to run apps form physical device:  |
Im clearly new to Flutter and Android dev, just wanna make this thing run, please.
Answer: Fixed my issue while discovering Android Studio was pointing to the wrong JDK(JDK v11, but i need JDK 17) when uninstalling different versions to different folders...
I fixed it by using the next:
Uninstalling JDK 11 (Because Android Studio always pointing out persistently old to new JDK on 'Tools > SDK Manager > Build, execution, deployment > Build Tools > Gradle JDK' and making sure only points out JDK v17 ONLY  like this:
Next I config my PATH environment system for Android Studio specifically creating new JDK_HOME pointing JDK 17 correctly pressing 'New' and adding name
JDK_HOME and my path where JDK installed:
C:\Users\{YourUserName}\.jdks\corretto-17.0.10

After I deleted 'jdk.table.xml' file(which Android Studio stores configuration, which is safe to delete because A.S. always rebuilds it when initiate):
C:\Users\{YouUserName}\AppData\Roaming\Google\AndroidStudio2022.1\options

Finally opened Visual Studio Code and told to point JDK 17 too from 'Terminal':
flutter config --jdk-dir C:\Users\{YourUserName}\.jdks\corretto-17.0.10

After that, both Android Studio, and Visual Studio Code pointed the correct JDK and moved on..


How many Stateless Widget Classes should a Flutter Developer put in one file?

Question: I'm currently working on a flutter application. I have a file with a big widget tree. In order to easier understand, read and maintain I wanted to "crop" the widget tree.
First what I did was to create multiple functions, which represented a bigger part of the tree such as _createFancyImage() or _createFancyContainer. After some research, I found out that such a design has some downsides (see: https://github.com/flutter/flutter/issues/19269). So then I decided to create StatelessWidgets instead. Because of the huge size of the widget tree, I broke it down to 3 logical StatelessWidgets. Now I can use FancyImage() or FancyContainer() which represent each a standalone widget.
As a beginner, I'm not sure whether I should keep those StatelessWidgetclasses within the same file. Alternatively, I could create independent files. One thing to clarify: I'm not using those fancy widgets somewhere else. Those are unique to this one big widget tree otherwise I could have outsourced them into a new folder such as "common_widgets" or "components".
Unfortunately, I couldn't find something within the Dart and Flutter Repo style guides nor on the internet.
I appreciate every suggestion.
Answer: You can add as much classes as you need in a single file. It depends on the developer's mindset.
But, let say if one of your class can be reused by other classes or packages then you should add it to another file for better separation.
I can favour you one approach is that your each file should have maximum one Stateful widget and as many Stateless widgets as you want for that corresponding widget will be a better scenario.
Still in some cases if you feel that more than enough stateless widgets has been added in a single file you should separate it in another file based on your choice.
Answer: I prefer to keep one public widget which having the same name as the filename and remaining private widgets.
comming to your ques is How many widgets in a single file?
Its actually depend there is no such rule to restrict the limit of file. Different authors having  different preference. I prefer try to keep 5-6 classes(widgets) and
each one having 5-6 functions.

Try to make a file single responsible i.e(5-6 classes together responsible for single functionality). Don't make god class which having unrelated concerns together later it will pains(haha)

If it's a common widget keep them separate to respect DRY principle(Don't repeat yourself)

If the widget is further divided into 3-4 widgets or it children widget change depend upon rest response keep seprate for good practise

Bonus Tip: try using code folding shortcuts to push a little more


Custom Prompt component in react-router-dom. Preventing the page reloading

Question: I'm using Prompt from react-router-dom to create my Custom Component for preventing users to leave from the Popup form.
const SignUpForm = () => {

  const { fields, onChange, onSubmit } = useFormState();

  const isEmpty = Object.keys(fields).length === 0;

  usePreventReload(!isEmpty);

  const promptMessage = useMemo(() => JSON.stringify({ fields }), [fields]);

  return (
    <form
      onSubmit={onSubmit}
      noValidate
      autoComplete="off"
    >

Sign Up

      <Prompt when={!isEmpty} message={promptMessage} />
      <TextField
        label="First Name"
        name="firstName"
        onChange={onChange}
      />
        ...

      <Button type="submit" variant="contained" color="secondary">
        Send
      </Button>
    </form>
  );
};

If you don't type anything in the form, then the popup can be closed without problems. And when you fill some value in any field and then try to close the popup you will see a custom pop-up with a warning that you can lose data.
It works great except one case - a page reload. To do this, I'm using the listener "beforeunload" but we can use only the system dialog in this case. But is it also possible to use Custom Component in this case?
I created a simple demo for you to get a better understanding - https://sin9k.com/custom-prompt
Does anyone know a workaround for this case?
Answer: You can capture window event "beforeUnload" to do this natively like below
so in componentDidMount you add this event listener like
window.addEventListener('beforeunload', this.pageRefreshFunction);

Then in componentWillUnmount you can remove this event listener because you don't want to leave it there when user go to other pages like
 window.removeEventListener('beforeunload', this.pageRefreshFunction);

and then pageRefreshFunction should have your popup trigger
But as per resources below the custom pop are not allowed anymore and you can only use native browser specific popups
https://developers.google.com/web/updates/2016/04/chrome-51-deprecations#remove_custom_messages_in_onbeforeunload_dialogs
https://developer.mozilla.org/en-US/docs/Web/API/Window/beforeunload_event
Comment: Thank you, as i thought. It's impossible to do custom component


Protect user login page with Basic Auth

Question: Is it possible to make Drupal more secure by making LOGIN page private? I would like to make /user/login page private with Basic Auth so an user and password are needed before I get in that page.
Or is there another ways to make the login more safe? Goal is to prevent hackers to get in. Also if there's some module that helps on this it would be nice.
Answer: From your question

Is there another ways to make the login more safe?

Yes,try one of the following modules
Login Security

Login Security module improves the security options in the login operation of a Drupal site. By default, Drupal introduces only basic access control denying IP access to the full content of the site.

Secure Login

Secure Login module enables the user login and other forms to be submitted securely via HTTPS, thus preventing passwords and other private user data from being transmitted in clear text. Secure Login module locks down not just the user/login page but also any page containing the user login block (or other forms that you configure to be secured).
Answer: You could also:

Define https protocol for the login page (and the backend) on the Vhost directives. see https://www.drupal.org/https-information. But secure login or password policy module seems great because they also take care of the cookie protection.
Increase the passwords complexity (lenght, signs, etc). The module Password policy could interest you.
Setup fail2ban and create a jail for Drupal to ban recurrent attacker (see Vincent article for example)
Answer: You may set up a Basic Auth on the /user/ URL.
See this question: https://serverfault.com/questions/532039/apache-basic-auth-for-a-particular-url
A proposed solution modified for this particular situation:
<code>#set an environtment variable "auth" if the request starts with "/user"
SetEnvIf Request_URI ^/user auth=1

AuthName "Please login to access"
AuthType Basic
AuthUserFile "/path/to/my/.htpasswd"

# First, allow everybody
Order Allow,Deny
Satisfy any
Allow from all
Require valid-user
# Then, deny only if required
Deny from env=auth
</code>
Comment: Hey, when I add that to .htaccess it displays it to all of the pages somehow, what to do?
Comment: Also where to store .htpasswd that its safe directory?
Comment: In all pages or just in all /user/* pages? Safe directory for .htpasswd depends on your system, it should be usualy outside your Drupal root directoy. For example in /etc/apache (if you have root privileges). Also you can name the file as you want.
Comment: In all pages. I cant find the /apache folder either I know theres the certain directory that the htpasswd goes.
Comment: Tunic, I've get it to work, but which is the right place to put .htpasswd file if I cant fine /apache folder? I dont have rights to it anyway.
Comment: Put it in a folder you have permissions, better if it's outside your docroot. Is not important where it is as long as the right path to the .htpasswd is used in the AuthUserFile param of the .htaccess file
Comment: This almost worked for me but I needed a different order for the rules, like so: #
    ```AuthName "Please login to access"
    AuthType Basic
    AuthUserFile path to file
    Require valid-user
    Order allow,deny
    Allow from all
    Deny from env=AUTH
    Satisfy any```
Answer: In Drupal 8 and 9, you can prevent anonymous access to the /user/login page using the Disable Login module. The module will require users to use an additional querystring and secret value for the querystring to access /user/login.
Admin user can set both the name of the QueryString and the value which will allow access to the login page.
For eg:
http://example.com/user/login
will give access denied.
http://example.com/user/login?key=secret
will allow the user to access the login page if key and secret are the name and value configured in the module.

Protect user login page with Basic Auth


Como é que a informação de charts poderão ser querys de sql?

Question: A minha dúvida consiste na criação de um chart de linhas onde cada valor é uma query que eu já construi!
Estou a utilizar o phpmyadmin que tem a BD (importada de um ficheiro sql)
E estou a usar:

link rel="stylesheet"href="//cdn.jsdelivr.net/chartist.js/latest/chartist.min.css"

e

script src="//cdn.jsdelivr.net/chartist.js/latest/chartist.min.js"

para criar os graficos!
Os labels são os meses do ano ou até mesmo os anos mas as series são as tais querys de sql que já tenho..
Ainda não tenho código para colocar aqui como dúvida, só queria uma ideia geral de como fazer para me guiar e orientar
A minha pergunta é, basta separar por virgulas as querys dentro das chavetas das series?
Answer: Eu já utilizei os charts do google, acho que vai te atender legal.
https://developers.google.com/chart/
Comment: obrigado pela ajuda mas fiquei na mesma situação... O meu problema aqui é que os meus valores para colocar no gráfico são querys que já fiz em sql


Drush 10 is not working - seems unable to properly find the site

Question: Greetings to all of you.
Drush seems unable to properly find the site.
The output of vendor/drush/drush/drush status
<code>PHP binary    : /usr/bin/php
PHP config    : /etc/php.ini
PHP OS        : Linux
Drush script  : /usr/share/site_dir/vendor/drush/drush/drush
Drush version : 10.5.0
Drush temp    : /tmp
Drush configs : /usr/share/nginx/site_dir/vendor/drush/drush/drush.yml
Drupal root   : /usr/share/nginx/site_dir/web/web
</code>
Please notice that the reported Drupal root shows web/web
<code>/usr/share/nginx/site_dir/web/web
</code>
In fact the Drupal root, seems to me, should be
<code>/usr/share/nginx/site_dir/web
</code>
Any ideas, please.
Many thanks.
Comment: Tanks @Jaypan. I found the problem: Somebody copied the file /usr/share/nginx/site_dir/composer.json to /usr/share/nginx/site_dir/web/composer.json
Comment: I'd do a search on the code base (specifically any scaffolding files) for the strings `'web'` and  `/web`, and see what you find. Maybe you need to remove it from some configuration file somewhere. Or it may be that you've misconfigured it in NGINX somewhere. Does the site load when you access the URL set in your NGINX config?
Comment: Also, what directory have you executed Drush from? And do you have Drupal Finder installed with composer in your project directory?
Comment: Does the site load when you access the URL set in your NGINX config?
Yes
What directory have you executed Drush from?
/usr/share/nginx/site_dir
Do you have Drupal Finder installed with composer in your project directory?
Can You be more specific, please.
Answer: This errors gets thrown when $alteredRoot and $root do not match during Preflight.
Often occurs with incorrect web-root settings in composer.json as can be seen here: https://www.drupal.org/forum/support/upgrading-drupal/2023-08-09/drush-unavailable-after-drupal-upgrade-from-9510-to-1012
In my case there was an extra composer.json in /web (just like the original poster). Removing this resolved it for me.
But other solutions offered in the D.O topic might be helpful for some, so thought to share this.