What are Drupal Apps?

Question: I've been seeing the use of Apps. What are drupal apps?
Are they just glorified Features with a backend UI? What are the differences between the two on a lower level? The apps project page only gives a pretty high level understanding.
Answer: Have you read the Open App Standard http://groups.drupal.org/open-app-standard/oas ? Here's an excerpt that might help:

What is an app? Before we say what an App is it's helpful to know who
  it's for: An App is for the end user, typically a non-technical user
  of a Drupal site such as the site administrator....
...Technically, an App is a collection of new or existing Drupal
  functionality, including Modules, Features, and Themes along with some
  simple additional code that publishes the existence of the App to the
  rest of the App infrastructure.
What distinguishes an App from a Module or Feature is that the App
  includes all of the additional polish, end-user UI, configuration UI,
  sample content, and documentation that are needed to fully meet a
  specific need. Whereas Modules and Features are more developer-centric
  and provide building blocks for larger functionality, Apps are larger
  blocks of completed functionality.
Comment: So an app is almost like an install profile, but you can install several apps to a single site?

What are Drupal Apps?


Unable to add changes to my repository

Question: I've modified a puppet module I previously cloned from it's own repository on github. The only thing I changed is a configuration file template, no big deal (pg_hba.conf.erb for the pgsql-aware).
The problem is that I'm not able to add the modified module to my repository.
[ngw@chienandalusia:~/outline.li]$ git status                     (08-13 15:18)
# On branch develop
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#   modified:   puppet/modules/postgresql (modified content)
#
no changes added to commit (use "git add" and/or "git commit -a")
[ngw@chienandalusia:~/outline.li]$ git add puppet                 (08-13 15:19)
[ngw@chienandalusia:~/outline.li]$ git add puppet/modules/postgresql
[ngw@chienandalusia:~/outline.li]$ git status                     (08-13 15:20)
# On branch develop
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#   modified:   puppet/modules/postgresql (modified content)
#
no changes added to commit (use "git add" and/or "git commit -a")

I currently don't see any reason for this behaviour and not being a git expert I'm not entirely sure how to debug this. FWIW there's nothing in my .gitignore that prevents the changes to be staged for commit, and git status apparently agrees with me. Even using the -f flag didn't help.
Does someone have any clue?
Comment: Does puppet/modules/postgresql/.git exist? Maybe Git submodules are in action.
Comment: mmm, yes I do have puppet/modules/postgresql/.git
Comment: Do you have any rule for this folder in your .gitignore file ?
Comment: @ngw: Why is there a `.git` folder in `puppet/modules/postgresql`?? There should only be one in the root (i.e. in the same folder where you have `puppet`).
Comment: @aaron-digulla because I've cloned the module in my own directory.
Comment: @tommyk nothing in .gitignore that could preclude the inclusion of that directory.
Comment: Does `git add -n puppet/modules/postgresql` say something useful?
Answer: puppet/modules/postgresql is a git submodule, changes to it do not affect git project. To make changes to a submodule you should cd to submodule directory and add/commit work, pull, push, go back to root and commit/push again.
Keep in mind that pushing submodule will push to a different remote then the root project.
See https://stackoverflow.com/a/5542964/2578489.

Unable to add changes to my repository


IIS URL Rewrite with a pound/number sign "#"

Question: I have a need to rewrite a url for an invoice that has a "#" in the querystring.  Evidently the "#" is causing IIS to balk somehow.  The rule I have currently is:
^invoice/([_0-9a-z-#]+)
and the action is:
invoice.aspx?id={R:1}
Pretty simple and works fine so long as there's no "#" in the invoice number. Is there any way to include this so it works?
Comment: Unless you properly URL-encode the `#`, you _don't have_ `#` in the path part of your URL to begin with. You would have a _fragment_, and that does not get send to the server in the first place, because it by definition is supposed to have relevance on the client side only.
Answer: Hash Tags in the URL serve a special purpose to the client browser, not to the server.so Browser did not anything after the '#' character.
To resolve this issue you can try this below url rewrite rule:
<code> <rule name="test # in query string" stopProcessing="true">
                <match url="(.*)" />
                <conditions>
                    <add input="{REQUEST_URI}" pattern="invoice/(.+)" />
                </conditions>
                <action type="Redirect" url="invoice.html?id={UrlEncode:{C:1}}" />
            </rule>
</code>
(.+) accessts all the chracter and {UrlEncode:{}} encode the url in the orignal manner.

Regards,
Jalpa


List unversionned files added after a specific date

Question: Using a git repository :
Is there a way to list all the unversioned files that have been added (created) after a given date ?
Edit : I'm on a windows machine. But I can use command such as grep or find or any command built in http://cmder.net/.
Answer: Try the following:
for a in $(git ls-files --others); do  find $a -mmin +120 -exec echo "$a was modified more than 120 minutes ago" \; ; done

To break that apart:
git ls-files --others: List all untracked files
      find $a -mmin +120: Check if the file was last modified 120 minutes ago
But I am not certain that answers your question if you were asking how to filter untracked files by when they were first untracked.


gcp container-os with OS login and IAM admin asking for sudo password?

Question: I'm running cos VERSION_ID=85 BUILD_ID=13310.1209.17 and I love the web ssh console. It was bugging me to enable OS Login to speed up authentication so I finally did it.
The user i'm logging in with has the following IAM roles:

Compute OS Admin Login
Owner

Before OS login, I could click the SSH button and open a web console and execute admin commands without using sudo. With OS Login I have to use sudo, but when I do sudo, it is asking me for a password. Since these SSH keys and PAM is managed by google what password is it looking for? How do i set this up correctly?
I turned off OS Login and I can now sudo without a password.
Comment: before turning on OS login, using latest cos, I would click the SSH button in the console and it would log me in and I could type sudo commands without a password. Once I turned on OS login I did the same thing and it asks me for a password.
Comment: With or without OS Login, `sudo` is required to run admin commands except if you are logged in as the user `root` or equivalent. More details are required for the command and the user where `sudo` is requesting a password. Provided that the user is part of the `sudoers` group, a password is not required. Note: the exact method and configuration is OS dependent.
Comment: Having password protected `sudo` makes your system more secure. Have you tried to change sudo configuration with `visudo`?
Answer: For anyone who might still be facing this issue:
You can get root priviliges without entering a password by using sudo su command.


Date #return_value to ordinary time()

Question: So I have included a date field in my custom block form and would like to use the date.
However it is saved as an array which I am finding a little difficult to work with:

The format for "#default_value" and "#return_value" is an
  array with three elements with the keys: 'year', month', and 'day'.
  For example, array('year' => 2007, 'month' => 2, 'day' => 15)

What is the best way of getting this "date array" into something I can use strtotime() on? Am I missing some Drupal way of allowing me to format this nicely into a string?
What I have working seems to be very long winded and static:
$startDate = variable_get('start_date', NULL);
if (isset($startDatea)) {
  $startDate = $startDatea['month'].'/'.$startDatea['day'].'/'.$startDatea['year'];
  $startDate = date('Y-m-d', strtotime($startDate));
}

I am sure there must be a nicer way.
Answer: For what it's worth you have a typo in your sample code, $startDate is being referenced as $startDatea - note the extra "a". As for a shorter version, try:
$startDate = variable_get('start_date', NULL);
$startDate = date('Y-m-d', mktime(0,0,0, $startDate['month'], $startDate['day'], $startDate['year']);

I suspect not jumping from numeric to string and then back again might save you a few cycles at the PHP layer.

Date #return_value to ordinary time()


Results related to The Happy Ending Problem

Question: Im giving a small talk for a combinatorics class on the Erdos-Szekeres conjecture regarding the happy ending problem (the paper is focused on recent work regarding the conjecture).  I always find that when the audience is not up for a long technical proof, and they aren't familiar with the field (in this case combinatorial geometry) that it is important to provide strong motivation for why the problem is important, and perhaps mention a couple interesting related results that have applications in other fields, or to other problems.
For the Happy Ending problem, I am aware of connections to the $n$-hole problem for points in general position, Ramsey theory (via the original paper by erdos and szekeres in 1935), and the generalization of the E&S theorem for convex bodies.  But Im not sure if these are good things to mention since the class does not know any Ramsey theory and convex bodies are probably too abstract to mention offhand.
So my question is: does anyone know of other interesting related results to the Happy Ending Problem, and the corresponding theorem and conjecture. Or, can you tell me succinctly why or why not this conjecture is important in mathematics.
I will accept anything along the lines of similar geometrically or combinatorially, directly related, or mentioned in a paper involving the E&S conjecture.
The original paper from 1935 is a goldmine of things like the ordered pigeonhole principle, but Im interested in things I havent read yet.
Comment: Maybe if you just simply told the story of why its called the Happy Ending problem, people would be interested in it. It was the cause of two people getting married right? Either way I'm sure if you do lecture on it, someone will ask why its called that.
Comment: @Eric yes, Im sure I will bring that up. It doesnt provide any mathematical motivation, but Im sure it will help the talk be entertaining.
Answer: You might look at this for some recent related work, and investigate its 14 references:
"Every Large Point Set contains Many Collinear Points or an Empty Pentagon." CCCG 2009, 99-102. The main result is exactly what is stated in the title.
Comment: This nice paper (Happy Endings for Flip Graphs) of David Eppstein mentions a version of the paper above in its references as well as showing other interesting connections with "flip graphs." http://arxiv.org/pdf/cs.CG/0610092


How to execute xq files on xml files?

Question: I have .xq files that I want to run again some XML data sets. Is there a tool that can be used to run Xquery files again XML data in command line on OSX?
I've already read this: How to execute XPath one-liners from shell?
Comment: "Is there a tool?" -- several. Have you decided which engine you want to use?
Answer: For BaseX, to evaluate file.xq against the contents in in.xml and write the result to out.xml:
basex -iin.xml -oout.xml file.xq

Much more documentation is available on the wiki.

For Saxon, to perform the same operation:
java net.sf.saxon.Query -s:in.xml -q:file.xq -o:out.xml

Documentation likewise available.
Comment: Thank-you. And for other readers, I just found the instructions to install BaseX here, http://macappstore.org/basex/
Comment: I'm having trouble installing for this to work on command line: http://stackoverflow.com/questions/34191925/how-to-install-basex-for-command-line
Comment: Saxon is probably the easiest to install and run if you want to evaluate queries over files from the command line.  If you use `brew`, the package `saxon` is a decently recent version (do **not** use `saxon-b`): "`brew install saxon`".
Comment: @FlorentGeorges: I used `brew install saxon`, but what's next? it only gives me executable `/usr/local/Cellar/saxon/126.96.36.199/bin/saxon` and that gives me back: `Command line option -q is not recognized. Options available:  -? -a -catalog -config -cr -diag -dtd -ea -expand -explain -export -ext -im -init -it -l -license -m -nogo -now -o -opt -or -outval -p -pack -quit -r -repeat -s -sa -scmin -strip -t -T -threads -TJ -TP -traceout -tree -u -val -versionmsg -warnings -x -xi -xmlversion -xsd -xsdversion -xsiloc -xsl -xsltversion -y` …any ideas?
Comment: Does it give a `saxon` executable?  Interesting.  But it seems it gives only access to the XSLT front-end class.

So either you use `java` with the proper classpath and class `net.sf.saxon.Query`, or you use the EXPath repository manager, that comes with a better saxon script, being a front-end for both XSLT and XQuery.

Go to http://expath.org/files/pkg, download the latest `expath-repo-installer-*.jar` and execute it.  Add the dir `bin/` where it has been installed to your PATH, and you're good to go!
Comment: @msciwoj - Just posted the above comment as an additional answer, for clarity.
Answer: If you like packaged bundles, you might be interested by the EXPath repository manager.  It comes with Saxon and Calabash (you can pick which to install), as well as scripts to run them with proper classpath, class, etc.  It comes with a graphical installer.
To install it:

go to http://expath.org/files/pkg
download the latest expath-repo-installer-*.jar
execute it (e.g. java -jar ~/Downloads/expath-repo-installer-*.jar)
(optional) add the dir bin/ where it has been installed to your PATH

On the shell, just invoke saxon --help (or ~/expath/pkg/bin/saxon --help or any ther place you installed it if you did not put it in the PATH), and follow the instructions.  All options like --xxx at the beginning of the command line are consumed by the script itself.  All the rest is passed as is to Saxon.
For instance to execute a query from the command line, with a specific document as the context item:
saxon --xq -s:data.xml -q:query.xq

How to execute xq files on xml files?


most efficient way to use open_mfdataset for opening many netcdf with xarray/dask (abnormaly slow in my case)

Question: I have a directory of 363 netcdf files corresponding to different times, (all files have a similar internal structure, with a "time" dimension of 1), 270MB each, for a total of about 100GB. I want to upload all these data in a single xarray (with dask arrays and chunks). It seems open_mfdataset is the appropriate solution, but it seems I am not using it properly because it is very slow.
# Import modules
import time
import numpy as np
import xarray as xr
from dask.distributed import Client

client = Client()

# Define variables of interest
vars = ['nitrogendioxide_tropospheric_column_qafiltered']

# Read data
start = time.time()
dir = '/data_directory/'
ds = xr.open_mfdataset('{}/*2019*.nc'.format(dir), engine='netcdf4', combine='nested', concat_dim='time', parallel=True)
ds.close()
print(' | size(ds)/duration = {:0.2f}GB / {:0.2f}s'.format(ds.nbytes / 1e9,time.time()-start))

The time required to do this is : size(ds)/duration = 98.83GB / 1746.73s. Why so slow?
Note that if I don't put the client = Client() and parallel=True, it does not change significantly the time, so I am a bit confused.
NB : This test is performed on an interactive session in a HPC facilities :
>>> client
<Client: 'tcp://127.0.0.1:43651' processes=4 threads=4, memory=33.78 GB>

NBbis : The xarray obtained is:
>>> ds
<xarray.Dataset>
Dimensions:                                                    (corner: 4, time: 363, x: 1028, x_b: 1029, y: 649, y_b: 650)
Coordinates:
    lat                                                        (y, x) float64 dask.array<chunksize=(649, 1028), meta=np.ndarray>
    lon                                                        (y, x) float64 dask.array<chunksize=(649, 1028), meta=np.ndarray>
  * time                                                       (time) datetime64[ns] 2019-01-01T05:00:00 ... 2019-12-31T05:00:00
Dimensions without coordinates: corner, x, x_b, y, y_b
Data variables:
    nitrogendioxide_tropospheric_column                        (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_tropospheric_column_precision              (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    qa_value                                                   (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    latitude_bounds                                            (time, corner, y, x) float64 dask.array<chunksize=(1, 4, 649, 1028), meta=np.ndarray>
    longitude_bounds                                           (time, corner, y, x) float64 dask.array<chunksize=(1, 4, 649, 1028), meta=np.ndarray>
    solar_zenith_angle                                         (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    solar_azimuth_angle                                        (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    viewing_zenith_angle                                       (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    viewing_azimuth_angle                                      (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    cloud_fraction_crb                                         (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_altitude                                           (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_albedo                                             (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_classification                                     (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_pressure                                           (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    cloud_radiance_fraction_nitrogendioxide_window             (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_stratospheric_column                       (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_stratospheric_column_precision             (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    degrees_of_freedom                                         (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    one                                                        (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    latitude_bounds_qafiltered                                 (time, corner, y, x) float64 dask.array<chunksize=(1, 4, 649, 1028), meta=np.ndarray>
    longitude_bounds_qafiltered                                (time, corner, y, x) float64 dask.array<chunksize=(1, 4, 649, 1028), meta=np.ndarray>
    nitrogendioxide_tropospheric_column_qafiltered             (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_tropospheric_column_precision_qafiltered   (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    qa_value_qafiltered                                        (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    cloud_radiance_fraction_nitrogendioxide_window_qafiltered  (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    cloud_fraction_crb_qafiltered                              (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_altitude_qafiltered                                (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_albedo_qafiltered                                  (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_classification_qafiltered                          (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    surface_pressure_qafiltered                                (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    solar_zenith_angle_qafiltered                              (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    solar_azimuth_angle_qafiltered                             (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    viewing_zenith_angle_qafiltered                            (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    viewing_azimuth_angle_qafiltered                           (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_stratospheric_column_qafiltered            (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    nitrogendioxide_stratospheric_column_precision_qafiltered  (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    degrees_of_freedom_qafiltered                              (time, y, x) float64 dask.array<chunksize=(1, 649, 1028), meta=np.ndarray>
    lat_b                                                      (time, y_b, x_b) float64 dask.array<chunksize=(1, 650, 1029), meta=np.ndarray>
    lon_b                                                      (time, y_b, x_b) float64 dask.array<chunksize=(1, 650, 1029), meta=np.ndarray>
Attributes:
    regrid_method:  conservative
    history:        read PRODUCT group...

I looked at the other posts but am not able to find an answer for this issue.
Thanks for your help.
Answer: See this github issue - there appear to be many people having problems with the performance of open_mfdataset, and no obvious solutions at the moment.


Can standardized $\beta$ coefficients in linear regression be used to estimate the $R^2$?

Question: I am trying to interpret the results of an article, where they applied multiple regression to predict various outcomes. However the $\beta$'s (standardized B coefficients defined as $\beta_{x_1} = B_{x_1} \cdot \frac{\mathrm{SD}_{x_1}}{\mathrm{SD}_y}$ where $y$ is the dependent variable and $x_1$ is a predictor) reported don't seem to match the reported $R^2$:

Despite $\beta$ 's of -0.83, -0.29, -0.16, -0.43, 0.25, and -0.29, the reported $R^2$ is only 0.20.
Also, the three predictors: weight, BMI and fat % are multi-collinear, correlated around r=0.8-0.9 with each other within sexes.
Is the $R^2$ value plausible with these $\beta$ 's, or is there no straight relationship between the $\beta$'s and the $R^2$?
Additionally, might problems with the multicollinear predictors affect the $\beta$ of a fourth predictor (VO2max), which is correlated around r=0.4 with the aforementioned three variables?
Comment: @RichardHardy I suspect that the confusion is that Sakari defined $\beta$ to be the *standardized* regression coefficient. In a bivariate linear regression the regression coefficient ($b$ in Sakari's notation) is $r_{xy}\frac{s_y}{s_x}$, where $r$ is the correlation and $s$ the standard deviation. To standardize a regression coeffcient we divide the coefficient with the standard deviation of $y$ and multiply with that standard deviation of $x$, so only the correlation is left. So Sakari is right.
Comment: No, in one regressor case $\beta$ is not equal to Pearson's correlation: $\beta=\frac{\text{Cov}(y,x)}{\text{Var}(x)}\neq\frac{\text{Cov}(y,x)}{ \sqrt{ \text{Var}(y)\times\text{Var}(x) } }=\rho(y,x)$. The relationship between $\beta$s and $R^2$ is not as simple.
Comment: ß stands for standardized b coefficients.

For a 1 predictor case ß is equal to pearson's r, which is directly relate to R-squared, however in this multivariate case, why don't high ß's imply a high R-squared?
Comment: What is $\beta$ in this context? A beta coefficient (standardised regression)? Or something else? If so then, them you can't really say anything all you get is an interpretation in terms of standard deviations. The fact that the coefficient implies large effects, does not imply a high $R^2$ value
Comment: I still dont see why you consider this to be wrong? If there is some summary statistics in the paper, you can simply check if the numbers add up. You even provided the formula to do so. You cannot conclude, simply because the effects are large in abosulte terms, that the models does a good job at explaning the variance in y.
Answer: The geometric interpretation of ordinary least squares regression provides the requisite insight.
Most of what we need to know can be seen in the case of two regressors $x_1$ and $x_2$ with response $y$.  The standardized coefficients, or "betas," arise when all three vectors are standardized to a common length (which we may take to be unity).  Thus, $x_1$ and $x_2$ are unit vectors in a plane $E^2$--they are located on the unit circle--and $y$ is a unit vector in a three dimensional Euclidean space $E^3$ containing that plane.  The fitted value $\hat y$ is the orthogonal (perpendicular) projection of $y$ onto $E^2$.  Because $R^2$ simply is the squared length of $\hat y$, we needn't even visualize all three dimensions: all the information we need can be drawn in that plane.
Orthogonal regressors
The nicest situation is when the regressors are orthogonal, as in the first figure.

In this and the rest of the figures I will consistently draw the unit disk in white and the regressors as black arrows.  $x_1$ will always point directly to the right.  The thick red arrows depict the components of $\hat y$ in the $x_1$ and $x_2$ directions: that is, $\beta_1 x_1$ and $\beta_2 x_2$.  The length of $\hat y$ is the radius of the gray circle on which it lies--but remember that $R^2$ is the square of that length.
The Pythagorean Theorem asserts
$$R^2 = |\hat y|^2 = |\beta_1 x_1|^2 + |\beta_2 x_2|^2 = \beta_1^2(1)+\beta_2^2(1) = \beta_1^2 + \beta_2^2.$$
Because the Pythagorean Theorem holds in any number of dimensions, this reasoning generalizes to any number of regressors, yielding our first result:

When the regressors are orthogonal, $R^2$ equals the sum of the squares of the betas.

An immediate corollary is that when there is just one regressor--univariate regression--$R^2$ is the square of the standardized slope.
Correlated
Negatively correlated regressors meet at angles greater than a right angle.

It is visually apparent in this image that the sum of squares of the betas is strictly greater than $R^2$.  This can be proven algebraically using the Law of Cosines or by working with matrix solution of the Normal Equations.
By making the two regressors almost parallel, we can position $\hat y$ near the origin (for an $R^2$ near $0$) while it continues to have large components in the $x_1$ and $x_2$ direction.  Thus, there is no limit to how small $R^2$ might be.

Let's memorialize this obvious result, our second generality:

When regressors are correlated, $R^2$ may be arbitrarily smaller than the sum of squares of the betas.

However, this is not a universal relation, as the next figure demonstrates.

Now $R^2$ strictly exceeds the sum of squares of the betas.  By drawing the two regressors close together and keeping $\hat y$ between them, we may make the betas both approach $1/2$, even when $R^2$ is close to $1$.  Further analysis may require some algebra: I take that up below.
I leave it to your imagination to construct similar examples with positively correlated regressors, which thereby meet at acute angles.
Notice that these conclusions are incomplete: there are limits to how much less $R^2$ may be compared to the sum of squares of the betas.  In particular, by examining the possibilities carefully, you may conclude (for a regression with two regressors) that

When the regressors are positively correlated and the betas have a common sign, or when the regressors are negatively correlated and the betas have different signs, $R^2$ must be at least as large as the sum of the squares of the betas.

Algebraic results
Generally, let the regressors be (column vectors) $x_1, x_2, \ldots, x_p$ and the response be $y$.  Standardization means (a) each is orthogonal to the vector $(1,1,\ldots,1)^\prime$ and (b) they have unit lengths:
$$|x_i|^2 = |y|^2 = 1.$$
Assemble the column vectors $x_i$ into an $n\times p$ matrix $X$.  The rules of matrix multiplication imply that
$$\Sigma = X^\prime X$$
is the correlation matrix of the $x_i$.  The betas are given by the Normal Equations,
$$\beta = (X^\prime X)^{-1} X^\prime y = \Sigma^{-1} (X^\prime y).$$
Moreover, by definition, the fit is
$$\hat y = X \beta = X (\Sigma ^{-1} X^\prime y).$$
Its squared length gives $R^2$ by definition:
$$R^2 = |\hat y|^2 = \hat y^\prime \hat y = (X\beta)^\prime (X\beta) = \beta^\prime (X^\prime X)\beta = \beta^\prime \Sigma\beta. $$
The geometric analysis suggested we look for inequalities relating $R^2$ and the sum of squares of the betas,
$$\sum_{i=1}^p \beta_i^2 = \beta^\prime \beta.$$
The $L_2$ norm of any matrix $A$ is given by the sum of squares of its coefficients (basically treating the matrix as a vector of $p^2$ components in a Euclidean space),
$$|A|_2^2 = \sum_{i,j} a_{ij}^2 = \operatorname{tr}(A^\prime A) = \operatorname{tr}(AA^\prime).$$
The Cauchy-Schwarz Inequality implies
$$R^2 = \operatorname{tr}(R^2) = \operatorname{tr}(\beta^\prime \Sigma \beta) = \operatorname{tr}(\Sigma \beta \beta^\prime) \le |\Sigma|_2 | \beta\beta^\prime|_2 = |\Sigma|_2 \beta^\prime \beta.$$
Since squared correlation coefficients cannot exceed $1$ and there are just $p^2$ of them in the $p\times p$ matrix $\Sigma$, $|\Sigma|_2$ cannot exceed $\sqrt{1\times p^2} = p$.  Therefore
$$R^2 \le p\, \beta^\prime \beta.$$
The inequality is attained, for instance, when all the $x_i$ are perfectly positively correlated.

There is an upper limit on how large $R^2$ may be. Its average value per regressor, $R^2/p$, cannot exceed the sum of squares of the standardized coefficients.

Conclusions
What may we conclude in general?  Evidently, information about the correlation structure of the regressors as well as the signs of the betas could be used either to bound the possible values of $R^2$ or even to compute it exactly.  Absent that full information, little can be said beyond the obvious fact that when the regressors are linearly independent, a single nonzero beta implies $\hat y$ is nonzero, demonstrating $R^2$ is nonzero.
One thing we can definitely conclude from the output in the question is that the data are correlated: because the sum of squares of the betas, equal to $1.1301$, exceeds the maximum possible value of $R^2$ (namely $1$), there must be some correlation.
Another thing is that since the largest beta (in size) is $-0.83$, whose square is $0.69$--far exceeding the reported $R^2$ of $0.20$--we may conclude that some of the regressors must be negatively correlated.  (In fact, $\text{VO}_{2\,\text{max}}$ is likely strongly negatively correlated with age, weight, and fat in any sample that covers a wide range of values of the latter.)
If there were only two regressors, we could deduce a great deal more about $R^2$ from knowledge of high regressor correlations and inspection of the betas, because this would enable us to draw an accurate sketch of how $x_1$, $x_2$, and $\hat y$ must be situated.  Unfortunately, the additional regressors in this six-variable problem complicate things considerably.  In analyzing any two of the variables, we have to "take out" or "control for" the other four regressors (the "covariates").  In so doing, we shorten all of $x_1$, $x_2$, and $y$ by unknown amounts (depending on how all three of those are related to the covariates), leaving us knowing almost nothing about the actual sizes of the vectors we are working with.
Comment: +1 but I don't understand why, in the non-orthogonal case, you project $\hat y$ vector *orthogonal* to the predictor axes, as opposed to making the projecting dotted lines go parallel to the other predictor. This sounds cumbersome but I think you will see what I mean. Your "projections" (two smaller red vectors) do not some up to get the big red $\hat y$ vector.
Comment: @amoeba Again you are correct.  At some risk of losing interested readers, but now feeling compelled to quantify the geometric intuition, I have tightened that conclusion and justified it with a little bit of algebra.  (I trust the algebra is correct!)
Comment: I think it's a very nice answer now!
Comment: Thanks a lot! As a sidenote, VO2max is negatively correlated with weight and BMI, since they are associated with higher lean body mass. In the said table VO2max actually corresponds to VO2max divided by weight (which is a bad way of scaling VO2max to body size). The VO2max/weight in the table is negatively correlated with all the other predictors, except sex, which might explain the high ß but low R-squared, as you mentioned.
Comment: @amoeba You are quite right.  I have been too hasty in creating these images!  I will (hopefully temporarily) delete this post until I get an opportunity to correct the problem.  Thank you for pointing this out.
Comment: @Amoeba I have corrected the pictures and modified the analysis to match them.  Although the details have changed substantially, the conclusions remain the same.
Comment: Thanks, @whuber. Here is one more thing: "By drawing the two regressors close together and keeping $\hat y$ between them, we may make the betas both approach $0$, even when $R^2$ is close to $1$" -- I don't see how this can happen. Wouldn't betas in this situation approach half the length of $\hat y$, their sum of squares approaching $R^2/2$? I don't think they will go to zero. They can probably become arbitrarily small, but only with enough (more than two) predictors.


Can I use algebraic equality operation for parametric equations of lines?

Question: I am working on below question from a linear algebra book that I am studying,
Suppose $X = X_0 + tV$ and $Y = Y_0 + sW$ are two parametric representations of the same line $l$ in $R^n$.
a. Show that there is a scalar $t_0$ so that $Y_0 = X_0 + t_0V$.
b. Show that $V$ and $W$ are parallel.
Because both equations are representation of the same line, can I claim that $X=Y$ or $X_0 + tV = Y_0 + sW$ in algebraic sense? Assuming it is possible I can prove (a) like this:
$$X_0 + tV = Y_0 + sW$$
$$Y_0 = X_0 + tV - sW$$
Since $W$ and $V$ are direction vectors of the same line, they are parallel, and I can say $W=cV$,
$$Y_0 = X_0 + tV - scV$$
$$Y_0 = X_0 + V(t - sc)$$
$t$, $s$, and $c$ are all scalars, so $t_0=t-sc$,
$$Y_0 = X_0 + t_0V$$
Here is my second question in regards to these proofs:
Using the same assumption that I can say $X=Y$ and the (a), I can prove (b) like this:
$$X_0 + tV = Y_0 + sW$$
Using (a) ($Y_0 = X_0 + t_0V$):
$$X_0 + tV = X_0 + t_0V + sW$$
$$W = \frac{(tV - t_0V)}{s}$$
$$W = V\frac{t-t_0}{s}$$
Which means $W$ is a scalar multiple of $V$ and are parallel.
When I am trying to prove (b) I am using my result from (a) [ $Y_0=X_0+t_0V$ ] which is concluded by the assumption that both vector directions $W$ and $V$ are parallel and infact in (b) I am trying to prove the same assumption that I made in (a) (that they are parallel). I am very new to this stuff and don't know what technical term is used to describe this situation, but it kind of looks like chicken and eggs problem to me. Is there anything wrong with the solutions that I provided?
Answer:
Can we claim that $X=Y$? Not in general.

$X=X_0+tV$ and $Y=Y_0+sW$ are arbitrary points on the same line determined by the values of $t,s$. If you take any point with position vector $A$ on the line, then $\exists s_1,t_1\in\Bbb R$ such that$$A=X_0+t_1V=Y_0+s_1W$$i.e. $X$ and $Y$ are equal for some pairs $s,t$ that describe the same point on the line and unequal for other pairs that describe different points.

In proving $(a)$, you have used the fact that $V,W$ are parallel because they are the direction vectors of the same line. This is not a rigorous proof, and you were supposed to prove it in $(b)$ so the question probably expects you to prove $(a)$ without using the fact that $V,W$ are parallel.

For proving $(a)$ without this assumption, note that for $s=0$, we get $Y=Y_0$ is a point on the line. Thus $\exists t_0\in\Bbb R$ such that$$Y_0=X_0+t_0V$$since $X_0+tV$ also describes points on the same line.

For $(b)$, slightly alter your proof like this: Take $s_1\in\Bbb R-\{0\}$. Then $\exists t_1\in\Bbb R$ such that$$X_0+t_1V=Y_0+s_1W$$Now substitute for $Y_0$ from $(a)$ to get $W=\left(\frac{t_1-t_0}{s_1}\right)V$. Note that $t_1\ne t_0$ since $W$ is not zero vector. Also note why taking $s_1\ne0$ was necessary. Thus $W$ is parallel to $V$.


Is it even possible to to add voices to this 2-part harmony?

Question: Soprano: C-B-A-A-G (descending)
Bass: A-B-C (ascending), then down to D and up to G.
Can you even add tenor and alto without breaking the rules of harmony? It seems like it is impossible.
So are there ways to harmonize this?
I did try it with 3 voices but ended up with B-D-B on the second beat. This is not correct as a sixth interval is good, but we have a G chord without G and with two thirds.
You don't want to end up with B-E-B since you'll end up with a fifth.
Comment: I would recommend adding a child's "Wow!", which for some reason harmonizes with anything, even [silence](https://www.cbc.ca/radio/q/blog/orchestra-seeks-child-who-shouted-wonder-filled-wow-at-end-of-concert-1.5128372)
Comment: "Harmony" is a very subjective term these days. Could you please be more specific as to what harmony rules you are referring to?
Comment: Post notation! Are we supposed to guess the rhythm of the parts?
Answer: I think B-D-B on the second beat shouldn't be disregarded too readily, but the implied harmony is rather E⁷ than G. This could be brought out even more by going full in with the leading tone:
X:1
L:1/4
M:C
K:C
%%score T1 Tn B
V:T1           clef=treble
V:Tn           clef=treble-8
V:B            clef=bass
% 1
[V:T1] c  B  A A  | G4
[V:Tn] E  ^G A c  | B4
[V:B]  A,,B,,C,D,,| G,,4

Comment: @ScottWallace yes, or rather, the bass should be an octave lower. Edited. Octave positions in ABC are always a trial&error for me...
Comment: Shouldn't your middle voice be an octave higher?  Otherwise you start out with a 2nd inversion chord.
Answer: Solutions
Here are two 4-part solutions.
This one include B-D-B on the second beat.
X: 0
T: 4-Voice Solution with B-D-B on Beat 2
M: 4/4
L: 1/4
K: none
V:Soprano name="S"
V:Alto name="A"
V:Tenor clef=treble-8 name="T"
V:Bass clef=bass name="B"
[V:Soprano]cBAA|G4||
[V:Alto]AGA^F|D4||
[V:Tenor]edcc|B4||
[V:Bass]A,,B,,C,D,,|G,,4||

And this one, based on @leftaroundabout's solution, includes B-E-B on the second beat.
X: 0
T: 4-Voice Solution with B-E-B on Beat 2
M: 4/4
L: 1/4
K: none
V:Soprano name="S"
V:Alto name="A"
V:Tenor clef=treble-8 name="T"
V:Bass clef=bass name="B"
[V:Soprano]cBAA|G4||
[V:Alto]EE_ED|_E4||
[V:Tenor]E^GAc|_B4||
[V:Bass]A,,B,,C,D,,|G,,4||

Discussion
As @Pyromonk indicated in the comments on the question:

"Harmony" is a very subjective term these days. Could you please be more specific as to what harmony rules you are referring to?

"Harmony" has a variety of definitions. There's Modal harmony, Tonal harmony, Jazz/Rock harmony, Microtonal harmony, and probably others I'm not thinking of or don't know about. So the system you're using is key. However, I'm allowing that you're asking about Common Practice (ca. 1600 - 1900) harmony. So, major/minor Tonality.
@leftaroundabout is also spot-on to point out that a too-restrictive approach to the "allowed" chords can paint one into a corner.
However, you are correct that in a 3-voice texture, if you require the second beat to be a G chord, then you can't use D, because you're missing the root of the chord. Doubling the third isn't a problem, though. What is not allowed is doubling a leading tone. In this case, the third is not functioning as a leading tone, so can be doubled.
The B-E-B solution for beat 2 is also okay. Fifths are okay; they just aren't to be used consecutively. It would be okay to ascend from E to F, because parallel fourths (with the bass, in this case) are allowed.
Another 3-voice problem is beat 4 moving to beat 1. Let's allow the goal is to end on a G chord. In that case, "the rules" pretty much require F# on beat 4 to serve as the leading tone. But that results in three Gs and no third in the chord. Allowed, but pretty stark.
Ultimately, I think four voices are actually easier to set here. The problem of 2-note chords can be solved in that way.


How to get the value from each field of a class with Byte Buddy?

Question: I'm writing a library that uses Java Agents with Byte Buddy to perform runtime code generation. I need to create an expression to get the value from each field of a class.
How can I create an expression to access the field value? Currently I'm exploring the FieldDescriptionAPI but I not found anything to do this.
Comment: What kind of instrumentation are you using? What have you tried?
Comment: I have another question. How can I get the `TypeDescription` for the class to be generated into a `MethodDelegation.to(AddFooMethod.class)`?
Comment: @RafaelWinterhalter I just need to create a method inside a class annotated with a specific annotation

For example:

    `@MyAnnotation public class Person {
        private String name;
        private String email;

        // method to be generated
        public void generatedMethod() {
            System.out.println("name " + this.name + " email " + this.email);
        }
    }`

Basically I want to generate a method inside a class that access all fields to get its value.
Answer: From the additional information, you provided on GitHub, this is how you would approach this:
To create a StringBuilder you call MethodCall.construct(StringBuilder.class.getConstructor())
You can then call the append to the string builder like: MethodCall.invoke(StringBuilder.class.getMethod("append", Object.class)).onMethodCall(...).withField(...) where you start with the initial string builder and then reiterate for every field or constant supplying the last input to the next instance.
You would have to adjust the method based on the field type, if the type is primitive, for this to work. In the end, you have to do a final method call on toString.


Is there any way to know which theme hook or template is being used to output certain area of a page

Question: Is there any way to know which theme hook or template is being used to output certain area of page? I am showing different menus on a page and i want to know which theme hook is being used to output particular menu and links of menu.
I tried dpm($variables) in function theme_menu_links($variables) in common.inc just to make sure if this theme hook is being used to output the links of menu but this theme hook is not being called and no output is printed. Then certainly this theme hook is overridden by other module or by theme.
Is it possible to know list of all available hooks and templates files for a particular are of a page like block, menu etc with the active theme hook used to output that area?
Answer: Yes, there is a module for that: Theme Developer.

Firebug for Drupal themeing

Note that this is a heavy module, enable it only when needed and on your local machine.
Answer: There is no single (active) hook to define an area (that's the idea of hooks). But there is one initial function defined by a module or theme and other modules and themes can alter the initial output.
If you have devel module installed, you can see the theme registry. This is a list of themable items with all files and (pre)process functions. So once a another hook alters an item it is listed there. Works great for entities, but I have never used it on menus. You can give it a try.
You'll find it at /devel/theme/registry
Comment: My mean by active hook is that the hook used to output that area at that time. As "Views" module show the active template along with all the possible templates to theme the style output and row style output under the view's "Theme: information" setting.
Comment: The template files as suggested by Views don't have much to do with hooks. There can be multiple hooks involved in preprocessing content, but it can (obviously) only be output in one template file. The active template is also visible in the theme registry.


How to create a shared disk across two or more virtual machines setup on ESX 4.1

Question: How to create a shared disk across two or more virtual machines  setup on ESX 4.1 ?
My setup :
1. Created three virtual machines on ESX 4.1
2. All the virtual machines have RHEL5U3 installed.
Question :
I wanted to create one disk of size 2GB and make it shared across above three virtual machines.
Comment: Are you trying to create a template or do you want all three of your virtual machines to mount the same block level device (disk) simultaneously?
Comment: Yes I wanted to mount same block level device on all the virtual machines (guest os) simultaneously.
Answer: Big giant caveat. You need to disable caching on the vm's to get a consistent view of the shared disk.
disable caching!
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
This would still require a clustered file system.
While yes it is possible, numerous vmware support documents mention corruption as a recurring theme.
They also recommend you use a RAW LUN instead of vmdk.
I would use something like drbd to do this.


Why not accounting for a factor unrelated to treatment makes p-values too conservative?

Question: I would like to know if a treatment works, but different subjects are measured across different sites. The site is unrelated to treatment, but it is related to the response (see the figure).

If I put site indicators to the model, p-values are uniformly distributed between 0-1 as expected. However, If I don't account for a site, I would expect that the p-values would be overly optimistic because I am not accounting for the dependence in the data, however when I simulate this dataset many times, I see that p-values are actually too conservative, so to get p < 0.05 is much less likely than a 5% (see fig 2). Why is that?

n_sims <- 25000 #this takes a minute

results <- data.frame(p_values = rep(NA, n_sims))
for(i in 1:n_sims){
  # create site effects, unrelated to treatment, but related to response
  site_effects <- data.frame(site_id = LETTERS,
                             site_effects = seq(-1, 1, length.out = 26))
  df <- data.frame(site_id = as.factor(rep(LETTERS, 50)))
  df <- merge(df, site_effects)
  # treatment is unrelated to response and site
  df$treatment <- as.factor(c('yes', 'no'))
  # response related to site, but not treatment
  df$response <- rnorm(nrow(df), mean=df$site_effects, sd = 1)
  # save the p-value for treatment
  results[i,] <- summary(lm(response ~ treatment, data=df))$coefficients[2,'Pr(>|t|)']
}

# making plots
library(ggplot2)

p1 <- ggplot(results, aes(x=p_values)) +
  geom_histogram(binwidth = 0.01, boundary=0) +
  ggtitle("Histogram of p-values")

p2 <- ggplot(results, aes(sample=p_values)) +
  geom_qq(distribution = qunif) +
  stat_qq_line(distribution = qunif, color='red') +
  coord_fixed() +
  ggtitle("qq plot of p-values")

Answer: As you (perhaps presciently) noted in your tags, this has to do with the type of standard errors used. The outcomes within a site are not independent, which means the standard OLS estimated standard errors will not be correct. Using cluster-robust standard errors (i.e., sandwich::vcovCL(., cluster = ~site_id)) produces the correct distribution of p-values, as shown in the plot below.

To get this plot, I replaced
results[i,] <- summary(lm(response ~ treatment, data=df))$coefficients[2,'Pr(>|t|)']

with
results[i,] <- lmtest::coeftest(lm(response ~ treatment, data=df),
                                vcov. = sandwich::vcovCL, cluster = ~site_id)[2,'Pr(>|t|)']

Comment: thanks for your answer. But why does it make p-values conservative and not optimistic? Which is what I would expect from dependent data.


Group By Linq só retorna o campo Id

Question: Tenho o seguinte método
public async Task<List<Fazenda>> GetAllFazendaTotal(int codigoTecnico)
    {
        var fazendas = await (from f in _apiBdContext.Fazendas
                              join a in _apiBdContext.Associados on f.CodigoAssociado equals a.Codigo
                              join pt in _apiBdContext.PaTecnico on a.CodigoPA equals pt.CodigoPA
                              where (pt.CodigoTecnico == codigoTecnico && f.Ativo == true)
                              group f by f.Codigo into p
                              select new Fazenda
                              {
                                  Codigo = p.Key,
                                  /*
                                  Ativo = f.Ativo,
                                  CodigoAssociado = f.CodigoAssociado,
                                  NomeFazenda = f.NomeFazenda == null ? "" : f.NomeFazenda,
                                  //NomeAssociado //Não se encontra na documentação do Bd, mas existe o campo no banco
                                  UF = f.UF == null ? "" : f.UF,
                                  CodigoIbgeCidade = f.CodigoIbgeCidade,
                                  AreaTotal = f.AreaTotal,
                                  TipoPosse = f.TipoPosse,
                                  ProprietarioTerra = f.ProprietarioTerra == null ? "" : f.ProprietarioTerra,
                                  RoteiroAcesso = f.RoteiroAcesso == null ? "" : f.RoteiroAcesso,
                                  Latitude = f.Latitude == null ? "" : f.Latitude,
                                  Longitude = f.Longitude == null ? "" : f.Longitude,
                                  Altitude = f.Altitude,
                                  */
                              }
                                )
                                .AsNoTracking()
                                .ToListAsync();

        return fazendas;
    }

Estou com problema na hora de retornar os dados, pois estavam vindo duplicados. Percebi que o problema era que estava faltando um group by, só que quando adiciono o group by não consigo retornar o resto dos campos, somente o Código que esta como key. Fiquei pesquisando como retornar esses campos que estão comentados, mas não achei nenhuma resposta que me ajudaria. Se alguém puder ajudar, sou iniciante e talvez esteja procurando errado.
Comment: *"ois estavam vindo duplicados. Percebi que o problema era que estava faltando um OrderBy"* order by não interfere nos duplicados só muda a ordem com que os dados são retornados, não seria um "group by" ?
Comment: Sim, é o group by, escrevi errado
Answer: Você pode tentar adicionar o restantes dos campos dentro do group by:
group f by (Campo1, Campo2, Campo3)

Você pode verificar a documentação da Microsoft
Assim como este link com mais alguns exemplos

Group By Linq só retorna o campo Id


What is $\lim_{(x,y)\to(0,0)} \frac{(x^3+y^3)}{(x^2-y^2)}$?

Question: In class, we were simply given that this limit is undefined since along the paths $y=\pm x$, the function is undefined.
Am I right to think that this should be the case for any function, where the denominator is $x^2-y^2$, regardless of what the numerator is?
Just wanted to see if this is a quick way to identify limits of this form.
Thanks for the discussion and help!
Answer: If you define $$\lim_{\langle x,y\rangle\to\langle a,b\rangle}f(x,y)\tag{1}$$ in such a way that it exists only when the function is defined in some open ball centred at $\langle a,b\rangle$, then what you wrote is correct. This is analogous to defining $$\lim_{x\to a}f(x)$$ only when $f(x)$ is defined in some open interval centred at $a$. However, just as we can talk about one-sided limits on the real line, it makes perfectly good sense to talk about $(1)$ whenever $f(x,y)$ is defined at points arbitrarily close to $\langle a,b\rangle$. In that case it's understood that we look only at the limit along 'paths' within the domain of $f$. On that understanding $$\lim_{\langle x,y\rangle\to\langle 0,0\rangle}\frac{x^3+y^3}{x^2-y^2}$$ still does not exist, but for a more fundamental reason.
Suppose that you approach the origin along the curve $y=\sin x$. Then by l'Hospital's rule you have
$$\begin{align*}
\lim_{\langle x,y\rangle\to\langle 0,0\rangle}\frac{x^3+\sin^3x}{x^2-\sin^2x}&=\lim_{\langle x,y\rangle\to\langle 0,0\rangle}\frac{3x^2+3\sin^2x\cos x}{2x-2\sin x\cos x}\\\\
&=\lim_{\langle x,y\rangle\to\langle 0,0\rangle}\frac{3x^2+\frac32\sin2x\cos x}{2x-\sin2x}\\\\
&=\lim_{\langle x,y\rangle\to\langle 0,0\rangle}\frac{6x-\frac32\sin2x\sin x+3\cos2x\cos x}{2-2\cos2x}
\end{align*}$$
which does not exist: the numerator approaches $3$ and the denominator, $0$. The problem is that this path, although it stays within the domain of the function, approaches the line $y=x$ so quickly as it approaches the origin that the denominator approaches $0$ much faster than the numerator, and therefore the function blows up as we approach the origin along this path.
Comment: @DayLateDon: Ouch. I sure do; thanks!
Comment: Compelet answer. :-)
Answer: Brian's answer is nice, so it got me thinking, what about approaching the point along other functions, $y=f(x)$, with $f(0)=0$. Let us plug that into the expression to get $\frac{x^3+f^3}{x^2-f^2}$. This will give $$\lim_{x \to 0} \frac{x^2-xf+f^2}{x-f}\;.$$
We will now perform a l'Hospital's rule to get $$\lim_{x \to 0} \frac{2x-xf'-f+2ff'}{1-f'}\;.$$
If at this point we assume that $f'(0)=1$, we have another $0/0$. If $f'\neq 1$, we get that the limit is zero.
So letus now hit this with another l'Hospital. This will give us
$$\lim_{x \to 0} \frac{2-2f'-xf''+2f''f'+2(f')^2}{-f''}\;. \qquad (*)$$
When evaluate at $x=0, f'=1$, we get  $$\lim_{x \to 0} -\frac{2f''+2}{f''}\;.$$
And you can decide what you want the limit to be by picking a value for $f''$.
Comment: I fixed an obvious typo (you had 'If $f'\ne0$' instead of $f'\ne1$) and moved the periods inside the displayed formulas.
Answer: It doesn't matter that along the lines $y = \pm x$, the function is undefined ; you can still stay inside the domain of $f(x,y) = \frac{x^3 + y^3}{x^2-y^2}$ and try to compute the limit, under the constraint that $y \neq \pm x$. The limit could still be defined at $(0,0)$.
But there are cases where your trick doesn't work : think of $\frac{x^4-y^4}{x^2-y^2} = x^2 + y^2$, which is continuous at $(0,0)$.
Hope that helps,
Answer: For your function, in the domain of $f$ (so $x\ne \pm y)$, to compute the limit you can set $x=r\cos\theta, y=r\sin\theta$, and plug it it. You get $\lim\limits_{r\to 0} \frac{r^3(cos^3\theta+sin^3\theta)}{r^2(cos^2\theta-sin^2\theta)} =\lim\limits_{r\to 0} \frac{r(cos^3\theta+sin^3\theta)}{(cos^2\theta-sin^2\theta)}$, and you can easily see that this is $0$ for any $\theta$ in the domain of $f$ (you need to avoid $\theta = \frac{n\pi}{2}-\pi/4$). Of course, if you are considering the whole plane, then the limit does not exist, because the function isn't even defined at $y=x$, so you can't compute the limit along that path.
Comment: There are paths within the domain of $f$ along which the limit is not $0$; e.g., $\theta(t)=\tan^{-1}\left(\frac{\sin t}t\right)$ and $r(t)=\sqrt{t^2+\sin^2t}$ as $t\to 0$.
Comment: My mistake. Also you need $\theta \ne \frac{n\pi}{2}-\pi/4$. I edited my post.
Comment: It's still wrong, I'm afraid. As @Baby Dragon and I showed, there are paths in the domain along which the limit is *not* $0$. Your calculation is valid only for straight-line paths.


Harmonizing All Music with the I,IV and V Chords?

Question: I read somewhere that any melody line can be harmonized using the I,IV and V  chords of the major scale.  The reasoning behind this being that any possible note played in the melody will exist in one of these chords.  My question is whether this is true, and if not, in what context would this idea fail?
Example
G+C+E (Ma)
G+B+D (ry)
E+G+C (had)
G+B+D (a)
G+C+E (lit)
G+C+E (tle)
G+C+E (lamb)
Comment: The letters G-C-E (Ma) are notes of the same chord.  It is the root C chord 2nd Inversion.
Comment: What are you proposing that you play six chords while singing the word Mary?
Comment: Any note can be harmonized against any chord or combination of chords. It's a matter of what you want the result to *sound like* — but that's a matter of opinion/intent.
Comment: F+A+D is not one of I, IV or V in key C.. Was that your intention to show?
Comment: Excellent observation.  I retrieved the example from the same place I got this idea from.  If I correct the chord for (ry) and (a), I can still use G-B-D (the V), though, according to the concept described in my question.  I tested this out on my guitar while playing the melody as the highest note and it seems to work (in this situation of course).
Answer: This is generally true of a melody in a major key. However, things can be more complicated in actual usage.
If the melody contains chromatic notes that are not in the key, the basic three chords may not work. Also if the melody modulates into an entirely different key than the original key, it won't work either. There are many songs where the melody does not stay in one key the whole time. In that case you need to identify what new key or keys that the melody has modulated into, and use the chords from the new key to harmonize that part of the melody.
You should also learn about the distinction between homophonic texture (the kind of texture used in your own example, where you have a different chord for every syllable or note in the melody) and melody with chordal accompaniment, which is what people are usually talking about when they say using three chords to harmonize a melody. In the latter, you usually only have one or two chords per measure, and each chord rings out for a few beats while the melody notes change above it.
Comment: +1 for actually mentioning all the caveats with this approach (must be in major key, avoid chromatic notes, and not modulate).
Answer: The I,IV,V are the basic chords in a scale. The other chords (ii,iii,vi, vii) can be 'created' from these chords by substituting some notes for some other. Let's take the C major scale:
I: C,E,G
IV: F,A,C
V: G,B,D
The remaining chords are:
ii: D,F,A -- Take IV, remove C and add D.
iii: E,G,B -- Take V, remove D and add E.
vi: A,C,E -- Take I, remove G and add A.
vii: B,D,F -- Take V, remove G and add F
So, in a way you could say that you could harmonize a simple song like Mary had a little lamb using only the 3 basic chords. You can use the other 4 chords when you want to spice up things a bit. If you have a song that sounds boring only with the basic chords, you can substitute some of the chords with some other. This is really common in music. An example you might have seen:
There is the usual progression I IV V, which is really really often substituted by I ii V.
Also, it is common after V to see vi instead of I.
Answer:
I read somewhere that any melody line can be harmonized using the I,IV
  and V chords of the major scale. The reasoning behind this being that
  any possible note played in the melody will exist in one of these
  chords.

Firstly, that reasoning only makes sense if the imagined melody line that stays within the major scale. The idea will 'fail' for any melody that uses notes outside the major scale.
Even with this restriction then, how profound is it to say that any major scale melody line can be harmonized using the I,IV and V chords?
Even if we restrict ourselves to sets of chords that contain the tonic (I), there are quite a few sets of chords from the major scale that contain all the notes of the major scale:
I,ii,iii
I,ii,V
I,ii,iii,IV
I,ii,vii (dim)
I,IV,V
I,ii,iii,V
I,ii,iii,vi
I,ii,IV,V
I,IV,vii (dim)
I,ii,iii,vii (dim)
I,iii,IV,V
I,ii,IV,vii (dim)
I,ii,V,vi
I,vi,vii (dim)
I,ii,iii,IV,V
I,ii,V,vii (dim)
I,iii,IV,vii (dim)
I,ii,iii,IV,vi
I,ii,vi,vii (dim)
I,IV,V,vi
I,ii,iii,IV,vii (dim)
I,ii,iii,V,vi
I,iii,vi,vii (dim)
I,IV,V,vii (dim)
I,ii,iii,V,vii (dim)
I,ii,IV,V,vi
I,IV,vi,vii (dim)
I,ii,iii,vi,vii (dim)
I,ii,IV,V,vii (dim)
I,iii,IV,V,vi
I,V,vi,vii (dim)
I,ii,IV,vi,vii (dim)
I,iii,IV,V,vii (dim)
I,ii,iii,IV,V,vi
I,ii,V,vi,vii (dim)
I,iii,IV,vi,vii (dim)
I,ii,iii,IV,V,vii (dim)
I,iii,V,vi,vii (dim)
I,ii,iii,IV,vi,vii (dim)
I,IV,V,vi,vii (dim)
I,ii,iii,V,vi,vii (dim)
I,ii,IV,V,vi,vii (dim)
I,iii,IV,V,vi,vii (dim)
I,ii,iii,IV,V,vi,vii (dim)
The set {I, IV, V} is one of the smallest suitable sets of chords, consisting of only three, but there are other sets of three chords that also contain all the notes of the major scale. The question arises, then : why single out {I, IV, V}?
There must be another criterion the author of this proposal is going by, other than simply that the set of chords must contain all the notes in the major scale. In other words, {I, IV, V} is somehow seen as providing more satisfying cadences and harmonic movements than those other groups...
...but then, if providing satisfying harmonic movements is a criteria, why restrict ourselves only to the three chord groups? Why not make a similar statement about {I,IV,V,vi}?
On the other hand, if we restrict ourselves to the three chord groups, it's possible that for some melodies, {I,ii,V} or {I,ii,vii (dim)} would provide more appropriate possibilities.
Possibly the significant thing about I, IV, V is the fact that 1) in itself  it is a common set of chords used to construct a progression (due to the strong relationship between I and IV and I and IV), and 2) it happens to be one of the smallest sets of chords that contains all the notes in the major scale.
Comment: Look at the circle of 5ths and you'll figure it out pretty fast.
Comment: Another way to look at it is that the circle of fifths is important because of the strength of the 2:3 frequency ratio relationship (which *is* the IV-I and I-V relationship I mentioned).
Comment: While that does exist, you're missing the bigger picture in the system. In the circle of 5ths you move up or down a 5th and building a scale off of it you get 1 note difference, but the tonic chords only share one of the previous notes due to how we build chords. Also while the the 4th and 5th degree are important, the 7th degree AKA the leading tone is the money maker which is why the tonic-dominant relationship works and make V-I and I-IV work so you have to look more towards ideas in the tonal system when talking about it.
Comment: The reason I'm kind of being a stickler about this is because you throw the question of "why" the system is used and never resolve it and seem to not grasp it yourself with how it concludes and the alternative suggestions that don't go anywhere which is why I almost down voted this. Remember whenever talking about I, IV, and V we are talking in a tonal context like it or not so an answer should come from a similar context which the understanding and  the explanation of tonic, dominant, and subdominant in the grand scheme is crucial.
Comment: I don't think I'm questioning any system or its use in my answer here, simply trying to examining what logic someone would be following if making the statement that "any melody line can be harmonized using the I,IV and V chords of the major scale". Can you clarify what part of my answer you think is confusing/confused?
Comment: You never answer these questions "The question arises, then : why single out {I, IV, V}?", "why restrict ourselves only to the three chord groups? Why not make a similar statement about {I,IV,V,vi}?" and you make a guess at it while coming up short. All the comments were to point you to the answer of those questions with specific terminology and tonal concepts which are absent from this answer which if you want to address those questions which fall a little outside of the scope of the question you need.
Comment: I do suggest a simple answer the to first question - I propose that it's that because *{I, IV, V} is somehow seen as providing more satisfying cadences and harmonic movements than those other groups*. I don't then go into much detail as to why *that* would be, but I'm seeing that as a tangential issue; a different question. The second question - *why restrict ourselves only to the three chord groups? Why not make a similar statement about {I,IV,V,vi}?* is rhetorical - I'm suggesting that that the logic of the statement in the OP itself doesn't make an answer to that question clear.
Comment: You might think I've answered the question in an odd way, but I've simply tried to reply in kind. Is the original statement true? *No* (depending on what exactly is meant by 'harmonize'). Is it true within some limits? *Yes*. Is it profound to point out the I, IV, V chords? *Maybe, to whatever extent that {I, IV, V} is seen as a special group in terms of harmonic function*. I've not really gone into *why* they might be seen as special - that seems to be taking a step away from what the question actually asks (as currently worded).
Comment: Technically, one could harmonize any diatonic melody with one chord, say C major is key and we just play Cmaj13 (with the 11th).
Answer:
I read somewhere that any melody line can be harmonized using the I,IV
  and V chords of the major scale. The reasoning behind this being that
  any possible note played in the melody will exist in one of these
  chords. My question is whether this is true, and if not, in what
  context would this idea fail?

I'm going to try and take a stab at this. I'll try and explain the core concepts of this question so you have a better idea of what is going on.
Firstly lets take melody.
Melodies when written correctly are built on chords. This is a big misconception that seems to be widespread but the first thing you have to decide on when writing a melody is the chord progression.
Ambiguity in regards to chords leads to melodies that have no proper foundation to build on. This is not chord in the way we guitarist think of them. When you jump from c to e to g in a melody then it is clear that the melody in that bar is built on a C Major chord.
There really is no good melody that would have only Primary Chord. Your natural chord progressions for melody writing would simply not allow you to only have Primary Chords in them.
It is simply just a sign of a good melody and harmony exercise to include Secondary Chords as well.
Harmony
Now harmony is when you are given a melody and asked to harmonize the melody for four voices. This is not really how music is written. It is all just an exercise to explain the core concepts of how multiple voices interact with each other.
When you think of harmony you need to think about how the chord interacts with the previous and the next chord. You need to think about the spacing, the doubling of the notes, the resolutions and the form.
Trough this all you have to write a melody within the boundaries of the rules of good harmony. You need that melody to make sense and interact with the given passage.
Where melody is just a single line of notes harmony is four lines of notes that all have to come together.
Now you ask...

I read somewhere that any melody line can be harmonized using the I,IV
  and V chords of the major scale.

I would say that this is somewhat general. Maybe if a melody only consisted of these three chords then maybe you could harmonize it with only these chords but as I say your melody chord progressions should be more than just Tonic, Sub Dominant and Dominant.

The reasoning behind this being that any possible note played in the
  melody will exist in one of these chords. My question is whether this
  is true, and if not, in what context would this idea fail?

The choosing of chords for harmonizing is much more than just choosing one of the three you mention. This is a gross over simplification of good harmony.
There is a plethora of chord progression that are important for harmony that do not have I, IV or V in them. You have the passing 6/4 built on the Leading Tone chord which you will never learn to use in this way.
You have a Imperfect Cadence that can be ii - V. You have an Interrupted Cadence that ca be V-vi. You have a lowered Leading Tone progression that is I - III - Iv with vi sometimes being substituted for I.
All of which are things that are often used in harmonization and all of which you will never learn with this misconception.
Truly harmony is much more than just a melody written of I-IV-V
Comment: I'm confused, are you saying that it's *true* that you have to decide on chords before you write a melody? Because I couldn't disagree more with that concept. Or are you saying it's a *misconception* that you have to write the chords first? Which I would agree with.
Comment: **"Melodies when written correctly are built on chords. This is a big misconception that seems to be widespread but the first thing you have to decide on when writing a melody is the chord progression."**  I think that in the context that this answer is written, this is a reasonable response, but this isn't at all universally true.  E.g., in older [counterpoint](https://en.wikipedia.org/wiki/Counterpoint), "harmonic, "vertical" features are considered secondary and almost incidental ... Counterpoint focuses on melodic interaction—only secondarily on the harmonies produced by that interaction."
Answer: I think theoretically yes, but it won't sound completely "right". Because say in a major key, if you took I IV V, these would only be the major sounding chords. So in a song that always sounds happy this would work.
But sometimes you want to throw in those minor chords (the ii,iii,vi) in order to create a sense of sadness that you otherwise couldn't with the I IV V.
After the I IV V the most common is the vi and then the ii.
Answer: I, IV and V contain, between them, all the notes of the scale.  So there's a degree of truth in the statement, but only if you restrict the melody to scale notes.  And only if you want a possible harmony, not a good one!


loading data from excel to sql

Question: I am loading data from excel sheet to sql using SSIS package. When I load data into table1 using the datatype varchar(255) for all fields, I had no problem.  But, when I tried to load data from table1 to table2 it showed an error : can not convert datatype varchar to numeric.
All of the fields in table2 have valid datatypes. Now when I look at the data in table1 for the field (its datatype is decimal(5,2) in table2) which was giving me that error, I saw one of the record had a value of "2.9999999999999999E-2" in table1. The same record in the excel sheet is 0.03.
In the same column there is a record with a value of 0.01. Why did it change the value for 0.03? Do I have to convert the data in excel sheet?  I want to load records from excel sheet the way they are.
I am using sql server 2005.
Thanks
Answer: Odds are, that the 0.01 value is in a text cell in excel, while the 0.03 is in a numeric cell.  Your library sees that it has a number, and tries to import it numerically.  Floating point numbers can't represent certain numbers perfectly, which leads to your error.
One way to solve this is to mark the cells in the original spreadsheet as text, or if your library has the capability, mark them as text when importing the values.
Answer: Have you considered checking the connection string, particularly IMEX=1?  This worked for me when I was facing similar issues in the past.
http://www.connectionstrings.com/excel
Answer: Sorry for the digging up of a 5 year old post, but it looks like a lot of people have looked at it.
I ran into a similar issue with numbers from excel coming over in the  Scientific Notation format
Try:
<code>CONVERT(FLOAT, [Your Column Here], 2)
</code>
Then wrap it into whatever type you need
Apparently when you get the Scientific Notation format don't work as expected.
This is how I found the answer see Quassnoi's post

loading data from excel to sql


Test of significance for a nonlinear trend in time series analyses, ARIMA

Question: I have water temperature data consisting of monthly means for 20 years. As one would expect there is a definite seasonal/cyclical pattern. I wish to model the time series data by fitting an ARIMA model.
Two questions:

Would this be an appropriate analysis for temperature data?
How do I test if there is a significant trend (a p-value would be nice!) of increasing water temperature in this data which is clearly nonlinear?
<code>  Year  Month   Temperature
  1953  March   21.88302419
  1953  April   21.22354167
  1953  May 19.98760753
  1953  June    18.39943056
  1953  July    16.94803763
  1953  August  16.71372312
  1953  September   17.46852778
  1953  October 18.12665323
  1953  November    19.29626389
  1953  December    19.9861828
  1954  January 20.86797043
  1954  February    21.91311012
  1954  March   22.3521371
  1954  April   20.94972222
  1954  May 20.34298387
  1954  June    18.39666667
  1954  July    17.18486559
  1954  August  17.13916667
  1954  September   17.35176389
  1954  October 18.30989247
  1954  November    19.14590278
  1954  December    20.25358871
  1955  January 21.8128629
  1955  February    22.73234195
  1955  March   21.60393817
  1955  April   20.92319444
  1955  May 20.09857527
  1955  June    18.16069444
  1955  July    17.24068641
  1955  August  17.14547043
  1955  September   16.676875
  1955  October 17.31141129
  1955  November    19.70297222
  1955  December    20.0072043
  1956  January 21.77598118
  1956  February    21.62849702
  1956  March   21.94706989
  1956  April   20.6965
  1956  May 19.05202957
  1956  June    17.81277778
  1956  July    17.3078629
  1956  August  17.35629032
  1956  September   17.84531944
  1956  October 18.0919086
  1956  November    19.68886111
  1956  December    20.42611559
  1957  January 21.82801075
  1957  February    22.76324405
  1957  March   22.8733109
  1957  April   21.89454167
  1957  May 20.41923387
  1957  June    18.85397222
  1957  July    18.36353495
  1957  August  17.7866218
  1957  September   18.141875
  1957  October 18.99646505
  1957  November    19.38388889
  1957  December    20.64235215
  1958  January 21.35995968
  1958  February    21.56925595
  1958  March   23.04430108
  1958  April   21.3755
  1958  May 20.06209677
  1958  June    18.8416968
  1958  July    17.77298387
  1958  August  17.71418011
  1958  September   17.39784722
  1958  October 17.88056452
  1958  November    19.88647222
  1958  December    21.30474462
  1959  January 22.51991925
  1959  February    23.35799107
  1959  March   22.5344086
  1959  April   22.08238889
  1959  May 20.04969086
  1959  June    19.00790278
  1959  July    18.27353495
  1959  August  17.55283984
  1959  September   17.22833333
  1959  October 18.01680108
  1959  November    18.6775
  1959  December    19.8469086
  1960  January 20.68198925
  1960  February    21.83154762
  1960  March   21.50378073
  1960  April   20.34733796
  1960  May 19.58559588
  1960  June    18.45406456
  1960  July    17.53125
  1960  August  16.96117944
  1960  September   16.98905903
  1960  October 17.73582997
  1960  November    18.65800694
  1960  December    19.32586694
  1961  January 19.71767473
  1961  February    21.04120651
  1961  March   21.35288306
  1961  April   20.30002431
  1961  May 18.6331754
  1961  June    18.62307639
  1961  July    18.2453629
  1961  August  16.99882056
  1961  September   17.33630194
  1961  October 16.92414798
  1961  November    18.00218056
  1961  December    18.73533154
  1962  January 19.35255376
  1962  February    20.21777565
  1962  March   21.21165323
  1962  April   20.21122222
  1962  May 19.51232527
  1962  June    18.25903472
  1962  July    17.53709677
  1962  August  16.88536713
  1962  September   16.02052778
  1962  October 17.28112903
  1962  November    18.13883333
  1962  December    19.95607527
  1963  January 21.23651882
  1963  February    19.9141523
  1963  March   21.89069892
  1963  April   21.20695833
  1963  May 19.74677419
  1963  June    18.01259722
  1963  July    17.27166667
  1963  August  16.63431452
  1963  September   16.78509722
  1963  October 18.14424731
  1963  November    19.29784722
  1963  December    18.66950269
  1964  January 22.3822043
  1964  February    21.98738095
  1964  March   22.06516129
  1964  April   20.77536364
  1964  May 19.60976277
  1964  June    17.69825347
  1964  July    17.47606586
  1964  August  16.81932997
  1964  September   17.18971597
  1964  October 17.78017742
  1964  November    19.022725
  1964  December    19.98706116
  1965  January 20.62890995
  1965  February    21.86117039
  1965  March   21.76144624
  1965  April   21.41317694
  1965  May 19.89478226
  1965  June    18.14042639
  1965  July    16.53863575
  1965  August  17.4485
  1965  September   17.38953973
  1965  October 17.79755645
  1965  November    19.54316042
  1965  December    20.5874375
  1966  January 21.33427016
  1966  February    22.30857515
  1966  March   21.52931855
  1966  April   21.37102778
  1966  May 20.02285954
  1966  June    18.68837361
  1966  July    18.03406653
  1966  August  17.54710551
  1966  September   18.25538665
  1966  October 18.1833172
  1966  November    19.20074583
  1966  December    20.33642742
  1967  January 21.07148185
  1967  February    22.28252799
  1967  March   21.35484005
  1967  April   20.18232083
  1967  May 18.87129435
  1967  June    18.12438472
  1967  July    17.34492215
  1967  August  17.27377688
  1967  September   16.92026389
  1967  October 17.54733871
  1967  November    18.37069444
  1967  December    19.83104839
  1968  January 21.33081989
  1968  February    21.68643155
  1968  March   21.64960081
  1968  April   20.69278889
  1968  May 18.98352957
  1968  June    18.3307
  1968  July    17.23593011
  1968  August  16.94599866
  1968  September   16.48676389
  1968  October 16.9316371
  1968  November    17.66956806
  1968  December    19.41569489
  1969  January 21.37015591
  1969  February    20.70526488
  1969  March   21.64435081
  1969  April   20.97368472
  1969  May 20.4197836
  1969  June    18.31870833
  1969  July    17.4229328
  1969  August  17.10003226
  1969  September   17.36448056
  1969  October 18.07489785
  1969  November    18.97591528
  1969  December    20.51862231
  1970  January 22.03919489
  1970  February    21.64364487
  1970  March   21.24479032
  1970  April   20.28298611
  1970  May 19.29274059
  1970  June    17.73382639
  1970  July    16.55055511
  1970  August  16.43872446
  1970  September   17.48788472
  1970  October 17.9552638
  1970  November    19.17982222
  1970  December    19.68649597
  1971  January 21.57199866
  1971  February    21.76181178
  1971  March   21.11755511
  1971  April   20.01336667
  1971  May 19.05388978
  1971  June    18.52273333
  1971  July    17.39878226
  1971  August  16.3497836
  1971  September   17.0700125
  1971  October 18.32019758
  1971  November    18.96098333
  1971  December    19.1836586
</code>
Comment: Are you willing to assume a parametric form of the trend (e.g. linear, quadratic etc. with unknown coefficients) or would you rather allow the trend to take whatever weird shape? The former case is simpler: you would include relevant variables in the model (e.g. a linearly or quadratically increasing deterministic series; this would turn the model from ARIMA to ARIMAX), and get the estimates of their coefficients (and p-values, too).
Comment: See e.g. Rob J Hyndman's [blog post](http://robjhyndman.com/hyndsight/arimax/). He also discusses R functions for this. Look at the documentation for these functions to see exactly how they work. The relevant trend variables could be $x_1=\{1,2,...,t\}$, $x_1=\{1^2,2^2,...,t^2\}$ for a linear and a quadratic term.
Comment: Proper transformations of the data including Generalized Least Squares (weighted regression) or properly segmenting the data due to parameter transiency while incorporating deterministic structure can be much more efficient. Post your data and we can have a comparative analysis highlighting the differences between different methods..
Comment: Dear Richard, thank you, as the temperature data are of seawater in a natural setting, I think I would be okay assuming they follow some sort of parametric trend. How would I go about determining/including relevant variables in the model as you allude to? Is there perhaps an example on the web that spells out how to do this?
Comment: Thank you Richard and IrishStat! I will post the data immediately in the form of an edit at the end of my original post (due to character limitations of comments), as requested. It would be great to see a comparative analysis, preferably done in R.
Answer: I took your 226 values and analyzed them with AUTOBOX. A reasonable ARIMA model  was developed BUT a statistically significant change point was detected using the CHOW Test  . The most recent 79 values were then examined to identify a suitable model. withe the following Actual/Fit and Forecast graph.  . The residuals from this model suggest randomness suggesting a sufficient model.   . The forecast plot/table is presented here   . Finally the Actual/Cleansed graph showing the two anomalous points is interesting and informative  . The statistical summary is here  and here   There is no suggestion in the data that would support a "trend" conclusion. Since there is no evidence of a transient error variance or a level shift in the model's errors , any number of simple/uncomplicated approaches should work with data if you ignore the change in parameters over time .
A question to the OP "Are you aware of anything that might have been responsible for the change in parameters ?"
EDIT: TO ANSWER @FORECASTER'S QUESTION REGARDING LIMITS.
I extended the forecast period to 200 periods and reported the multiplier (based upon the psi weights) of the error variance . In short the limits are diverging BUT slowly . Here is the results for the first 24 periods  and the periods  97 - 120  and finally the last set (175-200)
Comment: why would the prediction interval in autobox be constant, I would assume that as we go farther out into to the future, the uncertainty increases therefore the prediction interval be more wider? Thank you
Comment: Thank you IrishStat, very impressive, especially to a biologist! To answer your question, I am unaware of anything specific that may have been responsible for the change in parameters..the water temperature is simply at the mercy of the ocean currents in the area and the global atmosphere. When you say that there is no "trend" since there is no transient error variance or level shift in the model's errors which particular summary statistic/statistics are you looking at? And does the significant p-value for Differencing 1Autoregressive-Factor#1 simply mean that the fitted model is accurate?
Comment: @forecaster Confidence limits for a stationary process reach an asymptote.. sometimes quickly (e.g.  white noise) sometimes slowly.
Comment: @Sean The conclusion that there is no identifiable trend comes from the fact that there is no differencing involved in the model. If there was AND if a trend constant was included and significant then there would be a positive conclusion about the existence of trend. Alternatively if AUTOBOX identified the need for a deterministic variable as a result of it's Intervention Detection Procedure ala TSAY 1986 - like  x1={1,2,...,t}, (BUT it didn't) then one could conclude about the need for a  trend
Comment: @forecaster I should have added that this model is non-stationary with psi weights of      1.0
           .5763
           .3321
           .1914
           .1103
           .0636
           .0366
           .0211
           .0122
           .0070
           .0040
           .0023
          1.00
           .5771
           .3326
           .1916
           .1104
           .0636
           .0367
           .0211
           .0122
           .0070
           .0040
           .0023
          etc....
Comment: @forecaster I have added some output to my answer to more fully explain limits
Comment: @IrishStat I think I understand. So even though Differencing with a lag of 1 is significant (according to the summary stats) there is still no trend? So the ARIMA model would be of the form ARIMA(p,0,q) in this case? Hypothetically, what would the Differencing statistics show if there were a trend ?
Comment: correct ... the arima model is (1,0,0)(0,1,0) . The need for a trend would be suggested by the absolute value of (mean/SEM) in the residuals > 2.0 . The statistic in this case is .685943 suggesting non-significance.
Comment: @Sean  correction ! "The conclusion ....in the model " should have been "The conclusion that there is no identifiable trend CAN can come from a stationary model that has no differencing. Your model has differencing.BUT that is not sufficient."
Comment: @IrishStat thank you for your expertise! Earlier you asked if I was aware of anything that may have changed the parameters and I said I was not. Well, I have delved into the metadata and found that a change did occur in how the data were collected. Prior to 1965/4 temperatures were being recorded by TWO loggers side by side, after that time temperatures were recorded by only one logger.
Comment: Amazing that AUTOBOX was able to detect this as a structural break exactly at that time! As a result, I think the most suitable approach therefore is to use data from only one logger through time to avoid a structural break and rerun the analysis..
Comment: I took the most recent 79 values (starting at 1965/4) and found that yet another break point/change point was identified starting at 1968/5. These 42 observations yielded a very simple ARIMA model [(1-B**12)]Y(T) =+     [(1-  .658B** 1)]**-1  [A(T)]
Comment: time ran out on my edit.. here is the complete thought. I took the  recent 79 values (starting at 1965/4) & found that another break point/change point (visually obvious from a plot ) was identified starting at 1968/5.These 42 vlues yielded a very simple but sufficient ARIMA model                                           [(1-B**12)]Y(T) =+     [(1-  .658B** 1)]**-1  [A(T)] which generated essentially non-trending forecasts. The whole idea is to aid the "human eye" in identifying homogenous structure and not naively using all the  available data without an appropriate  challenge.


How can I enter a custom maintenance mode message for a non-English D8 site?

Question: I am playing around with a D8 site (8.0.0-beta6) and having trouble locating where some of the controls have gone. During the initial installation procedure I selected a non-English localization (Turkish). Subsequently putting the site into maintenance mode does indeed show the Turkish localized default maintenance message.
The trouble is I want a custom message. Entering one in the box on the maintenance page (I'm viewing the Administrator site using the same default locale) has no effect on the message displayed. Apparently this message changes the English only because if I add /en to by site URL I get the English maintenance mode page and header, but with my custom Turkish message below it.
Where does one change custom localized maintenance mode messages?
Comment: This is likely a core issue, I think I've recently seen a bug report about this. Yep, see https://www.drupal.org/node/2212069#comment-9623671, I think that is related.
Comment: A quick (and maybe dirty hack) is overwriting the maintenance template. It is located in core\modules\system\templates\maintenance-page.html.twig
Comment: @Robin It looks like I can also do it by enabling the interface translation module and overriding the translation for the default string, but both seem like crazy dirty hacks. I'm looking for what the _correct_ way to do this is. It doesn't seem like something that would require a hack—unless this is actually a bug.
Comment: @Berdir Thanks for tracking that down. It looks like the "answer" to this question is that I was doing it properly but there is a bug, so one of the hacks would be necessary at this moment or wait for a patch. Care to make that an answer or shall I?
Answer: This function should have worked as expected. There is not some feature somewhere else or another way this should have been done. The result described is the consequence of this bug in the Drupal 8 core. As of this writing a patch has been proposed but not settled on. Such are the consequences of playing around with beta releases!
In the mean time the only way to may a D8 site behave is to hack it using one of the following methods:

Manually implement one of the patches described in that issue.
Manually override the core template file (core\modules\system\templates\maintenance-page.html.twig).
Use the Interface Translation module to access the internal localization strings. Override the default maintenance mode string for the target language.

Eventually when that issue is closed and another release cycle happens it should be fixed and providing a custom message from the Admin page will work again as expected.


How to create a hyperlink insertion modal without losing text selection?

Question: I've got a React site which uses Slate-React (https://docs.slatejs.org/libraries/slate-react) to create a rich text input field. Slate uses the browser's prompt() function to apply hyperlinks to selected text, but I need to be able to style the prompt modal so I have to find an alternative. However, if I create a simple Javascript popup with an input field, the selected text is deselected as soon as the focus is applied to the modal so no link is inserted.
Is there a way to move focus to another element whilst still keeping the original text selected?
Essentially I'm trying to emulate the Insert Hyperlink functionality/styling that Google Docs uses:
Answer: you can have this selection in state, and pass some prop using this selection to apply link on this selection. You can do this by using Transforms.select(editor, thisSelection) followed by applying link
Answer: It's pretty hard, especially if you are making an accessible modal that captures input.  We have something like the following:
const handleSave = (inputText: string, url: string, caretPosition: number, remove = false) => {
        let removalOffset = 0;
        let prevPath = null;
        if (remove && Path.hasPrevious(linkPath)) {
          prevPath = Path.previous(linkPath);
          const prevNode = Node.first(editor, prevPath)[0];
          removalOffset = Node.string(prevNode).length;
        }
        Transforms.insertText(editor, inputText, { at: { path: linkPath, offset: 0 } });
        Transforms.delete(editor, {
          at: {
            anchor: { path: linkPath, offset: inputText.length },
            focus: { path: linkPath, offset: text.text.length + inputText.length }
          }
        });
        Transforms.setNodes(
          editor,
          remove ? { link: undefined, url: undefined, linkOpen: undefined } : { linkOpen: undefined, url },
          { at: linkPath }
        );
        handleClose();
        if (!remove) {
          setPointer(inputText, caretPosition, linkPath);
        } else if (prevPath !== null) {
          setPointer(inputText, removalOffset + caretPosition, prevPath);
        }
};

The setPointer function calls Transforms.select. However, you may need to run it asynchronously if you are capturing inputs in the modal for accessibility.


Need to move a debian server from i686 to x86_64 architecture

Question: I have a debian server that I need to move from one hosting provider to another. I don't really know how the old server was setup, all I know is that it's running a Ruby on Rails application with a lot of custom libraries installed and that I should prepare myself for a painful migration.
Old server:
-os: debian 5.0.9
-used disk space: 3.2GB
-architecture: i686
New server:
-os: debian 5.0.9
-free disk space: 10GB
-architecture: x86_64
As you can see the problem is that the servers are running different architectures.
Q: Is there anyway I could somehow migrate the old to the new server in a few steps (or am I just dreaming I could) ?
I was thinking maybe I could:
-get list of packages and gems installed on old server and use for loop to install them all on the new
-copy the disk content from old to new  server while excluding what is architecture-specific (the problem is that I don't really know what to exclude).
Comment: I don't see how this would be a painful migration; it's a pretty standard thing for a sysadmin to do.
Answer: There is a step-by-step guide in the Debian Wiki to upgrade from 32 to 64 bit. http://wiki.debian.org/Migrate32To64Bit
This is really ugly plus you have the problem with your Ruby installation. Are you sure your Ruby libs are running with 64 bit?
I would do definitley do a full backup/image/whatever of the machine, so you can rollback to a functional state in case things go wrong.
Answer: dpkg --get-selections > /backup/installed-software.log should dump everything installed to installed-software.log
dpkg --set-selections < /backup/installed-software.log then dselect should restore it.
You may also need to copy config files from /etc/ - this will depend on your specific set up.
you may also want to install ia-32-libs for supportng ia32 software on x86
Up until this point, you shouldn't have too many issues.
Now, debian does support running x86 libs on x64, so its a matter of copying over the files, looking at what breaks, and installing the appropriate libraries should do the trick
Alternately, a trickier, but 'simpler' option may be to set up a 32 bit chroot in the system, then simply copy over the setup of the old server into the newer one. Another install guide here


Why css hover is affecting only bookmarks?

Question: I have 5 links on site, they are styled with css, and they should turn black after mouse hovers on them. Links to extrenal pages doesn't turn black, only links to bookmarks on page are turning black.
I tried diffrent styling and nothing works. It works normally without React and without the server.
css styling:
.menu_link {
    border: none;
    text-decoration: none;
    color: darkgreen;
    height: 100%;
    width: 20%;
    text-align: center;
    font-size: calc(1em + 1vw);
    font-family: pokemon-hollow;
}

.menu_link:hover {
    color: black;
    background-color: beige;
}

.menu_link:visited, .menu_link:link {
    color: darkgreen;
}

MenuLink class:
import React from 'react';
import './MenuLink.css'

class MenuLink extends React.Component {
    render() {
        return (
            {this.props.name}
        )
    }
}

export default MenuLink;

Menu class:
import React from 'react';
import './Menu.css';
import '../MenuLink/MenuLink'
import MenuLink from '../MenuLink/MenuLink';

class Menu extends React.Component {
    render() {
        return (

                <MenuLink id="main_container" name="Home"/>
                <MenuLink href="https://bulbapedia.bulbagarden.net/wiki/Fennekin_(Pok%C3%A9mon)" name='Fennekin' target='_blank' />
                <MenuLink href="https://bulbapedia.bulbagarden.net/wiki/Braixen_(Pok%C3%A9mon)" name='Braixen' target='_blank' />
                <MenuLink href="https://bulbapedia.bulbagarden.net/wiki/Delphox_(Pok%C3%A9mon)" name='Delphox' target='_blank' />
                <MenuLink id="main_container" name="Galery"/>

        )
    }
}

export default Menu;

No errors, but when I force on hover on link in chrome developer tools i get this:
color: black;
Answer: You are overriding .menu_link hover styles with these
.menu_link:visited, .menu_link:link {
    color: darkgreen; /*this will override .menu_link:hover styles*/
}

If css specificity is exactly the same, order does matter. Styles declared later will be applied.
So change your css to the below:
.menu_link:visited, .menu_link:link,.menu_link {
    border: none;
    text-decoration: none;
    color: darkgreen;
    height: 100%;
    width: 20%;
    text-align: center;
    font-size: calc(1em + 1vw);
    font-family: pokemon-hollow;
}

.menu_link:hover {
    color: black;
    background-color: beige;
}

Comment: @DrunkenJanna those links does not have `href` attribute which is required for using pseudo-classes like `:link` and `:visited`. But for `:hover` it does not require
Comment: there is one more problem. Why the bookmark links weren't affected by wrong css?
Answer: The problem was the order of lines.
.menu_link:visited, .menu_link:link {
    color: darkgreen;
}

were after
.menu_link:hover {
    color: black;
    background-color: beige;
}

and they were overwriting hover selector. Changing the order to:
.menu_link:visited, .menu_link:link {
    color: darkgreen;
}
.menu_link:hover {
    color: black;
    background-color: beige;
}

was what fixed the problem.
Bookmarks links weren't affected, probably because they can't have state :visited or :link.