Plotting average shows (log) linear trend but fitting line has 0 p value

Question: I have some data and I am examining the relationship between two variables. When I form x-bins and take summary statistics of y in those bins, I see the plot below. The black line is the mean of each bin, the blue line is the median of each bin and the ranges are the lower and upper quartiles of the bins.

Here is a histogram of all my y values:

When I do linear regression with scipy.stats.linregress I get intercept -1.21, slope 1.69, r value .21 and p value 0. I think the slope and intercept sort of check out, but how do I interpret the pvalue of 0?
Comment: Remember that he p-value in this type of analysis is reported for a [test of the null hypothesis](https://en.wikipedia.org/wiki/Statistical_hypothesis_testing): the hypothesis that there is _no_ linear relationship between _y_ and _x_.
Comment: So if p was 1, that would mean I could reject the hypothesis with very high confidence that there is no linear relationship between x and y? p being 0 here means I have no confidence that there is not no linear relationship? Can you offer any guidance for how I could interpret the binned statistics vs the linear regression?
Answer: The p value in this case is the probability that you would find at least this big a linear relationship between $y$ and $x$ in a data sample of your size if, in fact, there was no such relationship (and the assumptions behind the test hold). So a p value of approximately 0 under that null hypothesis means that there is essentially no chance that there is no linear relationship: in other words, very strong evidence for some linear relationship. Your very large number of data points plays a big role in that conclusion.
In this particular case, the binning has shown further information that you might want to explore.
First, the strictly linear relationship seems to tail off at high absolute values of $x$ at both extremes. If you have a theoretical basis from your knowledge of the subject matter for why that would be the case, you could try a further transformation of the $x$ values in a linear regression, or a non-linear model of appropriate form, to deal with those extremes. Alternatively, if you have no theoretical basis for a particular form, you could use linear regression with spline terms to find an empirically based fit better than simply linear.
Second, the higher median than mean $y$ value in all of your bins suggests that the residuals between observed and predicted values might not have the normal distribution that is assumed in the calculation of p values. With this large a data set and this strong a relationship that shouldn't pose a practical problem, but it's something that might be important for other reasons in your application.
Comment: Thanks for answering! I would upvote it if I had more reputation. So my response to @Edm's comment is incorrect? The low p value actually means there *is* a linear relationship?
Comment: @GregoryStarr your response to my comment on your question was incorrect. A low p value for the test of the null hypothesis (no linear relationship) is evidence in favor of a linear relationship here. That's how these types of statistical tests work: you estimate the probability that you'd get that strong an apparent relationship by chance even if there wasn't really one.
Comment: Thanks for the help!


Query performance optimization for dynamically joined columns

Question: Current situation in SQL Server database
There is a table Entry with the following columns:

EntryID (int)
EntryName (nvarchar)
EntrySize (int)
EntryDate (datetime)

Further there should be the possibility to save additional metadata for an Entry. Names and values of these metadata should be free to choose and there should be the possibility to dynamically add those without changing the table structure of the database.
Each metadata key can be one of the following data types:

Text
Numeric value
DateTime
Boolean value (True/False)

Thus there is a table DataKey to represent the metadata names and datatypes with the following columns:

DataKeyID (int)
DataKeyName (nvarchar)
DataKeyType (smallint) 0: Text; 1: Numeric; 2: DateTime; 3: Bit

In table DataValue for each combination of Entry and DataKey values can be inserted depending on the data type of the metadata key. For each data type there is one nullable value column. This table has the following columns:

DataValueID (int)
EntryID (int) Foreign-Key
DataKeyID (int) Foreign-Key
TextValue (nvarchar) Nullable
NumericValue (float) Nullable
DateValue (datetime) Nullable
BoolValue (bit) Nullable

Image of the database structure:

TARGET
Target is to retrieve a list of entries fulfilling the specifications like in a WHERE clause. Like the following example:
Assumption:

Meta data key KeyName1 is text
Meta data key KeyName2 is DateTime
Meta data key KeyName3 is numeric
Meta data key KeyName4 is Boolean

Query:
... WHERE (KeyName1 = "Test12345" AND KeyName2 BETWEEN '01.09.2012 00:00:00' AND
'01.04.2013 23:59:00') OR (KeyName3 > 15.3 AND KeyName4 = True)

Target is to do these queries in a very efficient way, also with a large amount of data like

Number of entries > 2.000.000
Number of data keys between 50 und 100 or maybe > 100
Per entry at least a subset of values specified or maybe also a value for each key (2.000.000 * 100)

PROBLEM
The first problem arises when building the query. Normally queries require to have sets with columns that can be used in the WHERE clause. In this case the columns used in the queries are entries in table DataKey as well to be able to dynamically add metadata without having to change the database table structure.
During research a solution has been found using PIVOT table techniques at runtime. But it turned out that this solution is very slow when there is a large set of data in the database.
QUESTIONS

Is there a more efficient way or structure to save the data for this purpose?
How can the requirements listed above be fulfilled, also with regard to performance and time consumption when querying?

Here is a sql fiddle with the discribed database structure and some sample data: http://www.sqlfiddle.com/#!3/d1912/3
Comment: This is a well written question, and although you have bypassed the "not strongly typed" argument of the EAV vs normalised form debate. There are literally [thousands of articles/questions & answers/blogs](https://www.google.co.uk/search?q=sql+eav) comparing the two.
Comment: Another option might be to put the properties into an XML column containing all the properties in one XML as elements or maybe even attributes of the root element.
Comment: At the cost of some additional diskspace and query complexity, you could extract some of the EAV stuff into a normalised form (in a separate table), without sacrificing the flexibility you have. That's the sort of thing you can then update when you need to.
Comment: I am not sure, and never have used them myself, but maybe using `sparse columns` (see http://technet.microsoft.com/en-us/library/cc280604.aspx) for the additional properties in the main table would be an option? You would then not need any joins in your queries, and could have many, many property columns.
Answer: One of the fundamental flaws in an Entity Attribute Value design (which is what you have here) is the difficulty of efficient and performant querying.
The more efficient structure for storing data is to abandon EAV and use a normalised relational form. But that will necessarily involve changing the structure of the database when the data structures change (which should be self evident).
You could abandon your TextValue/NumericValue/DateValue/BoolValue fields and replace them with a single sql_variant column, which would reduce your query complexity slightly, but the fundamental problem will remain.
As a side note, storing all numerics as floats will cause problems if you ever have to deal with money.
Answer: I dont feel qualified to comment on what is best, or on design approaches. In fact I'm inclined not to answer at all. However I have thought about your problem and that
you've taken the time to describe it clearly, and this is how I would approach it.
I'd store each metadata datatype in its own table; So
Table MetaData_Text:
    ID int identity
    EntryID int
    KeyName nvarchar(50)
    KeyValue nvarchar(max)

MetaData_DateTime, MetaData_Boolean & MetaData_Numeric have the same structure as this, but with the appropriate different datatype of the KeyValue column in each case.
The relationship between an Entry & each of these tables is 0-Many; While every row in each of these tables belongs to one Entry.
To add a new metadata item for an entry, I'd just use a stored procedure taking EntryID, keyname & having optional parameters of possible metadata datatype:
 create procedure AddMetaData @entryid int, @keyname varchar(50), @textvalue varchar(max) = null, @datevalue datetime = null, @boolvalue bool = null, @numvalue float = null
 as ...

For querying, I would define a set of functions to manage each type of (a) metadata datatype & (b) test needing to be performed on that datatype, for example:
 create function MetaData_HasDate_EQ(@entryid int, @keyname varchar(50), @val datetime)
 returns bool
 as begin
     declare @rv bool
     select @rv = case when exists(
       select 1 from MetaData_DateTime where EntryID = @entryid and KeyName = @keyname and KeyValue = @val) then 1 else 0 end;
     return @rv
 end

and incorporate function references into required query logic as per
 SELECT ...
 FROM entry e ...
 WHERE (dbo.MetaData_HasText_EQ(e.EntryID, 'KeyName1', 'Test12345') <> 0
     AND dbo.MetaData_HasDate_Btwn(e.EntryID, 'KeyName2', '01.09.2012 00:00:00', '01.04.2013 23:59:00') <> 0)
   OR (dbo.MetaData_HasNum_GT(e.EntryID, 'KeyName3', 15.3) <> 0
     AND dbo.MetaData_HasBool_EQ(e.EntryID, 'KeyName4', 1) <> 0)

Answer: I believe that performance issues with that kind of data structure may require the structure to be reworked.
However, I think this fairly simple dynamic sql allows you to query as desired, and appears to run reasonably fast in a quick test I did with over 100,000 rows in the Entry table and 500,000 in the DataValue table.
-- !! CHANGE WHERE CONDITION AS APPROPRIATE
--declare @where nvarchar(max)='where Key0=0'
declare @where nvarchar(max)='where Key1<550'

declare @sql nvarchar(max)='select * from Entry e';

select @sql=@sql
    +' outer apply (select '+DataKeyName+'='
    +case DataKeyType when 0 then 'TextValue' when 1 then 'NumericValue' when 2 then 'DateValue' when 3 then 'BoolValue' end
    +' from DataValue v where v.EntryID=e.EntryID and v.DataKeyID='+cast(DataKeyID as varchar)
    +') '+DataKeyName+' '
from DataKey;

set @sql+=@where;

exec(@sql);

Comment: I had originally created a (generated) view based on flattening the DataValues first using MAX() but it turns out the query optimizer handles it exactly like the OUTER APPLY so I actually prefer your approach. Using a (generated) view rather than the dynamic SQL answers Robs requirements better IMHO so I took the liberty to edit your answer into one.
Comment: But strange but true my edit was rejected (**) for being 'too extensive and changing the original intention of the post'. Hmm, I don't really agree but seems I'll need to post my own answer then, sigh.
Comment: "Reasonably fast" is a vague concept. How fast did it run compared to a properly structured normalised database?
Comment: It ran in less than a second.
Answer: You have not specified any background information on how often the table is updated, how often new attributes are added and so on...
Looking at your inputs I think you could use a snapshot which flattens your normalised data. It is not ideal as columns will need to be added manually, but it can be extremely fast. The snapshot could be updated regularly with intervals depending on your users needs.
Answer: First to answer why do people use EAV or KVP even if it is so inefficient query-wise? Blogs and textbooks have many plausible reasons. But in real life, it is to avoid dealing with an uncooperative DBA.
For a small organization with small amount of data, it is ok to have a multi-use database (OLTP + DW), because inefficiencies are not noticeable. When your database gets large, it's time to replicate your online data into a data warehouse. In addition, if the data is meant for analytics, it should be replicated further from your relational data warehouse into a dimensional model or a flat and wide for consumption.
This are the data models I would expect from a large organization:

OLTP
Relational Data Warehouse
Dimensional Model for Reporting
Datamarts for Analytics.

So to answer your question, you shouldn't query against your KVP tables and creating a view on top of it doesn't make it better. It should be flattened out (i.e. pivot) into a physical table. What you have is a hybrid of 1 and 2. If there will be no users for #3, just build #4.
Answer: Based on Dan Belandi's answer I think the easiest way to use this would be by having a stored procedure/trigger that looks at the meta-data table and builds a view on the Data-table accordingly.
Code would then look like this:
 -- drop old view
IF object_id('EntryView') IS NOT NULL DROP VIEW [EntryView]
GO
-- create view based on current meta-information in [DataKey]
DECLARE @crlf char(2)
DECLARE @sql nvarchar(max)

SELECT @crlf = char(13) + char(10)

SELECT @sql = 'CREATE VIEW [EntryView]' + @crlf
            + 'AS' + @crlf
            + 'SELECT *' + @crlf
            + '  FROM [Entry] e'

SELECT @sql = @sql + @crlf
            + ' OUTER APPLY (SELECT '+ QuoteName(DataKeyName) + ' = ' + QuoteName((CASE DataKeyType WHEN 0 THEN 'TextValue'
                                                                                WHEN 1 THEN 'NumericValue'
                                                                                WHEN 2 THEN 'DateValue'
                                                                                WHEN 3 THEN 'BoolValue'
                                                                                ELSE '<Unknown>' END)) + @crlf
            + '                FROM [DataValue] v WHERE v.[EntryID] = e.[EntryID] AND v.[DataKeyID] = ' + CAST(DataKeyID as varchar) + ') AS ' + QuoteName(DataKeyName)
 FROM DataKey

--PRINT @sql
EXEC (@sql)

-- Example usage:
SELECT *
  FROM EntryView
 WHERE (Key1 = 0  AND Key2 BETWEEN '01.09.2012 00:00:00' AND '01.04.2013 23:59:00')
    OR (Key3 > 'Test15.3' AND Key4 LIKE '%1%')

Answer: I would use 4 tables, one for each data type:
MDat1
DataValueID (int)
EntryID (int) Foreign-Key
DataKeyID (int) Foreign-Key
TextValue (nvarchar) Nullable
MDat2
DataValueID (int)
EntryID (int) Foreign-Key
DataKeyID (int) Foreign-Key
NumericValue (float) Nullable
MDat3
DataValueID (int)
EntryID (int) Foreign-Key
DataKeyID (int) Foreign-Key
DateValue (datetime) Nullable
MDat4
DataValueID (int)
EntryID (int) Foreign-Key
DataKeyID (int) Foreign-Key
BoolValue (bit) Nullable

If i had partitioning available, i should use it on DataKeyID for all 4 tables.
Then i should used 4 views:
SELECT ... FROM Entry JOIN MDat1 ON ... EnMDat1
SELECT ... FROM Entry JOIN MDat2 ON ... EnMDat2
SELECT ... FROM Entry JOIN MDat3 ON ... EnMDat3
SELECT ... FROM Entry JOIN MDat4 ON ... EnMDat4

So this example:
WHERE (KeyName1 = "Test12345" AND KeyName2 BETWEEN '01.09.2012 00:00:00' AND
'01.04.2013 23:59:00') OR (KeyName3 > 15.3 AND KeyName4 = True)

Goes like:
...EnMDat1 JOIN EnMDat3 ON ... AND EnMDat1.TextValue ='Test12345' AND EnMDat3.DateValue BETWEEN '01.09.2012 00:00:00' AND
'01.04.2013 23:59:00')
...
UNION ALL
...
EnMDat2 JOIN EnMDat4 ON ... AND EnMDat2.NumericValue > 15.3 AND EnMDat4.BoolValue = True

This will work faster than one metadata table. However you shall need an engine to build the queries if you have many different scenarios at where clauses. You can also omit the views and write the statement from scratch each time.


Correct interpretation of estimates in poisson regression output

Question: I am learning to use and validate the Poisson regression model and interpret the results. I am using some data on grassland plant diversity in response to fertilizer and light. The experimental design is fully crossed; there were a total of32 plots, 8 received fertilizer and light(F+L+), 8 received fertilizer and no light (F+L-), 8 received no fertilizer and light (F-L+), and 8 received no fertilizer and no light(F-L-). The number of species was counted in each plot. In these grassland systems, there is typically a loss of species diversity in response to fertilization, which may be due to light competition. My goal is to estimate whether the loss of species can be prevented by restoring light to these areas.
Here is an example of the data:
structure(list(Fertilizer = structure(c(1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c("F-", "F+"
), class = "factor"), Light = structure(c(1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c("L-",
"L+"), class = "factor"), FL = structure(c(1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("F-L-",
"F-L+", "F+L-", "F+L+"), class = "factor"), LF = structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 2L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("L-F-",
"L-F+", "L+F-", "L+F+"), class = "factor"), Diversity = c(6L,
7L, 10L, 9L, 5L, 9L, 7L, 6L, 10L, 9L, 9L, 11L, 9L, 9L, 7L, 7L,
4L, 4L, 6L, 4L, 5L, 5L, 4L, 4L, 7L, 8L, 9L, 9L, 10L, 10L, 10L,
7L)), class = "data.frame", row.names = c(NA, -32L))

I fit a Poisson distribution model as follows:
PlantMod <- glm(Diversity~FL, data = plants, family = poisson (link = log))
Which gives the following output:
> summary(PlantMod)

Call:
glm(formula = Diversity ~ FL, family = poisson(link = log), data = plants)

Deviance Residuals:
     Min        1Q    Median        3Q       Max
-0.92920  -0.24454   0.04186   0.38080   0.91640

Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)   1.9981     0.1302  15.348   <2e-16 ***
FLF-L+        0.1851     0.1762   1.051   0.2933
FLF+L-       -0.4940     0.2115  -2.336   0.0195 *
FLF+L+        0.1710     0.1767   0.967   0.3334
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 21.2770  on 31  degrees of freedom
Residual deviance:  6.6221  on 28  degrees of freedom
AIC: 136.61

Number of Fisher Scoring iterations: 4

I want to make sure I am interpreting the estimate results correctly, I haven't been able to find good examples using crossed designs like this. Here are a few specific questions I have:

R chose (F-L-) as the baseline/reference group and gave an estimate of 1.9981. Is the exponential of this exp(1.9981) interpreted as the average effect on Y (Diversity), when the other estimators are at X=0? Does it matter which group is used as a baseline (is it related to the question you are trying to answer)?
From what I understand, the slopes can be interpreted as the average effect of the respective estimator on the fitted values for diversity (which incorporates all Y-values, or Diversity values, across all treatment groups). For example the estimated slope for the F-L+ group is 0.1851; since this is reported on the link scale, you could calculate 1-exp(0.1851)=-0.203, and conclude that the model would predict a 20.3% decrease in diversity in F-L+ plots when compared to F-L- plots?
I am not sure how to interpret the p-values in this case. Only (F-L-) and (F+L-) had significant p-values. Is this saying that the average of the (F-L+) and (F+L+) groups (which had the insignificant p-values) are not significantly different in diversity than the (F-L-) group, or that those groups didn't have a significant effect on the outcome of the actual response (Y or Diversity)?
advice on how to validate this model (if I used the best link function, and when I might consider switching to a different distribution, such as quasi-Poisson or whatever else) would also be greatly appreciated.
Answer: The coefficients basically tell you what is the response value to expect for each group, using one of them as reference. So in your design, it's essentially something close to the group mean.
You can first check the mean diversity per group:
with(plants,tapply(Diversity,FL,mean))
 F-L-  F-L+  F+L-  F+L+
7.375 8.875 4.500 8.750

And in log space:
log(with(plants,tapply(Diversity,FL,mean)))
    F-L-     F-L+     F+L-     F+L+
1.998096 2.183238 1.504077 2.169054

And you can see, the first intercept is roughly log (mean of F-L-) and coefficient for FLF-L+ is roughly 2.183238 - 1.998096 = 0.185142.
In your design, there is no slope to speak of. The coefficients tells you something like a log fold change over the reference group, for example, the coefficient of F-L+ is 0.185142, meaning you expect exp(0.185142) =  1.203389 fold over F-L-.
As for your remaining question, some groups might not show a difference over reference, you can always check later with posthoc, using multcomp package.
you should check whether it follows a poisson and in your case, it is actually underdispersed:
fit_q = glm(Diversity~FL, data = plants, family = quasipoisson)
summary(fit_q)

Call:
glm(formula = Diversity ~ FL, family = quasipoisson, data = plants)

Deviance Residuals:
     Min        1Q    Median        3Q       Max
-0.92920  -0.24454   0.04186   0.38080   0.91640

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  1.99810    0.06330  31.564  < 2e-16 ***
FLF-L+       0.18514    0.08566   2.161   0.0394 *
FLF+L-      -0.49402    0.10283  -4.804 4.74e-05 ***
FLF+L+       0.17096    0.08593   1.989   0.0565 .
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for quasipoisson family taken to be 0.2364279)

Are you only providing a selected subset of the data? It's very rare to see underdispersion unless there is some underlying reason for it.
Comment: No actually that was the full data set


Difference between distribution shift and data shift, concept drift and model drift

Question: Lately, I am seeing both terms used interchangeably in several scenarios.

Joaquin Quiñonero in MIT press (NIPS), Dataset Shift in ML
NeurIPS 2021 workshop in DistShift
Model drift: Towards Data Science

Are there differences in the definitions? Especially between data shift and distribution shift?
Answer: I'm not aware of a precise and accepted definition of each of these terms which sharply distinguishes them. There is an excellent blog post on the topic here. But broadly speaking:

Model drift: This refers to the general idea that in some cases model predictions deteriorate over time. I.e. the distribution of model predictions and distribution of true values drift apart from each other. This can happen for a number of reasons.

Concept Drift: This is drift due to the dependent variable. The distributions of data might be staying the same but the relationship between input and output has been altered. For example, in a model to detect fraudulent activity, there may be a change in the definition of what is considered fraudulent.
Data Drift: This is due to changes in the distributions of the input data. For example, using the fraud example again, we might see an increase in certain types of fraud which change the distributions of observations from what was seen in the training data.
Comment: I would say concept drift refers more to a change in the relationship between X and Y.
Comment: So concept drift referes to the Y, and data drift to the X?


How can gfortran tell if I am compiling f90 or f95 code?

Question: I understand gfortran can compile f90 or f95? How does it know which one it is compiling? Also can it compile f77 code? Does ubuntu already have a fortran compiler or do I need to download gfortran?
Comment: Also see [How can gfortran tell if I am compiling f90 or f95 code?](https://stackoverflow.com/q/10884260/608639), [Makefile with different source types](https://stackoverflow.com/q/8940552/608639), [cmake, fortran 2008, and .f08 file extension](https://stackoverflow.com/q/25005017/608639), [Correct suffix for Fortran 2003 source file - Intel Fortran compiler](https://stackoverflow.com/q/20269076/608639), etc.
Answer: gfortran can guess certain things from the file extension; if the file has an extension of .f, .f90, f95, .f03, or .f08 it will assume fixed (.f) or free format with the appropriate standards.  But you can force it to compile (say) fortran2003 code with the option -std=f2003.  Eg, from the documentation,

-std=std
Specify the standard to which the program is expected to conform, which may be one of f95, f2003, f2008, f2018, gnu, or legacy. The default value for std is gnu, which specifies a superset of the latest Fortran standard that includes all of the extensions supported by GNU Fortran, although warnings will be given for obsolete extensions not recommended for use in new code. The legacy value is equivalent but without the warnings for obsolete extensions, and may be useful for old non-standard programs. The f95, f2003, f2008, and f2018 values specify strict conformance to the Fortran 95, Fortran 2003, Fortran 2008 and Fortran 2018 standards, respectively; errors are given for all extensions beyond the relevant language standard, and warnings are given for the Fortran 77 features that are permitted but obsolescent in later standards. -std=f2008ts allows the Fortran 2008 standard including the additions of the Technical Specification (TS) 29113 on Further Interoperability of Fortran with C and TS 18508 on Additional Parallel Features in Fortran.

Note that, with a few exceptions, the fortran standards are highly backwards-compatable, so that much "nice", standard conforming, fortran-77 code is also valid fortran 2003 code.  The main problem with that is that much fortran code from the 80s and earlier is neither "nice" nor standard-conforming.
Comment: Can gfortran be persuaded that a .f file is Fortran 95 or higher?  `-std=f95` is not sufficient.
Answer: Yes, gfortran can compile FORTRAN 77 and Fortran 90,95 etc.   It has many features of 2003 & 2008 implemented. For Fortran 90/95/2003/2008 code a good extension to use is .f90.  Some other compilers don't recognize .f95, .f2003, etc.   And do you really want to rename your files when a new standard comes out?    Use .f or .for for FORTRAN 77.   The main difference is the source layout that the compiler will expect: free form for .f90, fixed form for .f or .for.  (Upper-case filetypes will cause the preprocessor to be run.)  You can force fixed-form layout with the options -ffixed-form  -ffixed-line-length-none.


How would this Markov chain problem be solved?

Question: I have the following problem:

Bob is a salesman. Each week, Bob will either make no money, make a small amount of money, or make a large amount of money. And if Bob makes money, he will either make \$100 (small amount) or \$1000 (large amount). For any given week, if Bob doesn't make any money in the previous week, then the probability that he will make a small amount of money is 0.6, and the probability that he will make a large amount of money is 0.3. For a week where he made a small amount of money in the previous week, the probability of making a small amount of money is 0.4 and the probability of making a large amount of money is 0.2. If Bob made a large amount of money in the previous week, then this week he will make a small amount of money with probability 0.3, and no money with probability 0.7. Let this be a Markov chain $(X_n, n \ge 0)$ with state space $\{0, 1, 2 \}$, where making no money is $0$, making a small amount of money is $1$, and making a large amount of money is $2$.

Assume that Bob earned a small amount of money this week. If we then wanted to calculate the expected value of the total number of times he earns money (both small and large amounts) over the next two weeks (after this week), then how would we do so? Is this a so-called 'hitting times' problem, or is it something else? In the field of stochastic processes, these things often go by different names, so I'm not exactly sure what it would be called; I would appreciate a link to some learning material, such as some online lecture notes or something similar.
Comment: The expected number of times something happens at some sequence of times is just the sum of the probabilities of those events. So here that's $P(X_1 \in \{ 1,2 \} \mid X_0=1)+P(X_2 \in \{ 1,2 \} \mid X_0=1 \}$
Answer: Let $Y_k$ be the indicator variable for $X_k \in \{1, 2\}$; then, the desired quantity is $$\operatorname{E}(Y_1 + Y_2 \mid X_0 = 1).$$
Since expectation is linear, and the expected value of an indicator for an event is the probability of said event, this boils down to computing
$$
P(X_1 \in \{1, 2\} \mid X_0 = 1) + P(X_2 \in \{1, 2\} \mid X_0 = 1)
$$
as pointed out by @Ian in the comments. We can further simplify this to
$$
\sum_{k=1}^{2} \sum_{j=1}^{2} P(X_k = j \mid X_0 = 1)
$$
since the events $X_k = 1$ and $X_k = 2$ are clearly disjoint.
From the information given in the question, the probability matrix for this Markov chain is
$$M = \begin{pmatrix}
0.1 & 0.6 & 0.3\\
0.4 & 0.4 & 0.2\\
0.7 & 0.3 & 0
\end{pmatrix}.$$
If we index the rows and columns from $0$ to $2$, then the $(i,j)$-entry of $M$ is $P(X_1 = j \mid X_0 = i)$.
Similarly for
$$M^2 = \begin{pmatrix}
0.46 & 0.39 & 0.15\\
0.34 & 0.46 & 0.2\\
0.19 & 0.54 & 0.27
\end{pmatrix},$$
the $(i,j)$-entry is $P(X_2 = j \mid X_0 = i)$. If we read the correct entries off of each matrix, we see that the desired sum is
$$
0.4 + 0.2 + 0.46 + 0.2 = 1.26.
$$


How to make Firebase Realtime Database connected to ESP8266 safe

Question: I'm planning on releasing a project that involves an ESP8266 connecting to a realtime database in Firebase. People will build the project, upload the code to their ESP8266 and download a Flutter Android App as a remote control for the whole project with Firebase as the backend.
The API key, database URL as well as the username and password will get sent to the ESP8266 from a Flutter app, so people can upload the code to their own ESP8266 without knowing the API key and database URL.
The problem here is that it would be a very simple hack to retrieve this data from the ESP8266 and create huge amounts of traffic in the database ending in a big bill at my end. There is a new thing called Firebase App Check but since the ESP8266 isn't an app it's useless.
How can I prevent this? Is there a different approach of doing this that I haven't thought of? Or is this not really a threat worth concidering?
Comment: https://firebase.google.com/docs/database/security would be a good place to start, including the video.
Answer: The question is how to securely store credentials on an ESP8266. I wish more people would consider this kind of question.
Unfortunately, the answer is that you can't. The ESP8266 lacks the hardware needed to securely store data. An attacker with physical access to it will be able to read its flash memory which the chip has no ability to encrypt.
Consider upgrading to the ESP32. The ESP32 includes a number of security features, allowing you to permanently program it with an unreadable encryption key so that it will only run signed firmware, and to encrypt flash memory where credentials would be stored. These features let you prevent the CPU from executing firmware which has been tampered with and keep sensitive configuration information secret even from attackers with physical access to the CPU.
Note that some of the security features are irreversible. They're set by changing "efuses" inside the chip, which can only be changed once. For instance, if you set a key to verify signed firmware the chip will from then on only ever be able to run signed firmware verified by that key and you will not be able to change the key or allow it to run unsigned firmware.
If you're going to be providing end users with processors you'll be able to enforce the security measures. If you're not and the users are providing their own hardware and running your firmware, you'll be able in the firmware to detect whether hardware security was enabled... but a malicious end user would be able to tamper with the firmware to not require hardware security.
Sticking with the ESP8266, you can make it more difficult for a casual attacker to retrieve credentials by encrypting them but you still have to store the encryption key. In this case an attacker wouldn't just find the credentials by searching the firmware for strings that resemble credentials, and they would have to decompile the firmware to find how to decrypt the credentials and locate the key. This is not strong security by any means; it's merely increasing the effort of a successful attack.
Ultimately only you can decide what your risks are. How many people will be getting these devices? How trusted are they? While one would be enough, the likelihood someone will attack one increases with the number. Only you can model the group of people with access to the devices, and their behavior.
You may also be able to limit your exposure by carefully adjusting permissions on the backend, setting up rate limiting (even if Firebase does't support this you could go through your own proxy which did) or using other back-end mechanisms.


When it comes to the laws of powers, is there a specific order?

Question: (I see an error I made, but I'd still like to know if there is a specific order.)
I have here $\left(6^{-36}\right)/\left(6^{-16}\right)\cdot\left(6^{16}\right)$.
If I do the division first, it's $-36$ minus $-16$, making an addition of plus $16$ for $6^{-20}$.
This times the $6^{16}$ equals $6^{-4}$.
But if I do the multiplication first, the powers of $16$ and $-16$ cancel out, leaving me with $6^{-36}$, which is a radically different answer.
When it comes to exponents, do I have to strictly go from left to right first, in terms of division/multiplication?
Comment: Exponents are not relevant here; but rather the ambiguity of $a/b\cdot c$.
Comment: I agree with David. The question really boils down to whether an expression like "$10/2 \cdot 5$" is equal to $1$ or $25$. I would say $25$ is the better answer.
Comment: Thats what i mean. A/B*C. Is there a specific order that must be followed? From left to right? Because if i recall, D/M and A/S can be done in any order as long as division and multiplication are done first.
Comment: Oh ok i see. Thank you guys.
Comment: Also related: [do values attached to integers have implicit parentheses](https://math.stackexchange.com/questions/16502/do-values-attached-to-integers-have-implicit-parentheses)
Comment: Welcome to Mathematics Stack Exchange! A quick [tour](https://math.stackexchange.com/tour) will enhance your experience. Here are helpful tips to [write a good question](https://math.meta.stackexchange.com/questions/9959/how-to-ask-a-good-question/9960#9960) and [write a good answer](https://stackoverflow.com/help/how-to-answer). For equations, please refer to this [MathJax tutorial](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference).
Answer: Evaluating $a/b\cdot c$ requires a bracketing convention. On BODMAS we divide first, giving $(a/b)c=ac/b$. On PEMDAS we multiply first, giving $a/(bc)$.
Comment: Both of them are supposed to be equal though no? Both division and multiplication have the same priority, so basically im just wondering if we do left to right as convention? Meaning both bodmas and pedmas would be the same. As in, bodmas isnt always division first and pedmas isnt always multiplication first.
Comment: @Dialectics The end result is that it differs based on context and the notation is ambiguous.  One should use enough parentheses so that what is written cannot be confused (*while also not using so many that it becomes unreadable*).  All that is important for *you* right now is understanding that this is a problem with ambiguity so that you can avoid writing something like this, and understanding how to evaluate either interpretation, be it $a/(bc)$ or $(a/b)c$
Comment: @Dialectics No, they're not of the same priority: that's why the results are so different.
Comment: @JMoravitz I was wondering how to tag someone haha. I understand how to evaluate a/(bc) and (a/b)c because the parenthesis make it obvious. Im not sure if its me who is going wrong or if the question in the textbook is simply too ambiguous.
Im provided (6^9)^-4  /  (6^2)^-8  *  (6^4)^4 in the book. Literally how its written. When we apply the exponent laws, we get what i provided in OP. Either the brackets are written around the base and new powers or they are removed altogether. There are no brackets to indicate the order of division or multiplication. In a case like, is it right to left
Comment: @J.G. So essentially the question is bunk/too ambiguous? its asking me to reduce to a single power then evaluate.
Comment: As already said, in a case like this it is ambiguous.  Know how to do it if you choose to interpret it from left to right.  Know how to do it if you choose to interpret it from right to left.  Know that the book did a poor job in writing the question by using ambiguous notation and that neither answer is "more correct" than the other and any answer given needs this warning that you assumed one or the other.  Know that in the future when presented with a similar question similarly ambiguously worded that you will have the same problem.
Comment: @JMoravitz Alright thank you man XD
Comment: There are some contexts where one reading is preferred or intended and the ambiguous notation is used despite the ambiguity.  E.g., in the case of $360/2\pi$ it should be clear from context that we are talking about converting an angle from radians to degrees and that it is meant to be read $360/(2\pi)$.  That doesn't necessarily excuse it as being poorly written.  In the real world, if you ever see an expression written this way you generally know what the expression was meant to represent as well and so could understand the author's intention.


If statements in masm: Syntax error in control flow directive

Question: I am trying to use the if macro in MASM, but I keep getting the following error message from the MASM assembler: Syntax error in control flow directive. I'm still not sure what the syntax error is here: what is the correct syntax for if statements in MASM?
.686p
.model flat,stdcall
.stack 2048

.data
X   byte         1;

ExitProcess proto, exitcode:dword

.code
start:

.IF(x > 1): ; "syntax error in control-flow directive"
    mov ah, x;
.ENDIF

invoke  ExitProcess, 0
end start

Answer: Remove the colon ":" after ).  Also, you define X as a capital letter, but use the lowercase x, this is not valid.  Assembly is case sensitive.


If the likelihood principle clashes with frequentist probability then do we discard one of them?

Question: In a comment recently posted here one commenter pointed to a blog by Larry Wasserman who points out (without any sources) that frequentist inference clashes with the likelihood principle.
The likelihood principle simply says that experiments yielding similar likelihood functions should yield similar inference.
Two parts to this question:

Which parts, flavour or school of frequentist inference specifically violate the likelihood principle?
If there is a clash, do we have to discard one or the other? If so, then which one? I will for the sake of discussion suggest that if we have to discard something then we should discard the parts of frequentist inference which clash, because Hacking and Royall have convinced me that the likelihood principle is axiomatic.
Comment: Hi, Stéphane. The problem is that Birnbaum proved that the Likelihood is equivalent to other two principles that are so natural that they should necessarily hold. We wrote a short review on this result. Here: http://www.ime.usp.br/~pmarques/papers/redux.pdf
Comment: I have never understand why the likelihood principle should be an axiom.
Comment: @Zen Now I have read more carefully your paper. That's true that it's difficult to disagree with the conditionality principle and the invariance principle.
Comment: LP is not that popular nowadays for practical reasons. By adopting it religiously you avoid the use of model-dependent priors such as the Jeffreys's prior, conjugate priors and hypothesis testing which can be useful in many contexts. I believe that statistics, [same as physics](http://en.wikipedia.org/wiki/Hilbert%27s_sixth_problem), cannot be axiomatised in a meaningful way (although this discussion may sound like [this](http://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F)). But it is important to identify advantages and disadvantages of different paradigms.
Comment: @Zen Thank you. At first glance the point I disagree with is this sentence written below the conditionality principle: "What matters is what actually happened". I should say instead "What matters is what actually happened among the issues which could have occur" (sorry if my english is not correct). That's what I claimed in my discussion with gui11aume: in a certain sense  the likelihood principle claims that the design of the experiment does not matter, and I cannot agree with this point.
Answer: The part of the Frequentist approach that clashes with the likelihood principle is the theory of statistical testing (and p-value computation). It is usually highlighted by the following example.
Suppose two Frequentist want to study a biased coin, which turns 'heads' with unknown propability $p$. They suspect that it is biased towards 'tail', so they postulate the same null hypothesis $p = 1/2$ and the same alternative hypothesis $p < 1/2$.
The first statistician flips the coin until 'heads' turns up, which happens to be 6 times. The second decides to flip the coin 6 times, and obtains only one 'heads' in the last throw.
According to the model of the first statistician, the p-value is computed as follows:
$$ p(1-p)^5 + p(1-p)^6 + ... = p(1-p)^5 \frac{1}{1-p} = p(1-p)^4. $$
According to the model of the second statistician, the p-value is computed as follows:
$$ {6 \choose 1} p(1-p)^5 + {6 \choose 0} (1-p)^6 = (5p + 1)(1-p)^5. $$
Replacing $p$ by $1/2$, the first finds a p-value equal to $1/2^5 = 0.03125$, the second finds a p-value equal to $7/2 \times 1/2^5 = 0.109375$.
So, they get different results because they did different things, right? But according to the likelihood principle, they should come to the same conclusion. Briefly, the likelihood principle states that likelihood is all that matters for inference. So the clash here comes from the fact that both observations have the same likelihood, proportional to $p(1-p)^5$ (likelihood is determined up to a proportionality constant).
As far as I know, the answer to your second question is more of a debated opinion. I personally try to avoid performing tests and computing p-values for the reason above, and for others explained in this blog post.
EDIT: Now that I think about it, estimations of $p$ by confidence intervals would also differ. Actually if the models are different, the CI differ by construction.
Comment: (+1) This sort of discrepancies usually appear when a *stopping rule* is involved in one of the models.
Comment: I am under the impression that the likelihood principle is obviously violated in frequentist statistics (hypothesis testing, confidence intervals) because we take into consideration the probability of each possible outcome, not only the likelihood based on the actual outcome. Right ?
Comment: @Stéphane Laurent yes, that's also how I understand it. James Berger has a nice quote in [Statistical Decision Theory and Bayesian Analysis](http://books.google.es/books/about/Statistical_Decision_Theory_and_Bayesian.html?id=oY_x7dE15_AC&redir_esc=y), which says that Frequentist sometimes reject hypothesis because of data that was never observed (it sounds better, but I cannot remember it).
Comment: @Scortchi Actually I was mistaken to think that one of the P-values points to the correct likelihood function and the other not: they both point to the same likelihood function which does present the evidence relevant to the probability of heads. You should ignore the last two sentences of my previous comment. (I can't edit it, can I?)
Comment: Thanks, gui11aume. Am I right to interpret that as an example where the 'meaning' of P-values varies with the intent of the experimenter? I assume that is the case when P-values are interpreted as a sort of threshold false positive error rate because they would have to be uniformly distributed under the null hypothesis? Is that needed with Fisher's approach where P-values are presented as indices of the strength of evidence?
Comment: @Michael Lew I think that the meaning of p-values always varies with the intent of the experimenter, because you need a model and an alternative hypothesis to compute it (which are up to the experimenter). I am not sure I follow your rationale on the second part of the comment.
Comment: @gui11aume On the other hand I think it is "normal" that statistical inference should take into account the set of all possible outcomes of the experiment. At the opposite the likelihood principle claims that the design of the experiment does not matter.
Comment: @Stéphane Laurent I agree that this is a concern (I am not fully convinced by the likelihood principle myself). Notice, still, that design matters to the likelihood principle if different designs give different likelihoods. The case above is contrived.
Comment: @gui11aume The second part of my comment relates to the fact that the Neyman-Pearson error-decision theoretic framework is not the entirety of frequentist statistics. P-values come from to Fisher's significance tests which are quite distinct from the Neyman-Pearson approach. Many people know of the wars between Fisher and Neyman, but unfortunately, fewer know what they were actually fighting about. This paper might make my concern clear:
Comment: @Michael Lew thank's for the link. Indeed I did not know the matter of the fight. And by the way, the link is broken (because of the '>' character), maybe you can fix it.
Comment: @gui11aume Sorry about the link. It's too late now to fix it. Here it is without the offending character: http://www.ncbi.nlm.nih.gov/pubmed/22394284
Comment: @Procrastinator I think that you've found the nub of the problem. In the example the first statistician uses what is called 'reverse binomial sampling', but it can just as well be thought of as a sequential scheme. His P-value is 'corrected' for that sequential stopping rule and so it becomes relevant to calculation of type I errors in the N-P scheme, but loses it relationship to strength of evidence to likelihood. The second statistician calculates an uncorrected (but correct!) P-value that fits with likelihood.
Comment: @MichaelLew: But both p-values would decrease as the number of observed heads decreases, just as the likelihood of the null hypothesis decreases - why does one "fit with likelihood" & the other not?
Comment: @MichaelLew: Thanks; that was puzzling me in your paper too.
Answer: I like the example by @gui11aume (+1), but it can make an impression that the difference in two $p$-values arises only due to the different stopping rules used by the two experimenters.
In fact, I believe it is a much more general phenomenon. Consider the second experimenter in @gui11aume's answer: the one who throws a coin six times and observes heads only in the last throw. The outcomes look like that: $$\mathrm{T \;\;\; T \;\;\;T \;\;\;T \;\;\;T \;\;\;H},$$ what is the $p$-value? The usual approach would be to compute the probability that a fair coin would result in one or less heads. There are $7$ possibilities out of total $64$ with one or less heads, hence the $p=7/64\approx 0.109$.
But why not take another test statistic? For example, in this experiment we observed five tails in a row. Let's take the length of the longest sequence of tails as the test statistic. There are $3$ possibilities with five or six tails in a row, hence $p=3/64\approx0.047$.
So if in this case the error rate were fixed at $\alpha=0.05$, then the choice of the test statistic can easily render the results either significant or not, and this has nothing to do with the stopping rules per se.

Speculative part
Now, philosophically, I would say that the frequentist choice of the test statistic is in some vague sense similar to the Bayesian choice of prior. We choose one or another test statistic because we believe that the unfair coin would behave in this or that particular way (and we want to have power to detect this behaviour). Isn't it similar to putting prior on the coin types?
If so, then the likelihood principle saying that all the evidence is in the likelihood does not clash with the $p$-values, because the $p$-value is then not only the "amount of evidence". It is "a measure of surprise", but something can only be a measure of surprise if it accounts for what we would be surprised about! The $p$-value attempts to combine in one scalar quantity both the evidence and some sort of prior expectations (as represented in the choice of the test statistic). If so, then it should not be compared to the likelihood itself, but perhaps rather to the posterior?
I would be very interested to hear some opinions about this speculative part, here or in chat.

Update following discussion with @MichaelLew
I am afraid that my example above missed the point of this debate. Choosing a different test statistic leads to a change in likelihood function as well. So two different $p$-values computed above correspond to two different likelihood functions, and hence cannot be an example of a "clash" between the likelihood principle and $p$-values. The beauty of the @gui11aume's example is that the likelihood function stays exactly the same, even though the $p$-values differ.
I still have to think what this means for my "speculative" part above.
Comment: Interesting thoughts. Yes, I agree that there need be no conflict between the LP and P-values as long as the P-values are not interpreted as _evidence_ in the same way as the likelihood function is. The likelihood function contains the evidence relevant to the parameter of interest _given the statistical model_. When you change the test statistic you change the model, so the likelihood function for your alternative model will (well, may) differ from the likelihood function for the original.
Comment: Michael, I am not sure what exactly "statistical model" means, but isn't a coin with heads probability $p$ already a model? How does changing the test statistic change the model?
Comment: Apart from that, I found this question because I was re-reading your "To P or not to P" paper (and googled "likelihood principle"). I generally like the paper, but I got completely confused by the section 4.4. You write that the p-values should not be "adjusted" by taking stopping rules into account; but I don't see any adjustments in the formulas 5-6. What would "unadjusted" p-values be? Do you mean that one of them is adjusted and another one not? If so, which one, and why not vice versa?
Comment: The statistical model is often ignored or tacitly assumed to be invariant. However, for the coins it includes a fixed unknown probability of heads, a random selection of observations, and, for the head out of trials test statistic, the binomial distribution of possible outcomes. I don't know what the distribution of the outcomes is for the tails in a row test statistic but I suspect it is different. Even if it is the same, the model which has your test statistic is not the same model as the original and so the likelihood function can be different even though it contains all of the evidence.
Comment: I'm nearly finished a complete reworking of that paper. It is relevant to this discussion but not yet ready for submission. (Is this chat?)
Comment: Hmm, you are right. Changing the test statistic changes the likelihood function, so in my example using two different test statistics yields two different p-values, but also leads to two different likelihood functions... I guess this basically kills my argument. If so, differing stopping rules do  indeed seem to be crucial. Re your paper: looking forward to reading the new version, because as I said the current section 4.4 confuses me. (This is not chat, [chat is here](http://chat.stackexchange.com), and a separate room for this topic will be created if we continue in comments long enough.)
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/21242/discussion-between-michael-lew-and-amoeba).


Distortion of intervals

Question: I'm wondering if this is a common phenomenon and if so how does it actually affect music:
When I play harmonic min2nd's the actual pitches I hear are not completely the individual pitches of the two notes but something in the middle. My ear averages the notes. The top note sounds slightly flat(probably around 10-20 cents) and the bottom note slightly sharp(probably less so). Of course, I do not hear the two pitches as if they were played melodically, there is inter modulation distortion going on which seems to act like the average of the two but I can force myself to hear the other pitches, but when I do so I always hear the top one a little flat (say I release the bottom note).
If this is a common phenomenon of the ears, which I believe it is although some individuals may be better trained or have better ears to distinguish min2nds and, of course, context matters (I'm assuming a synthetic piano), then how does this affect the harmonic aspects of theory? If one plays a closed voicing Cmaj7/B the C and B notes then, by this phenomenon, would produce sort of C|B(notation for note in between C and B) and this would throw off the entire harmonic expression of a Maj7 chord, out of context, at least, it would probably make the other notes sound sharp. Also, then it would affect voice leading and all that.
Maybe such things do actually go on in music and theory has not advanced enough to take them in to account? That is, in some sense we are actually dealing with something like 24-tet and harmonic m2nd's would be accessing notes in the extend divisions. It may not be 24-tet but whatever. The idea, though, is that it could dictate voice leading and resolutions. E.g., a closed Cmaj7/B would sound quite dissonant and suggest resolving probably to a G/B while a closed Cmaj7 would not have such a strong dissonance. Of course, then one would have to take into account the overtone series and compare any harmonic m2nd's as they too could inflect resolutions.
Trills seem to take advantage of this, typically trills on a m2nd are going to slightly alter the perceived pitch slightly below the principal tone (the subordinate tone pulls down the principal tone's pitch slightly allowing it to act sort of as a leading tone).
I've only heard this phenomenon talked about in one context in a book on blues where it suggested that this phenomenon existed and it was blues players trying to get at tones that 12-tet could not use. Guitar players would bend the notes microtonally to express them.
I'm not all that familiar with high note divisions of the octave which could possibly shine light on this besides being able to bend/slide notes on some stringed instruments. That is, has anyone explored higher order harmonic theory that can understand this phenomenon?
Comment: How do you recognize the perceived change of pitch? Do dyads sound out of tune to you, while single notes don't? Or do you hear melodic-like motion of a tone when you add a second tone to it? Also, could it be that what you observe are Combination Tones (https://en.wikipedia.org/wiki/Combination_tone)?
Answer: Two very nearby pitches combined can indeed result in a perceived single "average pitch", albeit with different tone character from the individual notes.
However, this generally requires the pitches to be really nearby, like ⅓ of a semitone. For a minor second, the tones normally won't blend together in this way but always be noticeable as a distinct dissonances – though I suppose it varies between listeners.
Regardless, minor seconds are such a strong, concentrated dissonance that they're avoided in most situations. maj7 chords are almost always voiced so as to not have a root note right above the j7 one. If the j7 note is in the bass, then it's actually hardly perceived as a maj7 chord at all, but rather as a sus4♭6♭9 kind of situation.
Trills are a bit of a different pair of shoes. In particular when playing fast trill on a string or wind instrument, then the waveform just continues through the trill. In the extreme case – trill on bass – this means the result is indeed almost indistinguishable from a single vibrato note of a pitch in the middle. Even in higher registers and ensemble playing, where a trill definitely does result in both individual pitches sounding seperately but simultaneously, the choppy nature means the effect is different from two instruments each playing a sustained note a semitone apart. This is a special case of the general phenomenon that for sufficiently short notes the pitch becomes more indistinct. There is in fact a fundemental physics reason for this (https://en.wikipedia.org/wiki/Uncertainty_principle#Harmonic_analysis), combined with human perception in particular.
Answer: The mathematical relation
sin(a + b) + sin(a - b) = 2sin(a)cos(b)
shows that the sum of two sound waves with different pitch is also the product of the average pitch with a changing volume.
This is more than just a perception; it is a physical fact. The same combination of sounds can be expressed in two different ways. It can also be measured and perceived in two different ways.
We are more likely to perceive the average when the pitches are close together. The volume changes are then known as 'beats', and we can tune instruments by listening to the beats and adjusting until they slow enough to be undetectable. When the pitches are far apart we are more likely to perceive the two separate parts of the sum.
Your ability to perceive the same sound combination as two different things reminds me of the pictures which can be seen  in two different ways, even though it is still the same picture.
Most variations in perceived pitch arising from related effects are usually taken as part of what a particular chord sounds like, without specifically trying to use the effect to get a quarter- tone sound. I am not a blues player, and it is interesting that blues players may be using the effect in this less usual way.

Distortion of intervals


Como converter array de strings para string?

Question: Em um programa para o Arduino, eu tenho um array de strings mas parece que a função onde vou usar esse dado só aceita string. Como faço para converter um array em uma string ?
File prog = SD.open("prog.bin");
String leiaS(void){

    return String(prog.read());

 }

   digitalWrite(leia(), inverte(leiaS()));

String inverte(String port){
     if(digitalRead(port)==HIGH)
      return LOW;
     else
      return HIGH;
  }

Erro:

could not convert 'leiaS' from 'String (*)()' to 'String'

Percebi que estava faltando um parênteses na chamada da função leiaS. Depois de corrigir e verificar apareceu o erro:

cannot convert 'String' to 'uint8_t {aka unsigned char}' for argument '2' to 'void digitalWrite(uint8_t, uint8_t)'
Comment: Pelo que pesquisei, não existe função pronta. Eu iria de `loop` dentro do array e concatenaria uma string mesmo.
Comment: Sim, mas para isso acredito que precisaria saber o tamanho do array. Sei que tem 1 byte, pois a função read() lê 1 byte de cada vez. O problema é quanto vai ser isso em strings. Segundo a documentação a função sizeof() retorna o numero de bytes. Não sei se serve. Se pudesse exemplificar eu agradeço.
Comment: Carlos, vc está usando `String` onde se espera numerais, me parece que vc precisa reescrever todo o seu código para se ter o resultado que deseja.
Answer: Você pode fazer uma iteração e ir lendo cada caracter que o read retorna e ir adicionando num array de char.
char arrayLeitura[20];
int index=0;
char charLeitura;

File prog = SD.open("prog.bin");
if (prog) {
    while (prog.available()) {
         charLeitura= prog.read();
         arrayLeitura[index++]=charLeitura;
    }
    prog.close();
}

Acredito que depois tu consegue dar um
return String(arrayLeitura);

Comment: O problema é que ai você está pegando mais de um byte. O byte na verdade é um numero. Só que parece que a função digitalWrite() só aceita string. E quando eu tento pegar um char e converter para string não funciona.
Comment: Dentro do `while` eu pego o byte e converto pra char, depois no final tu você pode fazer o meu return que vai retornar uma `String` dos array de `char`, você testou?
Comment: Tentei dessa forma `String leiaS(void){
    char bt = prog.read();
    char cod[1];
    cod[0] = bt;
    return String(cod);

  }`., que basicamente faz o mesmo, já que quero ler só um byte. Resulta no erro "could not convert 'leiaS' from 'String (*)()' to 'String'"
Answer: leiaS esta definido como funcao (de tipo String (*)(void))
String leiaS(void) { /* ... */ }

A referencia a leiaS na linha do digitalWrite
digitalWrite(leia(), inverte(leiaS)));

é como parametro para a funcao inverte.
Suponho que a funcao inverte esta definida como recebendo uma string: nao podes passar um ponteiro para uma funcao.
Comment: Deve ser isso. Tinha esquecido de colocar a função `inverte`. Atualizei.
Comment: Agora percebi que estava faltando um parenteses na função. Depois de corrigir, da o erro "
cannot convert 'String' to 'uint8_t {aka unsigned char}' for argument '2' to 'void digitalWrite(uint8_t, uint8_t)'"
Comment: agora tens o segundo parametro para funcao `digitalWrite` com tipo errado. A funcao esta definida para obter um valor de tipo `uint8_t` mas estas a passar um valor de tipo `String`. Este tipos nao sao compativeis, por isso o compilador nao pode fazer a conversao dum para o outro e da erro.
Comment: Troquei os retornos por `unsigned char` e resolveu


How to monitor number of connections to a remote computer for ASP.NET application?

Question: I am trying to solve non deterministic hangouts in AST.NET application. I have a sneaking suspicion that maxconnection limit is reached.
So my question is how can I verify my suspicion? Is there any performance counter or any method like ThreadPool.GetAvailableThreads to monitor number of simultaneous connections to a remote computer?
Also what does the "number of simultaneous connections" means? As I understand it's count per application domain. However doest it counts socket connection opened in .Net calls?
Answer: You can use Fiddler for monitoring http connections or TCPView For monitoring all the connection of the computer.
I am afraid there is no function for counting available connections, because it does depend on whole network infrastructure. You probably does not run out of connections on your computer, but router somewhere in the network will prevent establishing new connections.
Comment: I don't think fiddler is a option. First it only works for https(s) and second it's designed to be used as a proxy between browser and internet. I've never heard about usage for asp.net application although setting it as a proxy for web service calls is probably possible.
Comment: I assumed that your ASP.NET application uses http(s) connection. We use fiddler to debug CALDAV connections from our C# application, so it works even outside the browser (for applications reading proxy settings from windows).
Comment: If you want monitor all the connections, there is an application called TCPView by Sysinternals: http://technet.microsoft.com/en-us/sysinternals/bb897437
Comment: If a router had blocked the connection, you'd not be seeing an ASP.NET hang - you'd get a network error.  @Jakub: most web services run on HTTP; Fiddler will trap any outgoing Http traffic whether to internet or local network.
Comment: @Binus @Dan the hang is caused because .Net use internal connection pooling mechanism and the max pool size is defined in [maxConnection]. For more information about this topic see [this KB](http://support.microsoft.com/kb/821268)
Comment: @Binus @Dan i suppose that most of the connection are call to the internal service which doesn't comunicate via http. For our scenario TCPView works better. However I am no sure if all connection showen by TCPView are counted in maxconnection limit and vice versa.


Is there a way to figure out the right dimensions for images to be used on Adaptive cards for a MS teams app?

Question: We are trying to create an bot with a use of Adaptive cards on MS-Teams. The preview using adaptive card designer gives the proper image as required. But when used in the Teams app, the image looks totally distorted.
The preview image as in designer:

The image output as in the MS teams app:

Below is the Json used for the adaptive cards:
<code>{
"type": "AdaptiveCard",
"body": [
    {
        "type": "TextBlock",
        "text": "Howdy!",
        "weight": "Bolder",
        "color": "Accent",
        "size": "Large"
    },
    {
        "type": "TextBlock",
        "text": "How are you doing today? ",
        "wrap": true
    },
    {
        "type": "ColumnSet",
        "columns": [
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/2-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "awful",
                                "comment": ""
                            },
                            "title": "awful"
                        },
                        "size": "Large",
                        "width": "700px",
                        "height": "60px"
                    }
                ]
            },
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/3-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "bad",
                                "comment": ""
                            },
                            "title": "bad"
                        },
                        "size": "Large",
                        "width": "700px",
                        "height": "60px",
                        "spacing": "None"
                    }
                ]
            },
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/1-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "ok",
                                "comment": ""
                            },
                            "title": "ok"
                        },
                        "size": "Large",
                        "width": "700px",
                        "height": "60px"
                    }
                ]
            },
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/5-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "good",
                                "comment": ""
                            },
                            "title": "good"
                        },
                        "size": "Large",
                        "width": "700px",
                        "height": "60px"
                    }
                ]
            },
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/4-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "terrific",
                                "comment": ""
                            },
                            "title": "terrific"
                        },
                        "size": "Large",
                        "width": "700px",
                        "height": "60px"
                    }
                ]
            }
        ]
    },
    {
        "type": "Input.Text",
        "placeholder": "Add a comment",
        "isMultiline": true,
        "id": "comment"
    }
],
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.3",
"actions": [
    {
        "type": "Action.Submit",
        "title": "OK",
        "data": {
            "rating": "",
            "comment": ""
        }
    }
]}
</code>
Thanks in advance!!
Comment: We have raised a bug for the same.
Comment: I see '"width": "700px"' for -all- of the items - that doesn't look right based on the screenshot. Looks like all -together- would be 700px
Comment: We are able to repro this, we are checking this internally.
Comment: @Meghana-MSFT - Thank you . Hopefully a fix is provided . Right now we have managed to place the images in vertical order to make it look good.
Comment: This issue has been fixed, We have checked and its working fine. Attaching a piece of code for emoji height and width.       {"type":"Image","url":"https://howdy1.blob.core.windows.net/sample/2-removebg-preview.png","selectAction":{"type":"Action.Submit","data":{"rating":"awful","comment":""},"title":"awful"},"size":"Large","width":"70px","height":"60px"}
Answer: Moving comment to answer :
This issue has been fixed, We have checked and its working fine. Attaching a piece of code for emoji height and width. Please keep the width inline with height.
<code>{
"type": "AdaptiveCard",
"body": [
    {
        "type": "ColumnSet",
        "columns": [
            {
                "type": "Column",
                "width": "stretch",
                "items": [
                    {
                        "type": "Image",
                        "url": "https://howdy1.blob.core.windows.net/sample/2-removebg-preview.png",
                        "selectAction": {
                            "type": "Action.Submit",
                            "data": {
                                "rating": "awful",
                                "comment": ""
                            },
                            "title": "awful"
                        },
                        "size": "Large",
                        "width": "70px",
                        "height": "60px"
                    }
                ]
            }

        ]
    }
],
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.3",
}
</code>


python Pool with worker Processes

Question: I am trying to use a worker Pool in python using Process objects.  Each worker (a Process) does some initialization (takes a non-trivial amount of time), gets passed a series of jobs (ideally using map()), and returns something.  No communication is necessary beyond that.  However, I can't seem to figure out how to use map() to use my worker's compute() function.
from multiprocessing import Pool, Process

class Worker(Process):
    def __init__(self):
        print 'Worker started'
        # do some initialization here
        super(Worker, self).__init__()

    def compute(self, data):
        print 'Computing things!'
        return data * data

if __name__ == '__main__':
    # This works fine
    worker = Worker()
    print worker.compute(3)

    # workers get initialized fine
    pool = Pool(processes = 4,
                initializer = Worker)
    data = range(10)
    # How to use my worker pool?
    result = pool.map(compute, data)

Is a job queue the way to go instead, or can I use map()?
Comment: Fair enough, thanks.  The initialization takes a long time, so I only want to do it once per worker process.
Comment: All process objects are stateful.  You might want to remove that word from the title.  Also.  `compute` is a method of a Worker.  In the examples it's usually a completely stand-alone function. Why not write the compute function to simply include both initialization and processing?
Comment: You must want to emphasize the "gets passed a series of jobs" part of the question.  Since that wasn't obvious.
Answer: I would suggest that you use a Queue for this.
class Worker(Process):
    def __init__(self, queue):
        super(Worker, self).__init__()
        self.queue = queue

    def run(self):
        print('Worker started')
        # do some initialization here

        print('Computing things!')
        for data in iter(self.queue.get, None):
            # Use data

Now you can start a pile of these, all getting work from a single queue
request_queue = Queue()
for i in range(4):
    Worker(request_queue).start()
for data in the_real_source:
    request_queue.put(data)
# Sentinel objects to allow clean shutdown: 1 per worker.
for i in range(4):
    request_queue.put(None)

That kind of thing should allow you to amortize the expensive startup cost across multiple workers.
Comment: That's what I figured, thanks!  I ended up using a job queue (input) and result queue (output) to synchronize everything.
Comment: you example is awesome, i try right now how to input the sentinel objects when strg + c is pressed without an exepction
Comment: @S.Lott: Isn't it that Queue isn't pickle-able? and that's why you use [multiprocessing.Manager().Queue](http://stackoverflow.com/questions/3217002/how-do-you-pass-a-queue-reference-to-a-function-managed-by-pool-map-async)?
Comment: It is really *much more customizable* than the default `multiprocessing.Pool()`!!!
Comment: This way will create 4 separate processes. If you want a pool of workers to run in a single process, use Pool https://docs.python.org/2/library/multiprocessing.html#using-a-pool-of-workers
Answer: initializer expects an arbitrary callable that does initilization e.g., it can set some globals, not a Process subclass; map accepts an arbitrary iterable:
#!/usr/bin/env python
import multiprocessing as mp

def init(val):
    print('do some initialization here')

def compute(data):
    print('Computing things!')
    return data * data

def produce_data():
    yield -100
    for i in range(10):
        yield i
    yield 100

if __name__=="__main__":
  p = mp.Pool(initializer=init, initargs=('arg',))
  print(p.map(compute, produce_data()))

Answer: Since python 3.3 you can use starmap, also for using multiple arguments AND getting back the results in a very simplistic syntax:
import multiprocessing

nb_cores = multiprocessing.cpu_count()

def caps(nb, letter):
    print('Exec nb:', nb)
    return letter.upper()

if __name__ == '__main__':

    multiprocessing.freeze_support() # for Windows, also requires to be in the statement: if __name__ == '__main__'

    input_data = ['a','b','c','d','e','f','g','h']
    input_order = [1,2,3,4,5,6,7,8,9]

    with multiprocessing.Pool(processes=nb_cores) as pool: # auto closing workers
        results = pool.starmap(caps, zip(input_order, input_data))

    print(results)

python Pool with worker Processes


disabling few buttons from group of buttons React

Question: I have an array of data
<code>const projectTypeValues = [
{ name: 'Hour', value: 'hour'},
{ name: 'Day', value: 'day'},
{ name: 'Session', value: 'session'},
{ name: 'project', value: 'project'}]
</code>
From there I am showing the buttons in a material ui buttonGroup through an loop.
<code>{projectTypeValues.map(element=>
          <Button variant="contained" color="primary">{element.name}</Button>
        )}
</code>
how do i make a couple or few buttons from this loop disabled while others are active?
I want to show all the 4buttons but other 3 buttons will be disabled and only session will be active
Thanks for your help
Comment: you can keep another property in `projectTypeValues`, let say `disabled`. then you can set it direclty to the button. `{element.name}`
Comment: @AmilaSenadheera that would be so great, but in this scenario I cant do that, is there any way doing it with indexing or something?
Comment: can you explain which buttons you want to display and the condition for that please?
Comment: @AmilaSenadheera I want to show all the 4buttons but other 3 buttons will be disabled and only session will be active
Answer: Try like this
<code>{
    projectTypeValues.map(element => (
        <Button
            variant="contained"
            color="primary"
            disabled={element.value !== "session"}
        >
            {element.name}
        </Button>
    ));
}
</code>
Comment: @Zeed Tanue, check this out!!


How to add a link to slide elements in MD Slider?

Question: I have used the MD Slider module (which is Mega Slider, a paid module) in my websites to create slideshows.
But there is no default option to create a link for each slide.
Is there any way to hyperlink a slide?
Comment: I couldn't find any `MD Slider` module... are you referring to Mega Slider?
Comment: @NoSssweat - Thanks for the leads. I could not wait longer enough and created my own patch for the same: http://megadrupal.com/comment/13066#comment-13066
Comment: Yes, MD Slider is Mega Slider, its a paid module.
Comment: Apparently there are work arounds. [see this](http://megadrupal.com/comment/671) which is ridiculous for a paid module.
Answer: I have created a patch with the changes to convert a slide into hyperlink. Since I cannot upload a file here, I am copy-pasting the contents of my .patch file:
From 3f72f263b3673ce27ca4114963b007752b566c2a Mon Sep 17 00:00:00 2001
From: Raj Pawan Gumdal <email@example.com>
Date: Sat, 6 Aug 2016 11:02:04 +0530
Subject: [PATCH 1/2] Ported changes regarding MD_Slider done for an older
 version

---
 .../templates/front/front_layers_render.tpl.php    | 13 ++++----
 .../templates/front/front_slide_render.tpl.php     | 35 ++++++++++++++++++++--
 2 files changed, 41 insertions(+), 7 deletions(-)

diff --git a/sites/all/modules/md_slider/templates/front/front_layers_render.tpl.php b/sites/all/modules/md_slider/templates/front/front_layers_render.tpl.php
index a8dd7fc..16c30b4 100644
--- a/sites/all/modules/md_slider/templates/front/front_layers_render.tpl.php
+++ b/sites/all/modules/md_slider/templates/front/front_layers_render.tpl.php
@@ -11,11 +11,14 @@
 <?php endif;?>
   <div class="<?php print $class; ?>" <?php print $data; ?>>
     <?php if ($layer->type == 'text'): ?>
-        <?php if (isset($link)):?>
-            <a href="<?php print $link;?>"<?php if (isset($layer->link["target"]) && !empty($layer->link["target"])) print " target='{$layer->link["target"]}'";?>><?php print $layer->title;?>
-        <?php else:?>
-            <?php print $layer->title;?>
-        <?php endif; ?>
+    <?php /* Raj: Changes to make MD_Slider slide as hyperlink */?>
+       <?php if (strpos($layer->title,'URL:') === false):?>
+           <?php if (isset($link)):?>
+               <a href="<?php print $link;?>"<?php if (isset($layer->link["target"]) && !empty($layer->link["target"])) print " target='{$layer->link["target"]}'";?>><?php print $layer->title;?>
+           <?php else:?>
+               <?php print $layer->title;?>
+           <?php endif; ?>
+        <?php endif; ?>
     <?php elseif ($layer->type == 'image'): ?>
         <?php if (isset($link)):?>
             <a href="<?php print $link;?>"<?php if (isset($layer->link["target"]) && !empty($layer->link["target"])) print " target='{$layer->link["target"]}'";?>>
diff --git a/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php b/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
index 8df2c39..5dd71dd 100644
--- a/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
+++ b/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
@@ -18,10 +18,41 @@
       <div class="md-slider-overlay" style="background: <?php print $slide->settings['background_overlay']; ?>">

     <?php endif; ?>
-

+
+    <?php
+    // Raj: Check if there is a string which starts from "URL:". If so, extract the URL and make this div a hyperlink
+    $hasURL = false;
+    $URLString = '';
+    foreach ($slide->layers as $index => $layer):
+    $layerTitle = $layer["title"];
+    drupal_set_message("Layer object: $layer");
+    drupal_set_message("Layer title: $layerTitle");
+    // http://stackoverflow.com/a/4366748/260665
+    if (strpos($layerTitle,'URL:') !== false) {
+       $hasURL = true;
+       $URLString = str_replace('URL:', '', $layerTitle);
+       break;
+    }
+    endforeach;
+
+    if ($hasURL)
+    {
+       print "";
+    }
+    print "
";
+    foreach ($slide->layers as $index => $layer):
+    print theme('front_layers_render', array('index' => $index, 'layer' => $layer));
+    endforeach;
+    print "
";
+    if ($hasURL)
+    {
+       print "";
+    }
+    ?>
+
       <?php foreach ($slide->layers as $index => $layer):?>
         <?php print theme('front_layers_render', array('index' => $index, 'layer' => $layer));?>
       <?php endforeach;?>
-

+

 <?php endif;?>
--
2.7.4 (Apple Git-66)

From 74c3d0644fd505729163d4803c9a0f9cb014e679 Mon Sep 17 00:00:00 2001
From: Raj Pawan Gumdal <firstname.lastname@example.com>
Date: Sat, 6 Aug 2016 14:04:39 +0530
Subject: [PATCH 2/2] Finalized the changes to support hyperlinking whole slide

---
 .../templates/front/front_slide_render.tpl.php     | 30 +++++++++++++---------
 sites/all/themes/IndianSnakes/css/custom.css       |  6 +++++
 2 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php b/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
index 5dd71dd..764d46e 100644
--- a/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
+++ b/sites/all/modules/md_slider/templates/front/front_slide_render.tpl.php
@@ -9,6 +9,7 @@

 <?php if (!isset($slide->settings["disabled"]) || !$slide->settings["disabled"]):?>
   <div class="md-slide-item slide-<?php print $index+1;?>" <?php print $data_properties;?>>
+
     <div class="md-mainimg" style="<?php print $background_style;?>">
       <?php if (!empty($background_url)):?>
         <img src="<?php print $background_url;?>" alt="<?php print htmlentities($slide->settings["background_image_alt"], ENT_QUOTES, 'UTF-8');?>"<?php print $background_style;?> />
@@ -25,34 +26,39 @@
     $URLString = '';
     foreach ($slide->layers as $index => $layer):
     $layerTitle = $layer["title"];
-    drupal_set_message("Layer object: $layer");
-    drupal_set_message("Layer title: $layerTitle");
+//     drupal_set_message("Layer object: $layer");
+//     drupal_set_message("Layer title: $layerTitle");
     // http://stackoverflow.com/a/4366748/260665
-    if (strpos($layerTitle,'URL:') !== false) {
+    if (strpos($layerTitle,'URL:') === false) {
+    }
+    else {
        $hasURL = true;
-       $URLString = str_replace('URL:', '', $layerTitle);
+       $URLString = str_replace('URL:','',$layerTitle);
        break;
     }
     endforeach;
-
+//     drupal_set_message("URL: $URLString");
+//     drupal_set_message("Has URL: $hasURL");
+    $outputString = "";
     if ($hasURL)
     {
-       print "";
+       $outputString .= "";
     }
-    print "
";
+    $outputString .= "
";
     foreach ($slide->layers as $index => $layer):
-    print theme('front_layers_render', array('index' => $index, 'layer' => $layer));
+    $outputString .= theme('front_layers_render', array('index' => $index, 'layer' => $layer));
     endforeach;
-    print "
";
+    $outputString .= "
";
     if ($hasURL)
     {
-       print "";
+       $outputString .= "";
     }
+    print $outputString;
     ?>
-
+
+
 -->

 <?php endif;?>
diff --git a/sites/all/themes/IndianSnakes/css/custom.css b/sites/all/themes/IndianSnakes/css/custom.css
index 3d69b0d..3f0354e 100644
--- a/sites/all/themes/IndianSnakes/css/custom.css
+++ b/sites/all/themes/IndianSnakes/css/custom.css
@@ -27,3 +27,9 @@ display: inline-block;
 width: auto !important;
 height: 300px !important;
 }
+
+/* Reset the hyperlink style to normal inside slides */
+.md-slide-item a {
+    color: inherit;
+    text-decoration: inherit;
+}
\ No newline at end of file
--
2.7.4 (Apple Git-66)

I have defined a new "URL:" label protocol for each slide. If a slide has this label defined then the whole slide's div is converted into hyperlink with whatever link is provided after my URL: protocol. For example check this:

The code (patch) is configured such that label with text which begins with URL: is never rendered as text but it is used to convert the slide into a hyperlink.
For a live working example check: http://indiansnakes.org/homepage
Assumption:
The only limitation of this method is, you should not hyperlink any text or images which are used within this hyperlinked slide coz the anchor tags cannot be nested, apparently.
Edit: I thought its best to update the patch details in the MD Slider community as well, here it is: http://megadrupal.com/comment/13066#comment-13066
Also a gist is created with the contents of same patch: https://gist.github.com/gumdal/e580cd80199c871fa52f677538771551


Are there primes arbitrarily close to powers?

Question: For $n \geq 3$ let $r(n)$ be the previous prime to $n$; i.e., the largest prime strictly less than $n$. For example, $r(3) = 2$, $r(10) = 7$, and so on.
I have noticed that $r(n^p)$ is very close to $n^p$. In fact, I suspect that $$\lim_{n\to\infty}\frac{r(n^p)}{n^p} = 1$$ for any positive integer power $p$, where the convergence is faster if $p$ is big. Is this true? Are there effective bounds on the rate of convergence? Is there a simple proof of this fact?
Here's an argument that doesn't work: By Bertrand's postulate, there exists a prime between $n^p / 2$ and $n^p$ (roughly). Therefore $$\frac{r(n^p)}{n^p} \geq \frac{1}{2}.$$ But this is pretty far from $1$.
I think that I can prove this using some fancy number theory results, but they seem like sledgehammers. I'd like something simpler.
Comment: The fact that the limit approaches $1$ implies that the difference between a large prime and a large power can be a negligibly small fraction of that large power, but NOT that the difference can be made arbitrarily small, such as $<50$. Also, I don't see what is specially interesting about powers. Refinements to Bertrand's postulate suggest that $\lim_{n\to\infty}\frac{r(n)}{n} = 1$
Comment: so mersenne numbers and fermat numbers aren't in the picture.
Comment: @RoddyMacPhee - why are Mersenne-numbers/-primes out of the picture?
Comment: @Gottfried , because  them being infinite in the primes, would show there are infinitely many 1 away.
Comment: @RoddyMacPhee - yes, but why then *out of the picture* instead of *in the picture*. Mersenne-primes are perfect exemplars for what the OP is asking for, or not?
Comment: not all prime exponents will be close @Gottfried. Also n is not always 2. And they count as fancy in my books.
Comment: @RoddyMacPhee - ah, I overlooked the introductory condition $n \ge 3$ - this of course excludes Mersenne-primes. But what about $3^a-2$? $3^a-1$ might have many small primes as factors, so $3^a-2$ cannot have those factors and might more likely have few primefactors or may even be prime than the "common Joe". -------  But well, that all was just a short idea, it's not my current field of thinking and I better step out...
Comment: $a\equiv 3\bmod 4$ Always gives a factor of 5 for that @Gottfried . Don't feel bad I missed $p$ not being the previous prime somehow.
Answer: Notice that $r(m)=p_{\pi(m-1)}$ where $p_m$ is the $m$th prime number. (Since $m=n^p$ won't be prime in our case we will always have: $\pi(m)=\pi(m-1)$ and we can write everything more conveniently as $r(m)=p_{\pi(m)}$.)
Look at the number of primes less than $n^p$: that would be $\pi(n^p)\approx\frac{n^p}{\log(n^p)}$. Now you are looking for approximately the $\pi(n^p)$th prime number and $p_m\approx\,m\log(m)$, thus $p_{\pi(n^p)}\approx\,n^p-n^p\frac{\log(\log(n^p))}{\log(n^p)}$. Now your function is essentially $p_{\pi(n^p)}/n^p\approx{}1-\frac{\log(\log(n^p))}{\log(n^p)}$ which converges to $1$.
Comment: Technically, $r(m)=p_{\pi(m-1)}$.
Comment: Why isn't this a rigorous argument already? It seems pretty plausible to me! (I'm not immediately sure how to make the convergence *effective* from this, though.)
Comment: The limit being equal to 1 is immediate from the $\approx$ notation which implies that the two sides belong in the same asymptotic class as $n\to\infty$. The only issue with the proof is that it doesn't use the form $n^p$ at all! It simply holds for $n$ and $r(n^p)/n^p$ obviously converges too as a subsequence.
Comment: @J.G. Indeed, hopefully the asymptotic notation and the limits at the end save the day...
Answer: Though Μάρκος posted a great answer, for completeness I'll post the complicated proof that I know.
It suffices to show that there exists a prime between $x$ and $(1 + b(x))x$ for sufficiently large $x$, where $b(x) \to 0$ as $x \to \infty$. (Then $r(n^p) / n^p \geq \frac{1}{1 + b(n^p)}$, and letting $n \to \infty$ yields the result.) By some complicated number theory, we can take $b(x) = 1 + 1 / (2 \log^2 x)$ for $x \geq 3275$. This gives $$\frac{r(n^p)}{n^p} \geq \frac{1}{1 + \frac{1}{2 \log^2 n^p}}$$ for $n$ sufficiently large. This says that the limit is $1$, and that it approaches $1$ quicker if $p$ is large.
Comment: It's worth noting that your sufficient condition can be proved from the prime number theorem rather than Pierre Dusart's result - I added an update to my answer to cover this.
Comment: Brutal. Correct, but brutal.
Answer: To get the limit you only need the prime number theorem
$$\lim_{n\rightarrow \infty}\pi(n)/(n/\ln n)=1 \tag 1$$
but the deduction is a bit messy.
From (1) you can deduce that for any $\epsilon^{'}>0$ $$\lim_{n\rightarrow \infty}\pi(n(1+\epsilon^{'}))/(n(1+\epsilon^{'})/\ln n(1+\epsilon^{'}))=1$$ Now $\lim_{n\rightarrow \infty}\ln n(1+\epsilon^{'})/\ln n=1$ so $$\lim_{n\rightarrow \infty}\pi(n(1+\epsilon^{'}))/(n(1+\epsilon^{'})/\ln n)=1$$
This means that for any $\epsilon>0$ $\exists N \in \mathbb{N}$ s.t. $\forall n>N$  the following two inequalities hold:
\begin{align}
(1-\epsilon)(n/\ln(n))&<\pi(n)& &<(1+\epsilon)(n/\ln(n)) \\
(1-\epsilon)(1+\epsilon^{'})(n/\ln(n))&<\pi(n(1+\epsilon^{'}))& &<(1+\epsilon)(1+\epsilon^{'})(n/\ln(n))
\end{align}
Subtracting the two inequalities we obtain after some simplification:
$$\{\epsilon^{'}-\epsilon(\epsilon^{'}+2)\}<\frac{\pi(n(1+\epsilon^{'}))-\pi(n)}{(n/\ln(n))}<\{\epsilon^{'}+\epsilon(\epsilon^{'}+2)\}$$
This clearly implies that $$\lim_{n\rightarrow \infty}\frac{\pi(n(1+\epsilon^{'}))-\pi(n)}{(n/\ln(n))}=\epsilon^{'}$$ and in particular that $$\lim_{n\rightarrow \infty}\pi(n(1+\epsilon^{'}))-\pi(n)=\infty$$
This implies that for any $\epsilon^{'}>0$ $\pi(n(1+\epsilon^{'}))-\pi(n)>1$ for large enough $n$ and hence that there exists a prime $p$ with $n<p<n(1+\epsilon^{'}).$
Clearly then we have for $\epsilon^{'}>0$ and all large enough n $$n<r(n)<n(1+\epsilon^{'})$$ and thus also for any fixed $p\in \mathbb{N}$ $$1-\epsilon^{'}<\frac{1}{1+\epsilon^{'}}<n^p/r(n^p)<1$$.
which implies that $$\lim_{n\rightarrow \infty}n^p/r(n^p)=1$$
Update to show that the rwbogl's sufficient condition can be proved from the prime number theorem rather than Pierre Dusart's result.
From the PNT I showed that for any $\epsilon>0$ there exists a prime $p$ with $n<p<n(1+\epsilon)$ for some $n=N(\epsilon)$ where $N$ is strictly increasing as $\epsilon$ decreases to $0$. If we set $g(n)=N(1/n)$ where $n\in\mathbb{N}$ then $g\in\mathbb{N}$ increases monotonically and hence possesses a monotonically increasing pseudo-inverse $g^{-1}:\mathbb{N} \rightarrow \mathbb{N}$ s.t. $g(g^{-1}(x))=x$ for all $x\in\mathbb{N}.$
Substituting we have for any $n\in\mathbb{N}$ there exists a prime $p$ with $g(n)<p<g(n)(1+1/n)$. Setting $n=g^{-1}(x)$, $x\in\mathbb{Z}$ we have for any $x\in\mathbb{N}$ there exists a prime $p$ with $$x<p<x(1+1/g^{-1}(x))=x(1+f(x))$$ where $f(x)=1/g^{-1}(x)$ which decreases monotonically to 0 as $x\rightarrow \infty$.


Twitter REST API oauth '215 Bad Authentication data' using fetch in react-native

Question: I'm trying to build a react native app to show a user's twitter timeline, but can't manage to access the twitter REST api because I am getting a

215 Bad Authentication data

error.
I've signed in the user correctly, gotten all the access tokens, and using fetch to make a request.
I've also verified that all my keys and tokens are correct, yet I still can't manage to figure out why I'm getting this error, my code is available below;
Can anyone please advise me on how to debug this or tell me what's wrong with my code?
thanks.
code:
<code>let header = this._buildRequestHeader(twitter_token, twitter_tokenSecret);
    console.log(header);
     fetch('https://api.twitter.com/1.1/statuses/home_timeline.json', {
       method: 'GET',
       headers: {
         'Accept': '*/*',
         'Content-Type': 'application/x-www-form-urlencoded',
         'Authorization': ' '+header
       }
     }).then((response) => response.json())
        .then((json) => {
          console.log(json);
        })
</code>
_getBaseString:
<code>_getBaseString(method, url, parameter_string){
return method+'&'+
       encodeURIComponent(url)+'&'+encodeURIComponent(parameter_string);
</code>
}
_getSignature:
<code>_getSignature(user_auth_token, accesstoken_secret, data){
// let signing_key = encodeURIComponent(Constants.TWITTER_CONSUMER_SECRET)+'&'+
//                   encodeURIComponent(Constants.ACCESS_TOKEN_SECRET);
let signing_key = encodeURIComponent(Constants.TWITTER_CONSUMER_SECRET)+'&'+
                  encodeURIComponent(accesstoken_secret);
console.log('signing data');
console.log(data);
return this.b64EncodeUnicode(hmacsha1(signing_key, data));
</code>
}
_buildRequestHeader:
<code>_buildRequestHeader(user_auth_token, accesstoken_secret){
// https://dev.twitter.com/oauth/overview/creating-signatures
// https://dev.twitter.com/oauth/overview/authorizing-requests
let include_entities_key = encodeURIComponent('include_entities');
let include_entities_val = encodeURIComponent('false');

let oauth_consumer_key_key = encodeURIComponent('oauth_consumer_key');
let oauth_consumer_key_val = encodeURIComponent(Constants.TWITTER_COMSUMER_KEY);

let oauth_nonce_key = encodeURIComponent('oauth_nonce');
let oauth_nonce_val = encodeURIComponent(this._getNonce());

let oauth_signature_method_key = encodeURIComponent('oauth_signature_method');
let oauth_signature_method_val = encodeURIComponent('HMAC-SHA1');

let oauth_timestamp_key = encodeURIComponent('oauth_timestamp');
var val = Date.now() / 1000;
console.log(val);
console.log('parse'+parseInt(val));
let oauth_timestamp_val = encodeURIComponent(parseInt(val));

let oauth_token_key = encodeURIComponent('oauth_token');
// let oauth_token_val = encodeURIComponent(Constants.ACCESS_TOKEN);
let oauth_token_val = encodeURIComponent(user_auth_token);

let oauth_version_key = encodeURIComponent('oauth_version');
let oauth_version_val = encodeURIComponent('1.0');

// let parameter_string = include_entities_key+'='+include_entities_val+'&'+
let parameter_string =  oauth_consumer_key_key+'='+oauth_consumer_key_val+'&'+
                    oauth_nonce_key+'='+oauth_nonce_val+'&'+
                    oauth_signature_method_key+'='+oauth_signature_method_val+'&'+
                    oauth_timestamp_key+'='+oauth_timestamp_val+'&'+
                    oauth_token_key+'='+oauth_token_val+'&'+
                    oauth_version_key+'='+oauth_version_val;

let data = this._getBaseString('GET', 'https://api.twitter.com/1.1/statuses/home_timeline.json',
              parameter_string);

let signature = this._getSignature(user_auth_token, accesstoken_secret, data)
console.log('signature'+signature);
// 1499887682711
// 1318622958
let oauth_signature_key = encodeURIComponent('oauth_signature');
let oauth_signature_val = encodeURIComponent(signature);

let request_header_string = 'OAuth '+
                    oauth_consumer_key_key+'="'+oauth_consumer_key_val+'", '+
                    oauth_nonce_key+'="'+oauth_nonce_val+'", '+
                    oauth_signature_key+'="'+oauth_signature_val+'", '+
                    oauth_signature_method_key+'="'+oauth_signature_method_val+'", '+
                    oauth_timestamp_key+'="'+oauth_timestamp_val+'", '+
                    oauth_token_key+'="'+oauth_token_val+'", '+
                    oauth_version_key+'="'+oauth_version_val+'"';
return request_header_string;
</code>
}
But I get a 215 Bad Authentication data. Can someone please advise?
Comment: 'Authorization': ' '+header
Why this empty space before Header?
Comment: I realised that also contributed to the problem. Thanks.
Answer: I figured out what the problem was: The HMAC-SHA1 functionI was using was already doing the base-64 encoding for me.
and as Maciej Adamczewski pointed out, there was an unnecessary whitespace in the header string
Comment: :-) nice to here that I could help