Why should a developer use web services instead of direct connections to a db?

Question: I'm looking for a "top ten" list of reasons why we should be connecting to remote databases via web service instead of directly connecting to the db. This is an internal debate right now and I'm pro-web service but losing the argument. I have a basic grasp of WCF / web services, no one else does. We can do whatever we want moving forward but we need to stick with whatever we choose now.
Here is what I've come up with. Any more?

WCF web services can, if configured correctly, be more secure.
Changes to the DB only need to be made at the service level (config file or recompile service).
Once setup and hosted, web services are easier to consume.
Answer:
Security. You're not granting DB access to anyone but web server/app user.
This is extra important when you have tons of users. You avoid the pain of user/role maintenance on DB side.
DB load reduction. Web service can cache the data it retrieved from DB.
Database connection pooling (hat/tip @Dogs).
A web service can use a small pool of permanently opened DB connections. The helps in a variety of ways:

DB connection pool is limited on database server side.
opening a new DB connection is VERY costly (especially to database server).

Ability for fault tolerance - the service can switch between primary/DR data sources without having details of fail-over be implemented by service consumers.
Scalability - the service can spread requests between several parallel data sources without having details of the resource picking be implemented by service consumers.
Encapsulation. You can change underlying DB implementation without impacting service users.
Data enrichment (this includes anything from client customization to localization to internalization). Basically any of these might be useful but any of them is a major load on database and often very hard to implement inside a DB.
May or may not apply to you - certain architecture decisions are not DB acces friendly.
E.g. Java Servers running on Unix have an easy access to a database, whereas a java client running on a Windows PC is not database aware nor do you possibly want it to be.
Portability. Your clients may not all be on the same platform/architecture/language. Re-creating a good data access layer in each one of those is harder (since it must take into account such issues as above-mentioned failovers/etc...) than building a consumer layer for a web service.
Performance tuning. Assuming the alternative is clients running their own queries (and not pre-canned stored procedures), you can be 100% sure that they will start using less than optimal queries. Also, if the web service bounds the set of allowable queries, it can help with your database tuning significantly. I must add that this logic is equally applicable to stored procedures, not unique to web services.

A good list can also be found on this page: 'Encapsulating Database Access: An Agile "Best" Practice'
Just to be crystal clear - some of these issues may not be applicable to ALL situations. Some people don't care about portability. Some people don't need to worry about DB security. Some people don't need to worry about DB scalability.
Comment: Sorry, disagree. 1. So you grant DB access to a group instead of a single principal - no difference. 2. Any app can cache data. The kind of data that can be cached across multiple users will usually be low volume data in any case. 3. FT should be handled by the database in any case. 4. This is not an out of the box feature, and has to be programmed. 5. Your data access layer should be doing the encapsulation. 6. Same thing. 7. Really? JDBC doesn't run in a client? 8. Good point, when it matters, which is rare. 9. query vs. SP was not part of the question.
Comment: 1. Try managing that across 5000 users with hundrends of roles. Does not scale so well anymore. 2. Depends entirely on an app. Our current case has an instance of caching results of a query which in uber-optimized case takes 20 minutes to run and which we need to run 100s of times a day at least from different apps. 3. FT is handled on a bunch of levels. What do you mean "should be handled by a database"?
Comment: 4. Of course it has to be programmed. But it can be programmed into the service once, or into a zillion client apps on various platforms with varying capabilities. There's a major difference. Never mind the issues of configuration management for load balancing. 5. Same reasoning. You don't need to re-implement DAL. Matter of fact, you can just think of the web service as a portable DAL to ease your mind. 7. We don't want every client opening DB connections. Is that such a big thing to ask for? Again, you're forgetting points 1-5.
Comment: 8. >1 client architecture happens VERY often. Matter of fact I never worked on a project without such a situation in my life, but I'm centered in financial world. 9. It was not. I was basically playing devil's advocate.
Comment: I like this answer, but I think you skipped one of the most important points: Connection pooling. If you have 1,000,000 clients, you're going to either have 1,000,000 open connections, or millions of connections constantly being opened and closed. Basic intuition about computer organization tells us a well tuned pool of a couple hundred long-lived connections is astronomically more efficient than having a million long-lived or millions of short lived connections. The HikariCP docs to a great job of elaborating on this idea: https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing
Comment: @Dogs - yep, I alluded to it in one of the comments above but neglected to add to the answer itself. Feel free to edit in please :)
Answer: In my opinion, you should not automatically be exposing your database as a web service. If it turns out you need a service to expose your data, then write one, but not all database access should be done through web services.

There is no reason why a database connection should not be secure
You can encapsulate the database in a data access layer (possibly Entity Framework)
Web services are no easier to consume than a well-written data access layer.
Comment: Why necessarily XML? There's also much lighter to parse JSON, CSV for simple flat data, etc...
Comment: It's not "for no good reason".  As noted about, depending on your requirements and desires for future development, it might be necessary for your project.
Comment: I write my WS to consume a DAL. What port would you suggest exposing a DAL on?
Comment: @samus it doesn't matter.
Comment: "There is no reason why a database connection should not be secure", well, it is difficult for example to secure the connection between a Windows client and a database. Really difficult to implement a secure connection!
Answer:
Web Services form an API, defining the allowed interactions between external systems and the application's data.
Web Services decouple the database from external interactions and enabling the data layer to be managed independently of those outside influences.
Allowing access only by Web Services ensures that application logic gets the chance to execute, protecting data integrity.
Web Services allow the most appropriate authentication/authorization measures to be taken, as opposed to a database requiring a username and password/table level privileges.
Web Services provide an opportunity for automatic service discovery and configuration to be used.
Web Services traffic can be encrypted for transit over unsecured networks. Don't know of any direct DB connection solutions that enable that...?

Most of these points go for any formal API, not specifically Web Services.
Comment: Yeah, that was what I was going to say, if you have a single application accessing a database all your points are available for normal APIs as well.
Comment: "Web Services form an API, defining the allowed interactions between external systems and the application's data." You can do that with a database too.
Comment: "Allowing access only by Web Services ensures that application logic gets the chance to execute, protecting data integrity." -- I would argue that data integrity should be part of the DBMS only.
Comment: @Shoe It's nice to enforce as much integrity as possible by the DB, but as data starts populating and exposing shortcomings, like the need for some input validation, it's nice to enforce this at the application level (though I for one prefer enforcing on the client side, it sometimes needs to be dealt with on the server side if the validation rules are dependent on data known only to the server (kept from the client app)). Is it a big deal to change a DB's integrity rule set, I'd imagine it not being a trivial matter?
Answer: Writing a web service that simply wraps calls to stored procedures seems to be a misguided approach to designing a good DAL. More than likely, your stored procedures are legacy code left over from a older client-server systems i.e. business rules are buried in the SPs. If that's the case, you're really attempting is creating a silk purse from a sow's ear.
Moreover, your adding a SOAP message protocol layer that adds a performance hit to web apps that have been 'coerced' to dating this 'pig'. I'm working a project right now where our new MVC-4 app has been instructed to use such a DAL. We have the burden of having to change both the WebMethod signature and the SP signature whenever a new user story comes along necessitating such changes; which for us, is every single sprint. Inherent in such a passthru approach is two tightly coupled tiers.
Comment: Web API addressed the WCF/SOAP bloat issue. It's now like a light, fit, agile feline; just what it needs to RESTfully serve.
Comment: In theory, it is not wrong to call stored procs using web services.
Answer: 1)Headache of maintaining database is reduced from developers side so that they can focus only on developement.
2)Web service supports communication of different platforms (operating systems like windows,ios,android etc) in a very easy and effective method.
for example consider a situation when android application & ios application wants to communicate to a website which is java based so for communication of all the three things webservice is the best solution instead of maintaing three different databases.
Answer: In general

Web Service level promotes reuse of common data requests for multiple applications
Web Service can be set up with version management which deflects many issues arising from application level development. For example if I am new to a project which existing application do I use as a good model for configuring my application to use existing database sources.
Web Service has evolved to allow flexible options for sending requests and getting response results back in a common format such as JSON by using a simple URI which means
that client applications can be developed using a more common standard that encourages
dependable uniform interfaces.

I am just getting stared with ASP.NET Web Api and plan on making data services first.
I have recently been focusing on .NET MVC web applications with the use of the entity framework.

If you already use MVC the Web Api also uses MVC with the Api controller so the learning curve to build the services are fairly painless.

I recently found myself in a frustrating predicament with an MVC web app that I was building originally based on Oracle stored procedures. The original version as Oracle 9 or even earlier which presented another problem with Visual Studio 2012 pushing a more modern connection factory approach with load time assemblies finding the right dll files to use based on web config connections and TNS names.
Attempts to connect to the database failed with 'no longer supported' error messages. Out of curiosity I downloaded Oracle 12c and made some application level connections that worked nicely with my TNS names and the load assembly dll and I was able to work with Oracle with no problem.
There were some web services built that were working with connections to the older Oracle version. They were built with methods that were specifically mapped to selected tables however to my disappointment. I would have to write my own.
I was told that the group that was responsible for maintaining the Oracle databases that they would be writing new stored procedures to replace the older ones that I was using to abstract the client interface and business logic layers.
So my first thoughts were that all of the common data requests such as filling up drop down list or auto completes with enterprise wide data be done through data services that would call the Oracle stored procedures. Why repeat that process over each application and have each developer struggle with configuration and version/load assembly, TNS issues?
so....

For multiple database server issues such as using Oracle stored procedures in a .NET MVC application that might usually be using EF for the SQL Server data usage why not push those headaches up to Web Api service methods where those configuration issues can be isolated.
Again the client interfacing can be done using JavaScript, JQuery and JSON which you are already using if you are doing this using Web Api to make SQL Server data requests.

I am an Application Developer/Analyst and not a DBA so my perspective is one from experience with the never ending frustration of having to constantly modify applications when database tools evolve.


How to get $\sum_{k=1}^{n-1}n\binom{n-1}{k-1}y^k \bigg(\frac{1-z^k}{k}- \frac{1-z^n}{n}\bigg)$"?

Question: I asked a question here and got answer committing :

$$(1+yz^n)(1+y)^{n-1} - (1+yz)^n=\sum_{k=1}^{n-1}n\binom{n-1}{k-1}y^k \bigg(\frac{1-z^k}{k}-
\frac{1-z^n}{n}\bigg) \tag{1}$$and$$\frac{1-z^{n}}{n}-\frac{1-z^{n+1}}{n+1}=
\frac{(z-1)^2}{n(n+1)}\bigg(
\sum_{k=0}^{n-1} (k+1)z^k
\bigg) \tag{2}$$

My try:

Using binomial theorem, $$(1+yz^n)(1+y)^{n-1} - (1+yz)^n=(1+yz^n)\sum_{k=0}^{n-1}\binom{n-1}{k}y^{n-1-k}-\sum_{k=0}^n\binom{n}{k}(yz)^{n-k}$$
Using this,$$\frac{1-z^{n}}{n}-\frac{1-z^{n+1}}{n+1}=(1-z)\cdot\bigg(\frac{1+z+\dots+z^{n-1}}{n}-\frac{1+z+\dots+z^{n-1}+z^n}{n+1}\bigg)$$
I can't go ahead in both the cases. Please help me.
Comment: Equation (1) is correct. I do not know how this could help you : did you notice that (1-z^k)/k is the integral of z^(k-1) from z to 1 ?
Answer: $1.$
\begin{align*}
(1+yz^n)(1+y)^{n-1} - (1+yz)^n &= (1 + yz^n) \sum_{k=0}^{n-1} \binom{n-1}{k}y^k - \sum_{k=0}^{n} \binom{n}{k}(zy)^k \\
 &= \sum_{k=1}^{n-1}  y^k  \left(\binom{n-1}{k}  + z^n \binom{n-1}{k-1} - \binom{n}{k} z^k \right )\\
 &= \sum_{k=1}^{n-1} \binom{n-1}{k-1} y^k \left( \frac{n-k}{k } + z^n - \frac{n}{k} z^k \right ) \\
 &= \sum_{k=1}^{n-1} \binom{n-1}{k-1} y^k \left( \frac{n}{k}(1-z^k) - (1-z^n) \right )\\
\end{align*}
$2.$ $\displaystyle \sum_{k=0}^{n-1} (k+1)z^k$ is Arithmetico-geometric sequence can be evaluated as
$\displaystyle \frac{n z^{n+1}-n z^n-z^n+1}{(z-1)^2}$ just by differentiating both sides of $\displaystyle \sum z^k = \frac{1-z^{n+1}}{1-z}$
Also to proceed from your step,
\begin{align*}
\frac{1-z^{n}}{n}-\frac{1-z^{n+1}}{n+1} &= (1-z)\cdot\left( \frac{1+z+\dots+z^{n-1}}{n}-\frac{1+z+\dots+z^{n-1}+z^n}{n+1}\right ) \\
 &= \frac{(1-z)}{n(n+1)} \cdot \left( 1 + z + \dots + z^{n-1} - n z^n \right )\\
 &= \frac{(1-z)}{n(n+1)} \cdot \left( \sum_{k=0}^{n-1} (k+1)z^k - \sum_{k=0}^{n-1}k z^k - \left(  \sum_{k=0}^{n-1} (k+1)z^{k+1} - \sum_{k=0}^{n-1} k z^{k} \right ) \right )\\
\end{align*}
Comment: @Sush $$\sum_{k=0}^{n-1} \binom{n-1}{k} z^n y^{k+1} = \sum_{k=1}^{n-1} \binom{n-1}{k-1} z^n y^{k} +  z^n y^n$$. The last term get cancelled.
Comment: @Sush both has $n$ terms, the first term of second sum does not matter since $kz^k = 0$ when $k=0$. so first sum has n-terms and second sum has $n-1$ terms that get's cancelled. the the last term of first sum will only remain.
Comment: Sir, many many thanks.
Comment: @Sush added there.
Comment: Many thanks,Sir. How did you derive $$\sum_{k=0}^{n-2}\binom{n-1}{k}y^{k+1}z^n=\sum_{k=0}^{n-1}\binom{n-1}{k-1}y^k z^{n}$$ in second step of 1. ? Also, in the original equality, we start sum from $k=1$ and not from $k=0$. (If we start summing from $k=0$, we are forced to divide by $0$).
Comment: Sir, will you just answer my last query? How is $$nz^n=\sum_{k=0}^{n-1} (k+1)z^{k+1} - \sum_{k=0}^{n-1}k z^k $$
Comment: All my doubts are cleared!! This is the effect of having good company!! Hope I will have so calm and helpful instructor some day!! Many many thanks again.
Comment: @Sush lol ... you are welcome :)


Why is the trend of using percentiles to divide the Logit model different from the trend of using percentiles as the knots' restricted cubic spline?

Question: I am trying to analyze the effect of air pollutant on health.
First, I used the restricted cubic spline with three knots which were 25th, 50th, 75th percentiles of the pollutant and the result was Figure1: it looks a linear relationship. And then, I divided the data set into four parts by the 25th, 50th, 75th percentiles of the pollutant, and conducted  logsitic regression for each part, the result was Figure2.
I am confused that why the trend in Figure2 is not a ascending line? Because in my opinion, it should be the same as Figure1.
Please help me, it bothered me for several days.

And the restricted cubic spline model is :
k <- with(Data, quantile(Pollutant, c(.25, .50,.75)))
model_rcs <- lrm(disease~rcs(Pollutant,k)+temperature+humidity+age+gender+job,data=Data)
the logistic model is:
model_log <-glm(disease~Pollutant+temperature+humidity+as.factor(age)+as.factor(gender)+as.factor(job),family = binomial() ,data=Data_Q1) (or Data_Q2\Data_Q3\Data_Q4)
Comment: Presumably you have many other predictors too. The scope for those other predictors complicating or confusing results is enormous. If I understand this correctly, your second graph is a compilation of results from quite different model fits that don't know about each other. But in essence, it is hard to comment helpfully without more details. The limiting form "Why are my results puzzling?" is a question we all have from time to time but it's hard to answer.
Comment: Sure, it's the same covariates but what you estimate as the effects of pollution depends on those covariates too, and your models are not the same. Imagine a simpler case: relating people's weights to height, first overall and then in quantile bins of their height -- in a case where e.g. adults and children are mixed so height is confounded with age. You'll see different results with quite different models on the dataset and on subsets.
Comment: Thanks for your answer. Yes, your understand is right, in my model there are other covariates excluding the pollutant variables. And I added a detailed model to the problem.

Then I think you mean these covariates made  a difference. But  I used the same covariates in  two models, and they may  have a consistent effect.I still don't understand.
Comment: Thank you for the clear example. I seem to understand. In your example, age is the key variable for height and weight, then if I find the crucial variable for pollutant, though the whole is  divided, maybe the result will be better than now, is that so?
Comment: And I  read some researches that devided air pollutants into Levels by their  percentiles, and the concentration gradually rise as from the Level1 to Level4, then reserchers will conduct logistic regression for every dataset  respectively, and result shows the higher Level, the greater OR.
In my figure2, the result shows lower pollutant concentration has higher OR(Q1~Q2), this can not be explained, but in my figuer1, it can be explained.So I asked this question
Answer: The second graph is misleading because it assumes a discontinuous relationship when the true relationship is continuous.  Possibly even more problematic: you are constructing intervals from quartiles.  Quartiles tell us about the distribution of X and are completely disconnected from the shape of the relationship between X and Y.  How many points are near a given X=x should not inform us of how X relates to Y at x.
The knots in splines should not be confused with points of discontinuity in the estimated values of Y.  In a cubic spline function, the knots are points of discontinuity in the third derivative ("jolt") of the function.  The human eye cannot see discontinuities in the third derivative.  It can only see discontinuities in the function and in its first derivative (slope).  Details about this along with an graph illustrating the derivatives may be found in my RMS course notes.
Comment: Thanks for your answer. I will study this note.
Comment: 1. Are many / most splines currently used cubic splines for the above smoothness property? 2. Do you have any guidance for choosing how many knots to use (three above)?
Comment: https://hbiostat.org/rms see course notes
Answer: A few things to note. Most obvious, but probably least important, you model age as a continuous linear predictor in the first model but as a factor in the second model.
Second, note that your second model is actually a set of 4 separate models done on 4 different subsets the data. The effects of temperature, humidity, age, gender, and job are modeled separately within each subset, rather than all together as they are in the first model. That could certainly lead to this type of result. (That's pretty much what Nick Cox is saying in a comment that appeared as I was writing this.)
Third, the results from the separation into 4 models will differ depending on the distributions of the Pollutant values within each of the quartiles. For example, if for one or more quartiles they are bunched near one of the edges of the quartile cutoffs, comparing the results against the continuous fit could be quite misleading. (That's related to what Frank Harrell says in the first paragraph of his answer.)

Why is the trend of using percentiles to divide the Logit model different from the trend of using percentiles as the knots’ restricted cubic spline?


Magento 2: Get category description

Question: I am trying to load a collection of (sub) categories and display their description.
I tried the following things (without success). Used a collection with filter to get the root category. With this method only the root category gives back the correct description.
$cats = $this->categoryFactory
    ->create()
    ->setStoreId(1)
    ->getCollection()
    ->addAttributeToFilter('url_key',$this->getData( 'root_category_id' ))
    ->addAttributeToSelect(['description', 'url_key', 'name', 'store_id']);

var_dump($cats->getFirstItem()->getDescription()); // THIS WORKS!

// iterate subcats
foreach($cats->getFirstItem()->getChildrenCategories() as $subCat) {
    var_dump($subCat->getDescription()); // NULL
}

I am looking to get a category collection and filter by parent_id as I think this might work. However I don't get it to work. I have tried the following methods:
$cats = $this->categoryFactory
    ->create()
    ->setStoreId(1)
    ->getCollection()
    ->addAttributeToFilter('parent_id',$this->getCategory()->getId())
    ->addAttributeToSelect(['description', 'url_key', 'name', 'store_id']);

And:
$cats = $this->categoryFactory
    ->create()
    ->setStoreId(1)
    ->getCollection()
    ->addAttributeToSelect(['description', 'url_key', 'name', 'store_id']);
    ->getSelect()->where ("catalog_category_entity.parent_id = " . $this->getCategory()->getId());

When I try to use the collection, PHP throws this error. I suspect that happens because it wants to load too much categories?

Got error 'PHP message: PHP Fatal error:  Allowed memory size of
  792723456 bytes exhausted (tried to allocate 400572416 bytes)

I hope someone can steer me in the right direction.
Comment: I know I shouldn't be concatenating variables into SQL queries. So please don't mention, I'm doing this only for debugging-sake.
Answer: Here is a snippet I wrote from an old project, apologies I haven't tested it for you specifically but I do know that is still live on a production site today.
$this->_categoryRepository is an instance of Magento\Catalog\Model\CategoryRepository
/**
 * Get child categories of parent category id.
 * @param int $parentId
 * @return array
 */
public function getChildren(int $parentId): array
{
    $parent = $this->_categoryRepository->get($parentId);
    $childIds = explode(',', $parent->getChildren(false, true, true));

    $children = [];

    if (empty($childIds)) {
        return $children;
    }

    foreach ($childIds as $i => $id) {
        $children[] = $this->_categoryRepository->get($id);
    }

    return $children;
}

You'll notice I'm using the category repository rather than factory & collection. This stack overflow question has some more in-depth info on that.
Comment: Thank you very much. I will try it today and mark your answer as correct.
Comment: It's still giving me the "PHP Fatal error:  Out of memory (allocated 3223322624) (tried to allocate 3219132416 bytes)" exception. Even when I used `ini_set('memory_limit', -1)`. It's happening with the first call to `$this->categoryRepository->get($id);` when looping through the child ids. I suspect there is some kind of infinite loop going on.
Comment: it could be because you have too many `childIds`, so that when it loads them into an array it's using too much memory. if you run `count($childIds)` after the `explode` method what do you get? Also, try logging `$i` (in the foreach loop) to a file, so that when you run out of memory you can see how many children you can load until you fail. You can likely paginate your collection of categories if you are loading to many at once.
Comment: It has 9 children. The categories are also not containing any products. Not likely to fill up more than 3GB RAM ;-) .Even when  I add `break;` after the first time `$children[] = $this->_categoryRepository->get($id);` it is throwing the exception.
Comment: I've run into something like this before and the problem was that I had to `load($id)` the entity from a model. When loading the entity, the other attributes came with it (like the description that you're after). In the scope of your first example, try: `$subCat->load($subCat->getData('entity_id'));` Does that load more meta data from the subscription for you?

Magento 2: Get category description


How can I specify a level of a factor while in an lme?

Question: I have a dataset with repeated measures at different speeds. I've binned the speed ranges into 0-0.5, 0.5-1.5, and 1.5-infinity.
mydata`$`fact<−cut(mydata$section, breaks=c(0,0.5,1.5,Inf))

which gives the levels (0,0.5]  (0.5,1.5]  (1.5,Inf]
I first used the lme to test all speeds together to see if x and y are speed-dependent. Now I want to isolate speed bins, but I want to do it in the model instead of re-defining the factor. I can't seem to get it to work. "Fact" is what the speed bins correspond to.
lme(y ~ x + factor(repeatedmeasures) + fact, random = ~1 | z, data=mydata, na.action=na.omit)

Thanks for your help!
Comment: What do you mean by "isolate speed bins"? Do you mean that you want to fit a model for each bin?
Comment: Exactly. Apologies if I'm being naiive!
Comment: Why are you converting a continuous variable to discrete?  There are very few situations where this is appropriate.  It can very easily be manipulated to show spurious relationships.
Comment: Hadley, do you mean speed? It's biologically relevant to categorize the continuous variable into discrete bins. Between certain speeds there are locomotory gait transitions, so I am investigating each speed bin as to compare, say, walks with walks, rather than walks with runs.
Answer: If you just want to fit separate models for each level of your factor, then probably the easiest way is to use the subset= argument to lme (or any other GLM in R, btw). For example,
lme(y ~ x + factor(repeatedmeasures) + fact,
    random = ~1 | z, data=mydata, subset=as.numeric(fact) == 1)

should subset on the first level of your factor. I used as.numeric() because you didn't provide labels to your newly created factor. To do so, you can fill the labels= argument when calling cut(), such that the above code could read
lme(..., subset=fact == "low")

for example (assuming the first level of fact is named low). In both cases, you can easy set up a loop (using something like for i in 1:nlevels(fact) or for i in seq_along(levels(fact))) to iterate over your different models.
Comment: Thanks, I would've never have tried that! I will give it a go.
Comment: Can this also be done for pairwise.t.test? The structure doesn't accept any subset command. I had to try a roundabout method of re-defining the variables to include only data from "low" speed but I'm not sure I believe the results from the t-test.
Comment: I was able to get around it by using
i<-factor=="low"
pairwise.t.test(animals$feature[i], anotherfactor[i], p.adjust.method = "holm", pool.sd = TRUE, paired = FALSE, data=animals)

but if you have a better suggestion I'd appreciate it. Thanks!
Comment: @Alexis You can subset a data.frame at any time, for instance: `df <- data.frame(y=rnorm(20), g1=gl(2, 10, labels=letters[1:2]), g2=gl(5, 2, 20)); with(subset(df, g1=="a"), pairwise.t.test(y, g2))`.


How to maintain type parameter during TypeTag to Manifest conversion?

Question: Using the answer from this question, Is it possible to convert a TypeTag to a Manifest?, I can convert TypeTag to Manifest.
Unfortunately, using this method, you lose the Type Parameter. Due to the fact that you are using the runtimeClass to do the conversion. Here's a sample code that illustrate this point:
import scala.reflect.ClassTag
import scala.reflect.runtime.universe._

// From: https://stackoverflow.com/questions/23383814/is-it-possible-to-convert-a-typetag-to-a-manifest
def getManifestFromTypeTag[T:TypeTag] = {
  val t = typeTag[T]
  implicit val cl = ClassTag[T](t.mirror.runtimeClass(t.tpe))
  manifest[T]
}

// Soon to be deprecated way
def getManifest[T](implicit mf: Manifest[T]) = mf

getManifestFromTypeTag[String] == getManifest[String]
//evaluates to true

getManifestFromTypeTag[Map[String, Int]] == getManifest[Map[String, Int]]
//evalutes to false. Due the erasure.

Is there a way to retain the type parameters when converting from TypeTag to Manifest?
Answer: ManifestFactory has a method called classType that allows create Manifest with type arguments.
Here's an implementation:
def toManifest[T:TypeTag]: Manifest[T] = {
  val t = typeTag[T]
  val mirror = t.mirror
  def toManifestRec(t: Type): Manifest[_] = {
    val clazz = ClassTag[T](mirror.runtimeClass(t)).runtimeClass
    if (t.typeArgs.length == 1) {
      val arg = toManifestRec(t.typeArgs.head)
      ManifestFactory.classType(clazz, arg)
    } else if (t.typeArgs.length > 1) {
      val args = t.typeArgs.map(x => toManifestRec(x))
      ManifestFactory.classType(clazz, args.head, args.tail: _*)
    } else {
      ManifestFactory.classType(clazz)
    }
  }
  toManifestRec(t.tpe).asInstanceOf[Manifest[T]]
}

Comment: When I am using your code I get an error:  java.lang.ClassNotFoundException: scala.Any
Comment: Improved version of `toManifest` is here: https://stackoverflow.com/a/59673693/5249621


Use of the fact that every function is sum of an odd and an even function.

Question: It is well know that every real variable function $f$ can be written as a sum of an odd and an even function, namely $h$ and $g$ where: $$h(x) = {f(x)-f(-x)\over 2}\;\;\;\;\;\;\;\;\;\;\;\;g(x) = {f(x)+f(-x)\over 2}$$
Now what is the use of this fact? I told that to my $\color{red}{\rm high\; school}$ students but then I don't know what to do with this fact. Is it good for a graphing or to calculate the zeroes of a function or something else...?
Comment: Just of the top of my head. If the odd function $h$ is much more nicely behaved than the original function $f$ then there may be some integration tricks on symmetric intervals (or principal value integrals) such that $\int_{-a}^a f(x)dx = 2\int_0^a h(x)dx$. But then again, I can't think of an example when this occurs.
Comment: @Panda It should be $\int_{-a}^a f(x)dx = 2\int_0^a g(x)dx$, as $h(x)$ is odd and $g(x)$ is even
Comment: For an average high school precalculus class I would not worry about justifying this, but rather use it as a theoretical algebraic manipulation exercise or skip it altogether. Sure, tell them it's used occasionally in more advanced mathematical situations, but there are lots bigger fish to fry if you're looking for applications of certain techniques or results. For what it's worth, however, I made use of this in [7. Writing $R(t)$ as the Sum of an Even Function and an Odd Function](https://math.stackexchange.com/q/681893/13130) in a very extensive 2-part answer I wrote here a year ago.
Comment: I remember, about 42-43 years ago, this question on my calculus university exercise book, (Demidovic) to prove that any function can be written as the sum of an odd and an even function. I break my head on this problem, as in high school I never saw proofs in calculus. Result? I still remember this after 42 years...
Comment: It has a pedagogical value: By defining $h(x)$ and $g(x)$ in that way, you can learn something by doing the simple proof that $h(x)+g(x)=f(x)$; $h(x)$ odd; $g(x)$ even.  There might be some application with Fourier series, where odd functions can be expressed with sines and evens with cosines. I can think of more immediate applications of the fact that every function is the difference of two nonnegative functions: $$f(x) = \max[f(x),0] - \min[-f(x),0]$$
Comment: Most elementary applications are conceptual. It gives a nice way to introduce hyperbolic sine and cosine, [Mark Dominus](https://blog.plover.com/math/even-odd.html) uses it to explain features of graphs of functions. More serious applications are to signal and image processing, e.g. isolating asymmetries in face recognition, see [Digital Information Processing and Communications, p.489](https://books.google.com/books?id=kd0RBwAAQBAJ&source=gbs_navlinks_s).
Comment: @Conifold Yes, but I have a high school students.
Comment: In the Maclaurin expansion of $f(x)$, the even terms give the even function, and the odd the odd function. For example, one can see how the series for $\cos(x)$ and $\sin(x)$ follow from that of $e^{ix}$.
Answer: When you are confronted with a new mathematical problem there are a few helpful rules to get ahead. One of them is: Search for the symmetries! Often a problem has in its formulation an obvious symmetry $x\leftrightarrow -x$. Such a symmetry could force the solutions to be symmetric as well.
Another example: One of the most important functions in analysis is ${\rm cis}:\ t\mapsto e^{it}$. It has a fantastic functional equation: ${\rm cis}(s+t)={\rm cis}(s)\cdot {\rm cis}(t)$ for arbitrary complex $s$, $t$. Unfortunately this function ${\rm cis}$ is complex-valued. But ${\rm cis}$ has an even part $t\mapsto\cos t$ and (up to the factor $i$) an odd part $t\mapsto\sin t$. These symmetric, resp., antisymmetric parts of ${\rm cis}$ have an immense stating in all of analysis and its applications. Similarly with the even and odd parts $\cosh$ and $\sinh$ of $x\mapsto e^x$.
I'm not sure whether it is important make a general theory of this for high school students.
Comment: Thanks! Can you help me here: https://math.stackexchange.com/questions/2639166/what-is-the-largest-possible-number-of-moves-that-can-be-taken-to-color-the-whol
Answer: The idea it can be useful to generalize your result:
The group $\mathbb{Z}/2$ acts on $\mathbb{R}$ in the following way:
for each $x\in \mathbb{R}$ you have that $\bar{0}x:=x$ while $\bar{1}x:=-x$
The action can be induced also on the set $V$ of the function from $\mathbb{R}$ to $\mathbb{R}$, that is a $\mathbb{R}-$ vector space , in the following way:
for each $f\in V $ $\bar{0}f:=f$ while
$\bar{1}f=f(-\cdot)$
So you can observe that $\mathbb{Z}/2$ can be represented on the vector space $V$.
We denote $(\mathbb{Z}/2)^*$ the group of character of $\mathbb{Z}/2$ where a character is a morphism from $\mathbb{Z}/2$ to $\mathbb{R}$.
We define the following eigenspace related to $\mathbb{Z}/2$:
for each $\chi\in (\mathbb{Z}/2)^*$
$L_\chi:=\{f\in V: g f=\chi(g) f \forall g\in \mathbb{Z}/2\}$
You can observe that there are only 2 possibile character related to $\mathbb{Z}/2$:
$\chi_ 0:=1$
$\chi_1(\bar{0}):=1$ and $\chi_1(\bar{1})=-1$
Now you can observe that
$L_{\chi_0}= \{f: f(x)=f(-x) \forall x\in \mathbb{R}\}$
$L_{\chi_1}= \{f: f(x)=-f(-x) \forall x\in \mathbb{R}\}$
So you have that $L_{\chi_0}$ is the space of even function while $L_{\chi_1}$ is the space of odd function.
You have that for each $f\in V$ then
$\frac{f+\bar{1}f}{2}\in L_{\chi_0}$ while $\frac{f-\bar{1}f}{2}\in L_{\chi_1}$ and
$f= \frac{f+\bar{1}f}{2}+ \frac{f-\bar{1}f}{2}$
so you have that
$V=L_{\chi_0}\oplus L_{\chi_1}$
Now you can generalize this result to a general set $X$ when a finite abelian group $G$ acts on it:
For each $f\in V$, where $V$ is the $\mathbb{K}$ -vector space of the function from $X$ to $\mathbb{K}$, with $char(\mathbb{K})\neg | o(G)$,  you have that
$f_\chi:=\frac{1}{o(G)}\sum_{g\in G}\frac{1}{\chi(g)}g f\in L_{\chi}$ for each $\chi\in G^*$ and
$f=\sum_{\chi\in G^*} f_\chi$ so
$V=\oplus_{\chi\in G^*} L_\chi$
This is a useful result that often can be used in algebraic geometry to study some variety that can be viewed as quozient with respect to another variety and a finite abelian gruop $G$ acts on that variety.
Comment: Well, I don't understand a word of it. How can I use it for a high school students. Does in Italy do all this in high school?
Comment: Oooops sorry, I didn't read 'for high school'. However I'm an Italian boy and i' m sure that do not exist this concept for an high school
Answer: You could tie it to Fourier series. Not formally of course but just hint at the really neat idea that you can write a function as a series of sin (odd) and cos (even) functions.
You could also tie it to Taylor series (again not formally just a little introduction) where functions are the sums of odd powers (odd functions) and even powers (even functions). And, you could show in your examples that odd functions like sin(x) are the sums of odd powers in the Taylor series.
For actual applications, I personally have never had to use that fact to solve a problem, so I wouldn't worry about its application as a tool in High School math.


Finding index of a dictionary in a list by key value

Question: How could I find the index of 'node4' in my dataset a? I want node4 to be my input for this particular function.
ex:
a = [
   {'node1':['node2','node3','node5']},
   {'node2':['node1','node8','node10']},
   {'node3':['node4','node2']},
   {'node4':['node2','node1','node3']},
   {'node5':['DEADEND']},
   {'node6':['GOAL']}
    ....
    ]

Comment: Why is this a big list of 1-key dictionaries? One big dictionary might work better, but whatever the program, it's hard to think of a reason you'd build 1-key dicts for this.
Comment: Is this by chance the result of a network flow algorithm?
Comment: @JonathonReinhart it is actually
Comment: @Liondancer Which library are you using, NetworkX?
Comment: @JonathonReinhart nope i am not
Comment: @user2357112 I want to try your method. How would I convert my list into a dictionary?
Comment: @Liondancer: Well, you'd ideally build it as a dictionary in the first place, but to convert it, you'd do something like `dct = {}; for d in a: dct.update(d)`, except with proper indentation, better variable names, and a line break instead of the semicolon.
Answer: Use next and enumerate:
>>> next((i for i,x in enumerate(a) if 'node4' in x), None)
3

Answer: def findIndex(inKey):
    for idx, item in enumerate(a):
        if inKey in item:
            return idx
    return None

Comment: @thefourtheye `inkey in item` dictionary key lookup is O(1), while constructing a list/view and  lookup `inKey in item.keys()` is O(N)
Comment: @JonathonReinhart yes, I somewhat oversimplified, more details in https://wiki.python.org/moin/TimeComplexity
Comment: `if inKey in item` is going to be faster.
Comment: @AshwiniChaudhary Oh, why so? It doesnt have to get the list of keys explicitly?
Comment: @alko Are you sure `inKey in item` is O(1)? A typical hashtable implementation would calculate the hash O(1), check to see if that bucket is populated O(1), then *iterate* through all items in that bucket O(m), to see if the item being searched for exists. (m is some factor based on hash collision rate and bucket size). Eh, I'm probably just being pedantic.
Comment: @alko Average case O(1), worst case O(n). We both win :-)
Answer: I would build a dictionary with the indices:
>>> index = {key: i for i, elem in enumerate(a) for key in elem}
>>> index['node4']
3

This requires going over the entire list once to build the dictionary (which is the worst case  for one-time use anyway) but is then faster if you need it more than once.
Answer: Or just like that:
    [index for index, record in enumerate(a) if 'node4' in record.keys()][0]


validation regularexpression

Question: i have text box in that user have to enter only text. user shouldn't enter digits and special character also.help me thanku
Answer: Only letters and spaces, with at least one:
<code>^[A-Za-z ]+$</code>
Comment: thank u mr.Graham Clark. i have doubt this expression includes no special character.user should not enter  the special character also.

validation regularexpression


Should the Windows 7 local administrator profile be enabled if the computer is part of a Windows 2003 domain?

Question: Should the Windows 7 local administrator profile be enabled if the computer is part of a Windows 2003 domain?
Users are not administrators on their own workstations. I'm wondering if I need a local administrator account on the computer if I've already logged on once to the computer as the domain administrator.
Answer: Personally, I enable the local Administrator account via a startup script. (That script also sets the password-- depending on the Customer site it may be a unique password per machine, or a "blanket" password for all machines.)
This gives me an account to troubleshoot with if the machine's domain trust relationship gets broken. I need them very rarely, but they're nice to have when I do need them.
I've toyed with creating an Administrator-equivalent account with a randomly-chosen name and random per-machine password and leaving the stock Administrator account disabled but haven't decided to do that. The password also shouldn't be set using a startup script because it would be trivially easy for users to read the script (even if you restrict access to "Domain Computers"-- it's trivially easy for a user to get "Domain Computers" rights in most AD deploymnets). That'll probably be my next iteration...
Answer: No, it should not.  It's recommended by Microsoft to disable or rename the default Administrator account as a best practice.
That said, you do want a local administrator, otherwise you'll be in for a big hassle if the machine loses its domain trust (or domain connection) and you need to log in with administrative privileges to correct the issue.
You can both disable the default admin account and set up a new one with group policy.  I take the rename-it-to-a-fictional-character approach.

The path to do so (through Group Policy Management Editor) is: Computer Configuration --> Policies --> Windows Settings --> Security Settings --> Local Policies --> Security Options, and you'll see a bunch of relevant options, in particular, Accounts: Administrator account status and Rename administrator account.
Answer: You'll want to leave it there and enabled in case the machine looses the ability to talk to the domain for some reason (it happens).  Without it you'll have no way to remove the machine from the domain and join it back up to the domain.  Just make sure that it's got a strong password that the end users don't know what is.
Answer: It's up to you.
Reasons to keep:

If the domain membership fails, you need a local administrator to rejoin it to the domain.

Reasons to not keep:

If the domain membership fails, you can boot off a recovery CD to get at the files on the computer, then reimage it.


Por que usar/não usar * box-sizing?

Question: Já li muito a respeito de debates entre usar ou não usar a propriedade box-sizing: border-box; da seguinte maneira:
* {
    -webkit-box-sizing: border-box;
    -moz-box-sizing: border-box;
    box-sizing: border-box;
}

Sei que ela é bem influente, especialmente na elaboração do layout, como interação entre bordas, paddings, etc.
Mas por que usar ela deste modo? Ou por que não usar? Uma vez que estará aplicando o estilo a todos os elementos.
Em alguns casos que usei, pude notar uma imensa repetição das propriedades quando a hierarquia DOM era profunda.
Se não for usado deste modo, o mais correto seria definir apenas nas propriedades que vou precisar?
Ou melhor dizer, qual a real função da propriedade box-sizing? E qual é, de fato, o melhor modo de usar essa função?
Comment: Eu entendo que o uso do * ou não dá uma discussão interessante, independente do box-sizing, pelas questões de facilidade de aplicar em tudo versus o impacto no DOM. Já o box-sizing é uma propriedade como outra qualquer, que se usa conforme é necessária. Na verdade é uma propriedade que corrige uma das maiores ***adas do w3c, no meu ver (que o IE era o [único browser a fazer o que era sensato](https://en.wikipedia.org/wiki/Internet_Explorer_box_model_bug)).
Comment: Complementando: se for usar o *, tem uma proposta muito melhor neste artigo aqui, que muda o box do HTML todo, e herda no resto dos elementos (`inherit`): https://css-tricks.com/inheriting-box-sizing-probably-slightly-better-best-practice/
Comment: Eu sempre uso por padrão em todos os projetos. Particularmente acho um saco ter que calcular na mão o tamanho que a div terá de ficar só por que coloquei um border ou um padding
Comment: Pois então. Tentei começar um layout sem usar essa definição, pois a repetição dela me incomodava. Mas estou me incomodando mais com a falta dela do que com o excesso. Mesmo sem saber ao certo quais os reais prós e contras disso.
Comment: Segundo a [MDN](https://developer.mozilla.org/pt-BR/docs/Web/CSS/box-sizing)
A propriedade CSS box-sizing é utilizada para alterar a propriedade padrão da box model, usada para calcular larguras (widths) e alturas (heights) dos elementos. É possível de usar essa propriedade para emular o comportamento dos navegadores (browser) que não suportam corretamente a especificação da propriedade CSS box model.
Comment: @Bacco Muito bem colocado! Até o momento me parece o melhor método de se usar. O que eu achava que poderia atrapalhar, ou não ser a melhor prática, era pelo fato de repetir a propriedade de um modo, digamos, "descontrolado". Vou ver o que consigo montar em cima desse conceito!
Comment: O Foundation usa tanto no *, quanto no *:after, *:before. E eu uso assim também.
Comment: Eu só uso quando necessário. Não aplico isso ao `*`.
Answer: Este artigo do Sergio Lopes fala sobre o assunto e mostra como a especificação de box-model é confusa.

Você usa essa propriedade para definir que, o tamanho de um elemento deve ser calculado desde a sua borda. A forma mais simples de entender é com um snippet:

.a, .b {
  border: 20px solid #000;
  background: skyblue;
  font-size: 4em;
  margin: 2px;
  height: 100px;
  width: 300px
}

.b {
  box-sizing: border-box
}

sopt

sopt

Note que ambos os elementos possuem 300 pixels de largura.
A altura e a largura de .a são calculadas considerando somente o seu conteúdo (valor padrão: box-sizing:content-box), bordas e padding não são incluídos no cálculo e se você incluir um border: 20px solid #ccc o tamanho da sua caixa será 200px + 40px.
Com o border-box, o calculo será o tamanho da caixa incluindo a borda e margem interna.
Se fosse para responder "Por que usar?", eu diria simplesmente para evitar surpresas durante o desenvolvimento.
Comment: Isso é opção de quem desenvolve, não tem como responder. Chutando, talvez alguns desenvolvedores achem que aplicar uma propriedade em seletor universal seja ruim (do ponto de vista de performance). Mas [não é bem assim](http://www.paulirish.com/2012/box-sizing-border-box-ftw/).
Comment: Sim, esse conceito eu compreendi. Mas por que muitos utilizam ele aplicado ao `*` e outros já preferem aplicar diretamente no elemento que é necessário, seja através de um próprio atributo, classe complementar ou funções extras (como um @include / @extend no SASS)?
Answer: Tem dois grandes motivos para você querer padronizar o box model nos seus documentos, ou pelo menos numa parte deles:

Você quer ou precisa usar o modelo "tradicional" (border-box, o antigo modelo que todos usavam, e que sobreviveu por mais tempo no IE), que é mais intuitivo e pode ser a única solução em determinadas situações de leiautes intrincados.
Você está estilizando um formulário e nada parece fazer sentido. Isso porque elementos de formulários usam box models diferentes entre si, até hoje, em diversos browsers. Inclusive certos elementos ainda são fornecidos pelo sistema operacional, e não pelo browser (os elementos "substituídos", na linguagem do CSS).

Dá pra ver que no mínimo padronizar o box model dos formulários é uma decisão sensata. Quanto a padronizar tudo – alterando box-sizing para o elemento html, e os demais via herança com *, *:before, *:after, fica a seu critério. Independentemente do box model com o qual você decidir trabalhar, pode sempre surgir algum caso específico em que o outro box model seria uma mão na roda. Nesses casos você pode aplicar box-sizing individualmente ao que precisar.

Por que usar/não usar * box-sizing?


Show that $V_n(R)$ has one maximum and that it decreases for large $n$

Question: I am studying the volume of $n$-balls, and I have deduced the following formula for the volume of a $n$-dimensional ball of radius $R$:
$$V_{2k}(R) = \frac{(\pi R^2)^k}{k!} \hspace{1cm} V_{2k+1}(R) = 2R\frac{(4\pi R^2)^kk!}{(2k+1)!}$$
I now want to show that $V_n(R)$ tends to zero as $n$ tends to infinity, so I have to calculate $\displaystyle\lim_{k\rightarrow\infty} \frac{(\pi R^2)^k}{k!}$ and $\displaystyle\lim_{k\rightarrow\infty} 2R\frac{(4\pi R^2)^kk!}{(2k+1)!}$. However, I am not sure how to calculate the limits because of the factorials in them.
I also what to show that $V_n(R)$ has always one maximum in terms of $n$, that is, that there is a dimension where a ball with radius $R$ has the maximum volume, for example, for $R=1$, it is the 5th dimension ($n = 5$). However, again because of the factorials, I don't know how to calculate the derivative to optimize the function.
I would prefer a solution that does not use the gamma function, Stirling's approximation, or double factorials.
Comment: Use the ratio test.
Comment: For showing $\lim_{n\to\infty}V_n(R)=0$ for fixed $R$, you can either use Stirling (then it is trivial), or look at something like $n\ge\lceil\pi R^2\rceil^2$. For showing that it has a unique maximum, my strategy would be this: You try to prove that there exists a $n_0\in[1,\infty[$ such that $\frac{\partial}{\partial n}V_n(R)$ is positive on $[1,n_0[$ and negative on $]n_0,\infty[$. Then you argue that the unique maximum is achieved either at $\lceil n_0\rceil$ or $\lfloor n_0\rfloor$. However, you cannot proceed like this if you don't use the Gamma function (and I have no good alternative).
Answer: Since $R$ hass some fixed value, there exists $n=m$, for which $k=\pi R^2/m < 1$. Thus, for $n>m$:
$$
\frac{(\pi R^2)^n}{n!} = \left(\frac{(\pi R^2)^m}{m!}\right)\frac{\pi R^2}{m+1}\frac{\pi R^2}{m+2}\ldots\frac{\pi R^2}{n} < \left(\frac{(\pi R^2)^m}{m!}\right)\frac{\pi R^2}{m}\frac{\pi R^2}{m}\ldots\frac{\pi R^2}{m}=\left(\frac{(\pi R^2)^m}{m!}\right)k^{n-m}.
$$
In other words, your sequence is bounded by a geometric sequence with ratio $k<1$.
For the second sequence, you can do a similar trick, but first you need to simplify:
$$
\frac{n!}{(2n+1)!}=\frac{1}{(n+1)(n+2)\ldots(2n+1)}
$$
To address the question with one maximum, you can show that $K_n=V_{n+1}/V_n$ is a strictly decreasing sequence. So as long as $K_n>1$ $V_n$ is increasing and after $K_n<1$ volume is decreasing. However, note that there exist $R$ and $n$ such that $K_n(R) = 1$. Which means that $V_{n+1}(R)=V_n(R)$. In other words, volume can reach maximal value at two neighbour dimensions.
Answer: WLOG consider $R=1$. First the odd case:
$$V_{\text{odd}}(n)=\left|\frac{\pi^n}{n!}\right|\leq \left|\frac{\pi^n}{4^n}\right|$$
(For large enough $n$.) Obviously,
$$\lim_{n\to\infty}\left|\frac{\pi}{4}\right|^n=0$$
Hence $\lim_{n\to\infty}V_{\text{odd}}(n)=0$. Apply similar reasoning to the even case.


Specify forwarded port to be used in authorized_keys

Question: My Current SSH Situation.
I got a few SSH Clients and an SSH Server. I deploy an autosshserver.script to the clients. The script creates a reverse SSH tunnel (plus it forwards port 22 and 5900) to the server. On the other side I can connect via the server to the clients.
My Problem is now, controlling the ports, that are already used by the clients. I'm looking for an way, to specify those PORT_Variables on the Server. So I can always deploy the same script, but change the ports to be used on the Server side in the authorizedkeys for example.
Those are the relevant snippets from the autosshscript.
<code>##### VARIABLES ###########################################

PORT_SSH="{{  PORT_SSH  }}"
PORt_VNC="{{  PORT_VNC  }}"

AUTOSSH="/usr/bin/autossh"

AUTOSSH_PARAMS="-M 0 -f -2 -4 -N -C -n -T -p $HOST_PORT"

AUTOSSH_ADV_PARAMS="-o \"ServerAliveInterval 30\" -o \"ServerAliveCountMax 2\" -o \"ExitOnForwardFailure yes\" -o \"PubkeyAuthentication yes\" -o \"PasswordAuthentication no\" -o \"UserKnownHostsFile /dev/null\" -o \"StrictHostKeyChecking no\""

LOCAL_USER_KEYFILE="-i /home/FOOSSH/.ssh/id_rsa"

TUNNEL="-R 0.0.0.0:$(PORT_SSH):127.0.0.1:22 -R 0.0.0.0:$(PORT_VNC):127.0.0.1:5900"

HOST_LOGIN_USER="FOOSSH"

HOST_NAME="ssh.FOOSSH.eu"

##### LOGIC ###############################################
su -l $LOCAL_USER -m -c "$AUTOSSH \
$AUTOSSH_PARAMS \
$AUTOSSH_ADV_PARAMS \
$LOCAL_USER_KEYFILE \
$TUNNEL \
$HOST_LOGIN_USER@$HOST_NAME"
</code>
Thanks for help in advance. :)
Comment: It seems there is no such function. So my solutions is, to create a script.
Answer: So here is the solution. Not for using an SSH Build in function but by creating a workaround script.
Algorithm of the script:

diff <(current pub key lokal) <(pub key on server side) and read the comment, which defines the port that will be used to by the tunnel.
write variable into the autossh script.


Batch file to set the position of the curser on windows 7 OS. How?

Question: is it possible to make a batch file that sets the position of the mouse/curser to an (x,y) coordinate on the screen? and if so, then is it possible to click on something that is in that position? Any ideas would be helpful! Thanks!
Comment: Chances are, there's a better way to do whatever you're trying to accomplish here. Automating user interaction (like clicking on something displayed on the screen) is difficult because as soon as whatever UI element it is gets moved or something covers it, your batch file stops working (or worse).
Answer: No. You would have to have a seperate executable to do this for you.
It may be possible in VBScript.
P.S. If you're interesting in automating tasks I'd download something like a mouse recorder for something simple like that. If you're automating more complicated tasks, give a shot at Autoit.
Comment: ok how would you do it using vbscript? also, could you recommend a good tutorial for vbscript (im assuming vbscript is written in visual basic?)
Comment: i am not interested in automating tasks, i was just wondering if it was possible, ive seen a few custom programs for batch that do it, but again, i have no clue how to configure/setup custom commands/programs to work inside the command line...
Comment: VBscript is a "more extended" version of batch to most people. You can set datatypes, do more things, you can log the person out, you can make things run at certain times, heck, people have coded viruses in them (don't get any ideas..) Anyway, it's possible. You just have to have a seperate program do it for you. Batch 'programs' are really just 'scripts' that reads a file line by line. You can actually take (some) batch scripts, and type them in the command line, and they will have the same effect as if you ran the script. All batch is doing is running programs. Seriously, that's it!
Comment: Ok since ive posted this ive learned alot about vbscript, and python. So is it possible to accomplish this by means of a python script or vbs? If so , could you post an example to get me started! Thanks!
Answer: No and Yes.
You can't place the cursor at Position X,Y with pure batch, but you can replace the cursor to the home position in one line, and then you can rewrite the content.
For detecting mouse clicks with batch you need an external tool.
Try a look at Lingubender Tan's Batch tools
EDIT: You can place the cursor with pure batch to the home postion, but I need the help of the user, till now.
Comment: I found that you can get the mouse coordinates with this ultra complicated batch-file: http://ss64.org/viewtopic.php?id=1067 (A Tic-Tac-Toe Batch Game that uses mouse clicks...) or http://ss64.org/viewtopic.php?id=1062
Comment: But it isn't batch, it creates an executable exe file
Comment: (using Wscript/Cscript) I know, but it does it quickly and efficiently :D, its still pretty cool!


How to remove non-passive event listener?

Question: I'm developing a code editor. Actually, I'm at the end of the version 2.0.1 which is an important version containing new functionnalities. But just before to update the Google Chrome browser to his latest version (64...), I maked a few change in my code and I don't know anymore what is it.
After Chrome updated, when I open the editor file, I can not click on javascript button or links. This does not work on other browser too. I took a screen shoot of my console.
There are the events that I used : click, load, beforeunload, ready, keyup, keydown, online, offline, mousedown, mousemove, mouseover, mouseout, mouseup,  touchstart, touchmove, touchend, and the CodeMirror events.
Can someone help me to remove these non-passive events listener which are blocking events on my app ?
Comment: Aren't you using any kind of version control?
Comment: i guess the info is not enough to give a solution.
Comment: @José - why are you not trying the debugger of the Chrome Developer Tools? You are welcome to publish the source code on GitHub if you want, or better yet, a demo URL that reproduces the issue that can be debugged.
Comment: @Nirus, I really do not know what change I've made to create this issue.
Comment: @lxg I didn't use any version control and I don't use it. I'll think about it next time
Comment: What info can I give to get solution ?
Comment: @José: Do yourself a favour and start using Git (or any other VCS), it's really easy and a huge help when writing non-trivial software. Let your current situation be a lesson to you. :)
Comment: Thanks @lxg, I started using Git on VS Code
Comment: The console violation messages can sometimes be hints to slowness or bad behavior, but not always. They can sometimes be advices for speeding up the page and fixing them will not fix issues like the one you experience. Did you try to debug the issue regardless of those violations?
Comment: Also, without seeing code and having the option to debug it ourselves, there is little we can do because your question does not provide enough information in order to come up with a reason.
Comment: Ok @PhistucK. I tryed the default VS Code debugger but it doesn't help me anymore. Do you want the source code ?
Answer: I set a breakpoint on -
  $("span.jdb-toggle-fullScreen").click(function(){

And it never hit, so I set a breakpoint on the beginning of the containing function -
  $(".jdb-author-name").html(jdb.info.detail.author.name[1]).on("click", function() { window.open(jdb.info.detail.author.url); });

And it did hit. I stepped over the lines until I got to this code -
  // Say Hi Coder every 24h.
  if (jdb.cookie("Hello_World")) { return; }

The condition always returns true after you show some message/toast, I assume and when you load the page again anyhow for the next 24 hours, you will never continue running the containing function and you will never get to the code lines that add the event listeners to those buttons.
Solution -
Change -
  // Say Hi Coder every 24h.
  if (jdb.cookie("Hello_World")) { return; }
  iziToast.show({
    title: "Bonjour, cher Codeur! ",
    timeout: false, position: "topRight",
    onClosing: function() {
      jdb.cookie("Hello_World", "Yes", {
        expires: 1,
        path: "/"
      });
    }
  });

To -
  // Say Hi Coder every 24h.
  if (!jdb.cookie("Hello_World")) {
    iziToast.show({
      title: "Bonjour, cher Codeur! ",
      timeout: false, position: "topRight",
      onClosing: function() {
        jdb.cookie("Hello_World", "Yes", {
          expires: 1,
          path: "/"
        });
      }
    });
  }

In order not to run only the toast code when the cookie is there.
Comment: I used the Developer Tools, but I guess setting it in Visual Studio Code would have worked just as well (as long as it is connected to Chrome, of course). I never tried it.
Comment: In order to fix the violations, you need to make all of the code (yours as well as the CodeMirror, jQuery and any other third party code) do less while listening to event listeners, as well as change less styles/elements in the DOM and make those operations more targeted.
Comment: Oh yes, to say hello every day, I wanted use `If {} else {}` but I forgot the `else`.
In all ways `if (!jdb.cookie("Hello_World"))` is short and perfect.
I maked change and I can make an event now but I continue to see [Violation] ... for every event in the console.
Comment: So difficult. If that's all there is to it, thank you very much for your help, @PhistucK. I will try to reduce the events and DOM operations as I can.


Why does a model perform worse after reintroducing observations with missing data imputed?

Question: My dataset contains about 12% missing data, and much of the missing data is grouped along observations (not randomly scattered or along columns). I optimized a regression method after removing the observations with large amounts of missing data (and used KNN-impute to infer the remaining missing data). I am now trying knn-impute on the entire data set, without removing any data prior, and my previous optimal regression method is now performing worse. Why might this be?
Note: optimal regression method == optimizing parameters.
So is adding in this new data worse i) as it contains lots of missing data and knn-impute is the wrong method, or ii) I just need to search for new optimal parameters.
Answer: Presumably your model was optimised for one sample. If you add a large amount of new data it will generally perform worse. The performance could be even worse if the observations with missing data are systematically different in some way.


Why are media items no longer available at /media/{id}?

Question: I'm using the Media module, but I noticed there isn't a path like /media/%id where I can see the media and its fields.
The only path available seems to be media/%id/edit and media/%id/delete.
On Drupal 8, it was possible to access a media page. What's wrong?
Answer: This was an announced change that released in Drupal 8.7.0—"Media items no longer available at /media/{id}". The change added a new setting to enable the "id" route and turned it on for existing sites. New sites must have the setting enabled at /admin/config/media/media-settings.
Answer: Steps: To enable this '/media/%id' path:: In Drupal9/10
Enable the 'Media' module and then go to this page:
your_site_url/admin/config/media/media-settings
Enable this checkbox 'Standalone media URL'
Additonal info:
#1: Basically this route '/media/%id' disabled by core.
#2: May be in your case module 'Media' enabled, you just need to enable  this checkbox 'Standalone media URL' and save the form.


Wordpress redirecionando home para localhost

Question: Fiz a instalação do wordpress no meu localhost para testes, depois tentei migrar para um domínio, fiz as alterações em siteurl e home e atualizei também os links dos posts que tinha, está tudo funcionando corretamente com exceção da pagina home que deveria carregar em meudominio.com.br/ porem está redirecionando para localhost. As outras paginas estão tudo funcionando, por exemplo se eu acessar meudominio.com.br/quemsomos ele carrega normal, o admin também está normal.
Comment: O Search-Replace-DB permite procurar e substituir uma string por outra na base de dados. Tenta utilizar para encontrar e subtituir todos os links do localhost para meudominio.com.br https://github.com/interconnectit/Search-Replace-DB
Comment: Utilizei o Search-Replace-DB e ele não encontrou nenhuma referencia a localhost, não conheço bem a estrutura do wordpress, não sei se ele salva algo em arquivo.
Comment: Vê se tem alguma referência ao ip 127.0.0.1. Esse endereço equivale a `localhost`.
Answer: Vai em configurações > geral e altera os campos:
Endereço do WordPress (URL) e
Endereço do site (URL)


Sum of binomial coefficients $\sum_{k=0}^{n}\binom{k+m}{m}$

Question: Let $n,m \in \mathbb{N}$
Evaluate:
$$\sum_{k=0}^{n}\binom{k+m}{m}$$
I don't know what to do, because none of the identities I know match this sum.
Help is greatly appreciated.
Comment: See also [Proving that $\sum\limits_{k=0}^{n} {{m+k} \choose{m}} = { m+n+1 \choose m+1 }$](https://math.stackexchange.com/q/357063) and [Prove $\sum\limits_{i=0}^n\binom{i+k-1}{k-1}=\binom{n+k}{k}$ (a.k.a. Hockey-Stick Identity)](https://math.stackexchange.com/q/833451).  Found [using Approach0](https://approach0.xyz/search/?q=%24%5Csum_%7Bk%3D0%7D%5E%7Bn%7D%5Cbinom%7Bk%2Bm%7D%7Bm%7D%24&p=1).
Answer: Consider the recurrence identity
$$
   \binom{n+1}{m+1} - \binom{n}{m+1} = \binom{n}{m}
$$
for $n=k+m$. We get, by telescoping:
$$ \begin{eqnarray}
   \sum_{k=0}^n \binom{k+m}{m} &=& \sum_{k=0}^n \left( \binom{m+k+1}{m+1} - \binom{m+k}{m+1} \right)  = \sum_{k=0}^n \left(F(k+1)-F(k) \right)  \\ &=&  F(n+1)-F(0) = \binom{m+n+1}{m+1} - \binom{m}{m+1} \\ &=& \binom{m+n+1}{m+1}
\end{eqnarray}$$
Comment: It should be Binomial[m + n + 1, m + 1] in the last line, right?
Comment: @VincentPfenninger Yes, indeed. Thank you!
Answer: If you experiment with $m=0,1,2$, and $3$, trying the first few values of $n$ for each, you may be able to conjecture the identity, which is then very easy to prove by induction. Your summations are column sums of the matrix of binomial coefficients shown below:
$$\begin{array}{r|cc}
n\backslash k&0&1&2&3&4\\ \hline
0&1\\
1&1&1\\
2&1&2&1\\
3&1&3&3&1\\
4&1&4&6&4&1\\
5&1&5&10&10&5
\end{array}$$
E.g., with $m=2$ and $n=3$ you have
$$\sum_{k=0}^3\binom{k+2}2=\binom22+\binom32+\binom42=1+3+6=10\;.$$
HINT: The sum is a single binomial coefficient expressible in terms of $m$ and $n$.
Alternative HINT: You can try for a combinatorial explanation of the sum. $\binom{k+m}m$ is the number of ways to choose $m$ numbers from the set $\{1,2,\dots,k+m\}$. It's also the number of ways to choose $m+1$ numbers from the set $\{0,1,2,\dots,n+m\}$ with the stipulation that $k+m$ is the largest number chosen.
Answer: I have a better answer.
I made use of the rules that ${m+n} \choose {m}$ = sum all k (C(m,k)*C(n,m-k)),
Sum k=0 to n (C(k,m)) = C(n+1,m+1) to begin with since i started the equality backwards C(n+m+1,m+1) = sum k=0 to n (C(m+k,k)) .
It took me more than an hour to solve, and tears of joy I solved it so always try out for yourself.
I hope you can se the picture.
Ciao
Picture
Comment: Do not post links to images. Use MathJax to format equations.


Issue with multiplication by a fraction in pre-processor Substitution (#define)

Question: I'd like to start with the code
#define myConst (419*0.9)
#define myConst1 (419*.9)
#define myConst9 (myConst1*.9)
#define myConst11 (419*1.1)

int main()
{
    printf("myConst:%d\n",myConst);
    printf("myConst1:%d\n",myConst1);
    printf("myConst9:%d\n",myConst9);
    printf("myConst11:%d\n",myConst11);
    return 0;
}

Which gave me the output as

myConst:1913895624
  myConst1:1
  myConst9:1
  myConst11:1

I'm unaware about why multiplication by a fraction in #define is leading to a different value than expected? And also the value of myConst is varying continuously for example, some value I observed are -254802840 -1713343480 466029496. And also myConst1,myConst9 and myConst11 are giving me 1, which isn't logical according to me.
Any explanation regarding the same is most welcome.
Answer: The #define has no role in this; the problem is that you are passing a double when printf is expecting an int (due to the %d specifier); technically that is undefined behavior, so anything is allowed to happen.
Now, if this was built on 32 bit x86 probably you would get an int reinterpretation of the lower 32 bits of you float, since all parameters are normally passed on the stack. But I guess this is running on 64 bit x86, where parameters are passed in different registers if they are floating point. In this case, printf is just reading whatever garbage happens to be in some register at this time, there's not much logic involved in those values.
Notice that almost any compiler can warn you about this kind of mismatch between format string and values actually passed. For gcc and clang I suggest you to enable this warning (-Wformat, included in -Wall, which you should definitely enable), and possibly mark it as an actual error (-Werror=format), as there's no legitimate reason to do a broken printf like this.
Comment: You're right. But the whole point started at a GEL file of CCS where I'm comparing a value in a condition 'if(TIMER_TICS_FOR_100MS_9<(t1+(P2-P1)*419+419-t2)<TIMER_TICS_FOR_100MS_11)' where TIMER_TICS_FOR_100MS_9 and TIMER_TICS_FOR_100MS_11 are '#define TIMER_TICS_FOR_100MS (419*40)
#define TIMER_TICS_FOR_100MS_9 (TIMER_TICS_FOR_100MS*.9)
#define TIMER_TICS_FOR_100MS_11 (TIMER_TICS_FOR_100MS*1.1)' and middle part of condition resolved in to 16893 (which is int) and other two are double and from logs I can see the 16893 well in limits but need a type converted, any idea on this front.
Comment: even after converting them (t1, t2, t3, t4, P1, P2, P3, P4) to float, there is no use. CCS (Code Composer Studio) console output is __(t1+(P2-P1)*419+419-t2):16938.0
(TIMER_TICS_FOR_100MS_9):15084.0
(TIMER_TICS_FOR_100MS_11):18436.0__, which well lies in the range but it is not entering inside the if.
Comment: `a<b<c` does not do what you think it does... In C, it is interpreted as `(a<b)<c`, where `a<b` is 1 if a<b and 0 otherwise.
Comment: Yes, in my case a<b will return 1 and c (__18436.0__) is greater than 1, which should resolve to 1 finally. Note: This logic is in GEL script, which is C-like expression based language, I wonder if any limitation in it ?
Comment: Ops, I thought it was some kind of C IDE... Unfortunately I have no experience in GEL, so I cannot help you with that. If you have doubts specific to GEL you should open another question and tag it appropriately (the fact that the language is C-like isn't really relevant to obtain meaningful answers, especially on finer points like these).
Comment: Hey @Matteo, found out the issue, 1. The condition should be specified as **(a<b) && (b<c)** and 2. The type conversion need to performed if they're int (__as GEL is not giving any error/warning message about the same__). Thanks & appreciate your help.


Uncertainty in measurement error

Question: Imagine having a list of positions $\mathbf{x}$ and two different systems trying to estimate $\mathbf{x}$. One system is more precise than the other, and it will be used as ground truth. When measuring a position, it gives a result
$$
\mathbf{\alpha_i}\sim \mathbf{N}(x_i, 1).
$$
The second system is less precise, it gives a measurement
$$
\beta_i\sim\mathbf{N}(x_i, 3).
$$
The first system is considered to be ground truth because it is the best system we have. We now want to evaluate how well the second system works. To do this, we measure a large number of positions independently with both systems, take the difference between the results of the first and second system, and compute the percentage of differences which is smaller than a cut-off, e.g. smaller than 6.
Here is a computer simulation of the problem written in Python:
import numpy as np
import matplotlib.pyplot as plt

real = 100 * np.random.rand(100000)
gt = [np.random.normal(loc=x, scale=1) for x in real]
estimates = [np.random.normal(loc=x, scale=3) for x in real]

diff = np.array([np.abs(x - y) for x, y in zip(gt, estimates)])

plt.hist(diff, bins=100, density=True)
plt.show()

len(diff[diff < 6])/len(diff)
# Out: 0.94366

And the question is: What is the uncertainty of 0.94366 the uncertainty in the ground truth? Essentially, how do I compute the uncertainty of the area under the curve of the histogram above, to the left of $x=6$.
Note that the estimates, the values of the second system, are what they are. Even though they follow an error distribution, this distribution is not known and we don't wish to take it into consideration since the very point of this exercise is to find out how well the second system works.
Comment: @whuber sorry, I missed your comment. Imagine you want to sell a system to a customer. That customer wants to know what performance he can expect. You tell him that "95% of the measurements will have less than 1 cm error". In order to compute the error of the measurements done by the system, you need an even better system. That better system is the "ground truth". I'm asking this question because I'm concerned about the effect of having a "ground truth" that also has an error. I wonder what that means for the fraction of errors less than a threshold.
Comment: @whuber It's not just that you've told the customer to expect that performance. In fact, you are legally obligated to provide a system with that performance. This makes it even more important to understand what the effect of having a "ground truth" system with an error is, esp. if there are problems with this approach that could systematically affect whether the system performance beats the threshold.
Comment: I'm lost here, because I cannot see anything in the question that models what you are talking about.  Your two systems are perfectly *accurate* but have different *known* levels of *precision.*  The disconnect between what you are asking and what you stipulate in your model leads me to suspect that your model does not correctly express what you know.
Comment: @whuber If I'm not making sense it is because I don't know better. Yes, I know the two measurements exactly. But I also know that they are not exactly correct, they are merely realizations of a random variable. In my model, $x_i$ is the actual true value (as measured by a perfect system) and $\alpha_i$ and $\beta_i$ are the corresponding measurements by the two systems. That's as far as I've gotten with this problem. Do you understand the problem enough based on my previous comment to be able to suggest how I can think more fruitfully about the problem? Any clue would be much appreciated.
Comment: I am wondering whether you truly know the standard deviations of the two measurement systems (you specify that they are 1 and 3) or even whether you know both are accurate.  Could you tell us what the bases of those suppositions are?
Comment: @whuber In order to find that out, the typical method is to apply the same method again: we get a third system which is even better than the other two (it is possible by e.g. putting reflexive stickers on the object we're measuring, but in normal conditions we cannot use it). We assume that this is perfect, and estimate the standard deviation of the other two systems based on this. Then we assume that the standard deviations remain the same in normal working conditions. If we can still reason about the problem with the thresholds without knowing the deviations that would be interesting too.
Comment: @whuber Actually, the systems are calibrated against known measurements to remove systematic error so I think that it is correct to say that this is only matter of precision (and regardless of what the exact precisions are, one system is definitely known to perform much better than the other.)
Comment: If you're just interested in computing the area under your histogram, you could just [fit a function](https://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html) to the data and integrate that function in the desired interval
Comment: @Rickyfox Sorry, that was a mistake. I meant to ask, if I compute the area under the  histogram, what is the standard deviation of that area?
Comment: Have you considered repeating your computer experiment, say 100 times?
Comment: @Estacionario The simulation is just an example. The real life scenario is that I have two hardware systems. I want to evaluate the performance of one hardware system by using the other as ground truth. I cannot repeat the real-life physical experiment 100 times.
Comment: @C.E. what do you mean by the standard deviation of the area under the histogram?
Comment: @Rickyfox Each time I run the simulation, I'm going to get a new value for the area under the histogram for which $x < 6$ (or some other cut-off.) One time maybe it is 93.1, another 95.2, another 94.7, etc. So what is the standard deviation of those values if I were to run the simulation a hundred times? Since I cannot run my physical experiment a hundred times I would like to know, if possible, with some reasonable assumptions, what is the uncertainty (the standard deviation) of this area measurement when I run an experiment.
Comment: Could you explain what you mean by "ground truth" and what the point of counting small differences might be?
Answer: I'm not sure if this is what you aiming for but this is
one approach that might be suitable.
Start with modeling the over 6 error as a probabilistic event.
I.E, lets say that for each measure event $e_i$ there is a $p\in [0, 1]$ probability that true value $x_i$ minus measured value $y_i$ will be lower than 6.
According to this formalization, we are looking at a Bernoulli random variable with parameter $p$, lets name this RV $X$.
Now, in order to estimate $p$ using data we can use maximum likelihood.   This is a common method for parameters estimation, see this for example.
As for the uncertainty, we get it for free from the Bernoulli random variables variance definition - $VAR(X)=p*(1-p)$.

Uncertainty in measurement error


Pointer to C++ 2D Array

Question: I'm porting some C++ code to Actionscript 3 and just had a small question I'm confused about.
In one function, one of the parameters is  int* myPtr.  myPtr is the address of an element of a 2D-Array, &my2DArray[x][y].  x and y are also parameters of the function.  I'm just a little bit confused with what is being accessed when the code accesses, for example, myPtr[1]. I think this would be the next element in my2dArray, but I'm not sure if this would be my2DArray[x+1][y] or my2DArray[x][y+1].  Thanks for any help.
Additional info:
my2DArray is created by:
//initPtr is a int*, auxPtr is a int*, as is temp1
initPtr = (unsigned int *)NewPtr(
      sizeof(unsigned int) *
      X * Y);
  }

  auxPtr = initPtr ;

  for (i = 0; i < X; i++) {
    temp1 = auxPtr + i * Y;
    my2DArray[i] = (short *)temp1;
  }

unsigned char* NewPtr(
  int size)
{
  return ((unsigned char*)calloc(size, sizeof(unsigned char)));
}

Comment: Be careful: initPtr is allocated as an array of unsigned ints, yet it is being accessed as shorts through my2DArray.
Answer: It would be my2DArray[x][y+1].  C++ uses row-major order for multidimensional array indices.  That means that large jumps in memory are changes in the left-most index, and single element jumps are changes in the right-most index.
Answer: If myPtr truly is a pointer to an arbitrary cell in the array, then, unless you know the layout of the 2D array, you can't precisely. It is probably the next one in the row (assuming a standard C++ 2D array, but it might be down a row (if it were at the end of the row). Or, if the array is implemented as a 1D array of 1D arrays, then it might not even be a legal address (were it at the end of the row).

Pointer to C++ 2D Array


How to bootstrap a custom script and retrieve the session object?

Question: I'm writing a custom script that requires a proper Drupal 8 user session to work. This is the current version:
<?php

use Drupal\Core\DrupalKernel;
use Symfony\Component\HttpFoundation\Request;

$drupal_root = __DIR__;
define('DRUPAL_DIR', $drupal_root);

// Specify relative path to the drupal root.
$autoloader = require_once DRUPAL_DIR . '/autoload.php';
$request = Request::createFromGlobals();

// Bootstrap drupal to different levels
$kernel = DrupalKernel::createFromRequest($request, $autoloader, 'prod');
$kernel->boot();
$kernel->handlePageCache($request);
$kernel->prepareLegacyRequest($request);

// get session handler and retrieve session details
$session_handler = \Drupal::getContainer()->get('session_handler.storage');
$session_id = $session_handler->getId();
$logged_in = (bool) $session_handler->read($session_id);

// inspect values
var_dump($request->getSession());
var_dump($session_id);
var_dump($logged_in); exit;

I'd expect $request->getSession() to return something, $session_id to be the Drupal's session id, and the $logged_in variable to be truthy. Instead I keep getting this output:

NULL string(0) "" bool(false)

Update 1: I executed this on a browser, which I have an active session with my Drupal. This script is placed in a subfolder of the Drupal installation, so it should not be a cookie path issue.
Update 2: Added var_dump($request->getSession()) to the script.
Comment: Does the context you're executing this script in have an active session (and a method to carry it through to subsequent requests, like a browser)? For example if you're executing this via the CLI, that won't happen
Comment: @Clive: I executed this on a browser, which I have an active session with my Drupal. This script is placed in a subfolder of the Drupal installation, so it should not be a cookie path issue.
Comment: Is `$request->getSession()` also empty?
Comment: `var_dump($request->getSession());` returns `NULL`.
Answer: This is the code I came up with in the end. It works.
Feel free to suggest a more concise version :-)
<?php

use Drupal\Core\DrupalKernel;
use Symfony\Component\HttpFoundation\Request;

$drupal_root = realpath(__DIR__ . '/../');
$autoloader = require_once $drupal_root . '/autoload.php';

// Make sure Drupal operates correct when handle request instead of internal server error and twig error
chdir($drupal_root);

$kernel = new DrupalKernel('prod', $autoloader);

// Note: The session_name is generated from the string "{domain}/{script basepath}".
// Need to override the script filename and script name before creating request.
$_SERVER['SCRIPT_FILENAME'] = $drupal_root . '/index.php';
$_SERVER['SCRIPT_NAME'] = '/index.php';
$request = Request::createFromGlobals();
// Note: may restore script_name and script_filename after this line.

// This handle process would chain up and initialize all services
// in container, include the session handler.
$kernel->handle($request);

// now we can use session normally
$session_handler = \Drupal::getContainer()->get('session_handler.storage');
$session_id = $session_handler->getId();
$logged_in = (bool) $session_handler->read($session_id);

// restore app
chdir(__DIR__);

Comment: This has a few issues: 1) Drupal generates 500 error + twig error. 2) Generates page not found `/app/index.php`. 3) The session is not unserilized after read (no object). 4) the app is in a wrong state (you made not for that).


How essential is a purpose built Triathalon bike?

Question: I am a experienced marathon runner who is looking to get into Triathlons (long course and half -- ~50 mile bike), hoping to work up to an Ironman.
I have an older, but working, road bike that gets me around very well (to which I could attach Tri-style handlebars), so my question is:
how essential is having a purpose built triathlon bike if the goal is just to complete the race? Can I get away with a standard road bike?
Answer: The only question that matters here is what the regulations say for the competition(s) you're going to be entering. If your bike meets the regulations and you can complete the distance on it, then that's all that's essential to complete the race.
Answer: There is absolutely nothing wrong with riding the best bike you have in your events.  In beginner classifications it is not unknown to see rigid mountain bikes fitted with road tyres and clip-on aerobars.
Sure its not ideal, but if you're getting into a sport, there's no need to go silly buying gear.   On the other hand if you can find someone who did go a bit bonkers buying stuff and then left the sport, that's where you can gear up for cheap.
And the best feeling EVER is completing the cycling bit faster than someone on a better bike!  I've did a road race on my ~19 year old aluminium road bike, and while I was in the bottom 20% of finishers, I still managed to keep up with some better bikes.
Just remember, if you finish dead-last, you're still doing better than everyone who didn't try.


css border fades out

Question: I have a border created by the css below.  .left is a div and and .left-menu is an ASP.NET menu control nested inside .left.  Something I don't understand happens when the border is rendered in IE, FF and Chrome.  The border appears initially and then slowly fades away and disappears completely.  There is no javascript running.  It doesn't appear to be an animation and is NOT a gradient.  The border appears initially and then immediately begins to fade into the white background and then completely disappears.  The whole thing takes about 2-3 seconds.  If I change the color to something like #ff0000, the border remains and does NOT fade away.
.left
{
    min-height: 400px;
    padding: 0px;
    margin: 0px;
    padding-top: 50px;
    border-right: 2px solid #f1f1f2;
    float: left;
}

.left-menu, .left-menu ul
{
    width: 100% !important;
}

.left-menu ul li a
{
    padding: 10px;
    padding-left: 25px;
    padding-right: 50px;
}

Comment: Fades and disappears as in animates away, or it looks like a gradient of some sort? If it's the former, we need to see your jQuery or other JS code. If it's the latter, a screenshot or link to the site/example would be very helpful.
Comment: Sounds like you've got some javascript running wild on your div or on global borders. Can we see what's inside your head tag?
Answer: So I discovered the problem is, in fact, rogue javascript generated by the ASP.NET menu control.  The DisappearAfter property is being manipulated and actually fades out the border immediately after rendering.  Oddly enough, this functionality is buggy, since an unrelated border on the div above the menu is removed as well.  Also, changing the border color from the menu mouseover color to another color persists the border.

css border fades out


How to quickly tell what a URL maps to?

Question: I'm in a new position working on an existing Drupal 7 site that I didn't help build. None of the original devs are here. Amongst other modules, the site makes uses of panels, panelizer, aliases, and redirects.
I get requests from marketing to update or change a page, and they include a URL. The URL is often an alias, but sometimes it can be a panels page, or a drupal entity "overriden" by panelizer.
Is there an easy way to see what exactly a URL maps to when I get a URL from marketing?
Comment: What version of Drupal are you running and are just wanting to know the node ID? If so, see https://drupal.stackexchange.com/questions/6638/how-to-get-a-node-id-from-the-url and try `drupal_lookup_path()`.
Comment: @Christia a node ID is good, if I already know it's a node. But it could be a panels page or something panelized, like a taxonomy page. Drupal is obivously able to go through the appropriate objects to construct the page; I would like to tell me what it's doing so I can go to the object itself and start making changes.
Comment: Panel pages are still nodes but honestly, why don't you just go to the URL that marketing provides and edit it from there?
Comment: @Christia I go to the url that marketing provides and click "new draft" (forgot to mention, we also have workbench). The thing I need to change is not there. Since this is panelized, I click on "customize display". I find the field, and move it. But other nodes of the same kind have not changed. What I need to have done was go to the content type's "Full Page Override" and then "Content", to change all the content types.

Why do I have to play hide and seek just to change a page? Why can't drupal just tell me up front where the parts I need to change are stored?
Comment: With panels, that's the way you have do it. Drupal is a little different but you'll get used to it. :)  When you get used to that website, you can just go to structure > blocks or structure > views to find what you're looking for but what you're doing is still probably the easiest way. Edit: Deleted my comment so I could add to it.
Comment: @Christia Not used to it after 6 years. It's getting more than a little frustrating at this point.
Answer: There is no easy way. You have to have deep knowledge of the site's content to know how to edit what appears on the page. This is only obtained through poking around the site or asking a knowledgeable colleague.

How to quickly tell what a URL maps to?


Questions about Varnish file storage engine

Question: I have Varnish cache in front of some websites with the following storage engine configuration:

Storage: file
Size: 130GB

The server where it is installed has the following configuration:

RAM: 7.5 GB
SSD 1: 15GB (System)
SSD 2: 150GB (130GB for Varnish storage)
CPU: 4

Summary, it is a c4.xlarge on AWS.
Currently, Varnish is currently using 123GB of the allocated storage (130GB).
So, what will happen when Varnish use all the 130GB of allocated storage?

Is Varnish going to delete the storage.bin file and create a new one?
Is it gonna stop working?
Or what is going to happen?

Thank you for your help.
Comment: Varnish is a cache.  It'll do what caches do.  Expire stuff.
Answer: When the storage space is exhausted, Varnish won't create a new storage file or stop working. I will simply start evicting (i.e. 'nuking' using the Varnish terminology) objects cached in the storage when space for newer objects is required. You can check when that's happening looking at the MAIN.n_lru_nuked counter in varnishstat.
Comment: Thank you, you are right. I am currently checking varnishstat.


How does the enterState() Transitions work? (Slick2D)

Question: I have seen the transition thing with the enterState() function, but there you can also specify a out and in transition. How does this work?
Comment: Please clarify what you mean, by "how does this work?". Are you asking how to utilize the enter and exit state methods?
Comment: I am asking how to use it and what it does.
Comment: Has my answer helped you or am I still misunderstanding your question? If it helped, it would be nice to accept my answer, otherwise please expand upon what you wish to accomplish with transitions a bit more.
Answer: The slick2d game state methods of enterState() lets you set a transition between two gamestates. Let's say you have a splashscreen, and a Main Menu, where your New Game/Options/Exit choices reside. If you wish to have a fade effect between the splashscreen to the main menu, as most games tend to have a fade to black transition, you can call this in your update method after a certain time interval has passed in your splashscreen game state:
<code>statebasedgame.enterState(*state id you wish to enter*, new FadeOutTransition(), new FadeInTransition());
</code>
What this is telling the slick engine to do for you, is to fade out (exit effect) which will be a new FadeOutTranstion() class type, switch to the new game state class, and the fade in (enter effect) will be a new FadeInTranstion() class type.
There are a lot of transition effects prebuilt into slick, and just as well, you can even build your own if you want by extending the Transition class.
Good example of a personal made effect: http://slick.ninjacave.com/forum/viewtopic.php?t=1008
More examples of the transition effects that can be used: https://github.com/ariejan/slick2d/blob/master/src/org/newdawn/slick/tests/TransitionTest.java


Find a vector orthogonal to both $(1,2,-1,3)$ and $(1,0,-2,3)$

Question: So with my previous knowledge that the dot product of two orthogonal vectors should be equal to zero, I was wondering if the vector $(0,0,0,0)$ in $\mathbb{R}^4$ can be considered a vector orthogonal to the two vectors above, since its dot product with any other vector in $\mathbb{R}^4$ would be zero. Is my approach or intuition correct or logical?
Comment: Please, if you are ok, you can accept the answer and set it as solved. Thanks!
Answer: Yes, technically, the zero vector is orthogonal to both of your vectors. If this is a homework question, though, I'm certain that your professor intended for you to find a non-zero vector orthogonal to the two given vectors (which you can do fairly easily in this case by eyeballing the vectors, or more systematically using  e.g. Gram-Schmidt).
Answer: You can use Gram-Schmidt as suggested by @user7530. But you can also observe that any vector of the form $(2a,-a,3b,b)$ for $a,b\in\mathbb R$ is orthogonal to $(1,2,-1,3)$. And the dot-product of $(2a,-a,3b,b)$ with $(1,0,-2,3)$ is $2a-3b$. So choosing $a=3$ and $b=2$ gives $(6,-3,6,2)$.

Hope this helps.
Answer: Call the to-be-found-unknown-vector orthogonal to given vectors as $(x,y,z,w)$. Being orthogonal to a specific vector, say, $(a,b,c,d)$ means $ax+by+cz+dw=0$, (dot-product condition) and one more equation of the same kind for another orthogonality.
The problem boils down to solving a system of 2 equations in 4 variables; being homogeneous, zero vector is a solution, but as another answer points out, most likely you are expected to  find a non-zero solution.
Answer: Also note that you can solve this purely "3-dimensionally". Using the cross-product for vectors in $\mathbb{R}^3$, you see that $(-4,1,-2)$ is orthogonal to both $(1,2,-1)$ and $(1,0,-2)$. Now clearly $(-4,1,-2,0)$ is a solution to your problem.
Answer: The null vector is not a good choice since it is trivially orthogonal to every vector and no so much useful (it should be excluded by the problem set).
The problem to solve is to find a vector $v_3(a,b,c,d)$ such that $v_3\cdot v_1=0$ and $v_3 \cdot v_2=0$, that is equivalent to solve the system
$$Mv_3=\begin{bmatrix}
1 & 2 & -1&3\\
1 & 0 & -2&3
\end{bmatrix}\cdot\begin{bmatrix}
a\\b\\c\\
d
\end{bmatrix}=\begin{bmatrix}
0\\0\\0\\0
\end{bmatrix}$$
which has $\infty^2$ solutions.
You can also solve by inspection choosing a first vector orthogonal to $v_1$ keeping some free parameter and then making it orthogonal also to $v_2$, for example
$$v_3=(-2x,x,3y,y) \implies v_3\cdot v_1=0$$
then we need
$$v_3\cdot v_2=0\implies -2x-6y+3y=0\implies 2x=-3y \implies x=3 \quad y=-2 $$
thus
$$v_3=(-6,3,-6,-2)$$