SVM Hyperparameters Tuning

Question: I am using SVM classifier to classify data,
My dataset consist of about 1 milion samples,
Currently im in the stage of tunning the machine ,
Try to find the best parameters including a suitable kernel (and kernel parameters),
also the regularization parameter (C) and tolerance (epsilon).
My corrent approach is using a blackbox global optimization algorithm to find the best parameter set, i use k-fold cross validation as a minimazation function,
The optimization algorithms i have in stock: Cma-ES, Simplex, HillClimbing, Down hill, GA and Simulated Annealing.
The problem is that the crossvalidation is a very slow process which makes all of the algorithms run for hours or even days..
and from what i know crossvalidation is the only option i have..
I want to know if it's possible to run the optimization on only a small part of the dataset to improve the runtime? , or are the parameters i will find will not work well on the full, larger dataset.
Also are the kernel parameters have any correlation with the C parameter? is it possible to firstly tune the C parameter and only after one was found continue and optimize the kernel parameters?
I know hyperparameter tuning is a very common issue so how is that im feeling there is no "clean" solution for this problem..
It must be a way that makes it possible for large datasets,
Ill appreciate any kind of help and advices
Comment: I thought that if ill use it one line after writing "CrossValidation" it will be clear;)
Comment: Closely related: http://stats.stackexchange.com/questions/193306/optimization-when-cost-function-slow-to-evaluate/193310#193310
Comment: You can compute the whole regularization path for a SVM in one go per [Hastie et al 2004](http://www.jmlr.org/papers/volume5/hastie04a/hastie04a.pdf), the only remainder are the kernel hyperparameters.
Answer: My experience with SVM does not include 1M datasets. I work usually up to 50K datasets. So caveat emptor.
1) there is no way to decouple gamma from C. I answered your other question on that Are the kernel parameters and the regularization parameter correlated in SVM?
2) There is no epsilon for classification. This is a hyperparameter for regression only
3) on datasets up to 50K I found that PSO work better than simplex, and SA. I hav not tried CMA or GA. There is no downhill or hill climbing unless you use some approximations to the leave one out error there is no closed expression for the gradient.
4) You don't need to use a low variance CV -  I found that 2-fold is good enough. If you cannot afford the computational time of the 2-fold, than you could use a subsample, but in my experience that yields worse results.
5) IN MY EXPERIENCE, the error surface for the hyperparametrs is somewhat smooth - not convex, but smooth - there are no deep and narrow regions of low error that are worth spending a lot of computational time searching for them. Provided you are not selecting hyperparameters in a bad region of the error surface, there is no point in probing the surface too much. On that note, I would suggest a 5x5 grid search followed by another 5x5 grid around the minimum of the first grid - this should be enough.  You end up probing  the error surface 50 times,  and you probably get a result as good as any black box optimization with the same limit on the number of probing. And it is much easily to parallelize.
Comment: Thanks for the great answer, in 5 , you wrote 5x5 gridsearch , do you mean by that 5.parameters and 5 optional values each?
Comment: the two hyperparameters are C and gamma (for a RBF kernel). 5x5 means divide the range of C in 5 and the gamma in 5.
Answer: Your questions

"I want to know if it's possible to run the optimization on only a small part of the dataset to improve the runtime?"

As usual: "depends on the data". If you do this, try to make the smaller sample so that it has a similar distribution of labels ("stratified sampling"). This is a valid approach and can work well if your subset is sampled well.

"Also are the kernel parameters have any correlation with the C parameter? is it possible to firstly tune the C parameter and only after one was found continue and optimize the kernel parameters?"

I cannot give you a perfectly qualified answer here, but i know from personal experience:
The C-parameter can have a big impact on the runtime (assuming a fixed epsilon here). So if you did a full run once, you could at least remember some rough bounds for the C-value per kernel. Maybe you could avoid some "obviously unnecessary" combinations this way.
Bayesian Optimization
You could try Bayesian optimization [*]. The speedup will be greater, the more hyperparameter combinations (Kernal / C / epsilon) you have. The more combinations, the more crossvalidations have to be performed.
Bayesian optimization attempts to minimizes the number of evaluations and incorporate all knowledge (= all previous evaluations) into this task. Minimizing the number of evaluations also means fewer crossvalidations. So this is probably what you want.
There is even an implementation available (**) with Jasper Snoek, first author of the mentioned paper, involved (i do knot if the other authors are involved as well).
*: https://arxiv.org/abs/1206.2944
**: https://github.com/JasperSnoek/spearmint
Disclaimer: I have studied this for the last few weeks and am really enthusiastic about this ;) I also have experience with SVMs, unfortunately i have not tried both BO and SVM in combination.
Answer: Have you considered Random Search for the Hyper-Parameters, I know they have it in  Scikit-learn. I also found the paper that explains it by James Bergstra ,Yoshua Bengio. From my understanding the method works best when you have a low affective dimention compared to the number of hyper parameters.
Comment: Tanks but i honestly don't believe that random search can do better then one of the optimization algorithms i have mantioned , espiacially genetic algorithms in this task.
Comment: Have you tried cluster computing. I know Spark has SVM but I don't think they solve in in the primal form so you will not have access to the RBF kernel.
Answer: My understanding is that for non-linear SVM kernels training time scales as a square of the number of training examples, i.e. if you double the training examples training time will quadruple. Add in cross validation and your 1M training examples and it is not surprisingly that training is prohibitively computationally expensive.
If your training samples are unbalanced (i.e. for a classification problem a large majority of outcomes belong to one class), sub-sampling the majority class to reduce both class imbalance and therefore the total training set size might be optimal. In this case you're discarding data that may not be needed, but still producing a representative training set.
If your training samples are balanced, then cross validation on a reduced-size training set really defeats the purpose of cross validation - namely validating that the train/test split for each fold is representative of the distribution of the entire 1M record training set. Rather than reduce the size of the training set, I'd reduce the number of folds - or simply not do cross validation, but instead use a proportionally large (and therefore hopefully representative) test set, e.g. 20-40% of your 1M samples.
In general a Bayesian Optimisation method is much more efficient than either grid or random hyperparam optimisation as it is a guided search in hyperparam space. There are a variety of Bayesian and a few non-Bayesian hyper-parameter optimisation libraries to choose from. Have a look at:
http://fastml.com/optimizing-hyperparams-with-hyperopt/
I've used hyperopt with a lot of success. SMAC looks very interesting as it incorporates cross validation and will not necessarily validate every fold if it determines that a particular candidate set of hyperparams will not score better as compared to previous folds.
If you do want to use SVM with cross validation on 1M training examples then GPUs might be able to do the heavy computational lifting you require. Check out:
http://mklab.iti.gr/project/GPU-LIBSVM

SVM Hyperparameters Tuning


Find the MGF of X

Question: If $f(x) = (k + 1)x^2$ for $0 < x < 1$. Find the moment generating function of $X$.
Do i do the integral from o to 1 of the above function?
Comment: Why are people downvoting this without comment? This is one of the few introductory problems here on MSE where the OP has at least provided an idea that they have.
Answer: The moment generation function $G_X(t)$ is the expectation of $e^{Xt}$. This is
$$\int_0^1 e^{xt}(k+1)(x^2)\,dx.$$
Note that you will first want to find $k$.
Answer: You first need to determine the $k$ by forcing your distribution to integrate to one over the given interval of $x$, i.e.$$ \int_0^1 \left( k +1 \right) x^2 dx = 1 $$
Solve for $k$ and then proceed by taking the expectation again over that same interval after having plugged in the value of $k$.
Note that all expectations are taken over the probability space of the density function and the MGF is a special case of an expectation.
Comment: Yes $k=2$ and then your density function becomes $3x^2$ for $ 0<x<1$ . Integrate over $[0,1]$ now and you will get $1$. By expectation I meant the MGF though, so the expectation you need to take is $$ 3 \int_0^1  e^{tx} x^2 dx $$ And that will give you the moment generating function of that distribution.
Comment: @Xiong You can try integrating by parts twice.

Find the MGF of X


How related is the distribution of primes to the Riemann Hypothesis?

Question: I do not grasp all concepts of the Riemann Hypothesis (better yet: as a layman I barely grasp anything). However, I understand that there is a certain link between the Riemann Hypothesis and prime numbers and their distribution.
My question is:

Would a "formula" or other system that enables you to calculate the distribution of prime numbers enable mathematicians to solve the Riemann Hypothesis?
Are the directly linked, or does solving prime number distribution not automatically solve the Riemann Hypothesis?
Comment: I can't remember if there's actually an answer to your question in it, but I thought you might be interested in [this in-progress book](http://wstein.org/rh/) by Barry Mazur and William Stein.
Comment: @DylanMoreland: thanks for that link!
Comment: Just perusing that book-in-progress, I noticed this (p. 15): <<<>>> However, the first million digits of the $10^{10^{10^{10}}}$th prime are readily computed -- but I don't think that qualifies it as a "known" prime.
Answer:
would a 'formula' or other system that enables you to calculate the distribution
of prime numbers enable mathematicians to solve the Riemann Hypothesis?

There is an exact formula, known as "the explicit formula" of Riemann, for the prime number counting function $\pi(n)$  in terms of the zeros of $\zeta(s)$. (Really it uses a minor modification of $\pi(n)$, extended to positive real values of $n$, but the idea is the same.)
The explicit formula displays an equivalence between asymptotics of the prime number distribution and location of zeros of $\zeta(s)$.  Knowledge of the real part of the location of the zeta zeros translates into knowledge of the distribution of primes.  The closer the zeros are to the line with real part $1/2$, the better the control over the distribution of primes.
This is all in Riemann's paper approximately 150 years ago, that introduced the Riemann hypothesis. The prime number theorem is equivalent to a demonstration that no zeros have real part equal to $1$, which was done at the end of the 19th century.
The infinitude of primes is equivalent to the pole of $\zeta(s)$ at $s=1$, as was shown by Euler.
The difficulty in finding all the zeros is not the lack of a formula, but that the explicit formula relates two complicated sets without proving anything about either set individually. To restrict the location of zeta zeros through a formula for prime numbers, the prime formula would have to be strong enough to estimate $\pi(n)$ with an error of order $n^{1-\epsilon}$ for a positive $\epsilon$, which would be considered an incredible breakthrough. Using Riemann's explicit formula it would be possible to take any argument about the prime distribution and translate it relatively easily into an argument about the zeta function, so it's not the case that formulations in terms of primes are likely to be any more amenable to proof than talking about the zeta zeros.  In fact it is usually easier to start from the zeta function.
Comment: The explicit formula and the theory of the zeta function provide a tight relationship between the distribution of primes and the location of zeros of $\zeta(s)$.   A "formula for the prime number distribution already exists" in the sense that one can use this to compute the exact number of primes less than N  without finding them one by one.  But to prove RH you would need some additional facts about the primes (not in relation to zeta), or about the zeta function (not in relation to primes), or another relationship between primes and the zeta function in addition to the explicit formula.
Comment: Sorry, I do not understand this - but that is okay. Just to confirm: the answer would be "No, getting this formula does not help because it already exists. Calculating the prime number distribution with a formula does not solve the Riemann hypothesis.". Is that in simple words what you say?
Comment: For example, a proof that Li(x) is a good approximate formula for $\pi(x)$ would solve RH, but right now it is only known to be a consequence of RH:   if RH is true then this approximate formula for the prime number distribution has the expected level of accuracy.
Comment: You wrote "The closer the zeros are to the line with real part 1/2, the better the control over the distribution of primes" ? What is meant by control?
Answer: As lhf writes, there is a strong link between the error estimate in the prime number theorem and the Riemann hypothesis.  Indeed, RH is equivalent to a certain bound on this error estimate.
More precisely, the prime number theorem states that $\pi(x)$ (the number
of primes $\leq x$) is asymptotic to $\mathop{\mathrm {Li}}(x)$, and the Riemann hypothesis is equivalent to the statement that the error in this approximation is bounded (for large $x$)
by $x^{1/2 + \epsilon}$ for any $\epsilon > 0$.
Answer: The asymptotic prime number distribution has been known for over a century now. The Riemann Hypothesis is about the error term in that asymptotic equation. In this sense, they are very closely linked.


Problems with navigation from jsp to controller

Question: I can't navigate to my controller from the JSP. What am I missing?
I'm having an error status 404.
This is my index.html
<!DOCTYPE html>

<meta charset="ISO-8859-1">
<title>WELCOME PAGE</title>

WELCOME TO NUTS

<form action="remember.html">

    <input type="submit" value="GET STARTED">
</form>

This is my controller -----------------------------------------------
@Controller
public class ProductController {

     @RequestMapping(value = "/remember")
      public String list(Model model) {
         ServiceFacade facade = new ServiceFacadeImpl();
         List<Product> products = facade.getAllProducts();
         model.addAttribute("products", products);
         return "/jsp/LoginPage";
     }

}

This is the web.xml--------------------------------------------------
   <?xml version="1.0" encoding="UTF-8"?>
    <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">
      <display-name>NutsAboutCandyWebProject</display-name>
      <welcome-file-list>
        <welcome-file>index.html</welcome-file>
        <welcome-file>index.jsp</welcome-file>
      </welcome-file-list>

          <servlet>
            <servlet-name>appServlet</servlet-name>
            <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
            <init-param>
              <param-name>contextConfigLocation</param-name>
              <param-value>/WEB-INF/servlet-context.xml</param-value>
            </init-param>
            <load-on-startup>1</load-on-startup>
          </servlet>
</web-app>

This is my servlet-context.xml ----------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx"
  xsi:schemaLocation="http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.2.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd">

  <mvc:annotation-driven />

  <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
    <property name="prefix" value="/jsp" />
    <property name="suffix" value=".jsp" />
  </bean>

  <context:component-scan base-package="com.nutsaboutcandywebproject.controller"/>

  <bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost:3306/db_nutsaboutcandy"/>
    <property name="username" value="root"/>
    <property name="password" value=""/>
    <property name="validationQuery" value="SELECT 1"/>
  </bean>

  <bean id="mySessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
    <property name="dataSource" ref="myDataSource"/>
    <property name="packagesToScan">
      <array>
        <value>com.nutsaboutcandywebproject.model</value>
      </array>
    </property>
    <property name="hibernateProperties">
      <value>
        hibernate.dialect=org.hibernate.dialect.MySQLDialect
      </value>
    </property>
  </bean>

</beans>

EDIT 1:
web.xml configuration
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" id="WebApp_ID" version="2.5">
  <display-name>NutsAboutCandyWebProject</display-name>

  <servlet>
    <servlet-name>appServlet</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/servlet-context.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
 <servlet-mapping>
    <servlet-name>appServlet</servlet-name>
    <url-pattern>/</url-pattern>
 </servlet-mapping>

</web-app>

servlet-context.xml
  <?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:mvc="http://www.springframework.org/schema/mvc" xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx"
  xsi:schemaLocation="http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd
    http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.2.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd">

  <mvc:annotation-driven />

  <beans:bean   class="org.springframework.web.servlet.view.InternalResourceViewResolver">
        <beans:property name="prefix" value="/" />
        <beans:property name="suffix" value=".jsp" />
  </beans:bean>

  <context:component-scan base-package="com.nutsaboutcandywebproject.controller"/>

  <bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url" value="jdbc:mysql://localhost:3306/db_nutsaboutcandy"/>
    <property name="username" value="root"/>
    <property name="password" value=""/>
    <property name="validationQuery" value="SELECT 1"/>
  </bean>

  <bean id="mySessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
    <property name="dataSource" ref="myDataSource"/>
    <property name="packagesToScan">
      <array>
        <value>com.nutsaboutcandywebproject.model</value>
      </array>
    </property>
    <property name="hibernateProperties">
      <value>
        hibernate.dialect=org.hibernate.dialect.MySQLDialect
      </value>
    </property>
  </bean>

</beans>

Comment: Could it be as simple as `` vs `@RequestMapping(value = "/remember")` ?
Comment: it doesn't work either
Comment: Could You post Your other files? Configuration, LoginPage etc.
Comment: Show your configurations.
Comment: sorry for the delay guys.
Answer: the value you are returning is return "/jsp/LoginPage";
which means, your controller will be saying, "hey i have done my operation and returning you the view page LoginPage" so now, your view resolver's work is to add the appropriate suffix and render it.
Now,
1. have you declared a view resolver in your servlet-context.xml file?
2. had you declared it, you need not have to specify "/jsp/" this you can add as a prefix in your view resolver.
Also, i see that you are sending action = "remember.html" and your controller is looking for "/remember". both are different. meaning, you have to write something like "form action = "/remember" "
This means, /remember is an end point. not a html page.
Comment: Your last paragraph is false if they are using extension mapping for their servlet.
Comment: my LoginPage is placed inside a folder named jsp so that I have to put a prefix.
Comment: when you specify a request mapping, you hit an endpoint right? not a static resource right?
Comment: @Erika Is the JSP in `/jsp/jsp/..`. That's where your `ViewResolver` and controller will look for it currently.
Comment: @Erika you can specify that within your view resolver
Comment: `

`  this is how it looks... an example... you have to add that to your servlet-context.xml
Comment: @Erika your request mapping has no method `@RequestMapping(value = "/remember", method = RequestMethod.GET)`
Comment: @Erika once you do the changes, update your code so we can follow through
Comment: i see that the prefix value is not correct it should have been, `/jsp/` and then, your view resolver will be rendering jsp pages and not html pages
Comment: @Erika you have a view resolver that resolves your views to JSP. Your view resolver is actually looking for `/jsp/jsp/LoginPage.jsp`
Answer: Unless you haven't shown it to us, you haven't mapped your DispatcherServlet to handle any requests.
You only have
<servlet>
    <servlet-name>appServlet</servlet-name>
    <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
    <init-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/servlet-context.xml</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
</servlet>

which declares a Servlet, but nothing else.
You need a mapping, something like
<servlet-mapping>
    <servlet-name>appServlet</servlet-name>
    <url-pattern>*.html</url-pattern>
</servlet-mapping>

Comment: I've done this thing a while ago and it results to error 404 at the first run so I removed it..
Comment: @Erika If you remove it (as you currently have), your `DispatcherServlet` **won't handle any requests**. The idea is to do it _correctly_.
Comment: It produces a warning that no mapping was found for HTTP request with URI.........
Comment: @Erika That's progress. Now the `DispatcherServlet` is handling requests, but it doesn't have a `@Controller` handler method to handle the one you sent. What URI is it complaining about?
Comment: my context path is having that warning
Comment: No mapping found for HTTP request with URI [/NutsAboutCandyWebProject/] in DispatcherServlet with name 'appServlet'
Comment: @Erika What pattern did you use in `url-pattern`?
Comment:
appServlet
/*
Comment: @Erika Please read my answer. Mapping to `/*` will handle everything, skipping your ``. Map it to `/` so that it can be the default `Servlet` or map it to `*.html` for extension mapping, which is what your `form` and `@Controller` handler method are expecting.
Comment: still No mapping found for HTTP request with URI [/NutsAboutCandyWebProject/] in DispatcherServlet with name 'appServlet'
Comment: @Erika Edit your question. Add your new configuration below. Clean and redeploy your application.


What would happen if I disconnected a child domain from it's forest domain controllers?

Question: Windows 2003 native mode. One domain controller on-site, read-write, no RODCs. DNS is running on the DC onsite. Servers would be pointing to the DC for DNS and authentication.
How long would the child domain remain functional without it's ability to contact the forest level domain controllers? Could users still log in, up to several weeks after the disconnect, or would they experience Kerberos authentication issues?
Answer: Is this server a Global Catalog server? Is it the only domain controller in the child domain?
If those two are yes, it can be left like that for a while without any adverse effects. However, if it is a no to either than you could see issues really quick.
Comment: Yeah, the remaining DC, (there's only one) for the child domain in question, is a global catalog.

What kind of issues would I see, if it weren't?


How to make Microsoft Bundle output JavaScript not HTML script tags

Question: I am building a ASP.NET MVC4 based jQuery plug-in and I would like to use the standard Microsoft Bundler.
Now normally when I use the bundler the final usage is this:
@Scripts.Render(BundleConfig.jsBundleFile)

With a output like this in debug:

And a single tag to the minified and combined file in the official release.
However, because we are building a jQuery plug-in I want to use the minifier to insert the actual JavaScript from the various files i.e. in DEBUG, when I have Object1 defined in one file and Object2 defined in another then I get this:
/* Content from file #1 */
var Object1 = function() { /* Un-minified content of Object1 */ };
/* Content from file #2 */
var Object2 = function() { /* Un-minified content of Object2 */ };

and in the release I would get the minified content.
From the MSDN I saw that the bundler, by default can't do that. But I am wondering is there an extension that can do this or another minifing library that can? Though I'd prefer to stay with the Microsoft Bundler.
In my own research on how to extend the System.Web.Optimizer I found the CodePlex project of System.Web.Optimization, where it is said it's not open source as of yet, with makes the extending kinda difficult to anyone not really working on it.
EDIT :  So knowing that I can't really expand the System.Web.Optimizer I went with a compromise solution and this is what I ended up using.
In the Controller:
public JavaScriptResult jQueryComponent()
{
    JavaScriptResult ret = new JavaScriptResult();
    ClientSettings Model = new ClientSettings();

#if DEBUG
    List<string> jsFiles = App_Start.BundleConfig.Main.FilesToBeBundledJS;
    StringBuilder bundleBuilder = new StringBuilder();

    foreach (string file in jsFiles)
    {
        bundleBuilder.Append(System.IO.File.ReadAllText(Server.MapPath(file)));
    }

    ViewBag.bundledJS = bundleBuilder.ToString();
#else
    StringBuilder urlBuilder = new StringBuilder("http://localhost");

    int port = Request.Url.Port;
    if (port != -1) {
        urlBuilder.Append(':');
        urlBuilder.Append(port);
    }

    urlBuilder.Append(Scripts.Url(App_Start.BundleConfig.Main.jsBundleFile));

    WebClient wc = new WebClient();
    byte[] raw = wc.DownloadData(urlBuilder.ToString());
    string bundledJS = System.Text.Encoding.UTF8.GetString(raw);

    ViewBag.bundledJS = bundledJS;
#endif

    ret.Script = RenderRazorViewToString("~/Views/Main/jQueryComponent.cshtml", Model);

    return ret;
}

private string RenderRazorViewToString(string viewName, object model)
{
    ViewData.Model = model;
    using (StringWriter sw = new StringWriter())
    {
        ViewEngineResult viewResult = ViewEngines.Engines.FindPartialView(ControllerContext, viewName);
        ViewContext viewContext = new ViewContext(ControllerContext, viewResult.View, ViewData, TempData, sw);
        viewResult.View.Render(viewContext, sw);
        viewResult.ViewEngine.ReleaseView(ControllerContext, viewResult.View);
        return sw.GetStringBuilder().ToString();
    }
}

And in the View (jQueryComponent.cshtml):
@{
    Layout = null;
}

if (typeof jQuery != 'function') {
    throw "This is a jQuery plugin i.e. jQuery needs to be defined first";
}

(function ($) {

    //Adds the bundled JS into the plug-in
    @Html.Raw(ViewBag.bundledJS)

    $.fn.jsPlugIn = function() {
    /*
        Creates a new instance of [MainObject] with is defined in the bundle.
    */
        //uses the bundled JS
        var Main = new MainObject();
        Main.doSomething();

        return this;
    };

}(jQuery));

Any ideas how to improve this are welcome alongside the proper extention to the bundler.
Although works, this is far from ideal and I am well aware of this.
Please do not criticise this approach unless you have an alternative.
Comment: have you tried the Bundling features built in to the Visual studio Web Essentials plugin?  This bundler bundles at compile time instead of runtime, which seems to suit your needs. http://vswebessentials.com/features/general
Comment: @Andrew I am not really a big fan of plug-ins (they are kinda hard to push within the company, unless they are MS ones), but from a quick glanse it seams to me that this creates a separate minified file for the project, with I can include into my project. What I'd prefer is a solution like in the Web.Optimizer were I can update the files (on a deployed project) and have them combine into a single JS file for the client i.e. all the minification is done on runtime and not in the project deployment.
Comment: Why do you need all the .js in a simple javascript for developing a jquery plugin? I've developed many of them, and I can't see the reason. Perhaps instead of trying to do this you must change your way of facing this problem. If I understand why you think you need this, hopefully I'll be able to show you good alternatives. I've never needed it, and have several plugins in production.
Comment: It's not a component like a new scrollbar or anything like that. The customer is paying for the data that the plugin is showing and not the plugin itself. So the JS and ASP.NET are really linked. We need to be able to return nothing if the customer leaves us. The other big reason is that the code is OOP style JS, for example its not unusual for me to have 10 objects in 10 files for one project. This is why I can't use the minifiers own output. I need to incapsulate the code in the jQuery plugin. I also do not want to work with one file which has 10k+ lines.
Comment: @JotaBe I am not really looking for an alternative JS approache (see previous comment as to why) only a way to get the minified JS as a string in C#.
Answer: You can do what the BundleHandler does: Get a bundle based on the virtual path (same one you would use for
    @Script.Render), generate the response and write the contents.
@{
    var context = new BundleContext(this.Context, BundleTable.Bundles, string.Empty);
    var bundle = BundleTable.Bundles.GetBundleFor("~/VIRTUALPATH");
    var response = bundle.GenerateBundleResponse(context);
    var content = response.Content;

    this.WriteLiteral(content);
}

Or in short:
@Html.Raw(BundleTable.Bundles.GetBundleFor("~/VIRTUALPATH").GenerateBundleResponse(new BundleContext(this.Context, BundleTable.Bundles, string.Empty)).Content)

EDIT: To get unminified content in debug, we can do what Scripts does: Get the bundle again, but instead of generating the response, we iterate its files and write their contents (delimited).
@{
    var context = new BundleContext(this.Context, BundleTable.Bundles, string.Empty);
    var bundle = BundleTable.Bundles.GetBundleFor("~/VIRTUALPATH");

    if (BundleTable.EnableOptimizations)
    {
        var response = bundle.GenerateBundleResponse(context);
        var content = response.Content;

        this.WriteLiteral(content);
    }
    else
    {
        var files = bundle.EnumerateFiles(context);

        foreach (var file in files)
        {
            var stream = file.VirtualFile.Open();

            using (var reader = new StreamReader(stream))
            {
                this.Output.Write("{0}{1}", reader.ReadToEnd(), bundle.ConcatenationToken);
            }
        }
    }
}

Comment: This is almost exactly what I need (when you print the `content` through `@Html.Raw()` if you use `this.Write(content);` you get junk) it seams to print out only minified content. Do you know of a way how to get the unminified content in DEBUG?
Comment: Correct about the junk (HTML encoding), edited. Also added DEBUG handling.
Answer: It's easier to create your own class for doing this, and an stable minifier.
As to the minifier you can use one of these:

YUI compressor (used by Yahoo)
Microsoft AJAX Minifier (be aware that WebGrease already includes Ajax Minifier and you can get a conflict if you add both NuGet packages)

Your custom class should:

be static
contain a List<string> to hold the files in each bundle
conatin a dictionary, to hold the bundle names and its files: Dictionary<string,List<string>> with a bundle name string, and a list of file names List<string>.

To create the non-minified version you simply need to read all the files into a string with any of the classes available for this (for ezample StringBuilder, StringWriter, StreamReader, File). If you need to mimic the wildcard names in the bundler, use Directory, and perhaps Regex classes.
The result of the reading should be stored in an static dictionary, which holds the bundle name and the bundle content (for caching, and avoiding reading the same files over and over).
An finally, you need to minify it.
You can toggle the minificacion depending on two different things:

the definition of debug in the c# code #ifdef DEBUG ... #endif which depends on the solution configuration chosen in the configuration manager: debug/release
the compilation mode defined in web.config

Minifying with YUI compressor:
  JavaScriptCompressor jsCompressor = new JavaScriptCompressor();
  jsCompressor.ObfuscateJavascript = true; // Optional
  string minified = jsCompressor.Compress(originalScript);

Minifying with MS Ajax Minifier:
  Minifier min = new Minifier();
  string minified = min.MinifyJavaScript(originalScript);

You should also store the minified versions in an static dictionary to also cache them.
This is a basic working sample:
public class MyMinifier
{
    protected static Dictionary<string,List<string>> VirtualPathsInBundle
        = new Dictionary<string, List<string>>();

    protected static Dictionary<string, string> OriginalJavascriptInBundle
        = new Dictionary<string, string>();

    protected static Dictionary<string, string> MinifiedJavascriptInBundle
        = new Dictionary<string, string>();

    public static void AddBundle(string bundleName, params string[] virtualPaths)
    {
        VirtualPathsInBundle.Add(bundleName, virtualPaths.ToList());
    }

    public static string GetOriginalJavaScript(string bundleName)
    {
        if (!OriginalJavascriptInBundle.ContainsKey(bundleName))
        {
            var physicalFilePaths = VirtualPathsInBundle[bundleName]
                .Select(vp => HttpContext.Current.Server.MapPath(vp));
            // If you use wildcards, expand them here...
            StringBuilder scripts = new StringBuilder();
            foreach (var path in physicalFilePaths)
            {
                scripts.AppendFormat("// path: {0}", path);
                scripts.AppendLine();
                scripts.Append(File.ReadAllText(path));
            }
            OriginalJavascriptInBundle.Add(bundleName, scripts.ToString());
        }
        return OriginalJavascriptInBundle[bundleName];
    }

    public static string GetMinifiedJavaScript(string bundleName)
    {
        if (!MinifiedJavascriptInBundle.ContainsKey(bundleName))
        {
            Minifier minifier = new Minifier();
            MinifiedJavascriptInBundle[bundleName]
                = minifier.MinifyJavaScript(GetOriginalJavaScript(bundleName));
        }
        return MinifiedJavascriptInBundle[bundleName];
    }
}


Fatou's Lemma with $\max$

Question: Let $m$ be a probability measure on $W \subseteq \mathbb{R}^m$ so that $m(W)=1$.
For all $k \in \mathbb{Z}_{\geq 0}$ let $f_k: X \times W \rightarrow \mathbb{R}_{\geq 0}$ be measurable and locally bounded, where $X \subseteq \mathbb{R}^n$.
Consider a sequence $\{X_k\}_{k=0}^{\infty}$ of compact sets $X_k \subset X$ such that $X_{k} \supseteq X_{k+1} \supseteq \cdots \supseteq X_\infty = \{\bar x\}$.
Assume that there exists and integrable function $\bar{f}: W \rightarrow \mathbb{R}_{\geq 0}$ such that $f_k(x,w) \leq \bar f(w)$ for all $x \in \mathbb{R}^n$ and $w \in W$.
Say if the following is true.
$$ \limsup_{k \rightarrow \infty} \ \max_{x \in X_k} \int_{W} f_k(x,w) m(dw) \leq \int_W \limsup_{k \rightarrow \infty} f_k(\bar{x}, w) m(dw)  $$
Note: if $X_k = \{\bar{x}\}$ for all $k$, then the claim is true because of the Fatou's Lemma.
Answer: Try $f_k(x,w)=\bar f(w)\cdot\mathbf 1_{x\ne\bar x}$, assuming that $X_k\ne\{\bar x\}$ for every $k$. Then each maximum on the LHS is the integral of $\bar f$ and each limsup on the RHS is zero, hence LHS $\gt0$ and RHS $=0$ except in the trivial case where $m(\{\bar f\ne0\})=0$.

Fatou's Lemma with $\max$


ruamel.yaml.cmd rt breaks lists, if containing long string, or hash

Question: I just notices that the command line tool, called like this: "ruamel.yaml.cmd rt --save $YAML_FILE", will break lists that either contain long strings, or hashes:
Example list containing a hash:
Source:
telegraf::inputs:
  cpu:
    - percpu: true
      totalcpu: true
      report_active: true

output:
telegraf::inputs:
  cpu:
     - percpu: true
    totalcpu: true
    report_active: true

example list containing long string:
source:
rsyslog::config::snippets:
  00_forward:
    ensure: 'present'
    lines:
      - 'if $syslogfacility != 1 then {'
      - 'action(Name="collector-syslog" Type="omfwd" Target="%{hiera("rsyslog_server")}" Port="514" Action.ResumeInterval="5" Protocol="tcp")'
      - '}'

output:
rsyslog::config::snippets:
  00_forward:
    ensure: present
    lines:
       - if $syslogfacility != 1 then {
       - action(Name="collector-syslog" Type="omfwd" Target="%{hiera("rsyslog_server")}"
      Port="514" Action.ResumeInterval="5" Protocol="tcp")
       - '}'

I already created a bug report for this, but it was deleted with a comment pointing to https://yaml.readthedocs.io/en/latest/example.html?highlight=indent#output-of-dump-as-a-string.
But I am not sure how this code snipped should help me with the command line tool.
Or is the tool deprecated, and I have to roll my own?
Answer: The automatic detection of the indent seems incorrect for your input, as that input is inconsistent (your mappings are indented 2 positions and your sequences 4 positions with an offset for the block sequence indicator of 2). ruamel.yaml.cmd as on PyPI doesn't support different indentation levels for sequences and mappings (ruamel.yaml didn't when that was written, it does now).
Apart from that you cannot set the line width for the output in ruamel.yaml.cmd for older versions ( before 2020-12-01), and those versions are using the default 80 characters for the wrapping.
I recommend you upgrade to 0.5.6 and use the command line options:
yaml rt  --indent 2 --width 1024 --save <yourfile>

The appropriate repository for ruamel.yaml.cmd is https://sourceforge.net/p/ruamel-yaml-cmd/code/ci/default/tree/ . A bug report on ruamel.yaml which can only be used from a Python program, should include the minimal source code of the program that reproduces the error, and if not provided, issues will be removed as announced on its create issue page.
Comment: i give this a try. could you maybe fix the repo urls on pypi? they point to something that is not there anymore.
Comment: i tested this, and it is not doing the right thing :-/

with bsi 0, lists are not indented at all

i guess by mappings you mean type hash, and with sequences you mean type list/array?

if yes. the problem is that i have data structures that contain lists of hashes, see telegraf::inputs example from above.

they translate to [ordereddict([('percpu', True), ('totalcpu', True), ('report_active', True)])]

so it is a list of hashes. but when the lib dumps them, it gets the indention wrong.

i suspect the lib has problems dumping lists, that contain hashes.
Comment: There is now a ruamel.yaml.cmd 0.5.6 on pypi. I added a --width option to the rt command so try with `python -m ruamel.yaml.cmd rt --wdith 1024 --indent 2 --save yourfile`  ( tested with your 2nd source file for wrapping the long line)
Comment: I can dump that data structure without  a problem. ( `python -c "from ruamel.yaml import YAML; import sys;  a = {'list_of_hashes': [{'a': '1', 'b': 2, 'c': None}, {'d': '1', 'e': 2, 'f': True}, {'g': '1', 'h': 2, 'j': False}]}; YAML().dump(a, sys.stdout)"  `)
Comment: The links on PyPI have automatically updated with the new version. I had to move back from bitbucket to sourceforge, and set a forwarding link on bitbucket when I did so. Now it seems the braindead people at Bitbucket removed those links adding insult to injury
Comment: thanks for the width change. would it make sense to also break --bsi into two options, so that block and sequence indention can be set on there own?

using the lib directly, it is doing the right thing when calling "yaml.indent(mapping=2, sequence=4, offset=2)"
Comment: Yes it would, but that takes more of a rewrite. `ruamel.yaml.cmd` 's origin is from before the introduction of seperate indents for mappings and sequences, and I couldn't have gotten that out so quickly. I also need to change the "guess_indent" utility function to make that destinction.


Puxar um form C# em VB

Question: Puxar um form C# em VB
Bom eu queria saber como eu poderia ta puxando um form em C#
via Visual basic
exemplo:
Answer: Se você quiser incluir o arquivo Loading.cs no seu projeto Visual Basic, isso não é possível. Os compiladores lidam com apenas um tipo de linguagem no mesmo projeto. Você terá que reescrever o código em C# no seu projeto em VB.Net.
Alternativamente, você poderá referenciar o projeto em C# no seu VB.Net e utilizar o conteúdo do outro projeto no seu.
Para isso, referencie o projeto do C# para o projeto Visual Basic e instancie a classe do outro projeto no seu:
Imports <caminho.para.sua.form>  ' aqui é o namespace de onde sua classe Loading está
...
Dim janela As New Loading
janela.Show()

Observação: a classe Loading deve ser pública (public) para o outro projeto poder acessá-la.

Mas como referencio um projeto no outro?

Dentro do projeto em VB.NET, clique com o direito em referências:

Selecione o seu projeto em C# na aba "Projetos":

Clique em OK.
Comment: Deu o seguinte erro https://imgur.com/43qaXUU

ja deixei a classe loading public
Comment: @C0DE substitua o `` pelo caminho (namespace) de onde sua classe `Loading` está.
Comment: Não entendi poderia me falar um exemplo
Comment: @C0DE `Imports NomeDoProjeto.Janelas`, dentro de `Janelas` tem a classe `Loading`.
Comment: Parece que ele esta dando erro https://imgur.com/K522X1N
https://imgur.com/zm8bOUr
Comment: No `Loading.janelas` você precisa substituir com o namespace da classe `Loading`. Na sua pergunta, você não especifica o namespace. [Mas o que é um Namespace?](https://docs.microsoft.com/pt-br/dotnet/visual-basic/programming-guide/program-structure/namespaces)
Comment: https://imgur.com/GwwlyJd Minha namespace e essa

Puxar um form C# em VB


Including files in template.php. failed in some cases (when using some contrib modules)

Question: In template.php, in theme__preprocess_page function, I am including files (as my template.php grew big, I decided to separate some operations for readability) like this:
 function THEME_preprocess_page(&$variables) {
// .. skipped non-related code
 include  $_SERVER['DOCUMENT_ROOT'] . "/" . path_to_theme() . "/template_includes/file1.inc";

}

But sometimes some contrib modules complain about this. Currently I have an issue where Print module gives the following message:
Warning: include(): Failed opening
'/var/www/drupal/sites/all/modules/contrib/print/template_includes/file1.inc' **for inclusion** (include_path='.:/usr/share/php:/usr/share/pear') **in THEME_preprocess_page() (line 80 in** /var/www/drupal/sites/all/themes/THEME/template.php).

What causes paths mess?
Comment: u should simply add this(file) to your `.info` file.
Comment: @NikhilM I don't think this is a duplicate at all to be honest :/ This one is asking about how to load files, the other is asking what they should be named...
Comment: that simple ? :) I have bunch of them... will try and report back here.
Comment: possible duplicate of [Using the PHP include in your template: Naming the file .php or .inc?](http://drupal.stackexchange.com/questions/8236/using-the-php-include-in-your-template-naming-the-file-php-or-inc)
Comment: I think there are some circumstances where path_to_theme() may not work. You don't need the DOCUMENT_ROOT part in any case. Try using `include drupal_get_path('theme', THEMENAME) .  "/template_includes/file1.inc";` instead.
Comment: Thanks  Alfred ill try your suggestion and report here. Yes i agree doc root looks unecessary.
Comment: Agree with @Chapabu about this not being a dup.
Answer: If you read the docs for path_to_theme(), there is this tidbit:

It can point to the active theme or the module handling a themed implementation. For example, when invoked within the scope of a theming call it will depend on where the theming function is handled. If implemented from a module, it will point to the module. If implemented from the active theme, it will point to the active theme. When called outside the scope of a theming call, it will always point to the active theme.

The Print module invokes preprocess functions within its own preprocesss functions (look at the code), so I think that is why yours is getting called with a scope other than your theme.
You should really do your include as
include drupal_get_path('theme', 'YOURTHEME') . '/template_includes/file1.inc';

or
include dirname(__FILE__) . '/template_includes/file1.inc';

There is debate in the Drupal community which is best.
Personally, I think including code within a function is a really bad practice, and would recommend splitting out whole functions into separate file(s), and doing the include at the top.  If you go that route, then you using the second form is best.
Comment: yes, this is ultimate answer to my question+ thanks for the extra tip
Comment: reason why most of my includes are inside functions is because there are if conditions around appropriate inclusions.
Comment: and when using drupal_get_path... there is no more mess in inclusions! Ty again.


Check if Progress Bar Equals Amount in JQuery

Question: I know this should be simple, but I am trying to check if a progress-bar equals 100%. I can't seem to trigger an alert if it does.

var bar = $('.progress-bar').text(val);
var val = percentage + '%';
var percentage=1;
   if(bar == 100){
       alert("100");
   }

100%

Answer: $(document).ready(function(){

    var percentage = someFunctionToGetPercentage();

    $('.progress-bar').text(percentage + '%');

    if (parseInt($('.progress-bar').text()) == 100) { // or $('.progress-bar').text() == '100%'
        alert(100);
    }
});

Fiddle
Comment: http://jsfiddle.net/eb0xnyrk/ There is a problem with your understanding about js and html.. please read more tutorials about js/jquery in order to be able to create these scripts.. What you have tried is a basic script.. you should be able to do it by yourself
Comment: why not? what's the issue?
Comment: This script must be added in the head section after including the jquery framework


User defined signal method

Question: I've been following the APUE textbook where they throw in an example of printing a user-defined signal.
static void sig_usr(int);
/* one handler for both signals */
int
main(void)
{
    if (signal(SIGUSR1, sig_usr) == SIG_ERR)
        err_sys("can't catch SIGUSR1");
    if (signal(SIGUSR2, sig_usr) == SIG_ERR)
        err_sys("can't catch SIGUSR2");
    for ( ; ; )
        pause();
}
static void
sig_usr(int signo)
/* argument is signal number */
{
    if (signo == SIGUSR1)
        printf("received SIGUSR1\n");
    else if (signo == SIGUSR2)
        printf("received SIGUSR2\n");
    else
        err_dump("received signal %d\n", signo);
}

Can someone explain where SIGUSR1 and how SIGUSR2 are defined? I understand that the function just waits to catch the appropriate signal before listening for others but I don't understand how it knows which signal code is SIGUSR1 and SIGUSR2.
Answer: They are defined in the header file signal.h. You can programmatically send them by using kill command.
Comment: Thanks a lot, I knew I was missing something simple.

User defined signal method


Sonatype nexus 3 Remote connection pending

Question: I recently upgraded to nexus-3.0.0-M5 for our repository server and I noticed two problems with it.

Any repository (hosted) that I create, gets stuck in Remote Connection Pending state and it doesn't change no matter what I do.
I already have a huge storage/index (approximately 65GB that includes local and remote jars) that I want to use. In Nexus2.x versions, there was a property file called conf/nexus.properties where I could specify this and so all upgrades were quite simple and straightforward. But in Nexus3, I can't seem to configure these properties.

Any help in this regard would be useful, been banging my head on the wall since 2 days.
More Info
I am not sitting behind a proxy, all connections to my nexus repository are quite direct with authentication. I used this link to create a hosted repository (in case I was missing something) but it gets added fine, it just stays stuck in remote connection pending state.
I can separate out my local jars from the whole storage area that my previous nexus (nexus2.11) created but even those are about 1.4GB in size. If I could get nexus3 to some-how host these jars automatically, then I can spend the rest of the time in configuring the remote ones (bummer! but as long as my local jars are there, I'll live)
Answer: The "connection pending" message is normal in 3.0m6. It just means nothing has been downloaded through the proxy repository yet. Try running a build that retrieves artifacts through the proxy, the status will change once the first file of the artifact is downloaded.
Comment: Thanks for the response. You're right, I was able to download an artifact just fine and the status changed. That solves my first question. Any idea of the other question ?
Comment: Behavior confirmed for version 3.0.2-02.
Comment: Great appreciate for your reply


Removing a Mysql record

Question: I have a few thousand products in a mysql table for products.
Each product has a field for its manufacturer [manufacturer] stored as an integer - the manufacturer 'id'.
Over time some manufacturers have been deleted from the 'manufacturers' tables but the products have been left orphaned.
Is there a query I can run that will delete all orphaned products.
Something like

'Delete all products where products.manufacturer does not match
  manufacturers.id'.

One further complication. A product can have more than one manufacturer so the products.manufacturer can have data like this '1|75|432' where 1, 75, 432 are each manufacturers.
Thanks
Comment: It would be a good idea to change your table design, if you can. Columns with character delimited lists produce most time a lot of trouble.
Comment: What if product has, lets say, 3 manufacturers, but only one of them has been deleted? Do you delete it as well?
Answer: Try this,
DELETE FROM product
WHERE product_id not in (
    SELECT distinct p.product_id FROM Product p, manufacturer m
    WHERE p.manufacturer REGEXP '[[:<:]]' + m.manufacturer_id + '[[:>:]]'
);

PS: This will remove only those record, for which all the manufacturer in the list are deleted
Answer: You could use FIND_IN_SET in combination with REPLACE
DELETE
    p
FROM
    products p
LEFT JOIN
    manufacturer m
ON
    FIND_IN_SET(m.id, REPLACE(p.manufacturer, '|', ','))
WHERE
    m.id IS NULL

Explanation:
You've got to replace your limiter | with the only one that FIND_IN_SET works with, the comma , first. If there's no matching manufacturer in the manufacturers table, then the value of the id column of your manufacturers table) will be NULL.
see DELETE syntax:

The preceding examples use INNER JOIN, but multiple-table DELETE
  statements can use other types of join permitted in SELECT statements,
  such as LEFT JOIN. For example, to delete rows that exist in t1 that
  have no match in t2, use a LEFT JOIN:
DELETE t1 FROM t1 LEFT JOIN t2 ON t1.id=t2.id WHERE t2.id IS NULL;

see this fiddle for the SELECT statement.
Note
As I said in my comment, it would be better to change the database design. A product can have several manufacturers and a manufacturer can produce many products, the classical case for a junction table.
Answer: There are lots of preliminary things you should do:

1) Change the table design, tab limited multiple values is not a good
  idea, in-fact in a situation where you have to join with its value.
Sol: change the manufacturer_id to accommodate only a single value(FK
  with manufacturer id table).
2) You need some scripts which'll insert multiple rows for different
  tabbed values.
Anyway you need this for remaining records.
3) Now we come to point we can join manufacturer and product tables.

You can now do left join products upon manufacturer to do other tasks efficiently.

Removing a Mysql record


Would it be feasible to have Camera on the bat and ball?

Question: In current days, a camera is present everywhere - the stumps, boundary, umpire's hat etc. Would it be possible to have a camera on bat and ball as well? How would it disturb the aerodynamics of bat and ball?
Comment: Can I ask why you want cameras on the bat and ball? Is it just for an interesting perspective or are you trying to collect data about the game? In Major League Baseball, every pitch is tracked, enabling research about the skills of pitchers, batters, catchers, and umpires. I'm sure an equivalent system could be set up in cricket.
Comment: It is just for interesting perspective..
Answer: Technically possible, you could mount a small wireless camera in the ball but and something similar in the bat.
However in the ball it would only be able to face in one direction and there would be no gaurentee that would be of any interest.  The repeated impact forces would probably destroy it fairly quickly, and it would also alter the weight of the ball and probably change the way the ball is made to be able to fit the camera in so I doubt it will ever be attempted.
For the bat it would be easier to ensure it looked in the right direction but motion bluring would be an issue and then you would have the impact issues around the lifespan of the camera.  It would also introduce a structual weakness in to the bat.  Or if surface mounted would effect the pick up and stroke of the bat.
So its extremely unlikely to happen.


Best scoring performances in an NBA finals loss

Question: LeBron James of the Cleveland Cavaliers scored 51 points in a losing effort against the Golden State Warriors in game 1 of 2017 - 2018 NBA finals. That made James the 6th player ever to score 50+ points in the NBA finals, but he also became the first person in that list to lose the game.
What are the top 5 scoring performances in the NBA finals that resulted in a loss?
Answer: Using Basketball Reference's Player Game Finder with the following search criteria:

In a single game, from 1964 to 2019, in the playoffs, round is Finals, team lost game, sorted by descending Points

The top 5 scoring performances in the NBA finals that resulted in a loss are:
1) LeBron James - 51 points, Game 1, 2018 NBA Finals (Cavaliers at Warriors)
2) Stephen Curry - 47 points, Game 3, 2019 NBA Finals (Raptors at Warriors)
T3) Jerry West - 45 points, Game 4, 1966 NBA Finals (Celtics at Lakers)
T3)  Jerry West - 45 points, Game 2, 1965 NBA Finals (Lakers at Celtics)
T5) LeBron James - 44 points, Game 1, 2015 NBA Finals (Cavaliers at Warriors)
T5) Shaquille O'Neal - 44 points, Game 1, 2001 NBA Finals (76ers at Lakers)
T5) Michael Jordan - 44 points, Game 3, 1993 NBA Finals (Suns at Bulls)
T5) Rick Barry - 44 points, Game 6, 1967 NBA Finals (76ers at Warriors)
Comment: Thanks for the answer, please indicate the teams playing and I will accept your answer
Comment: @alamoot Teams have been added
Answer: #1. 63 points: Michael Jordan & Chicago Bulls

April 20, 1986 - Eastern Conference first round; Chicago Bulls Vs. Boston Celtics. Boston won: 135 - 131.

#2. 61 points - Ray Allen & Boston Celtics

April 30, 2009 - Eastern Conference First Round; Boston Celtics Vs. Chicago Bulls. Chicago won: 128 - 127

#3.1 51 points - Russell Westbrook & Oklahoma Thunder

April 19, 2017 - Western Conference Quarter Finals; Oklahoma Thunder Vs. Houston Rockets. Houston won: 115 - 111

#3.2 51 points - LeBron James & Cleveland Cavaliers

May 31, 2018 - NBA Finals; Cleveland Cavaliers Vs. Golden State Warriors. Golden State won: 124 - 114

#4.1 50 points -  Bill Cunningham & Philadelphia 76ers

April 1, 1970 - Eastern Division Semi Finals; Philadelphia 76ers Vs.  Milwaukee Bucks. Milwaukee won: 118 - 111

#4.2 50 points -  Michael Jordan & Chicago Bulls

May 5 1989 - Eastern Conference First Round; Chicago Bulls Vs. Cleveland Cavaliers. Cleveland won: 108 - 105

#4.3 50 points -  Kobe Bryant & Los Angeles Lakers

May 4, 2006 - Western Conference First Round; Los Angeles Lakers Vs. Phoenix Suns. Phoenix won: 126 - 118

Additional Reference:
List of National Basketball Association single-game playoff scoring leaders
Comment: Your response seems to be answering highest points scored in a loss in any playoff games, which is not exactly what I asked. My question is targeting exactly the NBA finals.


WMI query to find lack of drive letter for Group Policy

Question: I'm building a group policy to apply to a subset of users within my organization. This subset of users have no Home Directory set on their Active Directory object, whereas other users have an I:\ drive assigned.
I need to find a WMI query which will allow the User Configuration GPO to apply to a user with no I:\ drive.
I can easily find the inverse of that:
Select * From Win32_LogicalDisk Where DeviceID = 'I:'

But if I make this WHERE DeviceID <> 'I:\', it returns C:, D: and other mapped drives.
I thought about querying the user profile with something like this:
select RoamingPath from Win32_UserProfile where RoamingPath <> '\\*'

however that returns many results for system accounts and local administrator profiles, rather than the logged on user.
I've read that it is possible to query AD directly, but the command below gives me an "Invalid Class" error and I don't know if it would apply to the logged in user:
select * from ds_user where ADSIPath = LDAP://OU=CA,DC=global,DC=opus AND DS_homeDirectory='*'

I think ideally this would be solved by using Win32_LogicalDisk to count the total number of drives, a second query to get a count of I:\ drives (either 1 or 0), and then subtract the second result from the first. If the result is = the first query, I have my set of users. So far no luck on finding syntax to do this within a GPO wmi filter though.
Comment: Would you consider using PowerShell instead? You could do this in about two lines with powershell and the ActiveDirectory module.
Comment: I'd certainly consider PowerShell, although i don't know how it could be used as a filter to which users the GPO applies to.
Answer: I have determined that do accomplish this exactly as I've defined in my question is not really possible.
The two best options I have come across are:

Inverted WMI Filters - Create an environment variable per user/PC which produces a result based on required logic. Then use a regular WMI query to filter on that environment variable.
Scheduled PowerShell query - as Yanick suggested in the comments, create a PowerShell query which filters users based on required logic (i.e. homeDrive attribute being empty) and then add these user accounts into a group, which can then be used for Security Filtering on the GPO. This PowerShell is then scheduled on a LAN server to run repeatedly.

I've chosen option #2 in my environment.


menu with sub-menu hover-over effect

Question: I have the following lines in my website - http://codepen.io/anon/pen/LhDnu/ that creates a sidebar menu.
As the user hovers their mouse over a parent, listed item with child-links within it, a sub-menu appears.
However, when the mouse moves over to select one of the child-links, it removes the sub-menu listed items.
How can I modify the codepen above so that it will allow the user to select one of the sub-menu items?
Comment: The problem is that the sub menu is too far away, so you are no longer hovering over the parent when you move your mouse towards it. Depending on how you wanted it to look, you could either move the sub menu closer or leave it as it is but create a transparent "bridge" between them so that the hover effect doesn't die.
Comment: Thanks for the response John.  Could you please provide me with an example of how I would go about creating a transparent bridge?
Comment: It would be a particularly cumbersome fix and could easily cause usability issues of its own, but it would basically be a `position: absolute` block with no background-color (or 0 opacity), sort of wedged into the gap. You could create a pseudo element on either the top level or sub menu to achieve it.
Answer: Here is a working code for you, in css dropdown menus try to work with display:none & display:block not with the opacity
http://codepen.io/anon/pen/luGDK/
Comment: Thanks for the response Saif.  The sidebar needs the padding space around it to go with the visual consistency of the website and this may be why I'm still getting the same sort of results when hovering over the child items.  Using `display: none`, I also lose the gradual transition effect.
Answer: Nevermind.  Here is the solution - http://codepen.io/anon/pen/LzpyK/.

menu with sub-menu hover-over effect


window.open works different in chrome and firefox

Question: I want to open a new tab, if it has not been opened before.
If it has been already opened from the same opener, then show the tab (bring to front without reloading).
My code:
const openWindow = window.open(undefined, 'WindowName');

if (openWindow.location.href === 'about:blank') {
    openWindow.location.href = newWindowUrl;
}

In Chrome this will open a new tab and set the url.
Calling the same code again, will reopen the already available window, without reload.
In Firefox, this will open a new tab and set the url.
Calling the same code again, will do nothing.
How can I reopen the available window by window.name?
Answer: The solution for FireFox is:
const openWindow = window.open(undefined, 'WindowName');

if (openWindow.location.href === 'about:blank') {
    openWindow.location.href = newWindowUrl;
}

// ==> to open existing window in Firefox
openWindow.focus();


Wilcoxon signed-rank test. Is it necessary?

Question: I have a sample with millions of points. Every point $x$ has two values associated, let's say $A_{x}$ and $B_{x}$. I calculated the mean values of A and B in the sample:
$\overline{m}_{A} = 0.19 \quad \text{and} \quad \overline{m}_{B} = 0.21$
The distributions of the values are different and not normal. And the standard deviations of the means are negligible.
Can I say that the means are statistically significantly different? Should I calculate the Wilcoxon signed-rank test to prove it?
Comment: *Don't* use $\mu$ for *sample* means. The convention in statistics is usually that Greek letters are population parameters ($\mu_A$ is the population mean of the A's) and Roman letters are used for sample quantities ($\bar x_A$ as sample mean). The Wilcoxon signed rank test carries assumptions which - if your assertions are correct - will not be satisfied. Further, unless you make additional assumptions, it doesn't test mean shift, but something else. The sign test might have its assumptions satisfied, but it also won't be a test of means without further assumptions (that probably won't hold).
Comment: (1) You can't possibly satisfy the first assumption listed there; (2) ... which doesn't matter because the first assumption is nonsense; (3) There's a vital assumption that was removed more than a year ago - see [assumption 4](http://en.wikipedia.org/w/index.php?title=Wilcoxon_signed-rank_test&oldid=479227192#Assumptions) ... and from the look of the changes that are often made on that page, I would treat that particular page as pretty much a random phrase generator. e.g. I don't think there's been a single time on that page when *all* of the assumptions have been correct.
Comment: On topics where many people *think* they have expertise, thay want to jump in and "fix" it on Wikipedia. Basic stats is one of those -- so pages on topics like histograms or simple nonparametric tests get "fixed" by well-meaning people every day or so, often to its detriment. A good way to actually *understand* the assumptions is to work out how to enumerate the permutations for each value the test statistic takes. The required assumptions are usually immediately obvious. Failing that, get a good book by someone who has *published papers* in the area - in a good journal.
Comment: With millions of points, there's probably no point in a hypothesis test *at all* (unless the sds are gigantic relative to the difference in means), but in any case the important assumptions then will relate to bias in sampling and independence of the pair-differences. You don't need nonparametrics - under mild conditions (e.g. variances exist) you can rely on CLT and Slutsky's theorem to say a one-sample t-statistic has a normal distribution.
Comment: You *may* be able to get somewhere with a resampling based test, but even there, there are assumptions that mightn't be satisfied.
Comment: Thanks for the comments, but I don't see what assumptions aren't satisfied for the Wilcoxon signed rank test?
Comment: Okay, what are the assumptions?
Comment: I believe it satisfies all three assumptions enumerated in Wikipedia. However, I think you are correct about the fact that doesn't test mean shift. What test do you proposed for this?
Comment: Sometimes I feel about statistics like [him](http://en.wikipedia.org/wiki/The_Greatest_American_Hero) : I could do interesting things if only I had the (correct and clear) instructions. Anyway, I suppose that I should analyse the distribution of $\Delta_{x}$ as @January proposes. I could calculate $\overline{\Delta_{x}}$. And I suppose that with millions of points its standard error would be "negligible".
Answer: With millions of points the SEM will often be "negligible". However, you should ask yourself about the effect size, as "statistically significant" $\neq$ "significant". There is always an effect, and with sufficient sample size, you will be able to show that it is significant.
Last but not least, the values are paired ($A_x$ and $B_x$ are paired for a given $x$). Therefore, rather than comparing the distribution of $A$ and $B$, you probably should calculate the difference $\Delta_x = A_x - B_x$ and focus your analyses on that. For this, Wilcoxon signed rank test seems appropriate.
Finally, ask yourself the question -- aren't you comparing apples to oranges? What is the question that you want to answer? (apart from "do I need to run some statistics to get my paper accepted")
Comment: First thanks. Second, considering the central limit theorem, can't we calculate the standard error of the mean? Third, I want to focus on the difference $\Delta_{x}$, that's the reason why I consider to use the Wilcoxon signed rank test. And finally, I want to find what method is the best: A or B. And the best method will have the lowest mean.
Comment: You are right about the SEM. And you are right about the Wilcoxon signed rank test. I need to buy glasses (rank sum, signed rank...)


How exectue MySQL multi query, in the asp classic?

Question: asp --> odbc --> mysql, multi query problem.
dbCon.execute("update wp_posts set post_abc = '1234' where ID=1602';");: no problem
dbCon.execute("set @aa=1602;update wp_posts set post_abc = '1234' where ID=@aa;");: error
err message:

[MySQL][ODBC 5.2(w) Driver][mysqld-5.1.45p1-log]You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'update wp_posts........

How can I fix this? SOS!!!
Dislikes answers:
dbCon.execute("set @aa=1602;");
dbCon.execute("update wp_posts set post_abc = '1234' where ID=@aa;");

Answer: Add MULTI_STATEMENTS=1 in the connection-string, or enable multiple queries in the DSN entry by going to

Details -> Connection -> Allow multiple statements
Comment: MULTI_STATEMENTS=1 : good! good! good! good! good! good! good! thank-U! thank-U! thank-U! thank-U! thank-U! thank-U! thank-U! thank-U!^^


Find Laplace Transform using unit step function and t-shifting. (5.3-35, 5.3-36)

Question: How do the Laplace Transforms vary between the two following functions? What I am really asking is if I calculated the following Laplace Transform correctly...
$$\mathcal{L}\{(t-1)u(t-1)\}=\frac{e^{-s}}{s^2}$$
...what should the Laplace Transform be for this function:
$$\mathcal{L}\{(t)u(t-1)\}=?$$
Answer: The one sided Laplace transform is defined as
$$
\mathcal{L}\{f(t)\} = \int_0^{\infty}f(t)e^{-st}dt
$$
and the Unit step function is defined as
$$
\mathcal{U}(t)=
\begin{cases}
1, & t>0\\
0, & t<0
\end{cases}
$$
so the
$$
\mathcal{L}\{t\mathcal{U}(t-1)\} = \int_0^{\infty}t\mathcal{U}(t-1)e^{-st}dt =
\int_1^{\infty}te^{-st}dt = \mbox{???}
$$
Comment: Thank you for responding but I do not understand the answer. If I solve for the integral given will I get the Laplace for the second unshifted t function
Comment: I edited my comment. Please have a second look at it.
Comment: @JulesManson if you solve the integral given, you will still need to solve $\mathcal{L}\{-\mathcal{U}(t-1)\}$ since $\mathcal{L}\{(t-1)\mathcal{U}(t-1)\} = \mathcal{L}\{t\mathcal{U}(t-1)\}-\mathcal{L}\{\mathcal{U}(t-1)\}$
Comment: Thank you so much. That is what I needed. Do you mind if I ask. Are you a math teacher or an engineer?
Comment: I think it is wonderful that you are pursuing your PhD. I am currently working really hard on my BSME by attaining a minimum 3.0 GPA because I wish to pursue an MS afterwards. And who knows if I have the "right stuff" after all and if I am smarter than I give myself credit for and if the gods up there smile on me perhaps I may also attempt a PhD one day.
Comment: @JulesManson I have to get accepted to a school first. I will find out in March-April.
Comment: is this how to tag your user name to another posted question? Anyway I do not trust the answer I got on another question I posted and I would love for you to look at it. Here it is http://math.stackexchange.com/questions/1131483/find-laplace-transform-using-unit-step-function-and-t-shifting-5-3-38


Programmatically Setting Content Ranking Factors For Drupal Search

Question: I've added a dropdown to Drupal search page interface. I would like for users to be able to select this control to sort results according to Date or Keyword Relevance.
I'm able to get the dropdown value in a submit handler, however, I don't know how to programmatically alter the weight of 'keyword relevance' and 'date'.
Here is my submit handler. Any help would be greatly appreciated.
<code>function test_sort_form_submit_handler($form, &$form_state) {
  drupal_set_message("Drop Down Value Is: " . $form_state['values']['search_sort']);
}
</code>
Comment: Have you tried altering the form? I suppose you alter it to add the dropdown, so add #weight value to other form elements.
Comment: I would just have two pages with different views and link them.
Answer: I would do this with a view instead of trying to tweak Drupal's search page:

Create a view
Add the filter "Search: Search Keywords" and expose it
Add two sort criteria: your date field and the "Search: Score" field and expose both of them
In Advanced > Exposed form > settings, check "Allow people to choose the sort order" if you want to give people that option
Add the fields or setup the display as you prefer

You can set the default sort option by ordering the sort criteria and the asc/desc setting by choosing that option for the top criteria. You can also use the Better Exposed Filters module (full disclosure: I'm the maintainer) to combine the sort field and direction into a single dropdown and/or rewrite the text of the options.


Translation table: How do I restore a deleted metatag string?

Question: At the Translation table module (admin/config/regional/translate/translate), I was translating metatags. By accident I deleted one (metatag:global:og:street-address). I thought I just delete all the useless translations, but then it deleted the whole tag.
Is there a way to restore or (if easier) add it again to the table?
Comment: You are not able to restore deleted metatag if it is deleted from Cache.
Answer: Here are the steps to reproduce:

Enable Metatags, Entity Translation and Inline Entity Forms.
Make a node type translateable (using Entity Translation) and add an entity reference field that uses the inline_entity_form widget (it doesn't matter what it points to).
Create a node of that type, with custom meta tags and add one referenced entity, save it.
Add a translation in another language, translate the meta tags and save the translation (you can also translate the referenced entity, this isn't relevant for this bug though). At this point we have all content as it should be.
Now edit the translation again, leave the meta tags as they are (everything still looks fine) and make a modification to the referenced entity using the inline entity form widget, then save the entire form. The translated meta tags are gone and have been reset to the default values.

The cause of this issue is the following code in metatag_field_attach_form():
<?php
// Entity_Translation will trigger this hook again, skip it.
if (!empty($form_state['entity_translation']['is_translation'])) {
     return;
}
?>

This code works as intended when adding a translation, but not when editing a translation, because in that case $form_state['entity_translation']['is_translation'] is not defined (it gets defined later, only after the code above has already been executed). However, even though the logic is broken, the resulting behavior is correct, because when editing a translation metatag_field_attach_form() doesn't actually get called twice as mentioned in the code comment, so we really don't want to return yet.
The problem arises when using the inline_entity_form widget, which reloads part of the form using Drupal's standard ajax mechanism. This causes the form to be rebuilt, and the code above from metatag_field_attach_form() is run again, but this time after Entity Translation has set $form_state['entity_translation']['is_translation']. The code is now running as intended, but the resulting functionality is broken. Now the function returns prematurely, $form['#metatags'] doesn't get set, metatags' hook_form_alter() gets skipped and the form submit handler doesn't get set in metatag_metatags_form(), so meta tag information gets lost and is replaced with the default value.
The solution is to fix the code above to only return when a new translation is being added, not anytime a translation is edited:
<?php
// Entity_Translation will trigger this hook again when adding a new     translation, skip it.
if (isset($form_state['entity_translation']['is_translation']) &&     !empty($form_state['entity_translation']['is_translation'])) {
  return;
}
?>


Discrete Valuation Ring properties

Question: From Dummit and Foote problem about the Discrete Valuation Ring $R$ with field $K$

Prove that for each nonzero element $x \in K$ that either $x$ or $x^{-1}$ is in $R$

My attempt:
$v(1) = v(x\cdot x^{-1}) = v(x) + v(x^{-1}).$ I show $v(1) = 0$ via $v(1) = v(1\cdot 1) = v(1) + v(1) \implies v(1) = 0$. So we have $v(x) = -v(x^{-1})$ Then we can conclude if either $v(x) > 0$ or $v(x^{-1}) > 0$ then the other can't be in $R$.
But my problem comes if $v(x) = 0$, then both $x, x^{-1} \in R$. What's wrong with this?
Comment: Both $1$ and $1^{-1} \in R$.
Comment: Is that a problem?
Comment: Well, I guess that's my problem. It easily helps show the next part that a unit is in $R$ iff $v(x) =0$. But I guess I just have a problem semantically about the "or."
Answer: English is an awkward language for expressing logical connectives. Despite the use of "either ... or ...", the problem did not intend for $x \in R$ and $x^{-1} \in R$ to be exclusive; it merely wants you to show at least one of the two statements is true.
Comment: Thank you. As I commented earlier, my problem was with the wording.

Discrete Valuation Ring properties


Drupal Commerce Relationship Description

Question: A small question:
Shouldn't the Help text in the commerce_line_item module at line 1582 that defines the relationship between commerce_line_items and commerce_orders say something like "Relate this entity to the order to which it is attached".  As it now stands, the text says "Relate this entity to the line items reference by its commerce_line_items field".  Which doesn't seem to make much sense.
Answer: OK, I think I understand this now, and the Help text is actually correct as it stands.
Each order, it turns out, has a commerce_line_items [note the plural] field attached to it, and it is this field, of unlimited cardinality, that stores all of the commerce_order/commerce_line_item pairs that are associated with this order.  What this relationship does, then, is make the commerce_line_item reference field that is stored in the commerce_line_items field available to the view designer. Once this commerce_line_item reference field is available, all of the fields in the commerce_line_item itself become available.
The thing that was confusing is that the commerce_line_items field does not show up in a Site Builder's "Manage Fields" configuration screens, and I had not been able to find it in the code.  Many thanks to argiepiano, who showed me where to find the definition of the commerce_line_items field [in the commerce_order module], and the place where data is saved into that field [in the commerce_cart module].
So that is the way that this all works ...
... I think.


Pacific State - Is this out of tune, or is it a sophisticated performance playing with temperament?

Question: I heard the song "Pacific State" by the band "808 State" on the radio last night.
It sounded to me like the sax was badly out of tune.
But it then struck me that the performer might be playing that way deliberately.
Which is it? Is it that my ear is not sophisticated enough to appreciate the deliberate temperament chosen by the performer, or should I trust my initial reaction, which says that the sax player is just not very good?
Answer:
...should I trust my initial reaction, which says that the sax player is just not very good?

Yes! It's played by someone who by their own admission can't play the sax - producer and engineer Graham Massey:

"At that point, I was still recording with Biting Tongues, which had
  turned from a normal band into a sort of electronic duo with a
  jazz‑influenced saxophone front line. Howard Walmsley, who played the
  sax in Biting Tongues, had left his soprano from a session the night
  before, and so I picked it up and just had a go. I'd played the
  clarinet and was a bit of a have‑a‑go saxophone player, but I was not
  a real saxophone player. Still, I could crank out a melody, and I
  think my limited ability made it what it was."

(From Sound on Sound magazine).
In accordance with that 'experimental' attitude, I would imagine that it was simply played until the result was pleasing to Massey's ear, with only secondary consideration for whether or not it was in tune by some standard. Personally I rather like the intonation - I think it contributes to the somewhat melancholy, otherworldly feel of the track - but with this, as with everything else in music, de gustibus non est disputandum!
As a more general point, being (arguably) "out of tune" is something that is done deliberately in modern music in all sorts of contexts, for all sorts of reasons - whether it's simply the use of equal temperament, or blue notes being bent through a whole semitone, or the honky-tonk piano (and other chorus effects!), or rappers singing deliberately off key because being too in tune wouldn't sound 'street' - it's all out there.
Answer: Nope. It's simply out of tune.
Bloody awful piece of 'music'. Can't listen to it because of that hideous lead line.
Best guess is it's a found sample & no-one was smart enough to get it in tune, so they just left it. Some people have a very broad tolerance for tuning.
I don't :P
Comment: @Carl Witthoft You mean the stuff Jo Stafford did as Darlene Edwards?  https://www.youtube.com/watch?v=NySAbB2JLII
Comment: There's a wonderful performance by some female jazz singer in which she deliberately goes almost a full half-tone off pitch all the way through.  I wish like crazy I could find it again; I can't even remember who the performer was.
Comment: There was a lot of similar stuff in the 90s, when DJs first decided they could be musicians too... just without the talent. Tuning like this was just everywhere, as they all spun in their favourite bits of non-matching cr@p :\ I just dug out another of my 'favourites' Massive Attack, Unfinished Sympathy. That flown in 'yeah yeah' is just painful.
Comment: @LaurencePayne  That sure fits the bill.  Thanks!!


How to detect underscore in regex?

Question: I have a regex expression in order to test my password : at least one digit, at least one upper letter,at least one lower letter and at least one special character.
/^(?=.*?[A-Z])(?=.*?[a-z])(?=.*?[0-9])(?=.*?[^\w\s])/

But underscore is not recognized as special character.
How can I improve this ?
Comment: `[^\w\s]` does not match `_`. Replace with `[^a-zA-Z0-9\s]`
Comment: Note that imposing character-class password requirements like this is considered bad practice (see e.g. [NIST Guidelines for Memorized Secret Verifiers](https://pages.nist.gov/800-63-3/sp800-63b.html#-5112-memorized-secret-verifiers)).
Comment: @WiktorStribiżew, that works. Please post it as an answer. Thank you!
Answer: The problem here is that, for JavaScript, \w represents:
[a-zA-Z0-9_]

So you are avoiding _ implicitly.
Explaination
This part of your regex:
(?=.*?[^\w\s]) // Match special character

Is equivalent to:
(?=.*?[^a-zA-Z0-9_\s])

Bibliography

The \w metacharacter is used to find a word character.
A word character is a character from a-z, A-Z, 0-9, including the _
(underscore) character.

https://www.w3schools.com/jsref/jsref_regexp_wordchar.asp
Comment: I mean underscore is not a part of `special character`.
Comment: @Alexandru-IonutMihai The problem is that you are doing `[^\w\s]` which is equivalent to `[^a-zA-Z0-9_\s]`. So, the definition of _special character_ is not a letter, not a number, not space and not an underscore.

How to detect underscore in regex?


How to fix special characters in node titles?

Question: When I imported content from my legacy system using the migrate module some special characters didn't get imported correctly.
For example the " character got changed to "  and is showing as such in the node title.
Is there an easy option fix that when displaying or do I have to replace it at the database level?
Answer: Drupal does check_plain on titles which removes all html and does some conversion of chars (& -> &). Depending on how the migration went, that could be part of the problem (escaping escaped content). But it's hard to say what the problem is without knowing what you actually got in the database.
Drupal uses the philosophy to store everything raw in the database, and escape it on output. If the legacy system did it differently, you could have the data stored differently than what Drupal expects, and would then need to fix it at the database layer.
Comment: That is correct: User input is escaped when it is output, not when it is saved. Similarly, the node content is passed through the input filters when the node is rendered, not when the node content is saved in the database.
Comment: thanks. Looks like the data came over as '"' and got stored like that in Drupal too. Looks like I have to update the node table.


doctrine:migrations:diff give "No changes detected in your mapping information"

Question: I'm using doctrine in combination with symfony. For the database setup I'm using annotations. I created a table successfully but gave the wrong format integer for a field city which I need to change to string. My understanding was, that when I'm changing the annotations in the customers class from
class Customer{

  /**
   * @ORM\Column(type="integer", nullable=true)
   * @var string city
   */
  private $city;

}

to
class Customer{

  /**
   * @ORM\Column(nullable=true)
   * @var string city
   */
  private $city;

}

and then run
php bin/console doctrine:migrations:diff

all changes of the mapping should be recognized and a php file should be generated containing an ALTER TABLE query or similar. However, this command replies with a "No changes detected in your mapping information". What am I missing?
Comment: @ORM\Column(type="string")
Comment: type string is default. However, I tried your suggestion but the behavior remains the same.
Answer: I needed to first clear the cache with
php bin/console doctrine:cache:clear-metadata

Success!
Answer: You should add the annotation @Entity to the entity to be recognized as an entity in doctrine ORM.
<?php

/**
 * Customer
 *
 * @ORM\Table(name="customers")
 * @ORM\Entity(repositoryClass="YourBundle/Repository/YourRepositoryName")
 */
class Customer{

       /**
        * @ORM\Column(type="string", nullable=true)
        * @var string city
        */
        private $city;

}

You can also use this command to auto-generate entities with different required annotations:
bin/console doctrine:generate:entity

Comment: Sorry, I failed to give the full code. Of course there is the Entity annotation in my code.
Answer: This can also happened when your mapping is invalid.
A quick workaround would be to do
php bin/console doctrine:schema:validate
Then fix errors until you see
[OK] The mapping files are correct.
Now make php bin/console doctrine:migrations:diff again and it should work.
Answer: In my case, I had to add the --from-empty-schema flag.
doctrine:migrations:diff --from-empty-schema

Answer: Similar to @agoldev I needed to clear the cache. Just not the doctrine metadata cache but the symfony cache. But he lead me to find this solution.
php bin/console cache:clear

Then it worked.
Answer: In my case, i had mapping type 'attribute' in doctrine config file, but entity had annotations.
Answer: if you are using a complicated Symfony configuration check doctrine.orm.entity_managers.default.mappings config values in config.yml. Probably you have to add your new bundle here when it's not work.
Answer: In case it can help others, for me it was just a matter of a wrong namespace set in my entity class.
I had duplicated an entity from another package and forgot to update the namespace.
Answer: My problem was that I had placed my entity in bundle Doctrine folder but had to place it into Entity folder.


Sore fretting hand muscle

Question: When I play basic major scale from 8th fret, as I go up the tempo my left hand starts ache. The muscle that is tight is the base of the palm, the one that's used by Karate masters to brake bricks :-)
Neither my action is high, nor my left hand is weak. I have been practicing daily for the last 4 months and I did play guitar and bass during my teenage years (so muscles should have been adapted together with body development). I feel the pain mostly because of my pinky movements, when I stretch to fret with my index after pinky and vice-versa. I tried to change position to more classical one (bass on the left leg), but still, 60 bpm playing 16 notes and the palm is on fire. I have no other issues with my pinky when playing regular riffs. My hand is very small, glove size XS, but I don't think that this should be the limitation. Neither I feel that I fret hard. What can I do with my technique to be able to play major scales for at least 5 minutes 16th notes?
Answer:
Why do you want to play 16th notes at 60 bpm for 5min straight?  Is this a self imposed goal?
It would help if you post a pic or vid of your hand position while playing.  Even though you state you have played guitar and bass for a while and think you have good hand strength and good action you might have poor hand posture which is causing stress in the palm leading to your problem.
If it "burns" then it may be a pinched nerve and that's serious.  You should stop playing and see a doctor.  If it really is just muscle ache, and it is caused by the picky stretch you describe, you might have to adopt a new strategy for playing those notes, like shift instead of stretch.  I had a friend in high school who dislocated his pinky trying to develop a stretch on the upright.  It didn't hurt so he left it like that to get the reach he wanted.  I have no idea if he ever corrected it but he still plays professionally.

In short, I would recommend (1) posting a pic of your hand position for review and (2) see a doctor, possible (3) take lessons if you are not already and the instructor might be able to correct something right away.
Answer: Difficult to rectify without some pictorial evidence  - position, etc., but a possibility is the bass is too low, which will stretch that part of your fretting hand, and/or you need to raise the head of the bass. This may sound a bit daft, but try the whole bass at 45 degrees, instead of parallel to the floor. It changes all the angles (obviously!), making it more like an upright bass to play.
Another obvious is see a doctor, who is medically trained to diagnose problems and their causes. Yet another is to see a teacher, who should be experienced enough to see where any potential problems lie.
Comment: I recall working with a bass player in a big band decades ago who had a fender Precision electric bass and rigged up a stool with plank between his legs that allowed him to position it exactly like an upright.
Comment: I had a student who was struggling to play a tricky passage smoothly at a fair pace. I suggested holding his bass guitar pretty well upright, and he just laughed. After a bit of persuasion he re-positioned it - and played the passage perfectly. Every time after that, when he had problems, he'd look at me, shake his head, hold the bass almost upright, and succeed.
Answer: You mention playing a major scale on the 8th fret, can I assume you are talking about a C major scale starting on the E string with your 2nd finger and playing one finger per fret playing:
E: 2-4
A: 1-2-4
D: 1-3-4
What about when you play lower on the neck where the notes are further apart, is it worse there? I'm guessing it probably is.
You mentioned having small hands, if your fingers do not lay comfortably over 4 frets this will cause a lot of tension in the muscles especially for repetitive patterns.  One solution is to use upright bass fingering, only use the index, middle and the pinky and ring finger simultaneously. This involves having to shift a lot more but it becomes second nature after a while. The fingering for the same major scale would be this:
E: 1-4-(shift 1 fret down)
A: 1-(shift 1 fret up)-1-4 (shift 1 fret down)
D: 1-(shift 1 fret up)-2-4
If this isn't clear let me know. It seems like a lot of shifting, and it is but upright players are so used to this that it becomes second nature to us. (I play both)
Another option is if you are playing a 34" scale bass you can consider switching to a 30" scale bass so the frets are closer together. Hope this helps.
Answer: All the answers above are good involving technique.
Perhaps concentrate on how tightly you are gripping the  neck. It's really not about how much strength you have, or the size of your hand. Guitar is more about finesse, and you need to only press ever so lightly to make those notes sound. The muscle memory is created by repetition not by how strongly you are able to repeat the forms. If you feel that is not the issue, I agree with @tim above, you should seek a doctor to ensure you have no medical issue in your hands /tendons etc.
Good luck ~catz

Sore fretting hand muscle


Predicate logic: There is no largest prime number.

Question: I have come up with the following expression for "There is no largest prime number."
$$\neg \left( \exists q. \left( Prime(q) \wedge \left( \forall p. Prime(p) \wedge q \ge p \right) \right) \right)  \tag 1$$
However a solution given is:
$$\neg \left( \exists q. \left( Prime(q) \wedge \left( \forall p. Prime(p) \implies q \ge p \right) \right) \right)  \tag 2$$
The only difference is that there is an implication used instead of an conjunction.
Why is (1) wrong? How do the truth tables for (1) and (2) differ and why?
Answer: $(\forall p. Prime(p) \land q \ge p )$  is saying that every natural number (since the universal quantifier quantifies over your whole domain ... which is the natural numbers) is both a prime and smaller or equal to $q$. Well, just focusing on that first part: clearly it is not true that every natural number is prime. So with the conjunction instead of a conditional, that part ends up makes the rather trivially false statement that every number is a prime ... clearly not what you want.  What you want is that if you have some prime $p$, then that must be smaller or equal than the largest prime $q$.
Now, in predicate logic, you can't really use truth-tables anymore, because you start dealing with infinitely many different kinds of worlds (worlds in which there is 1 object, worlds in which there are 2, etc.)
However, we don't have to resort to a truth-table to show that these statements can take on different truth-values and are therefore not equivalent.  We just need to create some interpretation of the predicates that makes one true and the other false. Now, as to not be distracted by the predicates you are using, let's first abstract the 2 statements to:
$\neg \exists q (P(q) \land \forall p (P(p) \land R(p,q))$
$\neg \exists q (P(q) \to \forall p (P(p) \land R(p,q))$
In other words, we have some 1-place predicate $P$, and some 2-place predicate $R$ ... left to be interpreted any way we want.
OK, now ... let's consider a world where no objects have property $P$. Then clearly $P(q) \land \forall p (P(p) \land R(p,q)$ is false, and hence it is true that $\neg \exists q (P(q) \land \forall p (P(p) \land R(p,q))$. On the other hand, $P(q) \to \forall p (P(p) \land R(p,q)$ is vacuously true, and hence is is false that $\neg \exists q (P(q) \to \forall p (P(p) \land R(p,q))$. OK, so there is your counterexample to the claim of their equivalence
Comment: @FizzleDizzle In logic you are dealing with statements. Those statements are true or false depending on the situation. For example, the statement "It rains" would be true when it is raining, but false when it is not raining. Or, as another example: "The sum of the angles in a triangle equals the sum of two right angles" is true in Eucledian geometry, but false in non-Eucledian geometry. This is why we say that the truth of a claim depends on what world we evaluate it in.  (cont'd)
Comment: @FizzleDizzle Indeed, any "logically possible" world is a world without contradicitons. So, I could consider worlds where pigs fly, where grass is purple, etc.  The rows in a truth-table show these different kinds of worlds: A row where $P$ and $Q$ are both true represents the world (or rather, all kinds of worlds) where $P$ and $Q$ are both true. etc. And now we can also say things like: A statement is a tautology if and only if it is true in all possible worlds ... i.e. it is true under all circumstances/in all situations/no matter what
Comment: What do you mean by "world" and "different kinds of worlds"? (I haven't heard the word "world" in logic since I'm a complete beginner.)
Comment: @FizzleDizzle P.s. For an absolute beginner you're doing very well on these symbolizations!