What are the signs that a classification just wont work?

Question: Basically, I've a simple question:
Are there signs which tells me that I just can't classify my data?

To make things clear, I've a dataset whereby i try to predict the amount of tires changes.
But something tells me that this will be quite hard.
For example, here is a sneak peak of my training set.

2148 -- 0 tires changes
230 -- 1 tires change
1984 -- 2 tires changes
570 -- 3 tires changes
2791 -- 4 tires changes
889 -- 5 tires changes
2645 -- 6 tires changes
807 -- 7 tires changes
1819 -- 8 tires changes
512 -- 9 tires changes
1699 -- +9 tires changes

As you can see, it is very unbalanced, especially with the odd amount of tires changes.
Furthermore, in my dataset, I've +/- 150 features and 95% of those features are basically, dummy data e.g.

gear_4_dummy -- {0,1}
gear_5_dummy -- {0,1}
gear_6_dummy -- {0,1}
gear_7_dummy -- {0,1}
doors_3_dummy -- {0,1}
doors_5_dummy -- {0,1}
seats_2_dummy -- {0,1}
seats_5_dummy -- {0,1}
break_dummy -- {0,1}
berline_dummy -- {0,1}
sport_dummy -- {0,1}
...

and inly 5% of the data are numbers e.g.

price -- numeric
tire_size -- numeric
end_km -- numeric
km_a_year -- numeric
car_price -- numeric
...

Now the thing is, whatever I try the results are just awful...
I've tried Regression, I've tried classification, I've tried clustering.
But all of them won't give me a decent result.

The best thing I can do is +:- 20-35% of accuracy but that would almost be the same as guessing because of the unbalanced data ...

For example, this is what I did recently:

I created a training/val set (data > 2003 and data <=2012)
I created a test set (data > 2013)
I resampled the training/val set (randomly) in such a way that my target class (amount of tires changes) represented each class, equally
(This would mean that, every class contains 460 participants)
460 -- 0 tires changes -- (random sample)
460 -- 1 tires changes -- (doubled the amount)
460 -- 1 tires changes -- (random sample)
460 -- 1 tires changes -- (random sample)
...

By resampling the training set, I believe that the classifier won't be tempted to bias the even pair tires changes. (or with other words, odd pair tires changes are also important)
I trained / tested my dataset
Decision tree
Random forrest
Naive Bayesian
Logistic regression
Regression
...

Without any difference
But
When i took a look, and investigated my data even more then before. I saw something strange.
And that is basically the purpose of my question.
when I investigated the data (in weka) (I was working in python/notebooks/sklearn) I noticed something. And I wanted to ask if it is normally...
Thus when I looked at my data, I noticed that all of my targets are almost equally represented in each and every variable/parameter/bucket/data piece.

E.G here is a picture with my target data, after the resampling

And here are data pictures from other features, whereby the Target features are in coller
Start year

End year

Sales year

Lifecycle ages

Month vs new model

rw per

**end km **

**Euro  **

**4th quarter **

Tire diameter

As you can see,
It is just like that every column contains a proportional equal amount of target values. (I even would say, it looks like it is for 80-90% random) ...

In contrast, If you look at the Iris data set for example.

Then we see clearly, different things. Like for example, not every column contains a proportional equal amount of target values. And it makes much more sense that a classifier can classify this data set much better. (you can almost do it by just looking at it.
a picture of my confusion matrix

Nevertheless. If someone has some (good) tips or ideas, or an answer on my question. I will always appreciate that.
Regression (sklearn)
As mentioned before, I also tried to execute a regression model
But the results where also not that good. (probably because of the lack of numeric values?)
What I tried was:

Loop over all the Features

Calculate the P-value and MSE

Take the Feature with the best result
Loop over all the Features in combination with my previous best feature set

Calculate the P-value and MSE

Take the best new features set
Loop over all the Features in combination with my previous best feature set
...

The above algorithm is a bit simplified (I putted every improvement in a priority queue and took the best value, to calculate a new regression (i removed previous calculated feature sets))
Here under, you can see a list, of combinations and values
P-value, MSE, #Features, Features-used
0.23001432437,10.624070884777447,1,end_kms
0.252706916701,10.310964138620037,2,tire_diameter,end_kms
0.261575392281,10.188599117839662,3,tire_diameter,end_duration,end_kms
0.269643515788,10.077277155947542,4,personalcar_dummy,tire_diameter,end_duration,end_kms
0.276184144098,9.987031192941254,5,end_kms,tire_diameter,end_duration,schade_per,personalcar_dummy
0.280545142538,9.926859220879235,6,personalcar_dummy,end_duration,end_kms,schade_per,mnth_vs_new_model,tire_diameter
0.284413337973,9.873486822151795,7,personalcar_dummy,end_duration,end_kms,schade_per,seats2_dummy,mnth_vs_new_model,tire_diameter
0.287928576315,9.824984437619571,8,personalcar_dummy,merkcode_AL_dummy,end_duration,end_kms,schade_per,seats2_dummy,mnth_vs_new_model,tire_diameter
0.291212518569,9.779673419004128,9,personalcar_dummy,end_duration,end_kms,seats2_dummy,schade_per,berline_dummy,tire_diameter,mnth_vs_new_model,merkcode_AL_dummy
...
130 iterations further
...
0.323463084764,9.33468812619479,84,seats4_dummy,merkcode_LA_dummy,coupe_dummy,sale_mnth_1_dummy,end_duration,sale_year_2011_dummy,merkcode_RE_dummy,inzet_year_2012_dummy,euro5_dummy,lifecycle_age_class,sale_year_2010_dummy,seats2_dummy,tractie_a_dummy,berline_dummy,merkcode_IV_dummy,break_dummy,merkcode_AU_dummy,motor_pk,sale_year_2007_dummy,euro4_dummy,inzet_mnth_5_dummy,seats8_dummy,inzet_year_2007_dummy,cat_prijs,merkcode_RO_dummy,merkcode_SU_dummy,inzet_year_2009_dummy,euro3_dummy,sale_mnth_12_dummy,business_dummy,sale_year_2016_dummy,merkcode_VO_dummy,uitvoering_HYBRIDE SPORT_dummy,roadster_dummy,gear_auto_dummy,merkcode_MA_dummy,sale_b2c_dummy,tire_ratio,merkcode_PE_dummy,seats6_dummy,uitvoering_BUS_dummy,merkcode_NI_dummy,sale_mnth_5_dummy,merkcode_SS_dummy,inzet_year_2004_dummy,merkcode_DC_dummy,model_year_2008_dummy,gear7_dummy,kwart_Q4_dummy,sale_year_2012_dummy,model_year_2011_dummy,merkcode_SM_dummy,gear5_dummy,inzet_mnth_4_dummy,uitvoering_HYBRIDE_dummy,sale_mnth_7_dummy,schade_per,inzet_mnth_12_dummy,sale_year_2015_dummy,merkcode_TO_dummy,merkcode_AL_dummy,tire_diameter,inzet_year_2006_dummy,inzet_mnth_10_dummy,uitvoering_HYBRIDE HIGH_dummy,uitvoering_vip_dummy,rw_per,model_year_2012_dummy,kwart_Q2_dummy,seats5_dummy,personalcar_dummy,model_year_2007_dummy,merkcode_OP_dummy,merkcode_CV_dummy,merkcode_CH_dummy,seats9_dummy,end_kms,merkcode_FO_dummy,missing_gear_dummy,mnth_vs_new_model,seats7_dummy,sport_dummy,tire_width,merkcode_VW_dummy

Comment: Too long question, couldn't read it all, but I have a suggestion: find a more appropriate evaluation metric. Predicting (exactly) the number of tire changes might be unfeasible, but is it actually necessary? Perhaps an expected value and a prediction interval would be better. Also, give a look at ordered regression (ordered logistic regression) and poisson regression as well.
Comment: I would take @Firebug's comment seriously w.r.t. length. In particular, I do not think the extensive diagnostic figures are likely to get you a better answer, and may or may not be relevant to your issue. (At the least, you could consolidate the data distributions into one figure, cropping vs. full-on screen-caps; and also omit the Iris figures)
Comment: I'd recommend treating this as a regression problem; if you predict 3 changes and the actual is 4, that's pretty good. If you predict 3 and the actual is 9, that's not so great. And personally I'd try a randomForest -- as a regression, not as a classifier. It's plausible to me that most of your variables aren't important, e.g. number of seats.
Comment: Wouldn't "predict the amount of tires changes" be more a *regression* problem than a *classification* one?
Comment: @GeoMatt22 - My best p-value was 0,42 with a MSE of 10 :s
Comment: @GeoMatt22 - I've already to tried regression (sklearn) but even then, my values aren't that great at all. I'm just wondering, could it be, that the amount of tires changes - is basically a random event which has nothing to do with the type of car you're driving (random, unless i've better features)
Comment: Dieter: that sort of information (p-value) should be edited into your question, rather than as a comment. As for my comment, I meant you seem to be predicting a count, rather than a label. On your 2nd comment: I would be highly surprised if the # tire changes is unrelated to things like age and/or cumulative driving distance.
Comment: @GeoMatt22, I've added extra info about the regression. And yes, the end_km and tires width etc have an influence, but it isn't what i would expected
Comment: Also note that a regression problem for count data may require a different approach than for a continuous regression problem or a classification problem (e.g. see [here](https://stats.stackexchange.com/questions/3024/why-is-poisson-regression-used-for-count-data)).
Comment: But you may have a lot of randomness left -- you don't know the quality of the roads, whether the car is being driven past a lot of construction sites where there are nails, etc.
Comment: I also find it interesting how even tire changes are more likely. It makes sense actually. You can try to use this information as well, perhaps concatenate the "classes" into "1 OR 2 changes" instead for example.
Comment: @Firebug perhaps a result of sales pressure when replacing a problematic tire?
Answer: Besides poorly forming the overly long question you are using classification in an incorrect context.  You need risk estimation.  See http://www.fharrell.com/2017/01/classification-vs-prediction.html
Also be sure to use a proper accuracy score: http://www.fharrell.com/2017/03/damage-caused-by-classification.html
Answer: very long question.. however, let me tell that classification algorithms usually tries to pick the repeated patterns and adds rules accordingly. no matter how the format of your data looks like. It only depends on the classifier's ability to find these hidden patterns. Numerical values sometimes cause a problem if they vary widely and thus cause messy kind of logic. Sometimes you can limit the data ranges to reduce the classifier bias towards big values by either normalizing the data to limit their ranges and thus make the learning process easier or you can build your own rules to turn the numerical values to nominal ones to ease the learning process on your classifier. as an example for such rules:
if age ranges between 00-20 --> young
if age ranges between 20-40 --> Middle-age
if age ranges between 40-60 --> old

it may also fail in classification if the missing values of the data is huge . However, if this is the case, preprocessing your data before classification should be the best option for you.
Hope that helps.


Compare $L_2$ norm of Fourier series of $ f$ and $ f'$

Question: Let $f$ be a periodic continuous function on $[0,2\pi]$ with $f'$ continuous.
Let the Fourier series of $f$ be ${a_o\over 2} + \sum_{n=0}^\infty(a_n\cos nx  + b_n \sin nx).$
Is it true that $$||f-{a_0\over2}||^2 \leq ||f'||$$ , where ||.|| is the $L_2$ norm?
I have tried some functions and this seems to hold, but I don't know how to prove it.
My guess is to compare the Fourier series of $f$ and $f'$.
Let $s_n$ be the n-th partial sum of the Fourier series of $f$, then in $L_2$ $s_n$ converges to $f$. Let $\sum$ be the infinite summation part of the Fourier series ($s_n-{a_0\over2}$) then
$$||f-{a_0\over2}|| \leq ||f-s_n||+||\sum||$$
but since $||f-s_n||$ can be made as small as we want,
$$||f-{a_0\over2}|| \leq ||\sum||$$.
If I can compare $\sum_{n=0}^\infty(a_n\cos nx  + b_n \sin nx)$ and $f'$I can get somewhere....
Comment: What are your approaches?
Comment: What happens for $f (x)=\sin(x/2)$?
Comment: @ Friedrich Philipp, I added my approach so far.
@JeanMarie, I don't get what you're hinting at
Comment: @Niebla ; $\{\sin(nx)\}_{n \ge 1} \cup \{\cos(nx)\}_{n \ge 0}$ is an orthogonal family of elements of $L^2([0,2\pi])$, this means that $\|\sum_n (a_n \cos(nx)+b_n \sin(nx))\|^2 = \sum_n (|a_n|^2 \ \|\cos(nx)\|^2 +|b_n|^2 \ \|\sin(nx)\|^2 )$
Comment: @user1952009 I don't see where that leads. I mean, if I can compare term by term the Fourier coefficients of $f$ and $f'$ then I'm done, but I don't see how
Answer: I think I got it!
So I followed user1952009's suggestion.
Consider the Fourier series of f':
$$ {A_o\over 2} + \sum_{n=0}^\infty(A_n\cos nx  + B_n \sin nx)$$
Now, $A_o=\int_0^{2\pi}f'd{x}=f(2\pi)-f(0)=0  $ by periodicity of $f$
$A_n=\int_0^{2\pi}f'\cos{(nx)} dx$ integrating by part, this can be written in terms of f and you find $A_n=-nb_n$. Similarly $B_n=na_n$.
$f'$ is Riemann integrable, so its Fourier series converges to $f'$ in $L_2$ norm.
Also, $||\sin nx||=||\cos nx||$. Clearly $|a_n|\leq |na_n|$.
Therefore
$$||f-{a_n \over 2}||^2=||\sum_{n=0}^\infty(a_n\cos nx  + b_n \sin nx)||^2=\sum_{n=0}^\infty(|a_n|^2||\cos nx||^2  + |b_n|^2 ||\sin nx||^2) \leq $$
$$\leq\sum_{n=0}^\infty(|na_n|^2||\sin nx||^2  + |nb_n|^2 ||\cos nx||^2= ||f'||^2$$
And the desired inequality follows.
Let me know if you think that this works.
Answer: I believe your proof works, but it's probably overkill (for my taste at least). Once you realise that $\frac{a_0}{2}$ is just the mean value of $f$, you will see that your claimed inequality is nothing but the Poincaré(-Wirtinger) inequality, so you are done.


Quais são as causas que levam uma session php a expirar?

Question: No php as sessions são uteis pra segurar dados temporariamente. Mas as minhas estão expirando muito rápido (em menos de 15 minutos) e eu ainda não consegui entender o por que.
Meu session.cache_expire, de acordo com o phpinfo() está setado pra 240.
Desde já agradeço.
Comment: Você já viu se está setado como segundos ou minutos? Se for segundos, dá 4 minutos... Aumente, e faça o teste
Comment: De acordo com a documentação do php.net é em minutos. E eu testei aqui e ela expira antes desses 4 minutos, inclusive.
Comment: Tentou aumentar o tempo pra ver?
Comment: Tentei agora. Não resolveu.
Answer: Você tem que alterar o parâmetro session.cookie_lifetime mais informações no site do PHP

Update:
Tenta aumentar também session.gc_maxlifetime ex.
ini_set('session.gc_maxlifetime', 3600);
session_set_cookie_params(3600);
session_start();

Comment: Tentei, e mesmo assim ainda não resolveu.
Comment: Você esta usando um shared host? Se for isso a pasta pode estar sendo apagada pelo servidor?
Comment: Estou sim, se for o caso da pasta estar sendo apagada pelo servidor, o que eu poderia fazer para reverter?
Comment: Se der tentar criar um php.ini personalizado e mudar o path da session. Ou abrir um ticket pro suporte.
Comment: Vou tentar, conforme for, notifico aqui do que aconteceu.
Mesmo assim, muito obrigado!


Good sources to learn about Geometric Analysis

Question: So far the topics that have captured my interest the most are analysis and geometry. I want to learn a bit more about geometric and global analysis. There seems to be no shortage of geometry books but I was wondering what I should learn in the way of analysis. As of now I know some basic measure theory and functional analysis and was hoping for somebody to point me in the right direction. Thank you.
Comment: The book by J. Jost (Riemannian geometry and geometric analysis) is quite analytic, did you try that yet?
Answer: Here are three (somewhat standard) references:

Jurgen Jost, "Riemannian Geometry and Geometric Analysis."
Peter Li, "Lecture Notes on Geometric Analysis."
Thierry Aubin, "Nonlinear Analysis on Manifolds. Monge-Ampere Equations."

Jost's book is on its sixth edition.  Aubin's book has a first and second edition, although my understanding is that the first edition might actually be more suitable.
There are also some specialized texts that are nevertheless still written in an accessible manner, and might serve as introductions to geometric analysis.  Examples include:

Ben Andrews, Christoffer Hopper, "The Ricci Flow in Riemannian Geometry."
Peter Topping, "Lectures on Ricci Flow."
Jerry Kazdan, "Applications of Partial Differential Equations to Problems in Geometry."

Most of the above books only assume standard courses in real analysis and Riemannian geometry as pre-requisites.  In particular, I don't think that any PDE knowledge is explicitly assumed (except possibly for Kazdan's book).
Still, since you asked about analysis, here are some topics in analysis that I, as a beginner in geometric analysis, have found important:

The Spectral Theorem (in functional analysis)
Sobolev spaces
The Laplace equation (PDE)
The heat equation (PDE)
Second-order elliptic equations (PDE)
Calculus of variations
Hausdorff measure and rectifiable sets (Geometric measure theory)

Again, you don't need to know any of these topics to begin reading, but they couldn't hurt.
Comment: I've been planning on reading "Functional Analysis, Sobolev Spaces and Partial Differential Equations" by Haim Brezis for the first 4. What is good for Calculus of Variations?
Comment: @AbrahamRabinowitz: To be honest, I wish I knew.  There's a pretty substantial chapter on it in Evans' book on PDE, but other than that I don't know.


Simple JavaScript math not making sense

Question: I feel like I'm a novice again. I thought I was long past these problems. below is a simple script with two function neither of which work. What am I missing. Any help appreciated.

function calculator() {
  var bee = document.getElementById("beerPerc").value;
  var win = document.getElementById("winePerc").value;
  var liq = 100 - (bee + win);
  document.getElementById("liquorPerc").value = liq;
}

function calculator2() {
  document.getElementById("liquorPerc").value = parseInt(100 - (document.getElementById("beerPerc").value + document.getElementById("winePerc").value))
}

    <input type="number" id="beerPerc" value="50" onkeyup="calculator2()">   % of Beer Drinkers

    <input type="number" id="winePerc" value="30" onkeyup="calculator2()">   % of Wine Drinkers

    <input type="number" id="liquorPerc" onkeyup="calculator2()">   % of Liquor Drinkers

Comment: What "doesn't work" mean? Please tell us what  do you expect from the script and how does it actually behave.
Answer: Two things:

Case is significant. If you declare the variable bee, you can't read it with Bee.
.value is always a string. You need to convert the strings to numbers:
var bee = Number(document.getElementById("beerPerc").value);

If you don't do this, + will perform string concatenation, not addition.
You don't need to call parseInt() on the result of a numeric calculation, that's always a number.
function calculator2() {
    document.getElementById("liquorPerc").value = 100 - (parseInt(document.getElementById("beerPerc").value, 10) + parseInt(document.getElementById("winePerc").value, 10)))
}

Comment: Thanks Barmar for the revisit of the rules. much appreciated

Simple JavaScript math not making sense


Sandboxed Mac App with helper command line tool

Question: I am getting the following error when submitting my app

Invalid Signature - the main app bundle MyApp at path MyApp.app is signed but the signature is invalid. The following error(s) were reported from codesign: a sealed resource is missing or invalid In architecture: i386

I have a small console app that lives inside Resources folder, it does not need root privileges at all. I do not have the source code of this helper so I can't rebuild it.
If I remove the console app from my bundle the error goes away.
What this command line helper app does is it takes a file process it and creates a new one based on some information passed by args.
So the question is, how can we embed command line tools to our Sandboxed app and bypass the above error? Again I don't own the source code of this helper tool.
Answer: I just had this problem and was able to solve it by signing the app myself using codesign in Terminal.
Here is how you do it:
<code>codesign --entitlements path/to/.entitlementsFile -s "3rd Party Mac Developer Application: Your Company (or whatever the name of your certificate is in your keychain)" ./TheNameOfTheTool
</code>
Hope that helps!


DotNet Core calling method via onscreen button

Question: In my DotNet Core application, I have a button set up with the some javascript placed in for my OnClick event. It looks like this:

    @(Html.DevExtreme().Button()
                             .Text("Press me")
                             .Type(ButtonType.Normal)
                             .Width(90)
                             .OnClick("notify")
    )

The commented out lines are what I've tried to call my target method but neither have worked. The underlying method I want to call is this:
public void RunBatch()
{
    Console.WriteLine("Re-running batch");
    TestOutput print= new TestOutput ();
    print.TestMethod();
 }

Then, what TestMethod does:
public void ProcessAdjustedBatch()
{
    Console.WriteLine("I have been called from the datagrid!!!!");
}

So after I press the button, I would expect to see the following log messages:

Pressed
re-running batch
I have been called from the datagrid!!!!

But all that I see is Pressed in my dev log. How can I achieve my intended output?
Comment: `function notify()` is evaluated at the client side, but I assume the `public void RunBatch()` is part of your backend. Therefore the browser can't use the function `RunBatch`. You could Expose an HTTP Interface for this method and invoe it over HTTP
Comment: These are two dofferent parts. First one is writing to browser log and the second one is writing to Console. You are mixing up client and server.
Answer: Console.WriteLine() will never shown in client's browser because it runs server-side and opened in server's console window. It is necessary to use Debug.WriteLine() if you want to see debug messages in immediate output window.
Also note that var a = '<%=RunBatch()%>' only executes inside ASPX pages, it cannot be executed in Razor which uses @{ ... } for code segment, and both debug methods are server-side methods.
If you want to show server-side debug message in browser's console with console.log(), you need to create a controller action that executes both methods which have string return type and then use AJAX inside notify() function which outputs the result in success part.
Here is a simple example:
Controller Action
public IActionResult GetMessages()
{
    var messages = new StringBuilder();

    messages.Append(RunBatch());
    messages.Append("\n"); // use newline to separate messages from different methods
    messages.Append(ProcessAdjustedBatch());

    return Json(messages.ToString(), JsonRequestBehavior.AllowGet);
}

Debug Methods
public static string RunBatch()
{
    string batch = "Re-running batch";
    Debug.WriteLine(batch);

    TestOutput print = new TestOutput();
    print.TestMethod();

    return batch;
}

public static string ProcessAdjustedBatch()
{
    return "I have been called from the datagrid!!!!";
}

View (script tag)

Now the console output should be like this:
pressed
Re-running batch
I have been called from the datagrid!!!!

Live example: .NET Fiddle
Related issues:
Where does Console.WriteLine go in ASP.NET?
Asp.net mvc Console.WriteLine to browser
How to use Console.WriteLine in ASP.Net MVC 3


How to create an assignment expression for the enum using the .NET Expression API in C#

Question: Using the TypeBuilder (.NET 4.6) API I have created a Enum object at runtime; w.r.t. dynamic code generation here is the yield:
public enum WorkflowModelProperty
    {
        WorkflowModelDate   = 1,
        WorkflowModelTime   = 2,
        WorkflowModelHeader = 3,
        WorkflowModelTitle  = 4,
        WorkflowModelID     = 5,
        WorkflowModelOntologyRoot = 6,
    }

Here is the code used to generate the enum object:
// Create a dynamic assembly in the current application domain and allow it to be executed.

var dynamicAssemblyName = new AssemblyName(theAssemblyName);
var assemlbyBuilderFactory = await CurrentAppDomain.DefineDynamicAssembly, dynamicAssemblyName, BuilderAccess);

var moduleBuilderFactory = await assemlbyBuilderFactory.DefineDynamicModule(theLibraryModuleName, dynamicAssemblyName.Name + DLLPrefix);

TypeBuilder enumTypeBuilderFactory = moduleBuilderFactory.DefineType(theEnumDefinitionName, TypeAttributes.Public, typeof (Enum), null);

enumTypeBuilderFactory?.DefineField("value__", typeof(int), FieldAttributes.Private | FieldAttributes.SpecialName);

Here is the code that adds Field Members to the Enum TypeBuilder object:
EnumHostInfoSetCache = EnumHostInfoSetCacheBuilder.ToImmutable();

// Okay we need to check to see if the internal states validate the DSL model passed into us.

enumHostInfoSet = EnumHostInfoSetCache.SingleOrDefault(
    enumHostEntry =>
        (enumHostEntry.Value.GetValueOrDefault().EnumDefintionName ==
         localTypeDefinitionDSL.TheEnumDefinitionName)).Value;

var enumBuilder = await enumHostInfoSet.GetValueOrDefault().EnumTypeBuilder;

EnumFieldBuilderMaterializer = EnumFieldBuilderConnectedObserver?.
                            SubscribeOn(Scheduler.CurrentThread).Subscribe(
                                delegate(Tuple<string, TEnumBaseType> enumFieldMetadata)
                                {
                                    try
                                    {
                                        var enumFieldName  = enumFieldMetadata.GetEnumFieldMemberName();
                                        var enumFieldValue = enumFieldMetadata.GetEnumFieldMemberValue();

                                        // Code Generate the Enum Field, thus creating a literal definition
                                        var enumFieldBuilder = enumBuilder.DefineField(enumFieldName, enumBuilder,
                                            FieldAttributes.Public | FieldAttributes.Literal | FieldAttributes.Static);

                                        // Set the Constant value on
                                        enumFieldBuilder.SetConstant(enumFieldValue);

                                        // Okay, at this point the Enum Metadata DSL Model is hooked-up to our source here and is not connected to
                                        // the Hot Observable source here in the Agent; as we push data into memory the Model should load itself.

                                        EnumFieldBuilderCacheBuilder?.Add(enumFieldName, enumFieldBuilder);
                                    }
                                    catch (Exception e)
                                    {
                                        throw new ProtoNexusAPIException(e.Message);
                                    }
                                },
                                delegate(Exception exception)
                                {
                                    CodeGenExceptionHandler(exception);
                                },

At this point I've got a fully-baked instance of the generated Enum Type. Now I want to create a assignment of on a dynamically generated instance of the enum type - here's that code:
delegate() // OnCompleted() Implementation!
                                {
                                    // Step 0: Prepare for generation of Dynamic Enum Assignment via Expression APIs

                                    var codeForEnumType    = enumBuilder.CreateType();
                                    var enumInstance       = Activator.CreateInstance(Enum.GetUnderlyingType(codeForEnumType));
                                    var enumInstanceX      = Activator.CreateInstance(codeForEnumType);
                                    var recordOfEnumFields = enumBuilder.DeclaredFields;

                                    // Okay let's set-up for dynamic materialization of Enum Assignment Expression code:

                                    try
                                    {
                                        // Experimental code for learning and teaching...

                                        var rightSideExp = Enum.ToObject(codeForEnumType, 2); // I know I can get to this ordinal value

                                        // Step 1: Enum has been created and is ready for use.
                                        var enumCode = Expression.Variable(enumBuilder, "enumCode");
                                        // Step 2: Okay we are ready to make the Assignment via the Expression.Assign API. We can set this up a number of ways
                                        var enumAssignmentSource = Expression.Constant(rightSideExp);  // This is the value we want to assign
                                        var enumRightHandSideA   = Expression.Constant(Enum.ToObject(codeForEnumType, 1), codeForEnumType);
                                        //var enumRightHandSideB   = Expression.Constant(3, codeForEnumType); // This DOES NOT WORK! - Argument does not Match!
                                        var enumRightHandSideC   = Expression.Constant(enumInstanceX, codeForEnumType);
                                        // Step 3:
                                        var workFlowDisplayCommand = Expression.Assign(enumCode, enumRightHandSideA);  // DOES NOT WORK - Argument does not Match!
                                        workFlowDisplayCommand = Expression.Assign(enumCode, enumRightHandSideC);      // DOES NOT WORK
                                        workFlowDisplayCommand = Expression.Assign(enumCode, enumAssignmentSource);    // DOES NOT WORK
                                    }
                                    catch (Exception e)
                                    {
                                        Debug.WriteLine(e.Message);
                                    }

                                    localTypeDefinitionDSL.SetEnumType(codeForEnumType, ref recordOfEnumFields);
                                    localTypeDefinitionDSL.SetEnumTypeInstance(enumInstance);

                                    if (EnumFieldBuilderCacheBuilder != null)
                                    {
                                        try
                                        {
                                            // Setup the Rx HOT Subscription:
                                            // Step 1: Load-up our Immutable Collection
                                            EnumFieldBuilderCache = EnumFieldBuilderCacheBuilder?.ToImmutable();
                                            // Step 2: Create our Observable Source
                                            EnumFieldBuilderCollectionSource =
                                                EnumFieldBuilderCache?.ToObservable(Scheduler.CurrentThread);
                                            // Step 3: Convert the Collection Source to a Hot Observable
                                            EnumFieldBuilderSubscriptionSource =
                                                EnumFieldBuilderCollectionSource?.Publish();
                                            EnumFieldBuilderSubscriptionSource.ObserveOn(Scheduler.CurrentThread);
                                            localTypeDefinitionDSL.LoadFieldBuilderSubscription(
                                                EnumFieldBuilderSubscriptionSource);
                                        }
                                        catch (Exception e)
                                        {
                                            throw new ApplicationException(e.Message);
                                        }
                                    }

                                    EnumFieldBuilderMaterializer?.Dispose();
                                });

Here is the runtime error message: Expression of type 'WorkflowMessagingCommands' cannot be used for assignment to type 'WorkflowMessagingCommands'
What am I missing?!
Answer: In first step of your code, you must change the first parameter of Expression.Variable method. The first parameter of this method, should be your type, and you are not passing the properly type for variable creating.
You must use like this:
var enumCode = Expression.Variable(codeForEnumType, "enumCode");

Comment: Ah yes, I knew it had to be a slight oversight - thank you Alberto that did indeed fix the problem.
Comment: FYI - When I used the **EnumBuilder** in place of **TypeBuilder** this code does not work.


If $A= \{ x_1 , x_2 , x_3 , \ldots , x_n \}$. how many subsets are there?

Question: I have a question stating

Let $A= \{ x_1 , x_2 , x_3 , \ldots , x_n \}$ be a set consisting of $n$ distinct elements. How many subsets does it have? How many proper subsets?

My thought is that there would be subsets with $1$ element, $2$ elements, $3$ element and so on, up to $n$ elements. The number of subsets of each size would be:
$$\begin{array}{c|c}
\text{Subset size} & \text{no. of subsets} \\
\hline
1 & n \\ 2 & n-1 \\ 3 & n-2 \\ \vdots & \vdots \\ n-1 &n-(n-2) \\ n & n- (n-1)
\end{array}$$
From this it seems the number of subsets would be $\displaystyle \sum_{k=0}^{n-1} (n-k) $. And for proper subsets, I would just not include the subsets of size $n$ , so $\displaystyle \sum_{k=0}^{n-2} (n-k) $. Is this correct?
Comment: So $\{1,2,3\}$ has two size-two subsets? Which one of $\{1,2\}$, $\{1,3\}$ and $\{2,3\}$ isn't a subset?
Comment: for each subset, each of the $n$ elements can be in the subset, or not in.  $2^n$ number of subsets, including the empty set. Without the set itself, nor the empty set, we have $2^n - 2$ subsets.
Comment: You know, you can always just explicitly count them for small sets and look for a pattern if you're not sure.
Answer: I disagree. For example, the number of subsets of size 2 is $$ \binom{n}{2} = \frac{n(n-1)}{2} \ne n-1 $$ and in general you are looking at
$$
\sum_{k=0}^n \binom{n}{k} = 2^n,
$$
which is also easy to see from fundamentals by a simple counting argument -- each of the $n$ elements can be either included or excluded, independently of others. So you have $2$ choices $n$ times, a total of $2^n$.
Answer: Is a well-known fact that the answer is $2^n$. Proof by induction:

$A_1 = \{x_1\}$ has two subsets: $\emptyset,A_1$.
Suppose true up to $n$. The subsets of $A_{n+1} = \{x_1,\dots,x_{n+1}\}$ are the subsets of $A_n = \{x_1,\dots,x_n\}$ plus the subsets of $A_n$ union $\{x_{n+1}\}$.
Answer: Why do you think there will be $n-(k-1)$ subsets of size $k$?
Let's say you have a set of $25$ elements, $\{1,2,3,....., 25\}$ and you want is see who many subset of size $2$ there are.
You can have $\{1,2\}, \{1,3\}......, \{2, 25\}$.  That's $24 = n- 1$.  But those are all with the element $1$.  What about subsets without the element $1$.
$\{2,3\}...... , \{2,25\}$ is $23$  and $\{3,4\}...\{3,25\}$ is $23$. and so on all the way to $\{24,25\}$ being $1$.
So adding those up we have $24 + 23 + 22 + ... + 1 = 300$.
But what about subsets of size $3$.  $\{1,2,3\} ... \{1,2,25\}$ is $23$ and $\{1,3,4\}$ to $\{1,3,25\}$ is $22$ and $\{1,2,3\}...\{1,24,25\}$ is $23 + 22 + ... + 1 = 276$ and $\{2,3,4\}...\{2,24,25\}$ is $22 + .... + 1 = 253$. and so on so all of the subsets of size $3$ = \sum_{m=1}^{23} \sum_{i=1}^m i = \sum_{m=1}^{23} \frac {m(m+1)}2$.
And then to do subsets of size $4$ we get.... well, a headache.
A bit of thought is that choosing $k$ elements from $25$ (or $n$) is, quite literally choosing $k$ elements from $25$ (or $n$).  So the number of subsets of size $k$ is ${n \choose k}$.  Literally.
[This would imply that ${n \choose 3} = \sum_{i=1}^{n-2}\frac {i(i+1)}2$.  Does it?]
So, anyway the number of proper subsets would therefore be:  $\sum_{i=1}^{n-1} {n\choose i}$.
But... can that be simplified.  You should play with that for about 14 hours or so.
$n =2\implies \sum{n\choose i} = 2$
$n = 3\implies \sum{n\choose i} = {3\choose 1 } + {3\choose 2} = 6$.
$n = 4 \implies \sum{n\choose i} = {4\choose 1} + {4\choose 2} + {4\choose 3} = 4 + 6 + 4 = 14$.
Notice anything?
Well, here's a suggestion.  Try figuring the number of all subsets including the two improper subsets, $\emptyset$ and $A$.
Then the solution is $\sum_{i=0}^n{n\choose i}$.
Then the number of all subsets is
$n = 2; \sum {n\choose i} = {2 \choose 0} + {2\choose 1} + {2\choose 2} = 1+2+1 = 4$.
$n =3; \sum {n\choose i} = {3\choose 0}+ {3 \choose 1} + {3\choose 2} + {3\choose 3} = 1 + 3 + 3 + 1 = 8$.
And $n= 4; \sum {n\choose i} = {4\choose 0} + {4\choose 1} + {4\choose 2} + {4 \choose 3} + {4\choose 4} = 1 + 4 +6+ 4 + 1 = 16$.
Notice anything.
It seems that the number of all subsets is $2^n$.
Why would that be?  We could probably prove $\sum_{i=0}^n{n\choose i} = 2^n$ by induction.... But what does it mean?
well, consider this single sentence:
For any of the $n$ elements of $A$, either the element is, or is not, in  a specific subset of $A$.
Think about that.  If we had considered that from the beginning, the whole thing  could have probably been easier.
Answer: There are
$${n\choose 0},{n\choose 1},...,{n\choose n-1},{n\choose n}$$
subsets from $\{x_1,...,x_n\}$ with no element, one element,..., $(n-1)$ elements, and $n$ elements respectively. Therefore the total number of subsets is
$${n\choose 0}+{n\choose 1}+...+{n\choose n-1}+{n\choose n}=(1+1)^n=2^n$$


difference between sizeof and strlen in C linux

Question: The first printf statement is giving output 3 and second giving 20.
Can anybody please explain what's the difference between the two here?
char frame[20],str[20];
printf("\nstrlen(frame)= %d",strlen(frame));
printf("\nsizeof(frame) = %d",sizeof(frame));

Thanks :)
Comment: @Stolas none is the size of the pointer, though one is the size of the array.
Comment: One is length of the string the other size of the pointer. This is basic C knowledge :)
Comment: @user3542109 Sure, your program is executing undefined behavior. You're using an incorrect format-specifier. `sizeof()` evaluates to type `size_t`, yet your invoking `printf()` telling it the value is a system `int`. use `%zu` for your second `printf()`. *Both* lines invoke UB (the former invokes `strlen()` on a buffer with indeterminate content, the latter for the reasons I just said).
Comment: `strlen()` evals *content* at **run-time** for its computation. `sizeof` is deduced at **compile-time** and is content-agnostic. *Both* are covered in any remotely-decent text on the C language.
Comment: @Stolas , in addition to nos comment, sizeof pointer is 4 on my system.
Comment: @WhozCraig could you please tell me why sizeof is returning 3 even if it's compiled time.
Comment: I stand corrected. I forgot the array over the malloc.
Comment: @user3542109 You are getting `strlen(frame) --> 3` because of the _garbage_ in `frame[20]` when code runs as `frame` is not initialized.  Another day or on another system you may get something different - it is undefined behavior.
Comment: @WhozCraig thanks for pointing me over it. I usually don't do it. but will do from now :)
Answer: sizeof is a compile-time operator and determines the size in bytes that a type consumes. In the case of frame (char[20]) that is 20 bytes.
strlen is a run-time function and scans a given pointer until the first occurrence of a nul terminator '\0' returning the amount of characters until then.
Comment: @user3542109 Because frame[] is an uninitialized array, and it probably contains bogus characters up to frame[3], which is '\0'. If you would declare it as "char frame[20] = {0}", strlen() would return 0
Answer:
Because the contents of frame is not initialized, which means it is not a valid C string, so strlen(frame) could return any value, or crash. Actually, its behavior is undefined in this case.
Because frame is an array of 20 characters, therefore sizeof(frame) will return 20 * sizeof(char), which will always be 20 (sizeof(char) always equals 1).
Comment: It's always giving 3 only.
Comment: @user3542109 Have you tested it on another operating system? Or building it with another compiler?
Comment: no.. u mean it has a value for a (system,compiler) ?
Comment: @user3542109 No, it could have any value literally, it even could change from run to run of your program. It is one of the undefined behaviors of C.
Comment: ok.. I will try it tomorrow. Today it's giving 3 every time. Still thanks a lot for explaining beautifully :)
Answer: strlen actually gives you the length of the string, whereas sizeof gives you the size of the allocated memory in bytes. It is infact quite nicely explained here http://www.cplusplus.com/reference/cstring/strlen/ Extract given below.

The length of a C string is determined by the terminating null-character: A C string is as long as the number of characters between the beginning of the string and the terminating null character (without including the terminating null character itself).
This should not be confused with the size of the array that holds the string. For example:
char mystr[100]="test string";
defines an array of characters with a size of 100 chars, but the C string with which mystr has been initialized has a length of only 11 characters. Therefore, while sizeof(mystr) evaluates to 100, strlen(mystr) returns 11.

And yes as per the other comments, you are trying to get length for uninitialized strings and that leads to undefined behaviour, it can be 3 or anything else, depending on whatever garbage is present in the memory that got allocated for your string.


Posterior distribution

Question: I read the following in some document:

Let $Y$ be a random variable with distribution $\mathcal{N}(\theta,\sigma^{2})$. The variance $\sigma^{2}$ is known. Let $p(\theta) = 1$ a flat prior on $\theta$. Then, the posterior distribution $p(\theta \vert Y)$ is $\mathcal{N}(Y,1)$.

From Bayes' theorem, I know that:
$$
\begin{eqnarray*}
p(\theta \vert Y) & \propto & p(Y \vert \theta) p(\theta) \\
 & \propto & \exp \Big( -\frac{1}{2\sigma^{2}} (\theta-Y)^{2} \Big) \\
\end{eqnarray*}
$$
To me, the posterior distribution would be $\mathcal{N}(Y,\sigma^{2})$. I don't understand why the document says it is $\mathcal{N}(Y,1)$.
Answer: You are correct assuming we have only one observation. The true form of the posterior is $N(\bar{Y},\frac{\sigma^2}{n})$. To demonstrate:
$$
\begin{eqnarray*}
p(\theta \vert Y) & \propto & p(Y \vert \theta) p(\theta) \\
 & \propto & \prod_{i=1}^{n}\exp \Big( -\frac{1}{2\sigma^{2}} (\theta-Y_i)^{2} \Big) \\
 & \propto & \exp \Big( -\frac{1}{2\sigma^{2}} \sum_{i=1}^{n}(\theta-Y_i)^{2} \Big) \\
 & \propto & \exp \Big( -\frac{1}{2\sigma^{2}} (n\theta^2-2\theta\sum_{i=1}^{n}Y_i +Y_i^2) \Big) \\
 & \propto & \exp \Big( -\frac{1}{2\sigma^{2}} (n\theta^2-2\theta\sum_{i=1}^{n}Y_i ) \Big) \\
 & \propto & \exp \Big( -\frac{n}{2\sigma^{2}} (\theta-\bar{Y})^2 \Big) \\
\end{eqnarray*}
$$
This also agrees with our intuition that the variance of the posterior should shrink with the number of observations we have.

Posterior distribution


Como acrescentar Divs a partir de um numero passado do Select?

Question: Poderiam me ajudar com um problema que estou tendo?
Bom gente preciso criar um select com valores de 1 a N numeros, após o usuário selecionar um numero no select tem que criar n divs a partir do valor selecionado.
Segue o exemplo não funcionando ...

$("#qtdLote").on("change",function(){
    var num=parseInt($(this).val());

    for(var i=1;i<=num;i++){ // se não houver valor o loop não roda.
      $("#area").append("
 Div "+i+"
");
      // Append adiciona ao final do elemento desejado.
    }

 });
div{
  border:1px solid #000;
}

Escolha
<select id="qtdLote">
  <option>1</option>
  <option>2</option>
  <option>3</option>
</select>

Pode ver também no codepen: https://codepen.io/anon/pen/JZYdjv
Answer: Você pode fazer um loop com o valor do select.

$(function(){
  // Use desta forma caso suas divs fiquem dentro de um elemento apenas para elas:
  $("#seletor").on("change",function(){
    var num=parseInt($(this).val());

    $("#area").html(""); // Limpar o elemento pai.

    for(var i=1;i<=num;i++){ // se não houver valor o loop não roda.
      $("#area").append("
 Div "+i+"
");
      // Append adiciona ao final do elemento desejado.
    }

  });

  // Caso contrário você vai precisar identificar as divs criadas
  // como dar uma classe para elas. Vou colocar um exemplo abaixo.

  $("#seletor-com-classe").on("change",function(){
    var num=parseInt($(this).val());

    $(".criadoDinamicamente").remove(); // Remover divs.

    for(var i=1;i<=num;i++){ // se não houver valor o loop não roda.
      $("#area").append("
 Div "+i+"
");
      // Append adiciona ao final do elemento desejado.
    }

  });
});
#area{
  display:flex;
  flex-flow: column;
}
#area div{
  flex-grow:1;
  padding:10px;
  border-radius:3px;
  background: #eaeaea;
  text-align:center;
  margin:0 0 10px 0;
}

<select id="seletor">
  <option value="">Quantas divs criar?</option>
  <option value="1">1</option>
  <option value="2">2</option>
  <option value="3">3</option>
  <option value="14">14</option>
</select>

Comment: Opa Henrique, Muito Obrigado pela ajuda mas esta quase certo, exemplo se ele escolheu 2 aparece 2, ai no caso se ele escolher 3 aparece 3 e se ele escolher 1 aparece só uma,  ele mostra e esconde a partir do valor dado, pode me ajudar ?
Comment: Creio que entendi, editei o meu código ali para você ver como ficaria, tem 2 opções ali que podem lhe ser úteis
Comment: Caraaamba Henrique era isso Mesmo !!! Muuuito Obrigado mesmo.
Answer: Já tem uma resposta que lhe atende em termos de solução, mas é sempre importante percebermos os nossos erros para podermos evoluir e não os cometer no futuro.
Este é o seu código Javascript:
$("#qtdLote").on("change",function(){
    var num=parseInt($(this).val());

    for(var i=1;i<=num;i++){
      $("#area").append("
 Div "+i+"
");
    }
 });

Na verdade tá muito perto de estar a funcionar como pretende. O primeiro problema vem no html de facto, pois $("#area") não acerta no elemento que devia porque você definiu ele com um # a mais:

Retirando o # para ficar id="area" já preenche os elementos, de acordo com o que seleciona. Mas acaba por não limpar os anteriores cada vez que escolhe outra opção. Pode fazer isso adicionando um $("#area").html(''); antes do for.
A unica coisa que fica a faltar é correr o código mal inicia, que consegue fazer apenas com mais uma instrução, apos o código que já tem:
$("#qtdLote").trigger("change");

O trigger força programaticamente o evento passado como parâmetro, neste caso o change, como se tivesse sido o usuário a faze-lo.
Veja o seu código com essas 3 pequenas alterações a funcionar:

$("#qtdLote").on("change",function(){
    var num=parseInt($(this).val());
    $("#area").html(''); //limpar antes de gerar
    for(var i=1;i<=num;i++){
      $("#area").append("
 Div "+i+"
");
    }
 });

$("#qtdLote").trigger("change"); //forçar o trigger para quando abre a página
div{ border:1px solid #000; }

Escolha
<select id="qtdLote">
  <option>1</option>
  <option>2</option>
  <option>3</option>
</select>


nodejs to return json response from an api call made within nodejs

Question: How to return the response from the API call within nodejs. I can see the body from the response printed onto the console. But how do i print or return as final output when i call http://localhost:8080/
http.createServer(function (req, res) {
    res.writeHead(200, { 'Content-Type': 'text/json' });
    var options = {
        'method': 'POST',
        'hostname': 'XXX',
        'path': 'XXX',
        'headers': {
            'Content-Type': 'application/json',
            'api-key': 'XXX'
        },
        'maxRedirects': 20
    };
    var req = https.request(options, function (res) {
        var chunks = [];
        res.on("data", function (chunk) { chunks.push(chunk); });
        res.on("end", function (chunk) {
            var body = Buffer.concat(chunks);
            console.log(body.toString());

        });
        res.on("error", function (error) { console.error(error); });
    });
    var postData = JSON.stringify({ "TEST DATA"});
    req.write(postData);
    req.end();
}).listen(8080);

Answer: Change the 'Content-Type' in res.writeHead to 'application/json' since you are returning JSON data.
res.writeHead(200, { 'Content-Type': 'application/json' })

You can rename the res variable inside the https.request callback to apiResponse to avoid confusion between the incoming request's res and the API response. Then use res.end(body) to send the API response body back to the client. Something like this,
var apiRequest = https.request(options, function (apiResponse) {
    var chunks = []
    apiResponse.on("data", function (chunk) { chunks.push(chunk) })
    apiResponse.on("end", function () {
        var body = Buffer.concat(chunks)
        res.end(body) // Send API response to the client
    });
    apiResponse.on("error", function (error) { console.error(error) })
})


A proper way to reuse existing type definitions

Question: When using a react component that passes some props to another react child component, I find myself rewriting some type definitions to the parent that was already defined in the child component.
interface ParentProps {
    onChange: (value: string) => void; // I end up rewriting this, when it was already written on ChildProps interface.
}

const Parent: React.FC<ParentProps> = ({ onChange }) => {
    return <Child onChange={onChange} label="Label 1" />;
};

// Child component. Could be imported from a third party library.
interface ChildProps {
    onChange: (value: string) => void;
    label: string;
}
const Child: React.FC<ChildProps> = ({ onChange }) => {
    return <MyComponent onChange={onChange} />;
};

Are there any techniques to avoid rewriting type definitions?
Answer: Depends how much of ChildProps you want to reuse.
If you want to reuse just a couple of properties, you can use in indexed type query to get the type of a specific property:
interface ParentProps {
    onChange: ChildProps['onChange']
}

Or you can define ParentProps to extend ChildProps if you want to reuse all properties:
interface ParentProps extends ChildProps {
}

Or you can pick just a subset using Pick:
interface ParentProps extends Pick<ChildProps, 'onChange'>  { // Pick<ChildProps, 'onChange' | 'label'> to pick more
}

Or if you want to pick all except a subset you can use Omit
interface ParentProps extends Omit<ChildProps, 'label'>  { // Omit<ChildProps, 'onChange' | 'label'> to omit more
}


ERROR 2002 MYSQL socket /tmp/mysql.sock

Question: A have already read everything about this error but I coundn't solve the problem.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (38)

I'm trying to install mysql on my Mac OsX but when I try to connect to the db (mysql -u root -p) I see this problem, ERROR 2002.
I have Mysql installed in /usr/local/mysql
Comment: Have you checked that the `mysqld` process is **running**?
Comment: When I write: mysql start, mysql gives me this error ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (38)
Comment: when I write /Library/StartupItems/MySQLCOM/MySQLCOM start It says Starting MySQL database server but in MYSQLPANEL on mac says: The mysql server instance is stopped.
Comment: it's a bit tricky sometimes to start MySQL on Mac OS X, I've write you one possible way to do it, hope it helps.
Answer: You say you get the error when you run mysql -u root -p
This will look for a socket file to connect. When mysqld started up, it is possible for the socket file not to be made and still have mysqld start up ( See my post Percona-server time out on /etc/init.d/mysql start )
Try using one of the following to see if you can connect without a socket file

mysql -u root -p -h127.0.0.1
mysql -u root -p -h127.0.0.1 --protocol=tcp

If you can connect with one of these, the shutdown mysql like this
mysqladmin -u root -p -h127.0.0.1 --protocol=tcp shutdown

mysqld should shutdown properly. Afterwards, startup mysql as usual
Give it a Try !!!
Comment: finally i got my local mysql setup repaired! thats the solution for my problem here: https://serverfault.com/questions/987663/another-insert-into-overwrites-existing-data
Comment: I get this with the no-socket commands: ERROR 1045 (28000): Plugin caching_sha2_password could not be loaded: dlopen(/usr/local/Cellar/mariadb/10.4.6_1//usr/local/Cellar/mariadb/10.4.6_1/lib/plugin/caching_sha2_password.so, 2): image not found
Answer: Check the MySQL daemon is running
You can check if the mysqld daemon from **Terminal Application* running:
ps -feax | grep mysqld

And verifying that you get something similar to this:
    0  8308     1   0   0:00.08 ??         0:00.11 /bin/sh /usr/local/mysql/bin/mysqld_safe

or looking for a mysqld process on the Activity Monitor application
If it's not running  you have to start the daemon.
Start MySQL daemon
On Mac OSX you can start the daemon from the Terminal Application with something like this:
sudo /usr/local/mysql/bin/mysqld_safe &

Again check that now the MySQL daemon is running as described before.
Comment: Ok a  did this and now I have this: 0  8308     1   0   0:00.08 ??         0:00.11 /bin/sh /usr/local/mysql/bin/mysqld_safe
Comment: But still happen the same problem. There is 2 folders inside /usr/local/mysql: mysql/ and mysql-5.5.28-osx10.6-x86_64/. Is That a problem?
Comment: @RafaelPontes if you can see that output from the `ps` command is that the daemon is running, try @RolandoMySQLDBA solution... you can also check the configuration file (usually `/etc/my.cnf`) and be sure the `mysqld` daemon is using that socket (`/tmp/mysql.sock`)
Comment: Sorry, I thought that when I use sudo /usr/local/mysql/bin/mysqld_safe & it worked, but just now I saw this error:
Comment: mysqld_safe Logging to '/var/lib/mysql/Rafaels-MacBook-Pro.local.err'.
121219 17:35:59 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
121219 17:36:03 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Comment: @RafaelPontes can you show us the last lines of that error log ( `tail -20 /var/lib/mysql/Rafaels-MacBook-Pro.local.err`)
Answer: I know this is an older post, but I spent the last hour or so bouncing back and forth between serverfault, StackOverflow, and DataBase administrators trying to figure out how to solve this issue after installing on Mac (El Capitan) using homebrew.
At least in my case, the advice from this post worked:
# Run the following commands
mysqld stop
mysql.server start

From what I can gather, I think (at least in my case) brew mysql hadn't initialized the permanent socket and my network admin locked down the ability to connect new processes via /tmp/...
Answer: I am using MacOX El Capitan 10.11.3 and I faced the above issue and none of the answers worked independently. I then tried a combination of user153254 and Adam's and it worked like a charm!
mysqld stop
mysql.server start
mysql -u root

Avoiding mysql -u root -p did the trick
Answer: main reason is mariadb/mysql is not running.
brew services start mariadb

OR
brew services start mysql

I solved same issue in MariaDB with bellow command.  restart MariaDB/MySQL
launchctl unload -w ~/Library/LaunchAgents/homebrew.mxcl.mariadb.plist
launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mariadb.plist

I have installed MariaDB though homebrew. i have also restart my computer before running command.
Comment: nope, not working
Comment: @trainoasis brew services start mariadb close your terminal and  reopen and check
Answer: I had the exact same problem and I searched every possible site for solution to such problem and was not be able to solve it. However, today I did a simple command line from the Mac's Terminal that is I just typed in mysql -u root and then it solved the problem for me.
Let me know if this solution helps.
Answer: Provide hostname along with username and password
mysql -u <username> -p<password> -h <yourhostname>

ERROR 2002 MYSQL socket /tmp/mysql.sock


Same linear transformation, different basis.

Question:
Let $\beta=\{(1,0,0),(0,1,0),(0,0,1\}$be a basis of $\mathbb{R^3}$ and $g: \mathbb{R^3} \to \mathbb {R^3}$ a linear transformation, which matrix is:
$$G=\begin{bmatrix}1 & 0 &-1 \\ 6 & 4 & -2\\ 0 & 1 & -1 \end{bmatrix}$$
Now let $\lambda=\{(0,0,1),(0,1,0),(1,0,0)\}$ be a basis also.
What is the matrix of $M(g;\lambda, \lambda)$?

I started to write each vector of $\beta$ into $\lambda$ coordinates.
$(1,0,0)_{\beta}=(0,0,1)_{\lambda}$
$(0,1,0)_{\beta}=(0,1,0)_{\lambda}$
$(0,0,1)_{\beta}=(1,0,0)_{\lambda}$
So, the change of basis's matrix, from $\beta$ to $\lambda$ is:
$$T=\begin{bmatrix}0 & 0 & 1 \\ 0 & 1 & 0\\ 1 & 0 & 0 \end{bmatrix}$$
This last matrix transforms any vector on $\beta$ coordinates into a vector on $\lambda$ coordinates. Then I wrote this scheme:
$$[v]_{\lambda} \to C \to [v]_{\beta} \to G \to [u]_{\beta} \to T \to [u]_{\lambda}$$
Where C is the change of basis's matrix from $\lambda$ to the canonic basis.
My difficult is how to merge theese matrix in one. Can You help me? thanks.
Comment: http://en.wikipedia.org/wiki/Change_of_basis#Change_of_basis_2
Comment: Try writing out the scheme as the multiplication of matrices rather than with the arrows. E.g. instead of $[\vec{v}]_\lambda\to C \to [\vec{v}]_\beta$, write $[\vec{v}]_\beta=C[\vec{v}_\lambda]$, and go from there.
Answer: What you need is nothing else but the matrix product $TGT^{-1}$.
Let's call the standard basis vectors $e_1,\ e_2,\ e_3$. Then we have $\beta=(e_1,e_2,e_3)$ and $\lambda=(e_3,e_2,e_1)$.
Now you have luckily $T^{-1}=T$, as $T$ simply exchanges the basis vectors $e_1$ and $e_3$.
But, for this specific exercise, we can find the answer more directly:
We need $G'=M(g;\lambda,\lambda)$. Because $G\cdot e_i$ gives the $i$th column of the matrix $G$, we have
$$g(e_1)=G\cdot e_1 = e_1+6e_2$$
This means that the third column of $G'$ will be $\pmatrix{0\\6\\1}$, as these are the $\lambda$-coordinates of the $g$-image of the third basis vector of $\lambda$.
Similarly, you can see what happens to the rest of the matrix.


Which Machine Learning Technique to use for Demand Forecasting?

Question: I have a dataset with 2 discrete predictors and I need to forecast demand of workforce required. The features basically describe applicant behaviour(stage of application e.g. online assessment, pending invite, interview round 1, under review etc). There's another predictor variable that states time frame but in categories e.g. been in a certain stage of application for >1 and <10 days, > 10 days.
As seen above, the discrete predictors are not binary(e.g. Male/Female) and could be divided within multiple categories. I have data back from 2019-Now.
I need to forecast for anywhere upto 6 months. I'm not sure if this holds any importance but the data will be analysed on a weekly basis.
Since this is a demand forecasting problem, I thought an ARIMA/Holt-Winters approach would work but I'm unsure if the size of the dataset is large enough(roughly 2 and a half years). Since I have discreet features as well, I thought a random forest model might work better?
The data is quite seasonal as well, and that could serve as a key deciding factor as well.
I would appreciate any suggestions one could give me in terms of which model I should look into? If anyone has more suggestions than random forest or arima and holt winters then that would be of great usage as well!
Comment: Your first paragraph looks chopped off. Can you give some more details about your two predictors? ARIMA and HW would not be the tools of choice.
Comment: Thanks for the response Stephan, The features basically describe applicant behaviour(stage of application e.g. online assessment, pending invite, interview round 1, under review etc.). There's another predictor variable that states time frame but in categories e.g. (been in a certain stage of application for >1 and <10 days, > 10 days.
Comment: I have edited the main post as well
Comment: With such few predictors the best thing is to always make visualizations and tabulations. you mentioned discrete predictors so you could tabulate out all possible combinations if the number of categories is not too many. This will give you an idea of the signal in the data. Then start with basic linear regression and see how it performs.
Comment: Thanks. I'm a little confused: you want to forecast the *demand* of workforce required, but your predictors are not related to demand, but to workforce *supply* (since they are about the applicants). Can you clarify?
Comment: Meanwhile, as I wrote, ARIMA and H-W have major problems with "long" seasonalities, i.e., more than about 12 periods per cycle. Weekly data with yearly seasonality have about 52 periods per cycle. [Take a look at this](https://robjhyndman.com/hyndsight/longseasonality/), and note that this framework can also include explanatory variables. It's essentially [regression with ARIMA errors](https://robjhyndman.com/hyndsight/arimax/).
Comment: I'm sorry for the confusion(i'm quite new to data science and hence made errors)! basically there's a percentage of applicants that would be hired after every applicant behaviour stage(this percentage varies), however, using this percentage, we find the people who are hired out of total applicants(= x% * total applicants). The task is to predict that number of hired (which has in the past satisfied demand, hence I called it demand forecasting). So the number of applicants hired or demanded depends on those predictors. Or would you say I'm thinking of it in the wrong way?
Comment: @bdeonovic thank you! I'll definitely look into that
Comment: Thanks for the clarifications! I would indeed not call this "demand" forecasting, but it really doesn't matter. I hope my answer is helpful.
Answer: Your best bet, as bdeonovic notes, is likely linear regression. The total number of people hired would be your dependent variable. Time-lagged values of the numbers of applicants in the various stages would be the independent variables. You may need to forecast these predictors out themselved, especially for longer-term forecasts.
If your predictors themselves are already seasonal, then your model may not require additional "seasonality treatment". However, since your predictors will need to be forecasted themselves, you will in this case likely need seasonality models here. ARIMA and H-W have major problems with "long" seasonalities, i.e., more than about 12 periods per cycle. Weekly data with yearly seasonality have about 52 periods per cycle. Take a look at this, and note that this framework can also include explanatory variables. It's essentially regression with ARIMA errors. So you will need to plug various models together to forecast your predictors and feed them into the final model, adding a seasonal treatment where appropriate.
Once you use (potentially multiple) lags of your predictors, your models will likely be over-parameterized. Look into ways of regularizing them, like the Elastic Net, Ridge Regression or the Lasso. There are various R packages that can help you with this.
Comment: Thank you so much for this, it's a very helpful starting point, and I'll indeed look into this!


Using MySqlHook, how to output query results to string

Question: I am using My_Sql_Hook to get a connection to my test-db and I want to save the output of a query to a variable as a string. Is it possible to use something similar to the fetchall() function for My_Sql_Hook?
Answer: You've access to the get_cursor method (from DbapiHook) on the hook object, which will return the underlying Cursor object for a connection; you can assume that object to match the DBAPI.


set xlabel matplotlib python

Question: I've got a matplotlib question that I want to ask.
In the below image I want to 'enlarge' the label, instead of having a range(0,1,0.2), I want a range(0,1,0.01) and visualize every step(0.01, 0.02, 0.03, ...).
This is my plot code:
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

fig, ax = plt.subplots(figsize=(12, 6),dpi=800)
count = (frecuency/len(acousticness))
width = 0.005
print(type(frequency))

ax.set_ylim([0, 3])
xlim=np.arange(0,1,0.01)
my_colors = 'rgbkymc'
plt.bar(frecuency.index, frecuency,width,edgecolor="Orange",color=my_colors)
plt.ylim(0, 5)
plt.xlim(xlim)
# Add labels
plt.title('Histogram of Acousticness')
plt.xlabel('Acousticness')
plt.ylabel('Frecuency')

Also, an error ocurred on plt.xlim(xlim):
ValueError: too many values to unpack (expected 2)
Comment: `xlim` is `x limits`. That's not what you're looking for, try [xticks](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.xticks.html).
Comment: frequency spelling mistake
Comment: As sarema and BigBen told me, what I want to modify are the Xticks and not the xlimits.
Answer: xlim=np.arange(0,1,0.01)
plt.xlim(xlim)

You are calling the wrong method. You probably want to change the second line to
plt.xticks(xlim)

to make it work. (Then it is a good idea to change the variable name as well.)

set xlabel matplotlib python


Permutation of set

Question: Suppose I have a set which contain finite elements from $1,2,\dots,n$, where $n$ is odd. Let there are two permutations $(a_1,a_2,\dots,a_n)$ and $(b_1,b_2,b_3,\dots,b_n)$ defined on this set. Now I have to prove that $$(a_1-b_1)^2(a_2-b_2)^2(a_3-b_3)^2\cdots(a_n-b_n)^2$$ is a even number.
My findings/questions are as follows :-

Every permutation of a set is either a cycle or product of disjoint cycles, so these to permutations are either be cycle or product of disjoint cycles.
Can the product of square terms can be a cycle of the set?
Or the given product of square numbers can be converted onto any one of permutations, so that can claim to even or odd.

Please clear it will be helpful for me
Comment: Welcome to MSE. For some basic information about writing mathematics at this site see, *e.g.*, [basic help on mathjax notation](/help/notation), [mathjax tutorial and quick reference](//math.meta.stackexchange.com/q/5020), [main meta site math tutorial](//meta.stackexchange.com/a/70559) and [equation editing how-to](//math.meta.stackexchange.com/q/1773).
Answer: A product of numbers is even if and only if at least on of the numbers is even, so it suffices to show that $(a_i-b_i)^2$ is even for some $i$. The square of a number is even if and only if the number you start with is even, so in fact it suffices to show that $a_i-b_i$ is even for some $i$.
Now $a_i-b_i$ is even if and only if either both $a_i$ and $b_i$ are even, or they are both odd. Since $n$ is odd, there are more odd numbers than even numbers in $\{1,\dots,n\}$ so some pair of odd numbers must be matched up by the permutations by the pigeon hole principle. This proves the claim.
Answer: Since $n$ is odd, one of the factors $a_i-b_i$ is even, where $a_i,b_i$ are odd. Then the whole product is even.

Permutation of set


How to calculate log-normal parameters using the mean and std of the given distribution

Question: I have the mean (u) and the standard deviation (sd) of a continuous distribution (X). How do I solve for the mean (u_log) and standard deviation (sd_log) of the log of that continuous distribution (log(X))?
I am looking to draw random numbers from a log-normal distribution. The problem is, most statistical packages have a log-normal-random-number-generator function that takes the mean of log(X) and sd of log(X) as inputs.
I only have the mean of X and std of X.
How to I solve for mean(log(X)) and sd(log(X)) given those inputs.
Comment: see below for answer:
http://en.wikipedia.org/wiki/Log-normal_distribution

In contrast, the mean, standard deviation, and variance of the non-logarithmized sample values are respectively denoted m, s.d., and v in this article. The two sets of parameters can be related as (see also #Arithmetic moments below):[3]
Comment: This reads like routine bookwork. Is this for some subject?
Answer: Let m and s be the mean and sd of $X$ on the original scale.  The appropriate mean and sd on the log scale can be found after a little algebra to be
$E(\log(X)) = \log(m) - \frac{1}{2} \log [ (s/m)^2 +1]$
$sd(\log(X)) = \sqrt{\log [(s/m)^2 +1]}$


stackless python constructor

Question: import stackless
class MyTasklet(stackless.tasklet):
    def __init__(self, func, msg):
        pass

def foo():
    pass

msg = 'hello'
MyTasklet(foo, msg)()

I am using stackless python, this code generates the following error:
Traceback (most recent call last):
  File "E:/workspace/python/test/klass.py", line 11, in <module>
    MyTasklet(foo, msg)()
TypeError: tasklet() takes at most 1 argument (2 given)

It's very odd since I did not call the constructor of stackless.tasklet.
Anyone know what exactly the error is about.
Comment: The problem is that `stackless.tasklet` implements `__new__` and its `__new__` method accepts only an argument.
Answer: Since the problem has to do with sub-classing stackless.tasklet, you could just make MyTasklet a callable object instead like follows:
import stackless
class MyTasklet(object):
    def __init__(self, func, msg):
        self.func = func
        self.msg = msg

    def __call__(self):
      t = stackless.tasklet()
      t.bind(self.func)
      t.setup(self.msg)
      t.run()

def foo(msg):
  print msg

msg = 'hello'
MyTasklet(foo, msg)()

EDIT: Alternatively you can just override __new__ to pass the correct arguments to stackless.tasklet.__new__:
import stackless
class MyTasklet(stackless.tasklet):
  def __new__(cls, func, msg):
    return stackless.tasklet.__new__(MyTasklet, func)
  def __init__(self, func, msg):
    pass

def foo():
  pass

msg = 'hello'
MyTasklet(foo, msg)()

Comment: Thanks for your answer, but I am not asking for workarounds, I want to know why my code failed.
Comment: the reason you're code is failing is because `stackless.tasklet.__new__` takes only one parameter but you haven't overridden `__new__` in your subclass. I've edited my answer to reflect your request (i.e. a solution which keeps the subclassing)
Comment: got it, the constructor is in fact `__new__` not `__init__`.
Answer: Do not subclass stackless.tasklet.
It is not how you are meant to use Stackless Python - and more importantly, it is difficult and overly complicated compared to the easier way.  Anything you need to do, you should be able to do in f, the function you run in the tasklet.  If you do really really want to subclass anyway, search for existing code examples, which will already give you a basic working tasklet subclass.
If you really want help with this, no-one with Stackless Python experience really answers questions here.  The Stackless Python mailing list is where they are.
Comment: subclassing `stackless.tasklet` is fine, one just have to know which methods to override if one is going to change the arguments to the constructor.

stackless python constructor


Finding joint PDF of $\max$ and $\min$ of $n$ iid random variables

Question: The Problem: Suppose that $X_1,\dots,X_n$ are independent random variables with the same absolutely continuous distribution. Let $f$ denote their common marginal PDF. Set $Y=\min(X_1,\dots,X_n)$
and $Z=\max(X_1,\dots,X_n)$. Find the joint PDF of $(Y,Z)$.
My Attempt: First we find the joint CDF. For any $y,z\in\mathbb R$ we have that
\begin{equation*}\begin{split}
F_{YZ}((y,z))&=P(Y\leq y,Z\leq z)\\
&=P(\min(X_1,\dots,X_n)\leq y,\max(X_1,\dots,X_n)\leq z)\\
&=P(\min(X_1,\dots,X_n)\leq y,X_1\leq z,\dots,X_n\leq z)\\
&=[1-P(X_1>y)\cdots P(X_n>y)]F(z)^n\\
&=[1-(1-F(y))^n]F(z)^n,
\end{split}\end{equation*}
where in the fourth step we used the independence of the random variables $X_1,\dots,X_n$ and the fact that $P(A\cap B)=[1-P(A^\complement)]P(B)$ for any independent events $A,B$. By the
absolute continuity of the random variables $X_1,\dots,X_n$ we may differentiate the expression for $F_{YZ}$ to obtain the joint PDF of $(Y,Z)$,
\begin{equation*}\begin{split}
f_{YZ}(y,z)&=\frac{\partial^2}{\partial y\partial z}[1-(1-F(y))^n]F(z)^n\\
&=\frac{\partial}{\partial y}nf(z)[1-(1-F(y))^n]F(z)^{n-1}\\
&=n^2f(z)f(y)F(z)^{n-1}(1-F(y))^{n-1}\\
\end{split}\end{equation*}

Do you agree with my solution above? Any feedback is much appreciated. Thank you for your time.
Answer: Mind that $Y,Z$ are NOT independent.
$$ P(Y<y,Z<z) = P(Z<z)- P(Y>y, Z<z) = P\bigg( \bigcap_i \{X_i<z\}\bigg)- P\bigg(\bigcap_i \{X_i\in [y,z]\} \bigg) = \ F(z)^n- (F(z)-F(y))^n  $$
Taking the derivative w.r.t to $y$ and $z$ yields
$$ f_{Y,Z}(y,z)= \partial_y (nF(z)^{n-1}f(z) -n(F(z)-F(y))^{n-1} f(z)) = n(n-1)(F(z)-F(y))^{n-2}f(z)f(y) $$
Comment: yes, exactly. And due to the absolute continuity it doesn't matter if you write $\le$ or $<$.
Comment: Does the first step follow because $\{Z\leq z\}=\{Y\leq y,Z\leq z\}\cup\{Y>y,Z\leq z\}$ is a union of disjoint events?


CoreOS Fleet Unit file and Docker Dynamic Port issue

Question: I am currently trying to get a dynamic port so I can register it with etcd on CoreOS. This is proving very difficult.
Here is my Unit file
<code>[Unit]
Description=test Elastic Search Server
After=docker.service
Requires=docker.service

[Service]
User=core
ExecStartPre=/usr/bin/docker pull shizzle/elasticsearch-server
ExecStart=/bin/bash -c '/usr/bin/docker start -a %n || /usr/bin/docker run --name="%n" -p :9200 -p :9300 shizzle/elasticsearch-server /elasticsearch/bin/elasticsearch -Des.config=/etc/elasticsearch.yml'
ExecStartPost=-/usr/bin/etcdctl mkdir /%n
ExecStartPost=/usr/bin/etcdctl set /%n/host %H
ExecStartPost=/bin/bash -c "/usr/bin/etcdctl set /%n/port $(docker inspect --format='{{(index (index .NetworkSettings.Ports "9200/tcp") 0).HostPort}}' %n)"
ExecStop=/usr/bin/docker stop /%n
ExecStopPost=/usr/bin/etcdctl rm /%n
</code>
Unfortunately, the third ExecStartPost fails. Any help would be much appreciated, I'm stumped!
Answer: Why not use a host level registration service to register the containers with ports to etcd? Something like https://github.com/progrium/registrator might be able to do what you are trying to do in shell. Short of that, I have had success registering IP and port pairs in CoreOS using something like this:
<code>ExecStart=/bin/sh -c "port=$(docker inspect -f '{{range $i, $e := .NetworkSettings.Ports }}{{$p := index $e 0}}{{$p.HostPort}}{{end}}' %n); while netstat -lnt | grep :$port >/dev/null; do etcdctl set /app/upstream/%n $COREOS_PRIVATE_IPV4:$port --ttl 300 >/dev/null; sleep 200; done"
</code>
You might be able to adapt that to just do port.
Answer: You are not using absolute path for docker binary (/usr/bin/docker) can it be the cause?
What error does it print?
Comment: He is in a shell for the `ExecStartPost` command, so `$PATH` is available and the `docker` command doesn't need to be absolute.


TFS 2010 - can Build servers be a VM?

Question: I'm looking to setup a TFS environment.   I'm proposing the TFS 2010 machine be a virtual box.   can the Build server be a VM?
Answer: Yes, yes they can.
Comment: Simple. Authoritative. I like. :)

TFS 2010 - can Build servers be a VM?


How to make a dynamic question answer form using formArray in angular 6

Question: I am trying to make a form which contains a question and some options for that as answers. Adding Options are dynamic, so I'm trying to implement it using formArray.
But at the same time, I need to save which option is correct, so for that, I'm using radio buttons with each option.
Problem is when I'm changing the radio button it doesn't uncheck previous ones.
HTML:
<form [formGroup]="questionForm">
  <button type="button" class="iconBtn" title="Add Options" (click)="addOptions()">
    Add Option
  </button>

                        <input type="radio" [formControlName]="'isCorrect'" value="1" (change)="selectCorrectOption($event)"/>

                <input type="text" class="form-control" placeholder="Option Text" [formControlName]="'optionText'"/>

</form>

TypeScript (ts):
ngOnInit() {
  this.questionForm = this._fb.group({
    'options': this._fb.array([
      this._fb.group({
        'isCorrect': [0],
        'optionText': ['']
      })
    ])
  });
}

public addOptions() {
  let control = < FormArray > this.questionForm.controls.options;
  control.push(this._fb.group({
        'isCorrect': [0],
        'optionText': ['']
      })
   );
}

Any Solution would be appreciated.
Answer: You have to add name property in <input type="radio">
Answer: you have to change your template code to have the same formControlName for every radio button like:
<input type="radio" formControlName="isCorrect" value="1" />
<input type="radio" formControlName="isCorrect" value="0" />
<input type="radio" formControlName="isCorrect" value="0" />

and access the value via
questionForm.value.isCorrect

I found that adding the name attribute to the radio button tag is enough to meet the need of only having one button selected.
<input type="radio" name="isCorrect" formControlName="isCorrect" (change)="selectCorrectOption($event)"/>


Revisit: Probabiity of X dice results in a row

Question: This is a revist of
this question, which was closed because the posting was of low quality.
The original question is :

A (fair) 6 sided die is tossed $10$ times.  What is the probability that somewhere
in the sequence of $10$ throws, two consecutive 1's appear?

This posting is a self-answer question, which considers the following three distinct
attack weapons:

Stars and Bars Theory
Recursion
Inclusion-Exclusion

There may well be other viable approaches besides the three approaches above.  I am
posting this question because I found a few points of unusual interest:

Inclusion-Exclusion, which generalizes well, is normally the preferred approach.
I consider this problem an exception.  For example, suppose, for $~k \in \{1,2,\cdots,9\},~$
you let $S_k$ denote the subset of all possible rolls where there are consecutive $1$'s on
rolls $~k~$ and $~k+1~$.
The difficulty comes in trying to enumerate (for example)
$~\displaystyle
\left\{S_1 \cap S_2\right\}, ~\left\{S_1\cap S_3\right\},~\left\{S_1 \cap S_4\right\},~
\left\{S_1 \cap S_2 \cap S_4\right\},~$ and so forth.
So this is one of those unusual problems where Inclusion-Exclusion is not the way to go.

Stars and Bars can be employed, in a somewhat unusal and creative manner, with
reasonable elegance.

Recursion is glaringly easy for a problem of this nature.

In the posted answer, I consequently skip Inclusion-Exclusion, and post the two distinct
solutions of Stars and Bars and Recursion.
Answer: Inclusion-exclusion works, you just need to apply it in a clever way.
To fix this overlap problem, we use this clever trick. For each $i\in \{1,\dots,8\}$, let
$$
T_i=\{\text{sequences where the $i^\text{th}$ and $(i+1)^\text{st}$ rolls are $1$, but the $(i+2)^\text{nd}$ rolls is not $1$}\}
$$
and then define $T_9$ to just be your $S_9$, the set of rolls where dice $9$ and $10$ are both $1$. The point is that $T_i$ works just as well for PIE, because
$$
S_1\cup \dots \cup S_9=T_1\cup \dots \cup T_9,
$$since if you can find $(1,1)$ somewhere in your sequence, you can also find $(1,1,x)$ for some $x\neq 1$, except in the case where $(1,1)$ occurs at the end.
The size of the intersections then works as follows. Suppose we are counting the $k$-wise intersection $|T_{i(1)} \cap \dots \cap T_{i(k)}|$, for indices $i(1)<i(2)<\dots<i(k)$. If there are any consecutive indices which are at most $2$ apart, i.e. if there exists $j\in \{1,\dots,k-1\}$ such that
$$
i(j)\le i(j-1)+2,
$$
then we automatically have $|T_{i(1)} \cap \dots \cap T_{i(k)}|=0$. This is because a consecutive subsequence like $(1,1,x)$ (with $x\neq 1)$ at position $i(j-1)$ cannot overlap with a subsequence of $(1,1,x)$ or $(1,1)$ at $i(j)$. This is the benefit of using $T_i$ over $S_i$; many intersections just become zero.
If instead all of the indices are spaced three or more apart, then you can count the size of the intersection in two cases:

If the greatest index, $i(k)$, is less than $10$, then $|T_{i(1)} \cap \dots \cap T_{i(k)}|=5^{k}\cdot 6^{10-3k}$.

If instead $i(k)=10$, then $|T_{i(1)} \cap \dots \cap T_{i(k)}|=5^{k-1}\cdot 6^{10-3k+1}$.

Therefore, if we let
$$
\begin{align}
f(k)&=\text{# size $k$ subsets of $\{1,\dots,9\}$ which are spaced apart by three or more}\\
g(k)&=\text{# size $k$ subsets of $\{1,\dots,9,10\}$ which include $10$}\\
&\;\;\;\;\;\text{ and are spaced apart by three or more},
\end{align}
$$
then PIE gives us
$$
|T_1\cup \dots \cup T_9|= \sum_{k=1}^9(-1)^k\left[f(k)\cdot 5^{k}\cdot 6^{10-3k}+g(k)\cdot 5^{k-1}\cdot 6^{10-3k+1}\right]
$$
All that remains is to compute $f(k)$ and $g(k)$. These are counting problems which are not trivial, but at least much easier then the original question. You can show that
$$
f(k)=\binom{9-2(k-1)}{k},\qquad g(k)=\binom{7-2(k-2)}{k-1}
$$
Comment: This is interesting, and I will have to study this.  Assuming that it works, this is a nice rehab to Inclusion-Exclusion.
Answer:
A (fair) 6 sided die is tossed $10$ times.  What is the probability that somewhere
in the sequence of $10$ throws, two consecutive 1's appear?

I will express the probability as
$$1 - \frac{N}{D} ~: ~D = 6^{10}. \tag1 $$
So, the problem has been reduced to enumerating $~N,~$ which represents the number of
$10$-die-throw sequences that do not contain any occurrence of consecutive $~1$'s.

$\underline{\text{Method 1: Stars and Bars}}$
For Stars and Bars theory, see
this article and
this article.
For any satisfying (i.e. no consecutive $1$'s) sequence, there can be no more than
five occurrences of a $1$, among the $10$ die throws.  That is, if there are at least
six $1$'s, then two of them would have to be together (i.e. consecutive).
So, you have $6$ mutually exclusive cases, since the number of $1$'s must be some
element in $\{0,1,2,3,4,5\}.$  For $k \in \{0,1,2,3,4,5\}$, let $f(k)$ denote the
number of satisfying rolls, that contain $k$ occurrences of a $1$.
Then
$$N = \sum_{k=0}^5 f(k). \tag2 $$
In the remainder of this section, I will:

manually compute $f(2)$ and $f(4)$.
use these results to develop a pattern.
use this pattern to obtain a general formula for $f(k)$.
plug this general formula into the expression in (2) above to obtain a numeric answer.

To compute $f(2)$, consider the number of solutions to

$x_1 + x_2 + x_3 = (10-2) = 8.$

$x_1, x_2, x_3 \in \Bbb{Z_{\geq 0}}$.

$x_2 \geq 1.$

The idea is that the $2$ die rolls of $1$ create three disjoint regions, in the
sequence of $(10 - 2)$ rolls that are not equal to $1$.  These regions occur before
the first $1$, and after each $1$.  Specifying $x_2 \geq 1$ equates to not allowing the 1st and 2nd $1$'s to be right next to each other.
Setting $y_2 = x_2 - 1,$ you want the number of solutions to

$x_1 + y_2 + x_3 = (8-1) = 7.$

$x_1, y_2, x_3 \in \Bbb{Z_{\geq 0}}.$

Per Stars and Bars theory, you have
$$\binom {7 + 2}{2} = \binom{9}{2} ~\text{solutions}. $$
Each such solution admits $5^8$ possible values for the $8$ die throws that are
not a $1$.
Therefore,
$$f(2) = \binom{9}{2} \times 5^8.$$
To compute $f(4)$, consider the number of solutions to

$x_1 + x_2 + x_3  + x_4 + x_5 = (10-4) = 6.$

$x_1, x_2, x_3, x_4, x_5 \in \Bbb{Z_{\geq 0}}$.

$x_2,x_3,x_4 \geq 1.$

The idea is that the $4$ die rolls of $1$ create five disjoint regions, in the
sequence of $(10 - 4)$ rolls that are not equal to $1$.  These regions occur before
the first $1$, and after each $1$.  Specifying $x_2,x_3,x_4 \geq 1$ equates to not allowing any of the $1$'s to be right next to each other.
Setting $~y_i = x_i - 1,~ : ~i \in \{2,3,4\},~$ you want the number of solutions to

$x_1 + y_2 + y_3 + y_4 + x_5 = (6-3) = 3.$

$x_1, y_2, y_3, y_4, x_5 \in \Bbb{Z_{\geq 0}}.$

Per Stars and Bars theory, you have
$$\binom {3 + 4}{4} = \binom{7}{4} ~\text{solutions}. $$
Each such solution admits $5^6$ possible values for the $6$ die throws that are
not a $1$.
Therefore,
$$f(4) = \binom{7}{4} \times 5^6.$$
There is now a clear pattern for computing $f(k)$.
The equation
$$x_1 + \cdots + x_{k+1} = (10 - k)$$
changes to
$$x_1 + y_2 + \cdots + y_k + x_{k+1} = (10 -k) - (k-1) = 11 - 2k,$$
so that there are
$$\binom{[11 - 2k] + k}{k} = \binom{11 - k}{k} ~\text{solutions}.$$
Therefore,
$$f(k) = \binom{11 - k}{k} \times 5^{(10 - k)}.$$
So,
$$N = \left[\binom{11}{0} \times 5^{(10)}\right]
+ \left[\binom{10}{1} \times 5^{9}\right]
+ \left[\binom{9}{2} \times 5^{8}\right]$$
$$ + \left[\binom{8}{3} \times 5^{7}\right]
+ \left[\binom{7}{4} \times 5^{6}\right]
+ \left[\binom{6}{5} \times 5^{5}\right]$$
$$ = 48,300,000.$$

$\underline{\text{Method 2: Recursion}}$
Let $~f(n,1)~$ denote the number of satisfying die rolls (i.e. no consecutive $1$'s)
of length $n$ that end in $1$.
Let $~f(n,2)~$ denote the number of satisfying die rolls
of length $n$ that do not end in $1$.
Let $f(n)$ denote the number of satisfying die rolls
of length $n$.
Then:

$f(n) = f(n,1) + f(n,2).$

$f(n+1,1) = f(n,2).$
That is, for any sequence represented by $f(n,2)$,
you can append a $1$.

$f(n+1,2) = 5 \times f(n).$
That is, for any sequence represented by $f(n)$,
you can append any number other than $1$.

Then, the following table provides the enumeration.
\begin{array}{| r | r | r | r |}
  \hline
  n & f(n,1) & f(n,2) & f(n) \\
  \hline
  1 & 1 & 5 & 6 \\
  \hline
  2 & 5 & 30 & 35 \\
  \hline
  3 & 30 & 175 & 205 \\
  \hline
  4 & 175 & 1025 & 1200 \\
  \hline
  5 & 1025 & 6000 & 7025 \\
  \hline
  6 & 6000 & 35125 & 41125 \\
  \hline
  7 & 35125 & 205625 & 240750 \\
  \hline
  8 & 205625 & 1203750 & 1409375 \\
  \hline
  9 & 1203750 & 7046875 & 8250625 \\
  \hline
  10 & 7046875 & 41253125 & 48300000 \\
  \hline
\end{array}
See also, lulu's comment, immediately following this answer.  I totally overlooked her idea.

$\underline{\text{Final Computations}}$
Probability of a sequence of 10 die-rolls, with no consecutive 1's is
$$\frac{48300000}{6^{(10)}} = \frac{503125}{629856}.$$
Probability of at least one occurrence of consecutive $1$'s is
$$1 - \frac{503125}{629856} = \frac{126731}{629856}.$$
Comment: Noting that a good string of length at least $2$ must start either with $X$ or with $1X$, for $X\in \{2,3,4,5,6\}$, we see that $f_n=5(f_{n-1}+f_{n-2})$ (and of course $f_1=6, f_2=35$)  which is easily solved [in closed form](https://www.wolframalpha.com/input?i=solve+a%5Bn%5D%3D5*%28a%5Bn-1%5D%2Ba%5Bn-2%5D%29%2Ca%5B1%5D%3D6%2Ca%5B2%5D%3D35).
Comment: @lulu Interesting.  My experience with recursion is *hit-or-miss* superficial.
Comment: For binary strings, the same reasoning yields the Fibonacci numbers, so this sort of thing can be viewed as a generalization of those.


Can HQL substract or sum date fields?

Question: A have two entities. For example timing settings and orders.
@Entity
public class TimingSettings{

    @Basic
    private Long orderTimeout = 18000; //for example 18000 sec
    ....
}

@Entity
public class Order{

    @Basic
    @OneToOne
    private TimingSettings timingSettings;

    @Temporal(value = TemporalType.TIMESTAMP)
    @Column(nullable = false)
    protected Date time;
    ....
}

I can't select timeout before and calculate needed order time, because I don't known what orders have what timing settings.
Haw can I perform HQL same as the following:
select o from order o
left join o.timingSettings ts
where o.time < current_timestamp()+ ts.orderTimeout

Answer: This depends on whether you need this to work across all databases or not.
In the latter case (specific database) check your dialect for available date functions.
MySQL dialect, for example, supports timediff() and time_to_sec() functions, which would allow you to write your query as:
select o from order o
  left join o.timingSettings ts
 where time_to_sec(timediff(o.time, current_timestamp())) < ts.orderTimeout

If you need to do this for all databases (or if your database doesn't support needed date functions), Hibernate supports year(), month(), ... second() functions across all dialects. Using these will make your query rather verbose :-) plus you'll have issues with DST.
Answer: Problem is with arithmetics and timestamp, check this thread for some ideas.

Can HQL substract or sum date fields?


Criar variável de sessão ao fazer login - Laravel

Question: Bom dia, estou mexendo num site feito em Laravel, no qual não fui eu que fiz. Estou precisando fazer o seguinte:
Quando o usuário fizer o login, eu quero pegar um valor especifico (coluna "idLoja" ) daquele usuário no banco de dados, e criar uma variável de sessão, para que eu possa utilizar essa variável em outras páginas.
Como posso fazer isso?
Obs: Não conheço muito do laravel. Acho que o site foi feito no Laravel 4.
Comment: Isso resolveu o problema?
Answer: No Laravel 4, para usar as sessões, você deverá usar a classe Session.
Em seguida, mostrarei o que cada método faz em relação à essa classe:
Session::put - Adiciona um ou mais valores na sessão. Você pode utilizar o "dot notation" para poder facilitar a criação de arrays multidimensionais dentro da sessão.
Exemplo:
Session::put('usuario', ['nome' => 'Wallace']);

Session::put('usuario.nome', 'Wallace');

Session::get - Obtém os dados de um índice da sessão. Você também pode usar o DotNotation.
  $usuario = Session::get('usuario');

  var_dump($usuario['nome']);

  var_dump(Session::get('usuario.nome'));

Session::flush - Remove todos os dados da sessão.
Session::forget - Remove valores específicos da sessão:
  Session::forget('usuario.nome')

Veja mais na página do Laravel - Session


Multi-threading list pattern advice

Question: I have made an application which also contains a folder/file scanner. I'm coming across a problem with the threading structure.
How it works:
For each folder/file it finds it starts a thread. There is a function inside each thread that uses a list to check if a similar item has been found so that it can add to the existing item. If it's not found it will add the item to the earlier mentioned list. The threads are executed parallel (async).
Problem:
Because it's async it will sometimes fail on the listcheck. This is caused because there is a time period between the check and adding to the list. Something that can happen is that the check returns that there is not a similar item, while there certainly is. This will result in the same item occurring in the list.
I have also made it that threads wait on each other. I really like the effect this gives it on the frontend. (items nicely adding to the list real time). But this takes way to long for a lot of folders/files.
Now I'm thinking of making a mix between the functions, but i would really like to see a combination of the speed of async threads and the safety of waiting on each thread.
Anybody any idea?
Answer: You should lock the entire code part that checks the list and adds a value.
Something like this:
private void YourThreadMethod(object state)
{
    // long taking operation

    lock (dictionary)
    {
        if (!dictionary.ContainsKey(yourItemKey))
        {
            // construct object, long taking operation

            dictionary.Add(yourItemKey, createdObject);
        }
    }
}

In this way, every thread will have to wait until the list is free to use. If you want a more advanced solution, you could read into the ReaderWriterLockSlim class which gives a more fine grained solution.
Comment: Thanks for all the suggestions. I have looked at the problem again and found a possible solution. The problem was not related to the use of async lists (i was already using a custom made DispatchingObservableCollection) The solution is making the time period between the check and the add as short as possible. Now it's fast and the items are nicely added. But thanks for all the fast reactions! EDIT: ah i'm sorry. I did indeed implement your locking mechanism and now i see when i remove this it isn't working xD so it's a combination of the locking mechanism and the reduced time period!!!!!
Comment: Why not `ConcurrentDictionary`?
Comment: @AndreasNiedermair: Because that doesn't prevent the checking with delayed adding problem.
Comment: You are just constructing a lock around a non-atomic operation to make it "atomic" - what's the difference to `TryAdd`? Am I missing something - *This will result in the same item occurring in the list.* which is exactly the reason for `TryAdd`...??
Comment: Well, that could be useful if the construction of the object doesn't take too long so you could prepare it. I thought it would take some time to enumerate the directory info.
Comment: I am still not getting your demur :) ... The object needs to be constructed anyway to get the hash to store (`yourItemKey`) - additionally I would simply store a plain representation of an object instead of a full reference (you know, to avoid all the needed disposal and stuff...) as the key, and using a `byte` for the value is just as a marker.
Comment: @RolandMakkelie that's hence the non-atomic nature of checking and adding. Only atomic (or sealed/non-reorderable cpu-instructions like the ones from `Interlocked`) are guaranteed to be free from any race-condition. As an addition: You can even go for `dictionary[key] = value` if you do not have any special behaviour on existance, to provide the latest value to be always persisted (except in multi-core environments, where 2 threads can be executed in real parallel)
Comment: @AndreasNiedermair thanks for the explanation. It's not really about the latest value. I will give an example with the project i'm doing. The scanner is about a movie/serie scanner. In the beginning of a thread it's not known if it's about a serie or movie. On the end of the thread the movie/serie is added to the list. In the middle of the thread there is a check that searches the list to check if the serie already exists. If so: add it as an episode to the existing serie. I think with the locking mechanism it is going to work, but first I need to simplify the code.
Answer: The most sleekest approach is the usage of a ConcurrentDictionary<string, byte> when yourItemKey is type of string (otherwise adapt TKey and use a proper IEqualityComparer or implement IEquatable):
private readonly ConcurrentDictionary<string, byte> _list = new ConcurrentDictionary<string, byte>();
private void Foo(object state)
{
  // looong operation
  this._list.TryAdd(yourItemKey, 0);
}
public void Bar()
{
  // this is how to query the content
  this._list.Keys...;
}

The trick behind that is to not use a too complex object as the key, which may need disposal or has external references (I'd prefer any string representation), and a small type for the value, which just acts as a marker.
Answer: I would consider using one of the thread safe collections in C#. For your case something like a ConcurrentBag will be more efficient than using a lock.
In case there is a time delay between checking and adding, you can use ConcurrentDictionary. It has a TryAdd method which will return false if an item  with the same key is already in the dictionary.
Comment: That isn't a solution when there is time between checking and adding.
Comment: True, you can do that with `ConcurrentDictionary` though, it does the check and add in one atomic operation.
Comment: @NedStoyanov I wouldn't call that *atomic* per se, as it is not one operation. Internally a locking mechanism is used to ensure no interference. The lock acquisition is an atomic operation indeed (depending on the processor), but `TryAdd` is *just* thread-safe :)

Multi-threading list pattern advice


How do I solve this memory usage limited problem?

Question: I have more than 100,000 articles to move from Drupal 6 to Drupal 7.
How do I solve this memory usage limited problem?
module: migrate d2d
Answer: Currently migrate module does not support batch query for to get value of nodes. You need to apply one patch which work for you.
Comment: Where are these two patch file located? 2296187-10_2.5backport-do-not-test.patch and 2296187_09-batched_query_support.patch. Do I need to rename them?
Comment: you need to apply patch using git