Set thousands separator for C printf

Question: I have this C code:
locale_t myLocale = newlocale(LC_NUMERIC_MASK, "en_US", (locale_t) 0);
uselocale(myLocale);
ptrLocale = localeconv();
ptrLocale->thousands_sep = (char *) "'";

int i1 = snprintf( s1, sizeof(s1), "%'d", 123456789);

The output in s1 is 123,456,789.
Even I set ->thousands_sep to ' it is ignored. Is there a way to set any character as the thousands separator?
Comment: I think for C the decision to write your own formatting function is a good idea.
Comment: Don't you need to call useLocale *after* you modify the local structure to pass the new data into the runtime?
Comment: http://stackoverflow.com/a/24629137/1606345
Comment: Ok. then it is not possible. I keep using my own function which does not use malloc and is thread safe as well.
Comment: @VolAnd I found a new, better solution which works in C on linux and I posted it as a new answer.
Answer: Here is a very simple solution which works on each linux distribution and does not need - as my 1st answer - a glibc hack:

All these steps must be performed in the origin glibc directory - NOT in the build directory - after you built the glibc version using a separate build directory as suggested by this instructions.
My new locale file is called en_AT.

Create in the localedata/locales/ directory from an existing file en_US a new file en_AT .
Change all entries for thousands_sep to thousands_sep   "<U0027>" or whatever character you want to have as the thousands separator.
Change inside of the new file all occurrences of en_US to en_AT.
Add to the file localedata/SUPPORTED the line: en_AT.UTF-8/UTF-8 \.
Run in the build directory make localedata/install-locales.
The new locale will be then automatically added to the system and is instantly accessible for the program.

In the C/C++ program you switch to the new thousands separator character with:

setlocale( LC_ALL, "en_AT.UTF-8" );

using it with printf( "%'d", 1000000 ); which produces this output

1'000'000

Remark: When you need in the program different localizations which are determinated while the runtime you can use this example from the man pages where you load the requested locale and just replace the LC_NUMERIC settings from en_AT.
Answer: Function localeconv() just read locate settings, and ptrLocale->thousands_sep itself not changes that settings for current locale.
EDIT:
I do not know how to do this in C, but lots of examples with C++ output can be found.
See the following example in C++:
#include <iostream>
#include <locale>
using namespace std;

struct myseps : numpunct<char> {
   // use ' as separator
   char do_thousands_sep() const { return '\''; }

   // digits are grouped by 3
   string do_grouping() const { return "\3"; }
};

int main() {
  cout.imbue(locale(locale(), new myseps));
  cout << 1234567; // the result will be 1'234'567
}

EDIT 2:
The C++ reference said:

localeconv() returns a pointer to a filled-in object of type struct lconv. The values contained in the object can be overwritten by subsequent calls to localeconv and do not directly modify the object. Calls to setlocale with category values of LC_ALL, LC_MONETARY, or LC_NUMERIC overwrite the contents of the structure.

I tried the following example in MS Visual Studio 2012 (I understand that it is bad and unsafe style):
#include <stdio.h>
#include <locale.h>
#include <string.h>

int main() {
    setlocale(LC_NUMERIC, "");
    struct lconv *ptrLocale = localeconv();
    strcpy(ptrLocale->decimal_point, ":");
    strcpy(ptrLocale->thousands_sep, "'");
    char str[20];
    printf("%10.3lf \n", 13000.26);
    return 0;
}

and I saw the result:
  13000:260

therefore, it can be assumed that the changes of decimal_point and thousands_sep are possible through pointer received with localeconv(), but printf ignores thousands_sep.
EDIT 3:
Updated C++ example:
#include <iostream>
#include <locale>
#include <sstream>
using namespace std;

struct myseps : numpunct<char> {
   // use ' as separator
   char do_thousands_sep() const { return '\''; }

   // digits are grouped by 3
   string do_grouping() const { return "\3"; }
};

int main() {
  stringstream ss;
  ss.imbue(locale(locale(), new myseps));
  ss << 1234567;  // printing to string stream with formating
  printf("%s\n", ss.str().c_str()); // just output when ss.str() provide string, and c_str() converts it to char*
}

Comment: But which structure is `printf()` accessing? There must be a way how to override the thousand character. I diggered through the `printf()` from GNU glib library and it is not hardcoded there!
Comment: I suppose you need `setlocale()` function to change current locale
Comment: Also, check whether `snprintf` is locale-dependent function
Comment: @VoIAnd: Yes, but how do I set explicitly another thousand separator character? Calling `setlocale()` with a predefined locale "en_US", "de_DE", ... uses the separator as defined for the locale.
Comment: @AlBundy : I suppose, `printf`-family functions just ignore `ptrLocale->thousands_sep` settings. See **EDIT 2**
Comment: @VoIAnd: No, it does not. You get it with the ' flag and changing the locale from en_US to de_DE it also changes the character. I read the source code from GNU glib `printf()` and they respect it. My question is how do I get to the buffer for the thousand separator character so I can change it.
Comment: @Edit 2: This is funny. 1) In SLES I get Segmentation fault when I use `strcpy()` to overwrite it. 2) Try this format specifier: `printf("%'10.3lf\n", 13000.26);` - use the `'` character
Comment: That is really funny - `printf("%'10.3lf \n", 13000.26);` compiled with MSVS print out just `'10.3lf` .
Comment: Lets try another way: How can I in C++ use the `.imbue` functions but the result is returned as `std::string` or `char *` and I can provide more format specifier like this: `"%llu bytes loaded within %.4fms in thread #%03d"`
Comment: I can confirm the possibility to change the decimal separator, but could not find either how to use a thousand separator with `printf`.
Comment: BTW in my own tests, I did not use strcpy to set the decimal or thousand separator but simply `ptrLocale->decimal_point = ":"`. If thousand separator was empty string, using `strcpy` with a non empty source could lead to buffer overrun.
Comment: @VoIAnd: This is not exactly what I want. Using `imbue` is just another way as I had it till now with my self written functions. I have to think about it!
Comment: @those who are wondering how to use printf with thousand separators: you must `setlocale(LC_NUMERIC, "");` see answer to: "How can I format currency with commas in C?"
https://stackoverflow.com/a/11695126/1973022 .
Answer: This answer is derived from VolAnd's one.
According to this source, the thousand separator in only used with the non standard ' flag.
So if your printf is POSIX.1-2008 compatible, you could use :
setlocale(LC_NUMERIC, "");
struct lconv *ptrLocale = localeconv();
ptrLocale->decimal_point = ":";
ptrLocale->thousands_sep = "'";
char str[20];
printf("%'10.3lf \n", 13000.26);
return 0;

Comment: This is the code from my question where I am asking how to do it. Your code does not work for me. I have to use `setlocale(LC_NUMERIC, "en_US");` in order to see at least the American thousand separator character.
Comment: @AlBundy : with this code, I could successfully change the decimal separator. Unfortunately, the two systems on which I tried it do not support the `'` non standard flag (whatever locale I use).
Comment: Note that it depends on which standard you choose as to whether it is standard or not.  POSIX specifies that [`printf()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/printf.html) supports the `'` as indicating that the thousands separator should be printed appropriately.  Also, Mac OS X (10.10.5) and by inference BSD has a set of `_l` printing functions: `int printf_l(locale_t loc, const char * restrict format, ...);` and `int fprintf_l(FILE * restrict stream, locale_t loc, const char * restrict format, ...);` for example.  These are the best way to go if they're available.
Comment: More seriously, note that the POSIX specification for [`localeconv()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/localeconv.html) explicitly says: _The `localeconv()` function need not be thread-safe. … The `localeconv()` function shall return a pointer to the filled-in object. The application shall not modify the structure to which the return value points, nor any storage areas pointed to by pointers within the structure._  A portable application may not do as this answer suggests.
Answer: There is a really very dirty hack how to change the thousand separator character for printf():

Download the GNU libc.
run the configure --prefix=/usr/glibc-version command
run make -j 8
get the very long compiler command with all switches from the make output
write the C source file setMyThousandSeparator.c - content see below
compile this source file with the gcc switches from point 3.
in your normal C source code call setMyThousandSeparator("'") function before the printf() call.
link setMyThousandSeparator.o with your project.

For the moment I tried it when linking libc static but it works.
Content of setMyThousandSeparator.c:
#include <locale/localeinfo.h>

void setMyThousandSeparator(char * sMySeparator)
{
    _NL_CURRENT (LC_NUMERIC, THOUSANDS_SEP) = sMySeparator;
}

Info:
This solution is thread safe because it is accessing the same data as printf() does!
Answer: Here is a specialized C function which I'm using for uint64_t type, but it can be easily generalized. Basically, it injects the thousand separators into the string produced by snprintf().
This method is independent on LOCALE, C-standard used, etc - and of course, You don't have to recompile the GNU libc ;)
#if __WORDSIZE == 64
   #define PRT_U64 "lu"
#else
   #define PRT_U64 "llu"
#endif

char* th_sep_u64(uint64_t val, char* buf) {
   char tmpbuf[32]; //18'446'744'073'709'551'615 -> 26 chars
   int  nch, toffs, pos;
   pos   = 1;
   toffs = 31;
   nch   = snprintf(tmpbuf, 32, "%"PRT_U64, val);
   nch  -- ;
   buf[toffs] = 0;

   for (; nch>=0; --nch) {
      toffs -- ;
      buf[toffs] = tmpbuf[nch];
      if ((0 == (pos % 3)) && (nch > 0)) {
         toffs -- ;
         buf[toffs] = '\''; //inject the separator
      }
      pos ++ ;
   }
   buf += toffs;
   return buf;
}

Usage:
{
   char     cbuf[32];
   uint64_t val = 0xFFFFFFFFFFFFFFFFll;

   printf("%s", th_sep_u64(val, cbuf));

   //result: 18'446'744'073'709'551'615
}

Regards
Comment: This is a nice function but it becomes _tiresome_ when your format string contains lot of different formats. Recompiling GNU glibc can indeed become very tricky and I don't consider it as a good solution - 5 years later. Now, I solve it with building a new LOCALE which is really a very simple task and the huge advantage is, that the thousand separator works even in the Linux Bash command line so it becomes global.
Comment: Yeah, it all depends on what You need to achieve. I need to run my code on many machines, different OSes, so for me it would be _tiresome_ to build / install / change  LOCALE for each case.
Answer: Maybe "just" add a new printf specifier:
static int printf_arginfo_M(const struct printf_info *info, size_t n, int *argtypes, int *size) {

    if ( info->is_long_double ) {               // %llM
        size[0] = sizeof(long long);
        if ( n > 0 ) argtypes[0] = PA_INT | PA_FLAG_LONG_LONG;
    }
    else if ( info->is_long ) {                 // %lM
        size[0] = sizeof(long);
        if ( n > 0 ) argtypes[0] = PA_INT | PA_FLAG_LONG;
    }
    else {
        size[0] = sizeof(int);                  // %M
        if ( n > 0 ) argtypes[0] = PA_INT;
    }

    return 1;
}

static int printf_output_M(FILE *stream, const struct printf_info *info, const void *const args[])
{
    long long number;

    if ( info->is_long_double ) {               // %llM
        number = *(const long long*)(args[0]);
    }
    else if ( info->is_long ) {                 // %lM
        number = *(const long*)(args[0]);
    }
    else {                                      // %M
        number = *(const int*)(args[0]);
    }

    long long value = (number < 0) ? -number : number;
    int len;
    char buf[32];
    char *pos = &buf[31];
    int i = 0;

    *pos = '\0';

    do {
        if ( (i % 3 == 0) && (i > 0) ) *--pos = '.';
        *--pos = '0' + value % 10;
        value /= 10;
        i++;
    } while (value > 0);

    if (number < 0) *--pos = '-';

    len = fprintf(stream, "%s", pos);

    return len;
}

Usage:
register_printf_specifier('M', printf_output_M, printf_arginfo_M);

printf("%M\n", -1234567890);
printf("%lM\n", -1234567890123456789l);
printf("%llM\n", -1234567890123456789ll);

The downside is, gcc complains about the new specifier so that you might want to disable these warnings:
#pragma GCC diagnostic ignored "-Wformat"
#pragma GCC diagnostic ignored "-Wformat-extra-args"

Set thousands separator for C printf


Training on the full dataset after cross-validation?

Question: TL:DR: Is it ever a good idea to train an ML model on all the data available before shipping it to production? Put another way, is it ever ok to train on all data available and not check if the model overfits, or get a final read of the expected performance of the model?

Say I have a family of models parametrized by $\alpha$. I can do a search (e.g. a grid search) on $\alpha$ by, for example, running k-fold cross-validation for each candidate.
The point of using cross-validation for choosing $\alpha$ is that I can check if a learned model $\beta_i$ for that particular $\alpha_i$ had e.g. overfit, by testing it on the "unseen data" in each CV iteration (a validation set). After iterating through all $\alpha_i$'s, I could then choose a model $\beta_{\alpha^*}$ learned for the parameters $\alpha^*$ that seemed to do best on the grid search, e.g. on average across all folds.
Now, say that after model selection I would like to use all the the data that I have available in an attempt to ship the best possible model in production.  For this, I could use the parameters $\alpha^*$ that I chose via grid search with cross-validation, and then, after training the model on the full ($F$) dataset, I would a get a single new learned model $\beta^{F}_{\alpha^*}$
The problem is that, if I use my entire dataset for training,  I can't reliably check if this new learned model $\beta^{F}_{\alpha^*}$ overfits or how it may perform on unseen data. So is this at all good practice? What is a good way to think about this problem?
Comment: Almost an exact duplicate: http://stats.stackexchange.com/questions/52274 with lots of worthy answers. Perhaps these threads should be merged but I am not sure in which direction. Both have accepted answers that are very good.
Answer: The way to think of cross-validation is as estimating the performance obtained using a method for building a model, rather than for estimating the performance of a model.
If you use cross-validation to estimate the hyperparameters of a model (the $\alpha$s) and then use those hyper-parameters to fit a model to the whole dataset, then that is fine, provided that you recognise that the cross-validation estimate of performance is likely to be (possibly substantially) optimistically biased.  This is because part of the model (the hyper-parameters) have been selected to minimise the cross-validation performance, so if the cross-validation statistic has a non-zero variance (and it will) there is the possibility of over-fitting the model selection criterion.
If you want to choose the hyper-parameters and estimate the performance of the resulting model then you need to perform a nested cross-validation, where the outer cross-validation is used to assess the performance of the model, and in each fold cross-validation is used to determine the hyper-parameters separately in each fold.  You build the final model by using cross-validation on the whole set to choose the hyper-parameters and then build the classifier on the whole dataset using the optimized hyper-parameters.
This is of course computationally expensive, but worth it as the bias introduced by improper performance estimation can be large.  See my paper
G. C. Cawley and N. L. C. Talbot, Over-fitting in model selection and subsequent selection bias in performance evaluation, Journal of Machine Learning Research, 2010. Research, vol. 11, pp. 2079-2107, July 2010. (www, pdf)
However, it is still possible to have over-fitting in model selection (nested cross-validation just allows you to test for it).  A method I have found useful is to add a regularisation term to the cross-validation error that penalises hyper-parameter values likely to result in overly-complex models, see
G. C. Cawley and N. L. C. Talbot, Preventing over-fitting in model selection via Bayesian regularisation of the hyper-parameters, Journal of Machine Learning Research, volume 8, pages 841-861, April 2007. (www,pdf)
So the answers to your question are (i) yes, you should use the full dataset to produce your final model as the more data you use the more likely it is to generalise well but (ii) make sure you obtain an unbiased performance estimate via nested cross-validation and potentially consider penalising the cross-validation statistic to further avoid over-fitting in model selection.
Comment: +1: Answers the question: "If you use cross-validation to estimate the hyperparameters of a model (the αs) and then use those hyper-parameters to fit a model to the whole dataset, then that is fine…"
Comment: @soufanom, no, the use of "initial experiments" to make choices regarding the model is likely to result in over-fitting and almost certainly will introduce an optimistic bias into the performance analysis.  The cross-validation used for performance analysis must repeat EVERY step used in fitting the model independently in each fold.  The experiments in my paper show that kernel models can be very sensitive to this sort of bias, so it is vital to perform the model selection and performance evaluation with all possible rigour.
Comment: @DikranMarsupial. I don't quite get the third paragraph in your answer. If I do nested cross-validation, I will get a different set of hyperparameters for each fold of the **outer** CV (i.e. I get one set of hyperparameters from running the **inner** CV on a grid of parameters). How do I then choose the best set of hyperparameters?
Comment: For kernel methods, such as the SVM, it is often possible to perform leave-one-out cross-validation at almost no computational cost (see the papers listed in my answer).  I use this "virtual" leave-one-out cross-validation for tuning the hyper-parameters, nested in k-fold cross-validation for performance evaluation.  The cost is then quite reasonable.  In my opinion it is not acceptable to use any procedure where the performance evaluation is biased in any way by the tuning of the hyper-parameters.  It is worth the computational expense to get a reliable estimate.
Comment: cross-validation is essentially a means of estimating the performance of a *method* of fitting a model, rather than of the method itself.  So after performing nested cross-validation to get the performance estimate, just rebuild the final model using the entire dataset, using the procedure that you have cross-validated (which includes the selection of the hyper-parameters).
Comment: This is a very good answer (+1). I have been reviewing your answers on this topic and looked in your 2010 Cawley&Talbot paper too, because I came across a simple situation where I get exactly the same error estimate whether I do nested cross-validation or not. This made me wonder what are the situations in which nested CV would make a difference. I asked this question here: http://stats.stackexchange.com/questions/178174, and would be very interested in your input! Thank you.
Comment: @DikranMarsupial Thank you for this answer. In the comments above, you wrote: "after performing nested cross-validation to get the performance estimate, just rebuild the final model using the entire dataset, using the procedure that you have cross-validated." Just to be clear - you should only report the performance estimate from nested CV for the final model - right?
Comment: What could be a different approach from a nested cross-validation? Can we run some initial experiments and observe a proper setting for the hyper-parameter and then use cross-validation for the parameters?
Comment: Thanks for your direct response. I want to avoid nested-cross validation because of its high computational time cost. Lets say I have an alpha and lambda parameters. Can I loop over values of alpha and within every value we apply cross-validation for lambda? This might still more optimistic than the nested-cross validation but is it acceptable?
Comment: So, if we use nested cross validation, after performing it, we can build the final model on the whole dataset using the optimized hyper-parameters. But if the we use hold out to assess model performance and cross validation on the training set to tune the parameters, we can't. Why is that? I don't understand. Please see this question: http://stats.stackexchange.com/questions/153366/question-about-training-set-and-test-set. Thank you
Comment: I don't think I mention hold-out in my answer. but it depends on how cross-validation and hold-out are used.  If you cross-validate on all of the available data and then split the data into training and test sets and use the cross-validated hyper-parameters, that would give an optimistically biased performance estimate.  However, there is no reason why you shouldn't retrain on all of the available data.  The point is that cross-validation is an estimate of the performance of a method of generating a model, rather than of the model itself.
Comment: @BerkU. yes, that is what I would do.
Comment: This answer must be emphasized. People are usually clueless about what to do after nested cross-validation,
Comment: @DikranMarsupial: say I use nested CV to evaluate a modeling process that includes a search for some hyperP lambda in range(3,4). I get avg error over folds of 0.2 (unacceptable for application).  I redo the nested CV, but this time I search for lambda in range(1,6). I get error 0.01.  If I choose the second modeling process, I feel I'll fall victim to optimism mentioned in your paper.  How are we supposed to find search spaces, though, if not by this method..? https://stats.stackexchange.com/questions/282954/does-changing-the-parameter-search-space-after-nested-cv-introduce-optimistic-bi
Comment: Philosophically, why do you use a pseudonym, and post a paper of yours from which your true name can be inferred? (BTW: great answer, thank you, also for the link to the book)
Answer: Just to add to the answer by @mark999, Max Kuhn's caret package (Classification and Regression Training) is the most comprehensive source in R for model selection based on bootstrap cross validation or N-fold CV and some other schemes as well.
Not to disregard the greatness of the rms package, but caret lets you fit pretty much every learning method available in R, whereas validate only works with rms methods (I think).
The caret package is a single infrastructure to pre process data, fit and evaluate any popular model, hence it is simple to use for all methods and provides graphical assessment of many performance measures (something that next to the overfit problem might influence the model selection considerably as well) over your grid and variable importance.
See the package vignettes to get started (it is very simple to use)
Data Preprocessing
Variable Selection with caret
Model Building with caret
Variable Importance
You may also view the caret website for more information on the package, and specific implementation examples:
Offical caret website
Comment: Thanks. Do you know if, after model selection (which is done by calling `train`), there is a way in caret to train with the full dataset?
Comment: Not sure if that is a good idea or why you would want to that, but you can just fit the final model returned by train to the full data set.
Answer: I believe that Frank Harrell would recommend bootstrap validation rather than cross validation. Bootstrap validation would allow you to validate the model fitted on the full data set, and is more stable than cross validation. You can do it in R using validate in Harrell's rms package.
See the book "Regression Modeling Strategies" by Harrell and/or "An Introduction to the Bootstrap" by Efron and Tibshirani for more information.
Comment: To omit a next myth about "bad CV", this is a terminology problem -- Harrell's "cross validation" means N-fold CV, and "bootstrap validation" means resampling CV. Obviously I agree that this second flavor is more stable and overall nicer, but this is also a type of cross validation.
Comment: mark999 or @mbq, would you mind elaborating on how bootstrap would allow one to validate a model fitted on the full dataset?
Comment: @user27915816 Well, in principle nohow; the idea behind cross-validation is that you tests whether given *training method* is reliably making good models on a sets very similar to the final one, and, if so, generalise this observation to the full set with a silent assumptions that nothing strange will happen and that CV method you used is not somehow biased. This is of course almost always good enough, still you can never be *sure* that the model built on all the data you have is not overfitted.
Answer: I think you have a bunch of different questions here:

The problem is that, if I use all points in my dataset for training, I can't check if this new learned model βfull overfits!

The thing is, you can use (one) validation step only for one thing: either for parameter optimization, (x)or for estimating generalization performance.
So, if you do parameter optimization by cross validation (or any other kind of data-driven parameter determination), you need test samples that are independent of those training and optimization samples. Dikran calls it nested cross validation, another name is double cross validation. Or, of course, an independent test set.

So here's the question for this post : Is it a good idea to train with the full dataset after k-fold cross-validation?
  Or is it better instead to stick with one of the models learned in one of the cross-validation splits for αbest?

Using one of the cross validation models usually is worse than training on the full set (at least if your learning curve performance = f (nsamples) is still increasing. In practice, it is: if it wasn't, you would probably have set aside an independent test set.)
If you observe a large variation between the cross validation models (with the same parameters), then your models are unstable. In that case, aggregating the models can help and actually be better than using the one model trained on the whole data.
Update: This aggregation is the idea behind bagging applied to resampling without replacement (cross validation) instead of to resampling with replacement (bootstrap / out-of-bootstrap validation).
Here's a paper where we used this technique:
Beleites, C. & Salzer, R.: Assessing and improving the stability of chemometric models in small sample size situations, Anal Bioanal Chem, 390, 1261-1271 (2008).
DOI: 10.1007/s00216-007-1818-6

Perhaps most importantly, how can I train with all points in my dataset and still fight overfitting?

By being very conservative with the degrees of freedom allowed for the "best" model, i.e. by taking into account the (random) uncertainty on the optimization cross validation results. If the d.f. are actually appropriate for the cross validation models, chances are good that they are not too many for the larger training set.
The pitfall is that parameter optimization is actually multiple testing. You need to guard against accidentally good looking parameter sets.
Comment: More importantly: the iterated cross validation surrogate models share the same set of hyperparameters. That is, they are equivalent in all you deem to be important but the arbitrary selection of training and test cases. Selecting a "good" model thus in fact should primarily select a good test/training set combination - which is fundamentally what we usually do *not* want: we want a choice that is generalizing well and thus not only working for favorable cases. From this point of view, selecting a surrogate model from a "normal" cross validation does not make any sense to me.
Comment: @jpcgandre: (d.f.) I argue that choosing a model complexity that is appropriate for training on $1 - \frac{1}{k}$ of the data set (which I argue is almost as large as the whole data set), you may arrive at a bias towards slightly too restrictive models for training on the whole data set. However, I don't think this should matter in practice, the more so as my impression in my field is that we rather tend to err towards too complex models.
Comment: @jpcgandre: Choosing one of the surrogate models for further use is in fact a data-driven model selection, which means that you need an outer independent level of validation. And in general, unless you have enough cases so you can actually do statistically meaningful model comparisons on the basis of testing $\frac{1}{k}$ of the total sample size, IMHO you should not select.
Comment: @cbeleites can you elaborate on `If the d.f. are actually appropriate for the cross validation models`. In my understanding you are arguing that the train/validation sets are not very large when compared with the complete data set, am I right?
Comment: @Zhubarb Are you referring to 10x k-folds (5 for instance)? If so, I believe the best practice would be to use the model with best model selection parameters that comes out of each of the 10x k-folds. For each k-fold you get an average RMSE or R2 in the case of regression so you get already a kind of aggregated model. But care should be paid in the presence of unstable models. Perhaps the "best" model is not the one that comes from CV but the second or third, depending on the train/test data sets. Difficult to know without trying I think.
Comment: See `http://ai.stanford.edu/~ang/papers/cv-final.pdf`
Comment: @Zhubarb: aggregation is AFAIK more usually done with the predictions. For linear models, the results are the same if you average coefficients or predictions, but already for LR, the results will be different because of the nonlinear transformation. Please see the paper in the text for more details and an example.
Comment: `...If you observe a large variation between the cross validation models (with the same parameters), then your models are unstable. In that case, aggregating the models can help...` Can you explain this a bit more? e.g. if I am running logistic regression in a 10-k cross validated setup and end up with 10 sets of coefficients, do you recommend aggregating the coeff estimates to form a final model? If so, how can this be done, just taking the means?
Answer: What you do is not a cross validation, rather some kind of stochastic optimization.
The idea of CV is to simulate a performance on unseen data by performing several rounds of building the model on a subset of objects and testing on the remaining ones. The somewhat averaged results of all rounds are the approximation of  performance of a model trained on the whole set.
In your case of model selection, you should perform a full CV for each parameter set, and thus get a on-full-set performance approximation for each setup, so seemingly the thing you wanted to have.
However, note that it is not at all guaranteed that the model with best approximated accuracy will be the best in fact -- you may cross-validate the whole model selection procedure to see that there exist some range in parameter space for which the differences in model accuracies are not significant.
Comment: Thanks @mbq, but I'm not sure I follow. I **do** N-fold cross-validation for each point value of my grid search in the  hyperparameter space. The average result of the N-folds gives me the approximation that you mention, which I use to compare models and do model selection by selecting the model the best fits the validation set.

My question is about what happens when I train with the full dataset. I think the learned model changes (the $\vec\beta$ parameters of the learned model change), and in principle I have no way to know if I suffer from overfitting.
Comment: @AmV If so, ok -- as I wrote, CV already tests full set scenario, you cannot say more without new data. Again, you can at most do a nested CV to see  if there is no overfitting imposed by the model selection itself (if the selection gives very good improvement or the data is noisy the risk of this is quite big).
Answer: I found Moiser 1951, https://doi.org/10.1177/001316445101100101 claim:

obtain performance metrics using cross-validation
use the whole dataset to build a final, more accurate model

His last words are:

Possibly one of the other speakers knows a proof that the [coefficient of determination] R of the half sample's beta is always lower than of the full sample beta [where beta is the coefficients of a model]

Does somebody have a proof by now (in the year 2023)?
Comment: I posted this follow up question: https://stats.stackexchange.com/q/622128/298651


How to filter POST requests using pcap?

Question: I am playing around with the pcap library in C. I can filter http traffic using the filter expression "port 80" in the pcap_compile call, but now I'm wondering whether I can filter POST requests specifically.
Does anyone know of a filter expression for filtering http POST requests?
Answer: The following should match only POSTs:
<code>port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354
</code>
However, a HTTP request is likely to be split in various TCP packets, and thus you may not get the full request.

How to filter POST requests using pcap?


Imputation of missing response variables

Question: I am doing multiple imputation on a database of observations on hospital patients. There is one observation of many covariates per patient. There are 2 binary outcome variables:

Alive/Dead after 30 days
Died in hospital, or survived/discharged

Two seperate analysis models (logistic regression), with identical covariates, are to be run, each with one of these outcomes as the response. There are 7 covariates in the analysis models.
There is missing data of between 3 and 11% in the covariates and the outcomes.
In addition there are 7 further covariates that are to be used in the imputation model to predict the missingness in the covariates and outcomes.
My questions concerns the imputation of the two outcome variables. They are to be used as predictors for missingness in the covariates, as per standard practice, but they are highly collinear with each other. Is this a concern for the imputation model ? Is it valid/recommended to impute them both in the same model (to generate several complete datasets all containing  both outcomes) or should seperate imputations be performed for each of the outcomes (to generate two distinct sets of complete datasets, each distinct set having one of the outcomes) ? Any other suggestions for how to proceed would be welcome.
Comment: Do you want to do imputation of one of the outcomes (when one is missing) or are you only asking about the imputation of the covariates?
Comment: @Guillaume I am interested in both situations - imputing one when the other is missing, and also using both of them to impute the other covariates. That's where my concern about collinearity comes from)
Answer: As per the first answer, in general there is no reason not to impute all of your variables in one go, generating a single set of imputed datasets. Your two outcomes being strongly correlated should not be too much of an issue - when they serve as independent variables in the imputation model(s) for your missing covariates, their corresponding coefficients will be estimated imprecisely, but this isn't usually a problem in terms of drawing from the resulting imputation distribution.
There is a long discussion in the comments between AdamO and Joe King, and one of the discussions is about when complete case analysis / listwise deletion is unbiased. If data are MCAR, it is unbiased of course. If missingness is independent of the dependent variable in the model of interest, conditional on the covariates in the model of interest, it is also unbiased (this point is made by Little and Rubin, in their book). Depending on where the missingness occurs, this condition can sometimes correspond to an MAR mechanism and sometimes to a MNAR mechanism. It is not correct that complete case analysis is generally unbiased under MAR mechanisms.
For more on this, see for example this paper:
http://www.ncbi.nlm.nih.gov/pubmed/20842622
I also wrote a post about complete case analysis validity some time ago on my blog: http://thestatsgeek.com/2013/07/06/when-is-complete-case-analysis-unbiased/
Answer: In general, multiple imputation works by using all available information in the model to simulate the missing values: I use the word "simulate" because you're technically doing more than just prediction, which involves more parametric assumptions.
I assume the outcomes are either jointly missing or jointly observed, there are no cases where outcome A is known when outcome B isn't, or vice versa.
Collinearity is not an issue. If you are treating these outcomes as independent (reporting odds ratios from two logistic regression models with separate outcomes), then you don't even need to worry about whether contradictory outcomes are simulated: (e.g. patients both discharged living and died within 30 days -- I assume you wouldn't have observed that in the hospital).
Two overall thoughts about the analysis:

Why aren't you reporting a Cox proportional hazards model? Treating death/discharge as a 1/0 event indicator and time until the observed discharge or death as the time-to-event is a very similar, common, and preferred analysis. The hazard ratios approximate relative risks, just like odds ratios (for rare outcomes) and patients who are discharged should be censored. This way you use all available information about when patients were at risk for dying, it is a much more powerful analysis.
I don't really agree with the need for missing data methods. Complete case analyses of data (where rows of missing data are dropped from the analysis) are unbiased and require little validation of assumptions, unlike Multiple Imputation which has caveats of estimating and validating parametric models. 13% is fairly negligible when $n$ is 150 or more. As a rule of thumb, having 20 events (these are deaths) per variable in the adjusted model is sufficient for power considerations.
Comment: Thank you ! A few points in reply. 1: There are no rows where all data are missing. 2: Many rows have only 1 missing variable, so to exclude the row think leads to bias (they are not MCAR). 3:There are cases where a patient was discharged from the hospital, but died within 30 days, and also where a patient was alive at 30 days, but died in hospital (after 30 days, obviously). I think this precludes a Cox model ? 3: There are many cases where one outcome is recorded, but the other is missing. 4: My concern with collinearity is that these outcomes are used as predictors in the imputation model.
Comment: 1: okay. 2: You're wrong, see Rubin's Statistical Analysis with Missing Data 2nd ed. CC is unbiased with MAR data, MI propagates bias in NIM. 3: Read some NEJM / JAMA articles. If you observe patients after discharge, you should do time-to-death analyses, using time-varying covariates for discharge. You can model logistic regression for 30 day discharge among survivors, but be careful of interpreting results. 3: shouldn't matter, then, use one to predict the other and vice versa, they are strong proxies 4: that's not a collinearity issue. You use outcomes as predictors in MI.
Comment: Thanks again...2: That's not what I took home from the book...I thought it was clear on that. For example in my data, some variables are strongly predictive of missingness in others which violates MCAR but not MAR, and deleting them would certainly lead to bias. Also it would mean discarding around half of the entire dataset ! 3: I don't have time-to-death data. 4: The imputation model uses both these outcomes as predictors for imputation, but they are highly collinear - so there are 2 highly collinear predictors, used as covariates to impute (for example) a patient's age - that's my concern.
Comment: You should reread it. I don't have it on hand, so I can't cite anything. Just do a simulation and convince yourself. You originally reported 13% of the data missing, now you report 50% are missing. This changes the scope of my original recommendation. Please revise your post to more accurately describe the problem.
Comment: No, I am not saying there is 50% missing - I am saying there is around 50% of data that are complete (ie with no missing variables). There is between 3 and 11% of each variable missing - and that's what I said in my post. For the 50% of observations that are not complete, typically there is 1 or sometimes 2, and occasionally more, variables missing.
Comment: REgarding the question of listwise deletion / complete case analysis being biased under MAR, I don't have the book available either, but I found lots of references on the net including [this one](http://www.academia.edu/581063/Four_Techniques_for_Dealing_with_Missing_Data) _"Furthermore, if the data are MAR and not MCAR, listwise deletion will produce biased estimates (Allison, 2002; Little & Rubin, 2002)"_
Comment: This reference is unpublished and hence unreviewed. Can you provide a reference from a scholarly publication? In response to the previous comment, I still believe you have failed to describe the nature of your missing data adequately.
Comment: Sure, there are so many references ! I just chose the previous one, as that one specifically referred to Rubin and Little for the justification. [Here](http://os1.amc.nl/mediawiki/images/donders_-_jce_2006_missing_values.pdf) is a scholarly one:
**"In most situations, simple techniques for handling missing data (such as complete case analysis, overall mean imputation, and the missing-indicator method) produce biased results"**
Comment: And from the same paper **"Complete and available case analyses provide inefficient though valid results when missing data are MCAR, but biased results when missing data are MAR,"**
Comment: Not ragging on these Dutch guys but they show no analytical results to convince CC is all that bad. Their citations actually show that semiparametric data (GEE, Cox models) are biased but MLE like logistic regression is *unbiased in MAR data*. Read the book! Ambler does a better job: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.2515&rep=rep1&type=pdf. In the analysis of binary data, you get bias with small sample sizes, so poorly calibrated logistic regression models do exacerbate bias *that's already present in the imputed models*.
Comment: Unfortunately I don't live near a library that has this or similar books so until I can visit a University library I have to rely on online materials. I don't see how the article by Ambler et al that you linked to supports your view that CC is unbiased under MAR. They seem to clearly show that CC **is** biased, for example see Table 3/4/5/6. They also state, in the abstract **"complete case analysis may produce unreliable risk predictions and should be avoided"** and in the discussion **"ignoring missing data and performing a complete case analysis can lead to substantial bias"**
Comment: There are three objectives in an analysis: estimation of risk model coefficients, reliability of risk predictions, and inference of associations. Those conclusions were based on 2. If you're not familiar with small sample bias, you should look this up. This is the crux of their problem and you haven't described your data. As I said, you should report your number of "events per variable" in the analysis.
Comment: Also as I said, I suggest you post for us the results of a simulation study using your data and a controlled missing data mechanism to motivate the use of missing data methods. I feel this commentary is not being evaluated impartially, so I see no need to continue the discussion.
Comment: Thanks again. I'm sorry you don't see the need to continue the discussion. I am certainly trying to understand your commentary and take it on board but I have only begun learning statistics this year so I am not very experienced. I didn't notice where you asked previously for me to report the number of events per variable. Can you explain what that means ? Do you mean the total number of observations and the number of missing entries for each variable ? I hope you will reconsider continuing the discussion, as I am finding it extremely helpful - though somewhat confusing ;)


Função não definida com javascript

Question: Fiz um cshtml e nele há um botão que chama minha função que insere no BD. Acontece que quando eu clico no botão, me dá esse erro: Uncaught ReferenceError: InsereCadastro is not defined. O que eu preciso fazer para contornar esse erro? Achei que não fosse necessário mais nada, pois o botão chama a função. Veja meu código:
Meu cshtml:
<!DOCTYPE html>

    <meta name="viewport" content="width=device-width" />
    <title>Cadastro</title>

    <link href="~/Content/bootstrap.css" rel="stylesheet" />

    <form role="form" class="">

            Nome completo
            <input type="text" class="form-control col-lg-2" id="txtNmCadastro" name="nome" placeholder="Digite seu nome completo">

            Email
            <input type="email" class="form-control col-lg-2" id="txtEmail" name="email" placeholder="Digite seu e-mail válido">

            Endereço
            <input type="text" class="form-control col-lg-2" id="txtEndereco" name="Endereco" placeholder="Digite seu endereço">

            Bairro
            <input type="text" class="form-control col-lg-2" id="txtBairro" name="Bairro" placeholder="Digite o bairro">

            Cidade
            <input type="email" class="form-control col-lg-2" id="txtCidade" name="Cidade" placeholder="Digite a cidade">

            Telefone
            <input type="text" class="form-control col-lg-2" id="txtTelefone" name="Telefone" placeholder="Digite o número do seu telefone">

            Celular
            <input type="text" class="form-control col-lg-2" id="txtCelular" name="Celular" placeholder="Digite o número do seu celular">

            <input id="btnGravarCadastro" type="button" class="btn btn-success" value="Gravar" onclick=" return InsereCadastro();" />

        </form>

Meu arquivo .js(jquery)
function InsereCadastro() {

    var resultado = jQuery.parseJSON('{ "nmcadastro": "' +$("#txtNmCadastro").val() + '" , "email_cadastro": "' + $("#txtEmail").val() + '" , "end_cadastro": "' + $("#txtEndereco").val()
                                        + '" , "bairro_cadastro": "' + $("#txtBairro").val() + '" , "cidade_cadastro": "' + $("#txtCidade").val() + '" , "uf_cadastro": "'+ $("#cbxUf").val()
                                        + '" , "tel_cadastro": "' + $("#txtTelefone").val() + '" , "cel_cadastro": "' + $("#txtCelular").val() +'" }');

    $.ajax({
        url: '/Cadastro/InsereCadastro',
        datatype: 'json',
        contentType: "application/json; charset=utf-8",
        type: "POST",
        data: JSON.stringify({ _cadastro: resultado }),
        success: function (data) {

            alert('Registro gravado com sucesso.');

        },
        error: function (error) {

            alert('Erro ao tentar gravar o registro.');
        }

    });
}

O código de minha controller
[HttpPost]
        public void InsereCadastro(tbl_cadastro _cadastro)
        {

           using (INETGLOBALEntities db = new INETGLOBALEntities())
           {
               tbl_cadastro tb = new tbl_cadastro();

               try
               {
                   tb.nmcadastro = _cadastro.nmcadastro;
                   tb.email_cadastro = _cadastro.email_cadastro;
                   tb.end_cadastro = _cadastro.end_cadastro;
                   tb.bairro_cadastro = _cadastro.bairro_cadastro;
                   tb.cidade_cadastro = _cadastro.cidade_cadastro;
                   tb.uf_cadastro = _cadastro.uf_cadastro;
                   tb.tel_cadastro = _cadastro.tel_cadastro;
                   tb.cel_cadastro = _cadastro.cel_cadastro;

                   db.tbl_cadastro.Add(tb);
                   db.SaveChanges();
               }
               catch (Exception ex)
               {
                   Erro = "Erro na gravação do registro: " + ex.Message;
               }

           }

Comment: qual é o nome do arquivo onde está a função `InsereCadastro`?
Answer: Existe duas possibilidades

A função realmente não foi declarada, você esqueceu de chamar o seu arquivo javascript
Existe algum erro no seu arquivo javascript antes de vocẽ declarar a função

Você colocou esta função no seu arquivo Util.js?

Função não definida com javascript


Need a FullScreen JavaScript or any logic?

Question: Need a full screen where nothing appears excpet body part of my page. No task Bar at bottom and no title bar, addressbar, location bar, toolbar, command menu at top.
Need only HTML Body part to full Screen.
Googled a lot but could not get any appropriate which perform well in all browsers.
Comment: Pressing F11 does the job I think.
Comment: Full Screen API: http://johndyer.name/native-fullscreen-javascript-api-plus-jquery-plugin/
Comment: But I do not want user to press it manually. It should be done on page load, Any Idea? @SampritiPanda
Answer: You could try starting the browser up in Kiosk mode from the command line or a shortcut, most have browser have a way to do it.
For Chrome in Windows you can start it up from a command line with
START chrome.exe "THE-URL-YOU-WANT" --kiosk
or you can add --kiosk to the target in the desktop or taskbar shortcut properties (right-click -> properties) ie '"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --kiosk'
or on a Mac use
cd /Applications/Google\ Chrome.app/Contents/MacOS/
open -a Google\ Chrome --args --kiosk "THE-URL-YOU-WANT"
Answer: Different browsers have different methods for the full screen API.
// Webkit (works in Safari and Chrome )
element.webkitRequestFullScreen();
document.webkitCancelFullScreen();

// Firefox
element.mozRequestFullScreen();
document.mozCancelFullScreen();

For a more comprehensive guide, read through this article on the FullScreen API.
http://johndyer.name/native-fullscreen-javascript-api-plus-jquery-plugin/
Comment: This probabely does not work in IE. I need full support in all browser. @Sampriti Panda
Answer: I don't know if I understand your question correctly but if you need only a page to be fullscreen with javascript, you have to use a code like this:
element.addEventListener('click', function(){
      var domPrefixes = 'webkit moz o ms khtml'.split(' '), i = 0, prefix = null;
      while(!(prefix = previewFrame[domPrefixes[i] + 'RequestFullScreen'])){ i++; }
      prefix.apply(this);
});
Important: this code only works with a click or any other user interaction event. More informations at: https://developer.mozilla.org/en-US/docs/Web/API/Fullscreen_API


Is there module that will add a drop down widget that auto-populates with a list of the site users?

Question: Looking to have a "field widget" that creates a drop down populated with the site's users.  I'm not sure "field widget" is the right term.  I just mean a widget that's used in a field.
Does anyone know of such a module?
Answer: You're looking for the References  module

This project provides D7 versions of the 'node_reference' and 'user_reference' field types, that were part of the CCK package in D6, at functional parity with the D6 counterparts. See http://drupal.org/node/533222 for details.

Similar module is Entity Reference

Provides a field type that can reference arbitrary entities:

You can add a new field for user list by selecting User Reference as a field type and select list as Widget Type in your content type.
Here is the screenshot for that


Is there a logical difference between nested Min (), Max () functions in PHP?

Question:
Is there a logical difference between these two functions ?
min ( $a , $b , $c ) ;

and
min ( $a , min ( $b , $c ) ) ;

Edit
Currently developing a tiny class to manage a colour object for manipulating channel values between colour spaces like ; RGBtoHEX, RGBtoCMYK etc ... While I was looking for some sample code to make better understanding about conversion algorithms, many of functions written have the second method implemented to calculate min, max values of given RGB channel like ;
$min = min ( $R , min ( $G , $B ) ) ;

So ; is it safe to assume if the following function returns the same $min value ?
$min = min ( $R , $G , $B ) ;

Comment: No, their results should be identical. The first should be more efficient, though.
Comment: just min(2, 1, -2) == min(min(2,1), -2) ,These all are equal , leads to same results, also in min() You can compare unlimited number of values. (as per Documentation)
Comment: @raina77ow: You are totally right, they _should_ be identical. The truth is they are not: `min('0', 0, '.') !== min('0', min(0, '.'))` (as [PleaseStand suggests in the answer](http://stackoverflow.com/a/16878508/548696)). Proof: http://ideone.com/PZeSQI
Comment: The reason you may see `min(a,min(b,c))` is that in many libraries, the `min()` function can only accept two parameters. In PHP, they are identical, minus any odd edge cases regarding comparing mixed data types.
Comment: @Matthew [link](http://clarion.pesow.com/ENG/ALGORITHMS/algorithms_colors.htm) These are colour algorithms for an open source .NET extension. I have seen many ones written with just 2 parameters in various languages but there are many of them as well, written in PHP. I think there are a lot of similar **bad-formed** PHP functions around ported carelessly from any other language or from one another. Thanks for pointing this limitation out.
Comment: @Tadeck You're right, of course; I've thought about it, but totally forgot about `'0'`/`0` mix. And it's not even about mixed datatypes - it's about the rule that _order_ of operands influences the result of `min`. That, in turn, obviously gives such wonderful result when `'0'` and `0` are mixed.
Answer: There is a difference, although it is unlikely to be encountered in practice.
php> $a = '0'; $b = 0; $c = '.';
php> var_dump(min($b,$c));
int(0)
php> var_dump(min($a,min($b,$c)));
string(1) "0"
php> var_dump(min($a,$b,$c));
string(1) "."

Relevant quote from the documentation:

PHP will evaluate a non-numeric string as 0 if compared to integer, but still return the string if it's seen as the numerically lowest value. If multiple arguments evaluate to 0, min() will return the lowest alphanumerical string value if any strings are given, else a numeric 0 is returned.
Comment: I do not want to start flame war, but this is one of many inconsistencies in PHP that PHP developer needs to take into account. `min($a, $b, $c) !== min($a, min($b, $c))` is a very good example for that. Upvote goes to you.
Comment: So the result is dependent upon variables I think. It is nice to know that these two functions will not produce the same result **whatever** the case.
Comment: The true WTH of PHP is that `min($a, $b)` !== `min($b, $a)` in too many cases.


Como Ordenar Dados de Modelos Relacionados?

Question: Tenho o seguinte relacionamento na minha aplicação:
Modelo Cliente:
class Cliente extends Model
{
    protected $table = 'clientes';

    public function Usuario(){
        return $this->hasMany('App\Models\Admin\Usuario', 'id_cliente');
    }
}

Modelo Usuario:
class Usuario extends Model
{
    protected $table = 'usuarios';

    public function Cliente(){
        return $this->belongsTo('App\Models\Admin\Cliente', 'id_cliente');
    }
}

Porém preciso ordenar os resultados quando eu chamo todos os usuários do cliente na minha view:
@foreach($Cliente->Usuario as $usuario)...

Esse é apenas um exemplo, tenho diversos relacionamentos na minha aplicação que eu preciso aplicar esse filtro de ordenação, como eu posso fazer isso?
Comment: +1 importante essa pergunta.
Answer: No Laravel é possível fazer o carregamento prévio dos resultados dos relacionamentos de um determinado model. Assim, juntamente com esse carregamento, você pode também adicionar condições que serão aplicadas aos relacionamentos que serão carregados junto com sua consulta. Isso é feito através do método with (Sempre que quiser usá-lo, ele deve ser o primeiro método a ser chamado).
No controller, você pode fazer assim:
$with['usuarios'] = function ($query) {
    return $query->orderBy('nome');
};

$cliente = Cliente::with($with)->find($id);

Ou você também pode separar os dois, e fazer uma segunda query para os usuários baseado na query do cliente.
 $cliente = Cliente::find($id);

 $usuarios = $cliente->usuarios()->orderBy('nome')->get();

No segundo exemplo, você teria que fazer o foreach em $usuarios.
Ordenando a coleção
Também é possível ordenar os dados de relacionamento depois da conclusão da consulta.
Por exemplo, quero ordenar meus usuários que estão relacionados com um cliente pelo nome. E depois quero listá-los novamente e ordená-los pela data de atualização.
Ao invés de fazer duas consultas, você pode fazer assim:
#controller

 $cliente = Cliente::with('usuarios')->find($id);

 $ordenadores['nome'] = function ($usuario)
 {
      return $usuario->nome;
 };

 $ordenadores['atualizacao'] = function ($usuario)
 {
        return $usuario->created_at;
 };

 return view('minha_view', compact('cliente', 'ordenadores'));

Aí na view, você pode fazer assim:
#view

@foreach($cliente->usuarios->sortBy($ordenadores['nome']) as $usuario)

@endforeach

@foreach($client->usuarios->sortBy($ordenadores['atualizacao']) as $usuario)

@endforeach

Nota: Em alguns casos, não é necessário usar um Callback no sortBy. Basta passar o nome do campo como argumento de sortBy, para ordenar os dados na Collection.
@foreach($client->usuarios->sortBy('created_at') as $usuario)

@endforeach

Dica
Geralmente, para manter a legibilidade dos métodos do seu model, sugiro que utilize singular para relacionamentos (um para um) como belongsTo e hasOne, e plural para hasMany ou belongsToMany e afins (múltiplos relacionamentos).
Comment: @RaylanSoares esqueci de mencionar que internamente o Laravel já faz isso (transforma a key em callback) para usar no sortBy.
Comment: Se precisar, tembém tem o `sortByDesc('nome')`
Comment: muito bom mano, na verdade mandei um vai que cola e funcionou até sem fazer as modificações no controller, apenas colocando o nome do campo que eu queria ordenar no "sortBy". ->sortBy('nome'). Muito obrigado msm!
Comment: pow muito show mano


How do you find dependencies when installing software?

Question: In software installation tutorials, they always have a long list of dependencies. How do they find those dependencies?
For example, I'm starting on a fresh Ubuntu server, following directions to install rvm. They provide something like:
sudo apt-get install git-core libreadline5 libncurses5-dev libreadline5-dev build-essential zlib1g-dev libssl-dev libpcre3-dev libxml2-dev libxslt-dev;

I just want to understand how they find all of those dependencies. Copy and pasting works, but I think it's more important to understand it.
Answer: Each package comes with a list of its own dependencies. For debs, which Ubuntu uses, you can do:-
[andys@daedalus ~]$ apt-cache depends git-core
git-core
  Depends: git

The dependencies for a package are specified when the package is built. Some of them are specified manually by the package maintainer (the person who's responsible for building the package), and some are automatically determined when the package is built. If you download a Debian source package, and look in the debian/control file, you'll see a number of lines like Depends:, Conflicts:, Replaces:, etc.
So, back to the example - git-core depends on only one package, which is git. git is probably a metapackage of some sort, which in turn has a whole raft of dependencies, to make sure all the packages neccessary to run git are installed - not just to satisfy the dependencies of one package. If we look at the git package:-
[andys@daedalus ~]$ apt-cache depends git
git
  Depends: libc6
  Depends: libcurl3-gnutls
  <...snip...>
  Suggests: git-cvs
  Suggests: git-svn
  Suggests: git-email
  <...snip...>
  Recommends: rsync
  <...snip...>
  Conflicts: git-core
  <...snip...>
  Breaks: stgit
  Breaks: stgit-contrib
  Replaces: <cogito>
  Replaces: git-core

...you'll see that git itself has further dependencies. To complicate things further, there's different kinds of dependency other than just requiring another package to be installed...

Conflicts - this package conflicts with another package, and you probably won't be able to install the conflicting package without uninstalling this one, and vice versa. In this case, git conflicts with git-core. On my box here, the git-core package is marked as obsolete, which is probably why this is listed here.
Breaks - this package breaks another package, and installing it - although perhaps not requiring you to uninstall the package that will be broken - will cause problems with the software installed by that package. Again, looking at our example, the stgit and stgit-contrib packages will be considered broken once git is installed.
Replaces - this package replaces another package, meaning that the other package is obsolete and can probably be uninstalled. The git-core package shows up again here.

There are also 'soft' dependency types, such as Recommends and Suggests, which aren't required by the package you're installing, but are recommended or suggested, as the name suggests. Once again, looking at the example, the suggested and recommended packages are add-ons to git, or tools not directly related that will make using it easier.
Putting all this together, the package manager you use will start off with the package (or packages) you've asked it to install, and will look up the dependencies for it, and then any dependencies for those packages, and so on. It will also check for packages which conflict, break or replace the requested packages or their dependencies. Once it's happy that it's found a way it can satisfy your original request, it'll then go ahead with the install.
Different package managers have different ways of doing this dependency resolution. apt-get, aptitude and dselect, for example, all essentially do the same thing but the way they make their decision differs. You may find that aptitude and dselect will happily present you with a way to install a package whereas apt-get can't.
If you're interested in knowing more about Debian packages, have a look at the Debian New Maintainers' Guide, which goes over some of the guts of debs.
Comment: +1. Thorough, clear.
Answer: usually apt will do that for you. if you try to install a binary package that has unmet dependencies, apt will tell you which ones are missing and ask you for permission to install them as well.
most source distributions tell you in a README file what the -dev dependencies are.


Windows 2008 Server Core vs Full Installation

Question: Recently I have gone through some articles regarding Windows Server core installation, which is very light weight server and consume low resources when compared with full installation.
What my doubt is whether we REALLY get any benefit with "server core" installation? Because, generally the server will be in log off mode and it does not use any additional resources for GUI environment.
Answer: The biggest gain to Server Core is the reduced attack surface. The CPU/RAM gains are minimal, though important in certain scenarios (Hyper-V and very low memory systems/VMs spring to mind).
Answer: Core is designed to be small (disk space) and lightweight (cpu) therefore it is ideal to use in a VM for a simple, unattended task such as DHCP, DNS, print server, etc, almost as if it is an appliance. You would gain little from running Core on a physical machine, on modern hardware.
Answer: Depends what you do with it. Core also needs a lot less reboots (because less components get updates that may result in a need to reboot). I use core mostly for Hyper-V setups (well, Hyper-V server IS a server core).
The lower restart need is the main advantage. Driver update "issues" (until you find out how to acutally install new drivers properly) the main disdvantage.


Como pegar mais de um valor do banco de dados e salvar em uma ComboBox mais de um valor retornado?

Question: Estou com uma dúvida que é a seguinte, quando eu faço uma consulta no banco ele pode me retornar mais de dois nomes, e só consigo pegar um.
A idéia é filtrar os nomes dos médicos pelo tipo de médico(Neurocirurgião, Entre outros) e depois pegar esses nomes e colocar em uma ComboBox.
Aqui está o código porém retorna apenas um nome.
Criei uma função para isso.
    public String tipoMedico(String tipo)
    {
    try{
    Statement comando = cone.createStatement();
    ResultSet rs = comando.executeQuery("SELECT Nome,Tipo FROM `medicos`
    WHERE Tipo='"+tipo+"';");//Se tiver dois médicos do mesmo tipo, era
    para retornar mais de um nome, quero saber como pegar todos os nomes
    que retornar.

    while(rs.next()) {

              String Tipo = rs.getString("Tipo");
                String Nome1 = rs.getString("Nome");

                if(tipo.equals(Tipo)){

                 return Nome1;
                  }
    }}
    catch(SQLException e)
    {
      JOptionPane.showMessageDialog(null,"Erro na Conexão com o Banco de
    Dados");
    }
     return "Não encontrado";
    }

Depois chamar os Nomes em uma ComboBox.
   private void cbTipoActionPerformed(java.awt.event.ActionEvent evt) {
   String nome =
   dado.tipoMedico(cbTipo.getItemAt(cbTipo.getSelectedIndex()));
   String tipo = cbTipo.getItemAt(cbTipo.getSelectedIndex());
    switch(tipo)
   {
    case "Neurocirurgião": cbDoutor.removeAllItems();
    cbDoutor.addItem(nome);//Aqui eu adiciono o nome retornado do banco
    na comboBox
      break;
      case "Clinico Geral": cbDoutor.removeAllItems();
       cbDoutor.addItem(nome);
      break;

     }}

Comment: cara, o problema tá no seu return. Você tá familiarizado com a estrutura `LIst`?
Comment: Você tá retornando uma string simples do banco.Um combobox espera uma combomodel ou um vetor de strings.
Comment: Outra coisa, não se deve misturar as camadas, onde for pra pesquisar no banco, deve ser somente para consulta e funções de banco, botar optionpane em método de banco e com retorno definido é uma péssima prática.
Answer: Seu problema é que você retorna no primeiro incremento do ResultSet. Então sugiro que você retorne um List de String:
public List<String> tipoMedico(String tipo) {
List<String> medicos = new ArrayList<>();

try {
  Statement comando = cone.createStatement();
  ResultSet rs = comando.executeQuery("SELECT Nome,Tipo FROM `medicos` WHERE Tipo ='" + tipo + "'  ;");

  while (rs.next()) {
    String tipoEncontrado = rs.getString("Tipo");
    String nome = rs.getString("Nome");

    if (tipoEncontrado.equals(tipoEncontrado)) {
      medicos.add(nome);
    }
  }
} catch (SQLException e) {
  JOptionPane.showMessageDialog(null, "Erro na Conexão com o Banco de Dados");
}

return medicos;

}
E adicione o resultado ao combo da seguinte forma:
private void cbTipoActionPerformed(java.awt.event.ActionEvent evt) {
  List<String> nomes = dado.tipoMedico(cbTipo.getItemAt(cbTipo.getSelectedIndex()));
  String tipo = cbTipo.getItemAt(cbTipo.getSelectedIndex());

  cbDoutor.removeAllItems();

  for (String nome : nomes) {
    cbDoutor.addItem(nome);
  }
}

Comment: @AndersonHenrique entendo, mas fica a dica, sempre procure separar as responsabilidades, por exemplo, nesse seu código, eu teria uma classe Médico e outra para especialidade, geralmente a relação de médico para especialidade é nxn(médicos pode tem várias especialidades, e vice-versa).
Comment: Qual é a idéia de dar -1 nessa resposta e nem deixar um comentário do que pode ser melhorado ou que teoricamente está errado?
Comment: Nossa cara muito Obrigado Mesmo, vou tentar aqui, Enquanto ao DiegoFm nossa cara valeu pela dica, eu criei uma classe propria pro banco sim, mas não sabia dessa do JOptionPane muito obrigado, Depois volto com o resultado Obrigado Sorack
Comment: Não esqueça de marcar o ✅ e + 1 (útil) caso essa solução lhe atenda, para que possa ser utilizada como solução para outras pessoas também!
Comment: Deu certo de Listar mais que 1 valor, porém agora esta com um problema, toda vez que seleciono um tipo na check box ele está adicionando o mesmo nome novamente, se eu selecionar neurocirurgião 2x está aparecendo os nomes em dobro
Comment: Você tá dando o removeAllItens antes de adicionar?
Comment: @AndersonHenrique já cogitou criar um comboboxmodel baseado numa classe medico? Não é complicado e torna seu código mais organizado.
Comment: Era isso mesmo esqueci do removeAllItens, muito obrigado mesmo irei marcar aqui, onde marca isso? Sou novo aqui, criei hoje
Comment: Ao lado da resposta. Caso queira melhor a qualidade do código você pode criar um model como o @diegofm comentou e também separar as classes. Ah, no seu exemplo original tinham variáveis iniciando com letra maiúscula, pela convenção da antiga Sun você deve sempre iniciar nomes de variável com letra minúscula
Comment: @diegofm então cara, estou no inicio de java na faculdade, tem muitas coisas que não sei, pra falar verdade a maioria do que ja sei aprendi na internet mesmo


How do I conditionally add a field?

Question: I'm trying to find out any example of adding new conditional field using ajax in Drupal 8 form API, but I can't find it.
This is my code.
<code>$form['name'] = array(
    '#type' => 'textfield',
    '#title' => $this->t('Name'),
    '#maxlength' => 64,
    '#size' => 64,
    '#default_value' => $name,
);

$form['floor'] = array(
    '#type' => 'textfield',
    '#title' => $this->t('Floor'),
    '#default_value' => $floor,
);

$form['flat_per_floor'] = array(
    '#type' => 'textfield',
    '#title' => $this->t('Flat Per Floor'),
    '#default_value' => $flatPerFloor,
    '#ajax' => array(
        'callback' => 'Drupal\bms_flat\Form\FlatForm::flatGenerate',
        'effect' => 'fade',
        'event' => 'change',
        'method' => 'replace',
        'progress' => array(
            'type' => 'throbber',
            'message' => "Generating..",
        ),
    ),
);

if ($form_values['floor']) {
  $form['flats'] = array(
    '#type' => 'checkboxes',
    '#title' => $this->t('Content Type'),
    '#options' => node_type_get_names(),
  );
}

$form['submit'] = array(
    '#type' => 'submit',
    '#title' => $this->t('Submit'),
    '#default_value' => $this->t($submit),
);
</code>
This is the callback function.
<code>public function flatGenerate(array &$form, FormStateInterface $form_state) {
  // …
}
</code>
How do I add a new field after filling-up my Flat Per Floor field?
Comment: is flat per floor an optional field? or a required field?
Comment: It will be required field. But I didn't add the required option.
Comment: then you could go with my [answer here](http://drupal.stackexchange.com/questions/201372/create-dynamic-textfield?noredirect=1#answer-201494)
Answer: I don't have much experience in Drupal 8. Please refer my suggestion bellow.

Please add wrapper item for the ajax call
eg:

$form['flat_per_floor'] = array(
    '#type' => 'textfield',
    '#title' => $this->t('Flat Per Floor'),
    '#default_value' => $flatPerFloor,
    '#ajax' => array(
        'callback' => 'Drupal\bms_flat\Form\FlatForm::flatGenerate',
        'effect' => 'fade',
        'event' => 'change',
        'method' => 'replace',
        'progress' => array(
            'type' => 'throbber',
            'message' => "Generating..",
        ),
        'wrapper' => 'flat-per-floor-wrapper',
    ),
);

Add the wrapper container under the flat_per_floor field

$form['flat_per_floor_wrapper'] =  array(
  '#type' => 'container',
  '#attributes' => ['id' => 'flat-per-floor-wrapper']
);

For more details please refer the example module for drupal 8 and check the ajax form
https://www.drupal.org/project/examples

How do I conditionally add a field?


The sum of all odd digits of an input

Question: How to compute the sum of all 'odd digits of an input' using 'loop'. (For example, if the input is 32677, the sum would be 3 + 7 + 7 = 17.)
I can't quite figure out how to do that, can someone please help me out. This is what I have done so far, I don't know how to complete it or whether I have its right or wrong.
Any help would be appreciated!
System.out.println("Enter a number: ");
String input = in.nextLine();
int length = input.length();

int sum = 0;
int digits = 0;

for (int i = 0; i < length; i++) {
    if (length % 2 == 1) {
        digits += i;
        sum = digits++;
    }
}
System.out.println(sum);

Comment: why did you add this check: if(length%2==1){ ?
Comment: Does this answer your question? [Iterate through each digit in a number](https://stackoverflow.com/questions/5009591/iterate-through-each-digit-in-a-number)
Comment: You aren't too far, you found the main logic. Few steps are missing: inside your loop, you need to get the character indexed at `i` (e.g. with `String#charAt`), then you need to convert it into a number (e.g. with `Integer#valueOf`). Only then, you can do your modulo on it to know whether odd or even, and add it to the sum if odd.
Answer: Here comes a Java8-based solution:
final int result = input.chars()//make a stream of chars from string
                        .mapToObj(String::valueOf) // make every character a String to be able to use parseInt later
                        .mapToInt(Integer::parseInt) // transform character in int
                        .filter(i -> i % 2 == 1) // filter out even numbers
                        .sum();

Answer: You don't need to use String if your input is not so long.
also for safe side use long datatype.
Here is the working code with comments (explain each step).
long sumOddDigits(long value){
    long temp = value; // copy in temp variable
    long sum = 0;
    while(temp > 0){
      int digit = temp%10; // get last digit of number. example: 227 gives 7.
      temp = temp / 10; // remove that last digit from number.227 will be 22.
      if(digit % 2 == 1){
        sum += digit;
      }
    }
    return sum;
}

Comment: I think this is a really nice solution since you don't need any data conversion. If you use this solution you may also use `Scanner.nextInt()` and `Scanner.hasNextInt()` to make sure that the user can only input `Integer` values and catch inputs that are not `Integer` values.
Answer: Your interpretation of the digits inside of input is not working this way.
System.out.println("Enter a number: ");
String input = in.nextLine();
int length = input.length();

int sum = 0;

for (int i = 0; i < length; i++) {
    int digit = input.charAt(i) - '0';
    if (digit % 2 == 1) {
        System.out.println("Add digit: " + digit);
        sum += digit;
    }
}
System.out.println(sum);

Comment: You get a part of input, a sub string, between two indices. You start with the character at index i and end at index i + 1, so in the end you only get one character.
I would prefer charAt(i), but that returns a 'char' which is misinterpreted as Integer.
Comment: @Milgo, you could be using `charAt` too, and it would be even simpler to get the digit value: `int digit = input.charAt(i) - '0';` :)
Comment: Hi Milgo, what does +1 do exactly in this specific code: Integer digit = Integer.valueOf(input.substring(i, i + 1));
Am just intrested and want to learn
Comment: @AlexRudenko The output shows a totally wrong output, am I doing something wrong with your way?
Comment: @chabztar01, please clarify.  Both initial version and the updated one work fine for integer numbers without decimal or thousand separators.  Did you try to input a non-integer number containing other characters than digits?
Comment: @AlexRudenko I changed it to the following:

System.out.println("Enter a number: ");
        String input = in.nextLine();
        int length = input.length();

        int sum = 0;

        for (int i = 0; i < length; i++) {
            int digit = input.charAt(i);
            if (digit % 2 == 1) {
                System.out.println(digit);
                sum += digit;
            }
        }
        System.out.println(sum);

And the output showed the following:

Enter a number:
32677,
51,
55,
55,
161
Comment: @AlexRudenko sorry, it looks so messy when I put the code in the comment
Comment: @chabztar01, you need to subtract the value of `'0'` as it is written above to convert a char `'0', '1',..'9'` to integer number `0, 1, 2,.. 9`
Answer: In your loop, use Integer.parseInt(input.charAt(i)) to get the number at position i.
if(length%2==1){ that doesn't make sense here. You want to check if your number is odd, not the length of your string.

The sum of all odd digits of an input


Add relationships that has 3 layers, Parent -> Child1 -> Child2

Question: Recently I asked a question that was similar (Parent node needs to display the scheduled date that is referenced in the child node), but requirements have changed and now I'm dealing with the Node A (parent) -> Node B (child) -> Node C (grand-child). Bear with me as I explain this.
Node A is the main parent.  It contains an entity reference field that calls Node B.  I've successfully created the contextual filter and relationship to display the necessary information from both Node A and Node B.  Entering the mix is Node C.  Node C contains one piece of info that needs to be displayed.  There is an entity reference field in Node B that calls Node C.
I currently have a contextual filter and relationship on the Node B entity reference field, but the field from Node C isn't displaying.  I'm sure this has to do with there being 3 layers to this view.  Is there something here that I'm missing, or is views unable to handle the 3 layer relationship?
Answer: Well, that didn't take much time at all.  After doing a bit more searching around, I came up with the following solution.
Under "Relationships" I created another one on the entity reference field used in Node B.  Then I assigned the relationship that was created for the Node A entity reference field to this new relationship.  Note: the relationship is NOT required because some items do not contain the info from Node C.
Next, under "Fields" I updated the relationship used for the field containing the info from Node C to reference this new relationship. All fields display as/when they should.
I hope this helps someone else out.  Cheers.


Adapting generic code to different size of linked list in C

Question: For a programming assignment, we've been asked to read in some data from a text file and populate a linked list with the data. Here is the example code we've been given:
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

#define MAX_INPUT 20
#define EXTRA_CHARS 2

typedef struct listNode
{
   int data;
   struct listNode * next;
} ListNode;

typedef ListNode * ListNodePtr;

int main()
{
   ListNodePtr head, new, current, previous, next;
   unsigned listSize;

   int i, anInt;
   char str[MAX_INPUT];

   listSize = 0;
   head = NULL;

   while (fgets(str, MAX_INPUT+EXTRA_CHARS, stdin) != NULL)
   {
      /* Parsing the string to int */
      if(sscanf(str,"%d",&anInt) != 1)
      {
         sprintf(str, "Invalid input entered \n");
         exit(EXIT_FAILURE);
      }

      /* Creating the node using malloc(...) */

      if ( (new=malloc(sizeof(ListNode))) == NULL)
      {
         fprintf(stderr,"\nMemory Allocation for ListInsert failed!\n");
         fprintf(stderr,"Aborting data entry!\n");
         break;
      }

      current = head;
      previous = NULL;
      /* Search to find where in insert new list node */
      while (current != NULL && current->data < anInt)
      {
         previous = current;
         current = current->next;
      }

      new->data = anInt;
      new->next = current;
      listSize++;

      if (previous == NULL)
      {
         head = new;
      }
      else
      {
         previous->next = new;
      }
   }/*End of input loop */

   /* Display integers in linked list */
   current = head;
   while (current != NULL)
   {
      printf("%d\n", current->data);
      current = current->next;
   }

   /* Deallocate memory used by list nodes */
   current = head;
   while (current != NULL)
   {
      next = current->next;
      free(current);
      current = next;
   }

   return EXIT_SUCCESS;
}

Here's the problem I have with it. In every example of linked lists I've seen so far online or in books, the definition of a linked list is given as a struct that contains only one item of data and a pointer to the next node in the list. The problem is that we've been given the following structure definitions to populate with data:
typedef struct price
{
   unsigned dollars;
   unsigned cents;
} PriceType;

typedef struct item
{
   char itemID[ID_LEN + 1];
   char itemName[MAX_NAME_LEN + 1];
   PriceType prices[NUM_PRICES];
   char itemDescription[MAX_DESC_LEN + 1];
   ItemTypePtr nextItem;
} ItemType;

typedef struct category
{
   char categoryID[ID_LEN + 1];
   char categoryName[MAX_NAME_LEN + 1];
   char drinkType;      /* (H)ot or (C)old. */
   char categoryDescription[MAX_DESC_LEN + 1];
   CategoryTypePtr nextCategory;
   ItemTypePtr headItem;
   unsigned numItems;
} CategoryType;

typedef struct bcs
{
   CategoryTypePtr headCategory; /* Pointer to the next node */
   unsigned numCategories;
} BCSType;

This does not fit in with all the examples I've seen. So in the "generic" code above, do I have to do everything above, but replace the "new->data" part with, for example, "category->categoryID", and "category->categoryName" etc etc for all the members of the struct in order to populate the entire linked list with data?
Comment: "In EVERY example of linked lists I've seen online or in books" -- but those are all simplified examples. You need to learn to think abstractly ... clearly there is no reason in principle that there can't be more (or less) than one data item in a node. And you haven't read very widely if you've never seen such a thing ... examples are posted here at SO all the time.
Comment: Look at it very closely. You have a means to have them all wrapped up together thus allowing for a singular node link.
Comment: *but replace the "new->data" part with, for example, "category->categoryID", and "category->categoryName"* -- no, you don't have to replace anything ... the structures you were given already have all the "next" pointers they need, in the CategoryTypePtr and ItemTypePtr members.
Comment: @ Jim Balter but when I try to use the code, the compiler tells me that there's no member "data" in the structure, etc, so I must have to use names of members of individual structs instead of generic variables.
Comment: Like I said, the structures contain everything you need ... and that does not include a `data` member ... again, that's just a *simplified example*. " so I must have to use names of members of individual structs instead of generic variables" -- you must use the names of the structs and the names of their members that you were given. I don't know what you mean by "generic variables" ... "data" is *just an example*. Professional programmers rarely use meaningless names like that, they use meaningful names like in the struct definitions you were given.
Comment: Addendum: There are some linked list *libraries* that can handle arbitrary data and so they have "generic" `void*` pointers called `data`, but you aren't writing a library, you're writing an application that has specific data structures.
Comment: "when I try to use the code" -- I don't know what *you* mean by "use the code". You should be able to compile that code ... `ListNode` does contain a `data` member ... and run it. But obviously it won't do what you need to do ... **it's just an example**, a demonstration of how to allocate, store into, print, and free a list. Use it as a **usage guide**, but it's a bad idea to try to start with that code and then modify it. Start your program from scratch, referring to that code for the basic ideas of how to deal with linked lists.
Comment: *but replace the "new->data" part with, for example, "category->categoryID", and "category->categoryName"* -- sorry, I misunderstood what you were asking earlier. Yes, you would use references like that in your code to access the members of your structs, instead of `new->data`.
Comment: `fgets(str, MAX_INPUT+EXTRA_CHARS, stdin)` : `+EXTRA_CHARS` is bad idea.
Answer: Everything you need in the way of data structures was given to you. There's a top level item, BCSType, that leads to everything else. It has a linked list of categories with the head link in headCategory. And each CategoryType has a next pointer (nextCategory), and head link (headItem) to a linked list of ItemTypes, each of which has a next pointer, nextItem. You don't need to add anything to these structs and should not (and you'll get graded down if you do). Now you need to write the code that reads the data from the file and creates instances of these data structures, using the example code as a guideline for dealing with linked lists ... but you have to think, and be creative, to apply it to the three-level structure you've been given.
The vital thing here is to learn by doing. Try to write such code and don't give up, but if you truly get stuck you can, as a last resort, ask another question here ... but be very specific when you do, and include your code and error messages and all other relevant information about what you expected to happen and what actually happened instead. And be sure to learn how to use a debugger, and use it, before posting such a question.
Comment: OK thanks. I've been at this for days actually, and I've written several functions to do all that stuff, but to no avail. I posted this question because I thought there was something fundamental about linked lists that I was missing that was tripping me up. I'll go back to the drawing board, maybe start again. My code has become too large and messy.
Comment: @LeeCarbusier I advise initially writing your code top-down -- write calls to functions with names that indicate what they need to do, then write one of the functions that you haven't written yet, and repeat. If, in writing a function, you find you made a bad assumption about how to carve up the functionality, then go back and  modify the caller -- that's bottom-up. But primarily work top-down.
Comment: @LeeCarbusier Please see my latest comment under your question ... I had misunderstood you and so our exchange went off in the wrong direction ... that was entirely my fault, as you were spot on. I'm sorry if I misled you.
Comment: @LeeCarbusier "to no avail" -- As I said in my answer, you should be very specific about such things. Is your code not compiling? Is it compiling but crashing, or infinite looping, or producing the wrong results? You can append your code and what errors you are getting to this question and you may get help with it ... SO encourages posting actual code.
Comment: @LeeCarbusier as Jim suggested a good top down through a simple list will get your list-confidence up. There is no magic in lists, they are one of the most flexible data-structures in your toolbox. Start with a simple one-data item/next struct. You will find you have 2 basic functions in list creation (1) `create-node` where you **allocate** the node (struct) and storage for any members that require allocation; and then (2) **insert-node** where you check if list exist (if not, then node is 1st node), if it does, then just set the `->next` pointers as required. Post back when stuck.


Subseting data and summerise it based on many conditions and groups in R

Question: I need to subset a data frame and compute some summaries based on different groups combination of the two categorical variables: Membership and Sex.
For instance:
set.seed(123)

df <- data.frame( YEAR = c("1999", "1999", "1999", "1999", "2001", "2000","2000", "2000",
"1999","2000", "2000","1999", "2000", "2001", "2001", "2001", "2001"),
Sex=c("M", "M","M", "F", "M","M", "F","F", "F", "M", "F","F", "M", "M", "M", "F", "F"),
Membership = rep(c("YES", "NO"), each = 2),
Income  = sample(1:10000, 17),
Taxes = sample(1:100, 17))

For each year and each of the four possible group combinations (M, Yes), (M, NO), (F, Yes), (F, NO) I need to find out what is the ratio between the total income of each group and the grand income of all groups for that year. The same  applies for taxes column.
For instance, for 1999 and (M, N) the output should be (986+4233)/986+4233+9954+4776+8536) =0.1832
Any idea on how to get it?
Thank you
Comment: Please use `set.seed`.  Also, the example is giving some errors
Answer: I would do something like this...
library(tidyverse)
df %>%
  group_by(YEAR, Sex, Membership) %>%
  summarize(GroupIncome = sum(Income), GroupTaxes = sum(Taxes)) %>%
  group_by(YEAR) %>%
  mutate(TotalIncomeForYear = sum(GroupIncome),
         GroupPercentIncome = GroupIncome / TotalIncomeForYear,
         TotalTaxesForYear = sum(GroupTaxes),
         GroupPercentTaxes = GroupTaxes / TotalTaxesForYear) %>%
  ungroup()

Which yields a dataframe like this...
Comment: This is exactly what I need and it solved out my question entirely. thank you
Answer: The following code uses package dplyr to

Compute YEAR totals of Income and Taxes,
compute sums of Income and Taxes by groups of YEAR, Sex, Membership,
in the same instruction, divide these latter totals by the totals by YEAR computed earlier.

Ungroup the result.
library(dplyr)

df %>%
  group_by(YEAR) %>%
  mutate(SumYearIncome = sum(Income),
         SumYearTaxes = sum(Taxes)) %>%
  ungroup() %>%
  group_by(YEAR, Sex, Membership) %>%
  mutate(RatioGroupIncome = sum(Income)/SumYearIncome,
         RatioGroupTaxes = sum(Taxes)/SumYearTaxes) %>%
  ungroup() %>%
  select(-SumYearIncome, -SumYearTaxes)


Como fazer a junção de duas Arrays?

Question: Olá, Tenho o seguinte problema tenho um ArrayList que retorna varias pequenas arrays de duas posições
String[] temp = new String[2];

e tem o arrayList que salva adiciona varias "temp";
ArrayList lista = new ArrayList();
while(rs.next()){
   if(rs.getString("texto") != null){
    temp[0] = String.valueOf(rs.getInt("id"));
    temp[1] = rs.getString("texto");
    lista.add(temp);
   }else{
    temp[0] = String.valueOf(rs.getInt("id"));
    temp[1] = rs.getString("valor");
   }
}

Bom o temp[0] contém o ID de uma tabela meu desafio e pegar de dentro do ArrayList os valores e agrupalos por meio da posição do ID, assim ficariam apenas algumas arrays tipo a array tempx[] contem todos os temp[1] que tem temp[0]  com o mesmo valor. Que depois vai para outro Array List
Tipo algo assim
enquanto lista.get(a).temp[0] conter o mesmo identificador faça
tempx[i] = lista.get(i).temp[1]
i++;
Quando o lista.get(a).temp[0] mudar então  um novo arrayList deve receber essa nova array.
listaAtualizada.add(tempx);

Não tô muito bem das ideias hoje e tô sofrendo para fazer isso.
Answer:
             List tempA = new ArrayList();
             for(Object l: lista){
                 String[] arrayA = new String[2];
                 String[] ArrayB = (String[]) l;
                 int cont = 0;
                 for(Object l2:  lista){
                     String[] tw = (String[]) l2;
                     if(ArrayB[0].equalsIgnoreCase(tw[0])){
                         tempA.add(tw[1]);
                     }
                 }
                 tempX.add(tempA);
                 tempA = new ArrayList();
             }

Isso gera muitos elementos duplicados então eu removo eles usando

// adiciona todos os elementos incluindos duplicadas
            Set<String> hs = new HashSet<>();
            hs.addAll(tempX);
            tempX.clear();
            tempX.addAll(hs);

Como fazer a junção de duas Arrays?


How many strings are there (inclusion exclusion principle)

Question:
Q: What is the number of strings with the length of $8$ above $\left\{1,2,\cdots,10\right\}$ where $7,8$ appears at least one time?

So by using the inclusion exclusion principle:

$10^8$ is the total number of possibilities
$2\cdot 9^8$ - $7$ or $8$ doesn't appear
$8^8$ - both $7,8$ doesn't appear.

All in all we have:
$$ 10^8 - 2\cdot 9^8 + 8^8$$
"Alternative" answer:
Choose two places for $7,8$: $ 8\choose 2$.
Then, we have six more places to fill with no limitations.
Finally, we multiply by $2!$ (permutations number of $\left\{7,8\right\}$).
Therefore, $$2\cdot C(8, 2)\cdot 10^6$$
Question:
What is the difference between the two answers? (Their value isn't the same)
Answer: Your alternative solution counts some combinations more than once.
Pick locations $1$ and $2$ for $8$ and $7$, then you will get a string like this in your counts:
$$87111187$$
But you can get this string even when you pick the $7$th and $8$th locations for $8$ and $7$ and "fill" the remaining.


Yii2 formatter language

Question: This is my configurations:
'formatter' => [
            'class' => 'yii\i18n\Formatter',
            'dateFormat' => 'd MMMM Y',
            'locale' => 'ru-RU'

      ],

When I am trying this:
echo Yii::$app->formatter->asDate('2014-01-01');

I get:
01 Jan 2014

But I want the same in Russian.
Solved:
Open the file php.ini via OpenServer and uncomment ;extension=php_intl.dll.
Answer: the config you specified should output 01 янв. 2014
you are most likely missing php-intl extension
Answer: With formatter set to 'dateFormat' => 'php:d M Y' you can only get 01 Jan 2014 because PHP's date M stands for

A short textual representation of a month, three letters: Jan through Dec

If you want date to be formatted by intl for your language you need to set formatter with ICU:
'dateFormat' => 'd MMM Y'

like described at ICU User Guide.
Comment: Thanks a lot, anyway :)
Comment: I've edited my question. It doesn't work, too. I was trying this: uncomment `;extension=php_intl.dll`, but after reloading OpenServer it returns to comment. I don't understand how to change it with administration rules
Comment: @Bizley yii2 does [internal conversion](https://github.com/yiisoft/yii2/blob/af71e0fca0b8cc7ea11ce18824d3e9a7af6866ef/framework/i18n/Formatter.php#L631) to icu formats regardless
Comment: @csminb good to know, so it must be missing intl like you said.
Comment: How to uncomment this extension?
Comment: Okey, I've found how to do this.

Yii2 formatter language


Does Amazon remove EC2 instances?

Question: Does Amazon remove EC2 instances that that have not run for a period of time?
Answer: No. Instances can remain in the "Stopped" state indefinitely.

Does Amazon remove EC2 instances?