Picking cases to label for classification

Question: I'm working on a classification problem with the goal of diagnosing kidney diseases from clinical data.  For each patient, we have a large number of observations, and would like to determine whether a patient has a particular kidney disease.  In a minority of cases, the diagnosis is known, but usually it is not, so it seems like a positive/unlabeled classification task.
However, there's a twist: if we want, we can have an expert review the case and determine whether the disease is present or not.  This consumes time and other resources, so we don't want to do it for every patient, but we could do it for some.
One option would be to have the expert review a random sample of cases.  However, I wonder if there's a way to guide the process, so that the expert reviews the most informative cases that would provide the greatest increase in classifier accuracy.  We're open to any type of classifier.
Any suggestions for how to think about this problem?  Any methods or tools we should employ?
Also, how should we deal with our data as being not-quite-positive/unlabeled, since we actually have positive, negative and unlabeled cases.
If it matters, the prevalence of the disease varies depending on the particular disease we're looking at (there are several) but ranges from <1% to ~30% in our cohort.
Comment: You could use mixture models assuming your subjects are drawn from one of 2 subgroups in the population: diseased or not. If your response variable is say normally distributed (maybe some measure of kidney function over time) then you could fit a 2-component normal mixture distribution to the data where each group is modelled with a (for example) linear mixed effects model. You could then incorporate the expert by fitting 2 separate mixed models to only those cases reviewed (hence you know the 2 groups).
Comment: The parameter estimates obtained could then be used to build prior distributions on the model paramters so that you could then estimate within a
Bayesian framework the mixture model on the labelled and unlabelled data. In this way once the expert has done their classification, the classification of subjects
to diseased/un-diseased would be model-based. The accuracy of this classification could of course be checked against the cases the expert has reviewed.
Answer: So your problem is that you have labeled data, and unlabeled data. Look at 1st answer: https://stackoverflow.com/questions/19170603/what-is-the-difference-between-labeled-and-unlabled-data :

There are many active areas of research in machine learning that are
  aimed at integrating unlabeled and labeled data to build better and
  more accurate models of the world. Semi-supervised learning attempts
  to combine unlabeled and labeled data (or, more generally, sets of
  unlabeled data where only some data points have labels) into
  integrated models.

So you have to google for semi-supervised learning. This is way of state-of-the-art.
My way(without reading about semi-supervised learning so much), is to do unsupervised learning on whole data, to get clusters of similar cases. Then use humans(doctors) to describe clusters - is this cluster kidney disease or not?. Then you have data for supervised learning. And you can learn whatever you want Bayesian/LinearRegression/SVM classifiers.
Comment: Partitioning Around Medoids might be the right clustering algorithm for the job. I found success with it in a similar application, although I ended up using a very small number of clusters (4-8) relative to my sample (>1,000)
Answer: Semi-supervised learning methods may well be the way to go, but I don't know it.  A clustering approach to guiding doctors towards more useful cases to label (as @user1615070 suggests) also has some merits.
Let me suggest a different strategy.  In logistic regression, unlike linear regression, much of the information exists within a narrower range of your predictor variables.  Consider this contrived plot:

Note that the probability of 'success' goes from $.2$ at $x = -.73$ to $.8$ at $x = .73$, a range of less than $1.5$ on $x$.  It is within this range that you have the least information about the status of the true label for $y$.
With this idea in mind, I would use an iterative approach to gathering more labeled data:

Fit a logistic regression model to the labeled data you have.
Determine where the probable status of $y$ is unclear.
Sample unlabeled cases in the region where you are likely to gain the most information and have your experts label them manually.
Rinse and repeat as necessary.

Using this strategy, you should be able to converge on a reasonable model efficiently.
Note that this method assumes you already have some labeled cases with decent coverage of the predictor space.  From your description, I gather this is true.  However, if it isn't, you need a step 0 to precede the above.  If you have no idea from prior research where the probability shifts from undiseased to diseased, you would want to sample on a grid to get an initial sense of the target location.


Strict inequality in Reverse Fatou lemma: $\varlimsup \int f_n\le \int \varlimsup f_n$

Question: Let $\{f_n\}$ be a sequence of nonnegative functions dominated by some function
$$
g \in L^1.
$$
Then, the reverse Fatou lemma says
$$
\limsup \int f_n \le \int \limsup f_n.
$$
Is it possible to give an example where the inequality is strict?
I tried functions like
$$
f_n=\chi_{(n,n+1)},
$$
but the dominating function is not integrble.
Answer: $$f_{2n}=\chi_{[0,1]}\qquad f_{2n+1}=\chi_{[1,2]}\qquad g=\chi_{[0,2]}$$
Comment: Can you explain how this gives the answer?  Don't we end up with $\int$ lim sup $f_n(x) = 0$?
Comment: @JaneDoe No we do not end up with what you say. What is $\limsup f_n$ according to you?
Comment: Sorry, maybe I misunderstood. Are we taking the integral over $\mathbb{R}$ or over $[0,2]$? If we are taking it over the reals, I think $\limsup f_n$ should be 0 because $f_n (x) = 0$ $\forall x > 2$. However, if it is over $[0,2]$ then I think $\limsup f_n$ should be 1.
Comment: @JaneDoe You seem to have in mind $$\limsup_{x\to\infty}f_n(x)$$ for some unspecified value of $n$, that is, a limsup which is irrelevant, instead of, for each given $x$, $$h(x)=\limsup_{n\to\infty}f_n(x)$$ which defines the function of interest $$h=\limsup_{n\to\infty}f_n$$
Answer: What about $f_n(x)=x(-1)^n  \chi_{[-1,1]}(x)+\chi_{[-1,1]}, ~g=2\chi_{[-1,1]}$?
Comment: `nonnegative functions`.
Comment: sorry, but adding $1$ should fix that


gcc compile time notes/msg

Question: I know MSVC can do this via a pragma message -> http://support.microsoft.com/kb/155196
Does gcc have a way to print user created warnings or messages? (I couldn't find a solution with google :( )
Answer: <code>#warning "This will only hurt a little bit."
#error "Or maybe a lot!"
</code>

gcc compile time notes/msg


Setup Nagios to monitor custom SQL query

Question: I have Nagios (and nagiosql installed to help me out) but I'm a bit of a beginner and not sure what to do so here is my question:
How do I setup monitoring of a specific SQL query in Oracle Database using Nagios?
Example:
Table in Oracle contains only 1 row with 2 columns:
RUN_STAT | LAST_EXEC_TIMESTAMP
I wish to setup Nagios to execute SQL query on this table and return difference between current time and LAST_EXEC_TIMESTAMP in seconds.
If result at the time of execution is more than n seconds return critical, otherwise return OK.
How do I do that?
I already did the SQL part but I'm not sure how to pass it to Nagios - which plugin to use and how to handle output value :/
Answer: There is a standard plugin called "check_oracle" that might help you, and there are also a handful of Oracle-specific plugins on Nagios Exchange.
At first glance, it doesn't look like the stock check_oracle plugin allows you to run specific queries, but I'm not 100% sure as I've never used it.
Consol Labs also maintains a collection of "check_(db)_health" plugins that are quite excellent, and they have an Oracle-specific one. You can use "--mode sql" to run your own queries.
To use any of these, you must have the Oracle client set up already, and the proper ENV vars set, to connect to the Oracle DB.
If you can't connect directly to the Oracle DB from your Nagios box, your other option is to use NRPE/check_by_ssh to run a plugin locally, on the Oracle DB server itself.


How to add icons/images as indicators rather than list circles in bootstrap3 carousel slider

Question: Hi i am trying to make  a bootstrap3 carousel slider and I want to add some images as the slide indicators instead of the list default circles.
Html Markup :

if i add an img tag as below :

Its not showing anything. How can I achieve adding an image as the indicator rather than the  circle?
Answer: I think the most important thing is to override the width and height  properties on the li tags inside the .carousel-indicators.
To do so I've created an additional modifier css class called .carousel-indicators--thumbnails that is added to the .carousel-indicators div. This modifier class is used to override the li, .active and .active .thumbnail class.

.carousel-indicators.carousel-indicators--thumbnails li {
  width: 80px;
  height: 40px;
  margin: 0;
  border: none;
  border-radius: 0;
}
.carousel-indicators.carousel-indicators--thumbnails .active {
  background-color: transparent;
}
.carousel-indicators.carousel-indicators--thumbnails .active .thumbnail {
  border-color: #337ab7;
}
<link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css" rel="stylesheet" />

              Slide 1

              Slide 2

              Slide 3

          Previous

          Next


Preventing a Ghostscript command from entering the Ghostscript command line interface after execution

Question: Whenever I execute a Ghostscript command, like the one below, in the Windows command line window, upon completion it automatically enters into the Ghostscript command line interface (GS >, as shown in the screenshot below). I wonder if there is a way to prevent ghostscript from doing it.

EDIT:
Realized that the dNoCancel flag should be removed. Once it's removed, it no longer enters the gs command line interface.
Answer: What happens is that the interpreter starts up and, if any actions have been specified on the command line those are executed as part of initialisation. This includes running any files. Then the interpreter returns to the interactive prompt awaiting further input.
There is plenty of documentation available for Ghostscript (possibly too much). If you go here then the switches relating to interaction with the user are described.


Python - can not install some packages in latest version

Question: I have installed 3.6.1 version of python interpreter. I have a project that need to use "PIL" library so I try to install it but it gives me error, "Error occurred when installing package 'PIL'". I search it out why it's happening and I think it can only be installed on lower version of interpreter 2.7 or something lower. Now how can I install PIL in 3.6.1 version of interpreter? Should I downgrade my interpreter instead? But if I do, can it cause some other packages to be unable to install for it is only for 3.X version. Thanks in advance.
Comment: [Pillow](http://python-pillow.org/) is a fork of PIL that works for python 3 and python 2
Comment: what is a fork? sorry do not really know. So I should use Pillow instead? Is it the same as PIL?
Comment: are tried in linux or windows? if ubuntu put your command you tried
Comment: Sorry I forgot. Im using windows OS. 7 specifically.
Comment: thats why asked... in window you need to download and install... check my answer it will be in detail
Comment: Install using `pip install pillow` and then in your script you can use `from PIL import xxxx`
Comment: Pillow works very similar to PIL and it can be installed in windows using pip `pip install pillow`
Comment: [Fork definition](https://en.wikipedia.org/wiki/Fork_(software_development))
Comment: for windows better download from unoffcial libraries site and install
Comment: Now my problem is "cv2". I can not download and install it directly from the compiler. I tried to install "opencv" then write "from opencv import cv2" but error in 'cv2' part. How can I import cv2?
Answer: Install Pillow, the packaged and maintained fork of PIL, using:
pip install pillow

Answer: download pillow based on your version.  from python unofficial library ..
search pillow choose your python version and distribution from it... once you download,
move onto download path or
pip install ~/Downloads/Pillow‑4.2.1‑cp36‑cp36m‑win_amd64.whl


populate checkbox value in text area

Question: I am a drupal newbie and am having the following problem:
In my custom content type page, I have multiple check boxes and a text area, among other fields. What I want is that, I would like the values of the checked check-boxes to be pasted inside the text area. The checkboxes are created as a multi-value field (field type = list, widget = check-box). Basically what I want is, every time the user checks/unchecks one of the checkboxes, the text area should re-populate with values of all the checkboxes that are currently selected.
I tried using the dependency fields module, but it did not solve my problem. Have anyone encountered this need before?
Based on what I read, it seems this might be possible using hook_node_presave, however I am not sure where to add this. I am using a template called norma, so I edited its template.php and added a function like so:
function norma_node_presave($node)
{
 print_r($node);
 exit;
}

However, nothing has happened... :(
Any pointers on how to solve this is very much appreciated...
Comment: Are the text fields hidden? Or do you want it to be "check a checkbox" -> add the value of checkbox -> set field value.
Comment: it is a regular text area. Suppose there are three checkboxes with values 1,2 and 3 and suppose 1 and 3 are selected. Then in my text area I want the value to be "1 3"
Answer: hook_node_presave($node) should be included in a custom module and not in your theme !
Look at Creating Drupal 7.x modules if you want to try so.
In your hook do something like that :
function norma_node_presave($node){

  $value = array();
  foreach( $node->field_checkbox['und'] as $checkbox){
    $value[] = $checkbox['value'];
  }

  $node->field_textfield['und'][0]['value'] = implode('
', $value);
}

Comment: That is just what I was looking for, thanks a million!! :)
Answer: The best idea would be to use Javascript (so that as you check the checkbox the field value changes), you can do something like this.
$(document).ready(function() {
    $('#checkbox1').change(function() {
        $('.field1').val($(this).val());
    });
});

Or something like that, I can't guarantee that code works but it should get you started.

populate checkbox value in text area


Combined Binomial Distribution Problem.

Question: I have the following problem:
70% of women respond positively to a test, while only 40% of men do so. If 10 participants are selected (5 women and 5 men), what is the probability that only 1 man responds negatively?
Now what I have done is to have two distributions, one for women and one for men. Then I calculated P(Y=5) for women with p=0.7 and P(Y=4) for men with p=0.4, and since the events are mutually exclusive I´ve added the two, having a result of ~24%. Can this be correct...?
Comment: This method isn't quite right, but even if it were, it shouldn't give 24%. What are you doing to calculate $P_W(Y=5)$ and $P_M(Y=4)$?
Comment: P(Y=5) = 5C5 * 0.7^5 * 0.3^0 and P(Y=4) = 5C4 * 0.4^4 * 0.6^1
Comment: Oh yeah, that's right. I'll type up an answer now.
Answer: You have the right idea, but did some wrong calculations.
The probability of 4 men responding positively and one man responding negatively is $_5C_4(0.4)^4(0.6)^1$.
The probability of 5 women responding positively is $_5C_5(0.7)^5(0.3)^0$.
The product of these two calculations is what you are looking for.
Answer: The events of $5$ women out of $5$ respond positively and $4$ men out of $5$ respond positively are not mutually exclusive.  In fact it is perhaps reasonable to assume that they are independent, and so you can multiply the probabilities.
Comment: It's just one experiment, are they not mutually exclusive...?
Comment: For a positive outcome, you require *both* that exactly five women respond positively *and* that exactly four men respond positively. I think you might be mixing up "mutually exclusive" and "independent" (the latter of which you can probably assume, as paw points out).
Comment: In that case, when you consider two experiments, selecting 5 women who respond positively and selecting 4 men who do so also, then the probability is ~1.2907%. And in that case they might be independent events.
Comment: That's right. See my answer if you are still unclear.
Answer: Almost!
You've correctly figured out how to calculate the probability of picking five women and four men answering positively: $$P_W(X=5)= {5 \choose 5} (0.7)^5 (0.3)^0 \approx .168$$ $$P_M(X=4)= {5 \choose 4} (0.4)^4 (0.6)^1 \approx .077$$
Your mistake was in adding the two. You require that both of these events occur. Since they are independent, you may simply multiply the probabilities of the events occurring individually to obtain this: $$.168 \times .077 \approx .013$$

To more clearly see why you cannot add the two, recalculate your answer in the case where women answer positively $100\%$ of the time. You'll get a likelihood of more than $100\%$!

Combined Binomial Distribution Problem.


How do I change status from "Pending" to "Completed"?

Question: Ubercart is install without any payment gateway. The workflow is to take user orders via out website and payments are handled offline. Once we received payment and shipped the product, we'd like to mark the order as Completed.
Here is the doc:

When a customer moves from the checkout form to the review order page, a new order is created with their information in the default
  status for the in_checkout state. (By default, this is "In checkout.")
When a customer completes checkout, the order gets moved from the default in_checkout status to the default status for the post_checkout
  state. (By default, this is "Pending.")
The rest happens manually by order administrators, but the other two default statuses are "Processing" and "Completed." Processing is
  actually just another status in the post_checkout state while
  Completed is the default status for the completed state.

However, I don't see a status dropdown list or similar UI element for changing an order's status when editing an existing order:

http://shop.example.com/admin/store/orders/100001/edit

How can I change the status from Pending to Completed?
Answer: Under the drupal 7version of Ubercart 3, order status would be changed using the dropdown found under
http://shop.example.com/admin/store/orders/100001/view
Answer: We update status in
Home » Administration » Store » Orders
or:
admin/store/orders/view
At that interface, we see four icons.  If you hover over them you will see:

View order <order#>
Print order <order#>
Edit order <order#>
Delete order <order#>

Click on View order <order#>
The URL should now read:
admin/store/orders/<order#>/view
At the bottom of that screen you should see a drop down labeled:
Order status
with Completed as the last choice
Select the status you want, indicate whether or not you want a copy (good for troubleshooting and records) and then click on Update.


How to add optional parameter in JPQL?

Question: How to use/add optional parameter in JPQL?
<code>@Query(value = "SELECT stud FROM Student stud where stud.name = :studentName AND stud.age IN :studentAgeList")
List<Student> getStudents(
            @Param("studentName ") String studentName,
            @Param("studentAgeList") List<Integer> studentAgeList
         )
</code>
How to make studentAgeList parameter in above query ?
I tried below :
<code>@Query(value = "SELECT stud FROM Student stud where stud.name = :studentName AND (:studentAgeList IS NULL OR stud.age IN :studentAgeList))
List<Student> getStudents(
            @Param("studentName ") String studentName,
            @Param("studentAgeList") List<Integer> studentAgeList
         )
</code>
But getting error : unexpected AST node:
Tried above but getting error
Comment: I've tried it and the only problem I get is when I set an empty list in `studentAgeList` parameter. If I set NULL or a non-empty list, it works correctly. Could you show your stack trace ?
Comment: org.springframework.dao.InvalidDataAccessResourceUsageException: unexpected AST node: {vector}
Comment: This exception occurs when the program starts during JPA initialization or when you call the method `getStudents()` ?
Comment: getting this  when calling the getStudent()
Comment: Precisely, with what value for `studentAgeList` ?
Comment: check https://stackoverflow.com/questions/69610579/unexpected-ast-node-vector-in-spring-data-jpa. I think you will find a solution using `coalesce(:studentAgeList) is null`
Answer: JPQL does not support optional parameters, you can use overload methods with different queries or criteria API or JPA specifications.
may be get answer here: Set optional parameters in JPQL Query
Comment: Indeed, there could be 2 query methods defined (with or without the student list) but the question was more about the possibility to build a where clause testing the parameter it self.

How to add optional parameter in JPQL?


Do additional date range filters increase the performance?

Question: To an existing, large ElasticSearch 5 index, I want to add a date field, containing the date of the indexation of each document. Afterwards I want to query this index, to return all documents, created in the last minute.
In the ElasticSearch Ultimative Guide for version 1 it is mentioned, that adding additional filters for day, month and/or year can improve the performance drastically. Newer versions of the guide do not say so anymore.
Can I gain performance in ElasticSearch 5 with adding additional date filters?
Answer: Depending on the size of your index, it certainly would help restrain the amount of data that gets considered for your query.
If your index contains years of data, adding a filter to only consider the documents indexed during the last minute will considerably decrease the amount of data that will be focused on for your query.
It was the case with ES 1.x and it is still the case with ES 5.x, the only difference is that in 1.x you had to take care of properly ordering your filters to use the most aggressive filters first, which is not the case anymore in 5.x as ES is now smart enough to run the most aggressive filters first to make sure to weed out unnecessary data very early during the search process.
Comment: If I filter by `date > 2017-07-18 23:58:13`, will then adding filters like `date > 2017` boost performance (due to better caching?)
Comment: Not sure how both dates relate together?
Comment: The one date would be the exact second, that I want to filter on (unlikely to be cached), the other one would be only the year part of what I'm searching for (not exact, but very likely to be cached).
Comment: ES used to index numbers (and dates) the same way as text, but in recent versions they started doing it differently using BKD tree data structures, which provides a much better performance. You can read about it [here](https://www.elastic.co/blog/apache-lucene-numeric-filters), [here](https://www.elastic.co/blog/numeric-and-date-ranges-in-elasticsearch-just-another-brick-in-the-wall) and [here](https://www.elastic.co/blog/searching-numb3rs-in-5.0). All this to say that you can trust ES that if an optimization can be made from your queries, it will make it.


How to set korean keyboard layout by hs.keycodes.setLayout()?

Question: I want to change keyboard layout by using Hammerspoon.
To change Thai keyboard layout, this code works.
> hs.keycodes.setLayout("Thai")
true

But to change Korean keyboard, both codes doesn't work.
> hs.keycodes.setLayout("2-Set Korean")
false

> hs.keycodes.setLayout("Korean")
false

hs.keycodes.currentLayout() returns "2-Set Korean".
> hs.keycodes.currentLayout()
2-Set Korean

Where can I find corret keyboard layout name for each language?
Answer: Instead of setLayout() you need to use setMethod() for Korean input method.
hs.keycodes.setMethod("2-Set Korean")


Programmatically saving data in a field under user account

Question: QR codes have been printed out for a bunch of students in a university.
When they visit the site via those QR codes, and they are logged-in, the refcode query parameter used in the link needs to be saved for their account in a field_usr_ref_code field (a Text (plain) field).
In a .module file I am using the following code.
use Drupal\user\Entity\User;

/**
 * Implements hook_entity_load().
 */
function my_custom_module_entity_load() {
  $refcode = \Drupal::request()->query->get('refcode');
  $user_id = \Drupal::currentUser()->id();
  if ($user_id > '0' && isset($refcode)) {
    $account = User::load($user_id);
    $account->set('field_usr_ref_code', $refcode);
    $account->save();
  }
}

However, for some reason no data is being saved, without any logged error.
What is wrong with the code I am using?
Comment: Drupal 8 is end of life. Each day the risk increases that the site will be hacked.
Comment: Quotes are for strings,  not for numbers/integers. `$currentuserid > '0'` is best to have as `$currentuserid > 0`. In addition, `else { // Do nothing }` if you are  really doing nothing, then delete the else code, as having an else is not mandatory for if statements.
Comment: However, despite having those 2 blemishes that I pointed out in my previous comment, your code should've worked. So your issue is else where. In the hook function name, did you rename the `my_custom_module` part to your actual custom module's name? Did you make sure your module is indeed enabled? You could also add `\Drupal::logger('my_custom_module')->notice('My HOOK works!');` in your hook code to check that your hook is needed being executed. Then go to `/admin/reports/dblog` and check if it got logged.
Comment: @NoSssweat I have updated my code based on your input; And yes the module is enabled, all caches are cleared and the module is renamed as it should be... Still not working
Comment: Is the module's meta file named my_custom_module.info.yml?
Comment: Yes, the module is already working and contains another function which is: `function my_custom_module_form_alter(&$form, FormStateInterface $form_state, $form_id) {}`
Comment: `hook_entity_load()` is the wrong hook to use for code that should be executed when a page is visited.
Comment: Truly, and it has the wrong signature.
Comment: @apaderno can you suggest what function should be use instead ?
Answer: Use a request event subscriber with a priority of 299:
/src/EventSubscriber/MymoduleSubscriber.php
<?php

namespace Drupal\mymodule\EventSubscriber;

use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpKernel\KernelEvents;
use Symfony\Component\HttpKernel\Event\RequestEvent;

/**
 * Mymodule event subscriber.
 */
class MymoduleSubscriber implements EventSubscriberInterface {

  /**
   * Kernel request event handler.
   *
   * @param \Symfony\Component\HttpKernel\Event\RequestEvent $event
   *   The request event.
   */
  public function onKernelRequest(RequestEvent $event) {
    if (!$event->isMainRequest()) {
      return;
    }
    // put here your code
  }

  /**
   * {@inheritdoc}
   */
  public static function getSubscribedEvents() {
    return [
      KernelEvents::REQUEST => ['onKernelRequest', 299],
    ];
  }
}

mymodule.services.yml
services:
  mymodule.event_subscriber:
    class: Drupal\mymodule\EventSubscriber\MymoduleSubscriber
    tags:
      - { name: event_subscriber }

Drupal runs authentication with priority 300.
This code is for Drupal 9/10.
For Drupal 8 you need to use Symfony\Component\HttpKernel\Event\GetResponseEvent and $event->isMasterRequest().
Comment: For Drupal 8 you need to use Drupal 9


Error in connecting with pyspark to SQL Server

Question: I am trying to connect to a SQL Server from local with pyspark. I've downloaded the last driver version, but when running the code below this error shows:

Error: "The server selected protocol version TLS10 is not accepted by client preferences [TLS12]"

The code is as following (Spark session was initiated before and the driver is located in a supported path):
jdbcDF = spark.read \
   .format("jdbc") \
   .option("url","jdbc:sqlserver://<host>\\<port>;database=<database>;") \
   .option("dbtable",
) \
   .option("user", <username>) \
   .option("password", <password>) \
   .option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \
   .load()

I read that one possible solution would be to change some configuration from the SQL Server, but this is not possible for me. Is there any way of fixing it without changing anything in the SQL Server?
Comment: Why are you using TLS1.0?
Comment: I guess I'm using it by default, is it possible to change it?
Comment: What is the version of Windows that SQL Server is running on?
Comment: I think that the Windows version is Windows 8.1.
Answer: I think your question is similar to this one
From the above link:
TLS10 is disabled in Java 1.8 and above. You can enable it by remove TLSv1 from jdk.tls.disabledAlgorithms in the java.security file (This edit is done in your client machine, not SQL Server).
This worked in my case.


Result of parsing date is wrong by 16 hours 2 minutes

Question: I have a string 07/Dec/2016:07:38:59 1000.
I want to parse it to a date.
What do I do to become a date 07/Dec/2016 07:38:59 1000 as a result?
Now it prints out Wed Dec 07 23:40:59 CET 2016.
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class TimeParse {
    public static void main(String[] args) throws ParseException {
        String time ="07/Dec/2016:07:38:59 1000";
        SimpleDateFormat format = new SimpleDateFormat("dd/MMM/yyyy:hh:mm:ss");

        String dateString = format.format( new Date()   );
        Date   date       = format.parse ( "07/Dec/2016:07:38:59 1000" );

        System.out.println(date);

    }
}

Comment: Well you've got an extra ` 1000` at the end of your string. What's that meant to be for? You're also using 12-hour parsing (`hh`) when I strongly suspect you want 24 hour parsing (`HH`).
Comment: what is `tet` means ?
Comment: the extra 1000  is in the String.
Comment: what does the 1000 mean? millisecond timezone, what exactly ?
Comment: On my computer your program prints `Wed Dec 07 07:38:59 CET 2016`, which seems to be the result you expected. I have no idea why the result is 16 hours 2 minutes wrong on your computer. Sorry.
Comment: Can you reboot your computer and get the same result again? I cannot explain how it would help, but it's my best suggestion for something to try.
Comment: The other — and maybe really better — suggestion is you switch to using the Java 8 date and time classes. They are in the `java.time` package and are generally much more convenient to work with than `Date` and `SimpleDateFormat` and seldom give surprises.
Comment: My guess it the 1000 is a time zone offset indicator, +10:00. Just a guess, though. If the asker doesn't know, nobody here knows.
Answer: I'm assuming the 1000 is supposed to be milliseconds? if so, it's incorrect. it's like writing 13:60:60
This also means the time string format you entered is missing the milliseconds value.
It should be "dd/MMM/yyyy:hh:mm:ss SSS"
milliseconds value should not be more than 3 digits


Formula Workshop in Crystal Reports: Why does this not work in the formula workshop: {report_field1} or {report_field2}?

Question: In my report, {report_field1} shows up on a regular basis but when I change it to {report_field2}, it shows the values for it in the report but the places where {report_field1} should show it's values, it turns to 0.
So I am trying to make the report show that if {report_field1} values are 0 from the user, then it should show the {report_field2} values.
I tried to do it with an OR, AND statement but that does not work, is there any way to do what I am trying to do?
If there is no user input value for {report_field1} then it should print the user value {report_field2} and vice-versa
I am very new to Crystal Reports itself so I need some help with this.
Answer: Did you try to create and use a formula like this:
<code>IF {report_field1} = 0 THEN {report_field2} ELSE {report_field1}
</code>


Able to use custom variables in page.html.twig but not in menu.html.twig

Question: I have a module setting config variables which I'm trying to display in a twig template. When I use
function mytheme_preprocess_page(&$variables){
    $variables['facebook'] = \Drupal::config('social.settings')->get('facebook', 'No username');
    $variables['twitter'] = \Drupal::config('social.settings')->get('twitter', 'No username');
    $variables['instagram'] = \Drupal::config('social.settings')->get('instagram', 'No username');
    $variables['youtube'] = \Drupal::config('social.settings')->get('youtube', 'No username');
}

I'm able to display the variables in my page.html.twig template by invoking
{{ facebook }}{{ twitter }}{{ instagram }}{{ youtube }}

However I'd like menu.html.twig to invoke it. Even when adding a preprocess hook for menu like so
function mytheme_preprocess_menu(&$variables){
    $variables['facebook'] = \Drupal::config('social.settings')->get('facebook', 'No username');
    $variables['twitter'] = \Drupal::config('social.settings')->get('twitter', 'No username');
    $variables['instagram'] = \Drupal::config('social.settings')->get('instagram', 'No username');
    $variables['youtube'] = \Drupal::config('social.settings')->get('youtube', 'No username');
}

menu.html.twig produces nothing when invoking
{{ facebook }}{{ twitter }}{{ instagram }}{{ youtube }}

I know menu.html.twig is being used because I can make changes to it and the changes will appear. I also know that mytheme_preprocess_menu is being called because drupal_set_message works from it.
However {{ dump() }} from menu.html.twig is not displaying anything on the page.
Comment: This should work. Have you checked that both the preprocess function and the custom template are used?
Comment: If you don't use a debugger tool then the die function can be used to make sure the preprocess function is used, and you can insert some reason text in the template to make sure it is used.
Comment: P.S. you should pass the `\Drupal::config(...)` result to a variable.
Comment: I know that the function is being called because I added drupal_set_message to it and it's able to display the variables just fine.
Comment: Did you check that the template file is used?
Comment: Tes. I added random text to menu.html.twig and it shows up just fine
Comment: Also {{ dump() }} from menu.html.twig does not show anything
Comment: See my answer below.
Answer: The menu theme is declared using the variables key (see doc).
That means that only the keys present in the variables array can be passed to the template.
Any custom variable needs to be first added using hook_theme_registery_alter
Comment: I'm unable to find any reference to hook_theme_alter. Do you mean hook_theme_registry_alter()?
Comment: Yes, my mistake.
Comment: I'm not having any luck. Do I add the variables to hook_theme_registry_alter in the same way as I did in mytheme_preprocess_menu? Or is there another technique?
Comment: or does the menu not have access to the variables array from twig? I did check in the synchronization settings and the variables do appear there. Also after performing dump() on menu.html.twig only menu variables appear to be available.


How can I exclude certain routes from maintenance mode?

Question: I am trying to make certain routes exempt from maintenance mode by adding the option _maintenance_access to the route and setting it to TRUE.
I think (although I'm not 100%) that this works because in Drupal\Core\Site\MaintenanceMode there is the following method which checks for the _maintenance_access option and returns false if it is present -
public function applies(RouteMatchInterface $route_match) {
  if (!$this->state->get('system.maintenance_mode')) {
    return FALSE;
  }

  if ($route = $route_match->getRouteObject()) {
    if ($route->getOption('_maintenance_access')) {
      return FALSE;
    }
  }

  return TRUE;
}

I have followed this guide on drupal.org on how to alter existing routes
Here is my mymodule.services.yml file
services:
  mymodule.route_subscriber:
    class: Drupal\mymodule\Routing\MaintenanceModeRouteSubscriber
    tags:
      - { name: event_subscriber }

Here is my MaintenanceModeRouteSubscriber class
<?php

namespace Drupal\mymodule\Routing;

use Drupal\Core\Routing\RouteSubscriberBase;
use Symfony\Component\Routing\RouteCollection;

class MaintenanceModeRouteSubscriber extends RouteSubscriberBase {

  /**
   * {@inheritdoc}
   */
  protected function alterRoutes(RouteCollection $collection) {
    if ($route = $collection->get('entity.node.canonical')) {
      $route->addOptions(['_maintenance_access' => TRUE]);
    }
  }
}

But this is not working as when viewing this route (any node) it is still in maintenance mode.
Comment: Your code works fine for me. Do you see any errors when clearing the cache with `drush cr`? Try to print debug messages which you then see from the command line like `var_dump($route);`.
Comment: I'm not sure what is going on (no errors), what happens when you try to view a node when not logged in and when maintenance mode is on? Does it allow you to view it?
Comment: Yes, I can view nodes, other pages return the maintenance message. Did you try to print a debug message from alterRoutes() to see if it is working when clearing the cache?
Answer: You need to override the service directly.
mymodule.services.yml
services:
  maintenance_mode: # same ID as declared in core.services.yml
    class: Drupal\mymodule\Services\CustomMaintenanceMode # point to your class
    arguments: ['@state']

mymodule/src/Services/CustomMaintenaceMode.php
<?php

namespace Drupal\mymodule\Services;

use Drupal\Core\Session\AccountInterface;
use Drupal\Core\Site\MaintenanceMode;

class CustomMaintenanceMode extends MaintenanceMode {

  /**
   * {@inheritdoc}
   */
  public function exempt(AccountInterface $account) {
    $node = \Drupal::routeMatch()->getParameter('node');
    if ($node instanceof \Drupal\node\NodeInterface && $node->getType() == 'article') {
      return TRUE;
    }
    else {
      return $account->hasPermission('access site in maintenance mode');
    }
  }

}

Comment: Thank you, would overiding the service directly allow me to, say for instance, allow one node type and disallow another from the maintenance mode access?
Comment: Yes, but I'm babysitting right now, I'll show ya later.
Comment: @hck12685 after testing this out, turns out you need to override the `exempt()` method as overriding the `applies()` method to TRUE didn't do it.
Comment: Yes I figured that out, thanks. It does seem to be working now as the maintenance mode is gone, but now I am left with a "The requested page could not be found. " message. So some more debugging required. If it is anything related to this question I will post the solution back here.
Comment: Probably due to some contrib/custom module introducing that, because I'm not getting that on a fresh D8 install.
Comment: I assume you don't have that problem when logged in as admin and maintenance is on.
Comment: Yes that is correct, no problem when logged in as admin. I will try test on a new install. Thanks
Comment: Needing to use the "same ID as declared in core.services.yml" is in direct contradiction with the [documentation](https://www.drupal.org/docs/drupal-apis/routing-system/altering-existing-routes-and-adding-new-routes-based-on-dynamic-ones#s-altering-existing-routes) which claims it works the way the question's code is written.  Any insights?  Is the documentation incorrect?
Comment: @phils the documentation is correct. This just an alternate way of overriding a service which by some may not be considered best practice.


How do I fix this Java Loop Scanner

Question: This is part of a code that I am running and when the code runs it lets me enter a number but will not let me move on, it will just continue to print "Enter a student's total quiz score : " even though i have entered a value less than the perfect quiz score help!
  while (perfectQuizScore > quizScore ) {
        System.out.print("Enter a student's total quiz score : ");
        quizScore = Integer.parseInt(keyboard.nextLine()); // 6
        if (quizScore > perfectQuizScore) {
            System.out.println("Invalid value! Score must not be greater than perfect score.");
        }
        else if (quizScore < 0) {
            System.out.println("Invalid value! Value must be greater than 0.");
        }

    }

OUTPUT:
enter image description here
REVERSING THE SIGNS
enter image description here
Answer: If I'm understanding your question correctly... you want to exit the loop when the supplied quiz score is less than (or equal to) the perfect score? If so, you just need to change the sign in your while loop conditional (it is currently checking greater than).
Before:
  while (perfectQuizScore > quizScore ) {
After:
  while (perfectQuizScore < quizScore ) {
Now this is assuming that you initially set quizScore to a huge value (if you initialize it to 0 then the conditional will evaluate to false before entering the loop). A better solution is probably to use a do-while loop.
int quizScore = 0;
do {
   System.out.print("Enter a student's total quiz score : ");
   quizScore = Integer.parseInt(keyboard.nextLine()); // 6
   if (quizScore > perfectQuizScore) {
      System.out.println("Invalid value! Score must not be greater than perfect score.");
   }
   else if (quizScore < 0) {
      System.out.println("Invalid value! Value must be greater than 0.");
   }
} while (perfectQuizScore < quizScore or quizScore < 0);

Although I would personally probably structure this differently, to avoid writing your "bad" conditions down twice:
int quizScore = 0;
while (True) {
   System.out.print("Enter a student's total quiz score : ");
   quizScore = Integer.parseInt(keyboard.nextLine()); // 6
   if (quizScore > perfectQuizScore) {
      System.out.println("Invalid value! Score must not be greater than perfect score.");
   }
   else if (quizScore < 0) {
      System.out.println("Invalid value! Value must be greater than 0.");
   }
   else {
      // good input, exit the loop with a break statement
      break;
   }
}

Comment: here is the whole program https://pastebin.com/4fB61gdK
Comment: How do I convert it to while? cause i have to do while and do while
Comment: I've edited to give examples of both "while" and "do-while"
Answer: Maybe the following is what you searching:
    do {
      System.out.print("Enter a student's total quiz score : ");
      quizScore = scnr.nextInt();
      if (quizScore > perfectQuizScore) {
          System.out.println("Invalid value! Score must not be greater than perfect score.");
      }
      else if (quizScore <= 0) {
          System.out.println("Invalid value! Value must be greater than 0.");
      } else {
          break; // get out of the loop
      }
    } while ( quizScore <= 0 || quizScore > perfectQuizScore );

Do the check (while) after getting the first time, also change the checks to stay on the loop until the user write a valid value.
The do {...} will let the code inside run one time before the while checks.
Comment: THIS WORKS WHEN CONVERTING TO WHILE while ( quizScore <= 0 || quizScore > perfectQuizScore );

How do I fix this Java Loop Scanner


Enviar Input de texto em um FormData (angular)

Question: Estou com um formulário de upload de uma imagem, mas também quero passa um campo de texto junto a imagem, a api esta funcionando perfeitamente e quando faço o envio pelo Insomnia ele cria tudo direitinho no banco, mas ao fazer as requisições no frontEnd ele só envia a imagem.
exemplo da service que estou utilizando
 importarArquivo(file:File, links: string ): Observable<HttpEvent<any>> {
    // instanciando formData para utilizar no req
    const formData: FormData = new FormData();

    formData.append('imagens', file);
     formData.append('links', links);

    const req = new HttpRequest('POST',`${environment.apiUrl}/banners`, formData );

    return this.http.request(req);
}

Answer: Primeiramente tente converter o arquivo para Base4:
static converterArquivoParaBase64(file: File): Promise<string | ArrayBuffer> {
    return new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.readAsDataURL(file);
        reader.onload = () => resolve(reader.result);
        reader.onerror = error => reject(error);
    });
}

Após isto, na sua requisição, envie um json contendo o arquivo + propriedades que deseja:
salvar(link: string, base64File: string): Observable<HttpEvent<any>> {
   let parametros = {
            "imagem": base64File,
            "link": "wwww.imagem.com"
   }
   const req = new HttpRequest('POST',`${environment.apiUrl}/banners`, parametros);
   return this.http.request(req);
}

Obviamente seu backend deverá tratar o arquivo base64 em conformidade com seu objetivo. Convertendo de volta ao arquivo original, salvando na base de dados ou qualquer outra intenção.


What is the .dockerfile extension?

Question: Visual Studio Code (1.22.2) offers a file extension named .dockerfile in the the save dialog. What is a file with this extension? A Dockerfile is in all documentation and examples, that I've seen so far, only called Dockerfile.

If I enter Dockerfile as a file name, a file named Dockerfile.dockerfile is created.
Comment: Have you tried it to see if the file actually gets a `.dockerfile` extension? Perhaps it's just an UI thing (e.g. the part inside the parentheses couldn't be left empty).?
Comment: Yes, I've tried it. If I enter `Dockerfile` as a file name, a file named `Dockerfile.dockerfile` is created. I'll add that to the question.
Comment: Windows operates on files with extensions so my guess is they introduced such extension by themselves. Docker implies the `Dockerfile` name, but you can name the file as you want, as long as you provide it's name to the related docker command.
Answer: It appears that "*.dockerfile" is simply an alternative to the conventional "Dockerfile" name. This is perhaps useful if you want to keep a collection of dockerfiles in the same directory. Note the -f/--file option in docker help build:

-f, --file string             Name of the Dockerfile (Default is 'PATH/Dockerfile')

In other words, you are not required to use the name "Dockerfile", and the VSCode extension will correctly syntax-highlight any file ending in ".dockerfile".
Comment: This is the correct answer. If you have multiple dockerfiles in a single directory, you can give those abitrary names with the `.dockerfile` extension, and then build those by running `docker build . -f "filename.dockerfile"`
Answer: Dockerfile
A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession. Docker images are the basis of containers. An Image is an ordered collection of root filesystem changes and the corresponding execution parameters for use within a container runtime. An image typically contains a union of layered filesystems stacked on top of each other. An image does not have state and it never changes.More on
Dockerfile extension
A Dockerfile has no extension . if your using docker on docker on windows use notepad ++ to create a dockerfile while saving select "All type " and save the file name as "Dockerfile".
Mongodb/Dockerfile
Comment: This doesn't answer the question. As you can see, there _is_ a `.dockerfile` extension, even if docker files usually don't have one.
Comment: This answer reads like a mix of some statements not answering the question and some statements advertising docker.
Comment: Docker can deal with files simply named `Dockerfile`, not requiring an extension. However, let's say you have a directory with 2 or more different dockerfiles, you can give those an arbitrary name with the `.dockerfile` extension, and then build those by running `docker build . -f "filename.dockerfile"`. Your answer is therefore incomplete at best.
Answer: Using the .dockerfile extension tells VSCode that the file is a DockerFile for code highlighting and linting
Comment: VSCode does code highlighting and linting for files called `Dockerfile` as well.
Answer: What worked for me was to save the file in VS Code as a Dockerfile.  But, you need to remove the .dockerfile extension that VS Code puts on it before running the $docker-compose up command:
Answer: Even though VSCode can deal with extensionless files just fine, some major parts of the Windows operating system can't. Try double clicking a Dockerfile (without extension) in the Windows Explorer. You will always be asked which program you want to open it in because Windows can't map extensionless files to a default program.
My guess is that because of this problem, Microsoft would like for all files to have an extension and uses VSCode to nudge people towards using a file extension for Dockerfiles, ignoring the fact that this contradicts the de facto standard.
Comment: Double click is for running executable programs. What do you expect to happen when you double click on it? Build the image? Run the image? Or just open a text editor?
Comment: @OneCricketeer You can easily configure Windows to open any file (with a file extension) with any program on a double click. You cannot do that with a file which has no file extension. I consider your first sentence to be incorrect/incomplete. As to your question: I expect the default program for a Dockerfile to be a text editor.
Answer: Dockerfile doesn't have any extensions.
As you can see from hte documentation, https://docs.docker.com/compose/gettingstarted/, it doesn't have any extensions.

What is the .dockerfile extension?


Promises em NodeJS

Question: Estou aprendendo Node e apanhando um pouco da programação assíncrona.
Pra aprender, eu peguei o seguinte exemplo

console.log("1");
setTimeout(function(){console.log("2");},3000);
console.log("3");
setTimeout(function(){console.log("4");},1000);

A saída que eu queria, é que fosse 1, 2, 3, e 4. Mas sai 1, 3, 4, 2 por conta da programação assíncrona. Pesquisei na internet e achei alguns lugares citando promises, mas os exemplos que eu achei de promise são um pouco complexo e não entendi bem. Creio que não deve ser difícil resolver esse problema, porém não achei algo muito bem explicado.
Se alguém puder me mostrar um código pra resolver isso eu agradeceria, obrigado
Answer: Tenho uns exemplos prontos que encadeiam dois timers, espero que seja útil. O primeiro usa Promises, esperando um tempo, retornando um resultado aleatório e simulando falhas para que se veja o que acontece quando a promessa não é "cumprida":
"use strict";

function op1()
{
        return new Promise((fulfill, reject) => {
                setTimeout(() => {
                        let n = Math.random();
                        if (n >= 0.75) {
                                reject("op1 failed");
                        } else {
                                fulfill(n);
                        }
                }, 1000);
        });
}

function op2(x)
{
        return new Promise((fulfill, reject) => {
                setTimeout(() => {
                        let n = Math.random();
                        if (n <= 0.25) {
                                reject("op2. failed");
                        } else {
                                fulfill(Math.floor(x / 0.75 * 1000));
                        }
                }, 1000);
        });
}

console.log("Start");

op1()
.then(op2)
.then((res) => {
        console.log("Sucess: " + res);
})
.catch((err) => {
        console.log("Error: " + err);
});

O final do código pode ser substituído pela sintaxe async/await. Infelizmente, para esperar (await) pelo resultado de uma função assíncrona, também é preciso que await esteja dentro de uma função async, por isso a função anônima:
(async () => {
        try {
                let x = await op1();
                let res = await op2(x);
                console.log("Sucess: " + res)
        } catch (err) {
                console.log("Error: " + err)
        }
})();

Comment: @MatheusRocha O *fulfill* utilizado no exemplo é o que mais normalmente é chamado de `resolve` no comum dos códigos com Promisses
Comment: Amigo, desculpe encher o saco, mas nesse caso ele executa a op1, que pega um valor random, se for maior ou igual que 0.75 ele dá erro, se não ele da um fullfill(n). O que seria esse fullfill? Obrigado!
Comment: "Fulfill" quer dizer "cumprir", no sentido de "cumprir a promessa". Na verdade é um callback passado como parâmetro passado pela Promise e a variável poderia ter qualquer outro nome, tipo "ok", "bom", "cumprir", o que você desejar.
Answer: O problema começa no facto de o 2 e o 4 serem escritos assincronamente, logo se deixar tudo rolar normalmente serão escritos no fim.
Começo por dizer que esse exemplo ainda é fazível sem Promises, mas não é uma solução escalável:

console.log("1");
setTimeout(() => {
    console.log("2");
    console.log("3");
    setTimeout(() => console.log("4"),1000);
},1500);

Exemplo simples
Antes de começar com Promises tem de perceber para que servem e como se utilizam em cenários simples mesmo. Imagine que quer mostrar um numero na consola utilizando uma Promise e um setTimeout para que a escrita seja temporizada.
Nesse caso poderia fazer assim:

const timer = new Promise((resolve, reject) => {
  setTimeout(() => resolve(10), 1000);
});

timer.then(valor => console.log(valor));

Repare que a Promise recebe dois callbacks um de sucesso normalmente chamado de resolve e um de erro normalmente chamado de reject. Quando faz resolve dentro da Promise está a executar a função recebida passando o valor de sucesso. Depois chamamos o .then passando a função de sucesso que é:
valor => console.log(valor)

Que faz log do valor recebido. Optei por escrever com Arrow Function para ser mais simples e compacto, mas também pode ser escrito com função normal:
timer.then(function(valor){
    console.log(valor);
});

O mesmo se aplica ao resto das funções no exemplo. Veja o mesmo exemplo totalmente sem Arrow Functions:

const timer = new Promise(function(resolve, reject){
  setTimeout(function(){
    resolve(10)
  }, 1000);
});

timer.then(function(valor){
  console.log(valor)
});

Encadeando Promisses
Um dos objetivos principais das Promises é conseguir encadear as várias chamadas com then evitando o aninhamento. O aninhamento seria maior consoante a quantidade de coisas encadeadas que pretende fazer. Se algum erro surgir em algum dos promises no encadeamento ele será apanhado pelo catch.
Veja o primeiro exemplo rescrito com Promises:

const timer2 = new Promise((resolve, reject) => {
  setTimeout(() => resolve("2"),1500);
});

console.log("1");
timer2.then(valor => {
  console.log(valor);
  console.log("3");
  return new Promise((resolve, reject) => {
    setTimeout(() => resolve("4"),1000);
  });
}).
then(valor => console.log(valor));

Async e Await
Apenas como nota final, deixo aqui a dica que async e await são bem mais fáceis de trabalhar, ficando num estilo mais perto do convencional quando não se trabalha com funções assincronas. É importante de perceber primeiro como funcionam as Promises pois esta alternativa é baseada nelas.
Não irei entrar em detalhes pois o @epx já deu uma ideia de seguir por este caminho.
Comment: Nossa, muito obrigado pela sua resposta, Isac, foi muito esclarecedora, simples e objetivo. Muito obriagdo de verdade!
Comment: @MatheusRocha Sem problema, estamos ca para ajudar

Promises em NodeJS


Why isn't this parser for tracking parentheses working?

Question: I was wondering why the parser portion(separated out) does not work. It is printing everything as if I did not add the parser. In my original code(not parser indicated), it converts (a)n or (a){n} to \pochhammer{a}{n} but it does not do this for patterns with nested parentheses inside the a. The parser is supposed to ignore the nested parentheses in something like "(a(b))n or (a(b)){n} and print out \pochhammer{a(b)}{n}. Thanks.
while (scanner.hasNextLine()) {
        String line = scanner.nextLine();
        Pattern cpochhammer = Pattern.compile("\\(([^)]+)\\)_\\{([^}]+)\\}");
        Matcher pochhammer = cpochhammer.matcher(line);
        StringBuffer rplcmntBfr = new StringBuffer();

parser:
        char toCheck = '(';
        char toChecko = ')';
        int count = 0;
        for (int i = 0; i < line.length(); i++) {
            if (line.charAt(i) == toCheck) {
                count++;
            }
            if (line.charAt(i) == toChecko) {
                count--;
            }
            if (count == 0 && line.indexOf(")_") == i) {
                Pattern dnestedParen = Pattern.compile("\\((.+)\\)_\\{([^}]+)\\}");
                Matcher dmnestedParen = dnestedParen.matcher(rplcmntBfr.toString());
                if (dmnestedParen.find())
                    dmnestedParen.appendReplacement(rplcmntBfr,
                            "\\\\pochhammer{$1}{$2}");
            }
        }

end parser
while (pochhammer.find()) {
            pochhammer.appendReplacement(rplcmntBfr,
                    "\\\\pochhammer{$1}{$2}");
        }
        pochhammer.appendTail(rplcmntBfr);
        Pattern npochhammer = Pattern.compile("\\(([^)]+)\\)_(.)");
        Matcher ppochhammer = npochhammer.matcher(rplcmntBfr.toString());
        rplcmntBfr.setLength(0);

parser:
        count = 0;
        for (int i = 0; i < line.length(); i++) {
            if (line.charAt(i) == toCheck) {
                count++;
            }
            if (line.charAt(i) == toChecko) {
                count--;
            }
            if (count == 0 && line.indexOf(")_") == i) {
                Pattern nestedParen = Pattern.compile("\\((.+)\\)_(.)");
                Matcher mnestedParen = nestedParen.matcher(rplcmntBfr.toString());
                if(mnestedParen.find())
                    mnestedParen.appendReplacement(rplcmntBfr,
                        "\\\\pochhammer{$1}{$2}");
            }
        }

end parser
        while (ppochhammer.find()) {
            ppochhammer.appendReplacement(rplcmntBfr,
                    "\\\\pochhammer{$1}{$2}");
        }
        ppochhammer.appendTail(rplcmntBfr);
        writer.println(rplcmntBfr);
    }

Comment: What does your debugger show?
Comment: Detecting nested parens with regex is very difficult unless using an extended regex library that will allow for recursion or counting of pairs.  Perl's library is capable, not certain if Java's can handle that.
Comment: Any particular reason for this  if (count == 0 && line.indexOf(")_") == i) instead of    if (count == 0 && lne.charAt(i)==toChecko&& line.charAt(i+1) == "_" )??  As it is written it scans then entire line repeatedly on every run of the loop.  If you really want to use indexOf do it before the loop and store the index.  This also assumes only one pair per line as written.  No idea what you are parsing though.
Comment: Third time now. If you are NOT validating the text and are trying to **ignore** nested paren's/braces, is simply _parens_ - _not parens_ - _parens_ `\([^()]+\)` Whats wrong?
Answer: Because it isn't a parser. It's a scanner, using regular expressions, and regular expressions cannot count anything, let alone parentheses. To parse nested syntax of any kind you need an explicit or implicit stack, and that takes you out of the land of regular expressions altogether into the world of parsers. Have a look at the Dijkstra Shunting-yard algorithm, or recursive descent expression parsing.


Taxonomy Sidebar view with Nodes

Question: I'm building a sidebar for my taxonomy and node pages.
I've got it as

Show (View Setup): Taxonomy Terms
Unformatted List
Show: Fields
Relationship: Taxonomy term: Representative node

Fields:

Taxonomy term: Name
(Representative node) Content: Title

This works pretty well, each row starts with the taxonomy term name as a link and is followed by a node from that term. The trouble is, it'll only show the first node. How do I get it to display all nodes?
Answer: The Representative node relationship will show you a single representative node for your term, so that's not what you need in this case. Instead, try this:
Relationship: Taxonomy term: Content with term
Your results will now look like this:
Term A
Node 1
Term A
Node 2
Term B
Node 3 etc
Assuming this is not what you want, you can next add some grouping like this:
Under Format: settings set Grouping field Nr. 1 to Taxonomy term: Name.
Finally, in Fields, set the Taxonomy term: Name field to Exclude from display. Now you should have a list of node titles grouped by taxonomy term.

Taxonomy Sidebar view with Nodes


@Autowired bean is coming as null at static method

Question: i have two class name with Bean class and Util class. At Util class i want to call the method from Bean Class. But the thing is when i @Autowired the Bean class at Util class. There is null pointer exception because of static method at Util class. Here is my code.
public class Util{

@Autowired
private static BaseServiceCommonPropertiesBean baseServiceCommonPropertiesBean;

private static String PASSPHRASE = baseServiceCommonPropertiesBean.getBassURL();
System.out.print(PASSPHRASE);

}

Here is for BaseServiceCommonPropertiesBean class
public class BaseServiceCommonPropertiesBean {

@Value("#{baseServiceapplicationProperties['mobi.sc.travellers.amr.email.base.url']}")
    private String baseUrl;

public String getBaseUrl(){
        return baseUrl;
    }

}

Whenever the system read the baseServiceCommonPropertiesBean.getPassPhrase() method. It goes out and stop working . I tried @Postconstruct annotation before it does not work. Thanks.
Comment: Yes, you cannot use @Autowired on static fields.   What do you mean "@Postconstruct annotation before it does not work" ?   (it most certainly does behave according to it's documentation) - can you edit the question to show us your attempt?   More to the point, WHY do you want PASSPHRASE set statically - ie, why is a `getPassphrase()` method on the instance is not sufficient?
Comment: One of the benefits of spring is not having to use static fields.
Comment: Either getPassphrase() or all of the get method from BaseServiceCommonPropertiesBean cannot used.  and for PostConstruct i used below code before it does not work also. still come as null pointer exception. Thanks.

`
@Component
public class Util{

  private static BaseServiceCommonPropertiesBean baseServiceCommonPropertiesBean;

  @Autowired
  private BaseServiceCommonPropertiesBean bean;

  @PostConstruct
  private void initStaticBean () {
     baseServiceCommonPropertiesBean = this.bean;
  }

}
`
Comment: Don't use `static`.
Comment: Possible duplicate of  https://stackoverflow.com/questions/19946852/calling-non-static-method-from-static-class-in-java-with-spring-autowiring
Answer: You cannot @Autowired static fields, either remove static from BaseServiceCommonPropertiesBean or rewrite your Util to be something like below:
@Component
public class Util{

private static BaseServiceCommonPropertiesBean baseServiceCommonPropertiesBean;

@Autowired
    public void setBaseServiceCommonPropertiesBean(BaseServiceCommonPropertiesBean baseServiceCommonPropertiesBean){
        Util.baseServiceCommonPropertiesBean = baseServiceCommonPropertiesBean;
    }

private static String PASSPHRASE = baseServiceCommonPropertiesBean.getBassURL();
System.out.print(PASSPHRASE);

}

Comment: Thanks for the help. I tried the way you mention. But still having error and nothing coming out @3_eyed_raven.
Comment: how exactly should it work, if Util is not scanned for creation and autowiring by spring?  To my knowledge you cannot simply put @Autowired and call it a day
Comment: @KaungKhantZaw - I haven't tried it myself, the answer is a general guidance, try adding @ Component to Util class. Also, why not use non-static Util?


How does the product of two polynomials from different rings of polynomials make sense?

Question: I have asked a question here and gave a comment regarding an answer by Captain Lama. He seems to be busy or not interested in my question any more that he has not responded to my comment, so I have no choice but to post another related question in this thread.
I think it is not good to copy relevant concept/information (which I typed carefully) from that question to this one. The copying will make this thread unnecessarily long. If you are interested in answering this question, please take a look at the old one.

My textbook states that

$p=\sum_{\alpha} p_{\alpha} X^{\alpha} \in  K\left[X_{1}, \ldots, X_{m}\right]$ can be written in the form $p=\sum_{j=0}^{n} q_{j} X_{m}^{j}$ for suitable $n \in \mathbb{N}$ and $q_{j} \in K\left[X_{1}, \dots, X_{m-1}\right]$.

From Captain Lama's answer here, I got that $X^j_m \in K[X_1,\dots,X_m]$ and $q_j \in K[X_1,\cdots,X_{m-1}]$.
My question: $K[X_1,\dots,X_m] \neq K[X_1,\cdots,X_{m-1}]$ and thus they are two different rings of polynomials. How can the product $q_{j} X_{m}^{j}$ make sense?
Thank you for your help!
Comment: One of the rings is a subring of the other.
Comment: @RolfHoyer In my textbook, *a formal power series in $m$ indeterminates over $K$* is a function from $\Bbb N^m$ to $K$. Please check if I understand your ideas correctly. By *One of the rings is a subring of the other*, you meant that $K[X_1,\dots,X_{m-1}]$ is not a subset $K[X_1,\cdots,X_m]$ and that there is a ring-isomorphism $f:K[X_1,\dots,X_{m-1}] \to K[X_1,\cdots,X_m]$. You meant that $q_{j} X_{m}^{j}$ denotes the product of $f(q_j) \in K[X_1,\dots,X_m]$ and $X_{m}^{j} \in K[X_1,\dots,X_m]$.
Answer: What is happening here is that the polynomial $p \in K[X_1, \ldots, X_m]$ is being written as a polynomial in the one variable $X_m$, whose coefficients are polynomials in $K[X_1, \ldots, X_{m-1}]$.  So $q_j X_m^j$ is not a product of elements of different rings, but rather a coefficient $q_j$ times a variable $X_m$ raised to a power, and a polynomial ring can have coefficients in any ring.  So the fact that we can make polynomials of the form $\sum_{j = 1}^{n} q_j X_m^j$ makes sense for the same reason that it makes sense to consider polynomials with coefficients in say, the integers, or any arbitrary ring.
In other words, what this lemma is saying is we can write $K[X_1, \ldots, X_m]$ as $K[X_1, \ldots, X_{m - 1}][X_{m}]$, or more succinctly as $R[X_m]$, where $R$ is the ring $K[X_1, \ldots, X_{m - 1}]$.
Comment: Did you meant by *we can write $K[X_1, \ldots, X_m]$ as $K[X_1, \ldots, X_{m - 1}][X_{m}]$* that $K[X_1, \ldots, X_m] = K[X_1, \ldots, X_{m - 1}][X_{m}]$, or just there is an isomorphism between $K[X_1, \ldots, X_m]$ and $K[X_1, \ldots, X_{m - 1}][X_{m}]$?
Comment: There is a canonical isomorphism between them.  The question of being "equal" is a little more subtle.  Technically the first is the polynomials in $m$ variables over a field, and the second is the polynomials in one variable over a ring, so they aren't "exactly" the same thing
Comment: Please correct me If I'm wrong. I think that $p=\sum_{\alpha} p_{\alpha} X^{\alpha} \neq p' = \sum_{j=0}^{n} q_{j} X_{m}^{j}$. By *$p=\sum_{\alpha} p_{\alpha} X^{\alpha} \in K\left[X_{1}, \ldots, X_{m}\right]$ can be written in the form $p=\sum_{j=0}^{n} q_{j} X_{m}^{j}$ for suitable $n \in \mathbb{N}$ and $q_{j} \in K\left[X_{1}, \ldots, X_{m-1}\right]$*, the authors mean that there is a canonical isomorphism between $K[X_1, \ldots, X_m]$ and $K[X_1, \ldots, X_{m - 1}][X_{m}]$.
Comment: This is correct
Comment: Thank you so much for your dedicated help ;)


Examples of when PCA would be preferred over NMF

Question: What are some specific examples of when PCA should be used instead of NMF?
PCA is a widely used method for dimension reduction in data science, machine learning, and bioinformatics. NMF is also a popular method for dimension reduction, much like PCA, and can be used for many of the same types of analyses (e.g. graph-based clustering, trajectory inference, a denoised embedding for reduction with UMAP, etc.) but with very different theoretical and practical characteristics:

PCA components are orthogonal while NMF factors are colinear. Orthogonality means that information in a given component is dependent on information in preceding components. Colinearity means that factors are additive in parallel, and parts of the whole.
PCA values are signed while NMF values are >= 0. This means PCA fits to missing signal while NMF imputes signal.
PCA is exact while NMF is approximate (requires random initialization). However, PCA is only exact within a dataset and components are components are rarely comparable between datasets, while NMF can isolate factors that are similar in independent models with substantial differences.
PCA explains variation, NMF explains additive signal. PCA is essentially sequential normalizations of data along axes of variation while NMF isolates distinct and interpretable signals within the data.
It is difficult to estimate an appropriate number of PCs to use, while NMF almost always has an optimal rank that can be determined by cross-validation.

I have developed NMF and PCA implementations from scratch, but find myself preferring NMF for almost every problem I address. Most of my data is non-negative count data or some normalization of count data, but this leaves me wondering when PCA is actually more useful than NMF. Given it's historical prescience, wide use (or abuse), and mathematical utility, there must be some practical applications where PCA, but not NMF, is the go-to method?  I have to teach a class on data mining, and I'd like at least one or two examples where PCA is both appropriate to use and the preferred method.
Comment: Re your list of characteristics: Much of this looks confused to me or at least using nonstandard terminology. For example, collinear factors are statistically dependent, not independent. Neither PCA nor NMF imply how to impute a missing value, and I don't know what you mean by "upward" - from where? What result does this refer to: "NMF can capture robust signals across similar datasets" - I'm not aware of anything in this respect that NMF can do and PCA can't. "NMF isolates generating processes" - unless you assume a data generating model, this cannot be taken for granted.
Comment: Dimension reduction routines (there are dozens) are rarely an end in themselves but a single step in a multi-step process. If the results are of interest then PCA, as with most linear methods, lends itself to their commonsense interpretability. This is valuable when explaining things to nontechnical audiences. In addition, PCA does not perform well with mixtures of variables. Continuously distributed information with few outliers work best. Introducing discrete data, e.g., 0,1 dummies, will result in heavily biased results. NMF may be more flexible in this regard.
Comment: I'm not sure everybody knows what NMF is; explaining abbreviation is preferable.
Comment: The list still looks like a sales pitch for NMF with a number of unclear, questionable, and imprecise statements. "Colinearity means that factors are (...) parts of the whole"?? "This means PCA fits to missing signal while NMF imputes signal" - "This means" seems nonsensical, and neither is it clear what "fits to missing signal" means. The last statement is relative to what dimension reduction is used for, and statements in between sound as if they are backed up by theory but I'm not so sure they are.
Comment: @ChristianHennig thanks for the great critiques!  Question edited for clarity and language.  As an example where "NMF can capture robust signals across similar datasets" and not PCA, one might perform dimensional reduction using NMF or PCA on a single-cell transcriptomics experiment from lung and skin -- very different, but with some overlapping cell types.  PCA components will not at all align, while a subset of NMF factors will (i.e. those factors describing information about cell types in common).  This is simply a feature of orthogonality vs. colinearity.
Comment: Presenting PC scores with zero mean is just a strong convention. You can use any linear rescaling and preserve the information. That said, I can't see that this convention bites in any practical sense, unless through the fact that you may need to explain it.
Answer: I'd normally prefer PCA for low-d (usually 2-d) data visualisation, as orthogonal components are more intuitive in this respect.
I also suspect that PCA will usually work better for dimension reduction of the x-space in linear regression, because collinearity is unhelpful in regression, although I normally use neither for this task (I have seen data in which Principal Components Regression did a pretty good job, but even there something else such as Partial Least Squares could be found that is slightly better using cross-validation for comparison; I don't think NMF would've been any good in these situations though).
Answer: NMF stands for "nonnegative matrix factorization", which means that your input data must be nonnegative. The factorization you get is also nonnegative, so you don't have to worry about the interpretability of your results compared to PCA that generally requires other post processing steps to interpret the results. If your data are negative, NMF is not suited then.
The NMF optimization problem is flexible, you can add additional constraints and regularization terms to the objective function to specify the solution you want. There are many NMF extensions and variants among them orthogonal NMF, which aims to produce a unique solution.
Numerous studies have demonstrated the effectiveness of NMF over PCA.


Views data export remove tags from xml output

Question: I want to remove <no-name> tag generated by views data export on xml output
Any ideas please?
I have already tried to override template output for the field but no results :(
Answer: I have found the problem, i was overriding the template but with a the wrong code, i was using the alternative style template with these variables:
<?php print $header ?>
<?php print $body ?>
<?php print $footer ?>

Instead these should be used :
// Short tags act bad below in the html so we print it here.
print '<?xml version="1.0" encoding="UTF-8" ?>';
?>

<<?php print $root_node; ?>>

<?php
/**
 * @file views-view-table.tpl.php
 * Template to display a view as a table.
 *
 * - $title : The title of this group of rows.  May be empty.
 * - $rows: An array of row items. Each row is an array of content
 *   keyed by field ID.
 * - $header: an array of headers(labels) for fields.
 * - $themed_rows: a array of rows with themed fields.
 * @ingroup views_templates
 */
?>
<?php foreach ($themed_rows as $count => $row): ?>
  <<?php print $item_node; ?>>
<?php foreach ($row as $field => $content): ?>
    <<?php print $xml_tag[$field]; ?>><?php print $content; ?></<?php print $xml_tag[$field]; ?>>
<?php endforeach; ?>
  </<?php print $item_node; ?>>
<?php endforeach; ?>

</<?php print $root_node; ?>>

I'm posting so maybe it will be helpful for someone else.


Standardize non-normal predictors before performing binomial GLMM using mean and sd?

Question: I am planning to predict a binomial variable (1/0, a used point by an animal or point available to an animal in its range) using several continuous, distance-based predictor variables (distance to habitat type) and interactions (whether that habitat was available to an animal in its range). The data for each predictor variable are non-normal. I will use SAS to perform a binomial GLMM with a logit link.
Several sources (e.g., Gelman and Hill 2007) suggest that standardizing predictors by subtracting the mean and dividing by two standard deviations prior to including them in the model can help interpret parameter estimates, especially when interactions are present (but here, Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?, @bluepole disagrees).
I read these two posts:
Standardizing non-normal data for use in distance-based classifier  and
Standardizing data,
but I didn't find a satisfactory answer to my question:
Is it appropriate or acceptable to "standardize" non-normal data using the mean, when we all know that non-normal data aren't well represented by that mean? If it isn't appropriate, what are my options? Could I use the median and quantiles in a similar way? Or would I attempt to (this is getting messy) transform the data to a normal distribution and then try to standardize (yuck).
Mirkin's book (Clustering for Data Mining: A Data Recovery Approach, 2005)
suggests that means and standard deviations may not be appropriate for scaling...
Comment: Keep in mind that linear transformations of the predictors in a linear model don't change the inferences; they _only_ change the interpretation.
Answer: The predictor variables do not need to be transformed or standardized.  The standardization can sometimes help with interpretation or comparing effect size between predictors, but is not needed.
If you choose to standardize then standardize based on what will help you the most in your interpretation.  If you subtract the mean of each predictor variable from that variable, then the intercept will represent the predicted value when all the variables are at their means.  If you subtract the median from each predictor then the overall intercept will represent the predicted value when all the predictors are at their medians.  If you don't subtract anything then the intercept represents the prediction when all the predictors are at a value of 0 (which may be interesting, or it may be meaningless, depends on the variables being used and the range of values observed).
Without dividing the predictors by anything, the coefficient represents the change in the log-odds (exponentiate for the odds ratio) for a 1 unit increase in the predictor while holding everything else constant.  The only way to hold an interaction term constant while increasing one of the variables included in the interaction is when the other variable is fixed at 0, so this is part of the justification for centering/scaling/normalizing, you can think of the main effect slope in terms of a 1 unit increase while holding everything else at the mean/median/whatever.  In general it is difficult to really understand the effects of individual variables by their slopes when there are interactions or transformations, better is to make predictions at meaningful values of the predictor variables and create a table and/or graph to show how the predictions change with the predictors.  Scaling a predictor variable is most meaningful when the units that the variable are measured in are different from a meaningful differences, for example if you measure time in days but the observed times range over 10 years then looking at the effect of increasing time by one day is probably less meaningful than looking at the effect of changing time by 1 year (or month, or decade, or ...), so scaling by something like 365 or 365.24 will be more meaningful.  Scaling by a standard deviation will put you in a reasonable scale, but it may be more difficult to interpret than a fixed value.  Dividing by something like the inter quartile range will have a similar effect (and may be easier or harder to interpret/explain).  If you scale I would suggest scaling by something with some scientific meaning rather than something derived solely from the data.
Transforming a predictor to normal without any other justification is just going to complicate any interpretation without gaining anything useful.
Comment: Minor exceptions/caveats: (1) if you have a very difficult model to fit, centering and/or scaling may help with *numerical* stability; (2) if you force the correlations between random slopes and intercepts to be zero, then centering continuous predictors actually changes the model.
Comment: @Nova, It looks like your features are already comparable.  Standardization probably will not help much (will not really hurt either).  The real advantage of standardization is when you have different variables on very different scales that you want to compare on similar scales.
Comment: I have no problem with that.  You might want to write up your decision, and justification, as an answer (comments are ephemeral).
Comment: @Nova, it does not really change the response since the response shows why the advice is in the literature.  When comparing relative importance the standardization by the SD has more justification, but with skewness the effect of the different SD's could be questionable.  The nice thing about the normalized predictors is that they become unitless, changing a predictor from days to years will also change the slope (but not its significance), the slope on the normalized value will be more constant and comparable (but choosing a common scale with scientific meaning will be better if available).
Comment: @GregSnow this is a great answer but the second paragraph is huge and hard to read. I didn't want to stick my nose in with an edit but I would highly recommend breaking it up.
Comment: Also, +1 for "make predictions at meaningful values of the predictor variables and create a table and/or graph to show how the predictions change with the predictors." Nothing illuminates a model like putting it to work.
Comment: @GregSnow, all of my predictor variables are "distance to habitat feature"*"Availability of that feature" including things like "forest type" or "road". I am definitely graphing the results to show how the density of a certain feature (which vary widely throughout animal ranges) influences the random slopes for each animal (and it makes sense, thank goodness - when standardized by mean/2*sd, haven't tried not standardized yet). My main goal is to understand the selection of animals to each habitat feature, so I want the features to be comparable to each other.
Comment: Thanks for your help here Greg. Because the magnitude of the measurements are quite different, standardizing does make the regression coefficients much easier to interpret (when I tried without standardizing, regression coefficients are very small). Unless you or others,  incl. @BenBolker, think it's a bad idea, I may go with standardizing by subtracting the median instead of the mean, and dividing by 1.5 the interquartile range (equivalent to 2*standard deviations for a normal distribution). This seems comparable to what is done in the literature but seems more appropriate for my data.
Comment: Does it change your response if I say I am not interested in intercepts but in random slopes? I'd like to know the relative "importance" of each predictor variable in comparison to each other.
Comment: I understand, but I'm still not sure where to take your advice... it seems like (let me know if I'm wrong) subtracting the median and dividing by the interquartile range would be a possible solution, but I've never seen this before.
Answer: I am going to sum up the comments that helped me make a final decision and post what I've decided to do in the end. I want to be able to compare my regression coefficients to each other and want to ensure that regression coefficients are sensible (without standardizing, they are very small). My predictor variables are of the same measurement unit (kilometers) but differ in scale by orders of magnitude. Also, Ben Bolker pointed out that standardizing helps with numerical stability if the model is difficult to fit. Greg Snow noted that the slopes on normalized values will be more constant and comparable when data are standardized. This is very important in my analysis.
I wanted to come as close to standardizing by subtracting the mean from each value and dividing by 2 standard deviations, as suggested in Gelman and Hill 2007. However as Greg Snow pointed out in comments, this gets a bit questionable when input data have some skewness (which my data do).
In the end I've decided to standardize by subtracting the median from each value and dividing by 1.5 * the interquartile range (IQR). This is approximately equal to standardizing as above if the distributions are normal, but is likely more appropriate for data that are not normal. (See http://www.physics.csbsju.edu/stats/box2.html: IQR = 1.35*standard deviation).


nginx configuring uwsgi and gzip buffers for different usage scenarios

Question: I'm trying to configure two URL path groups for relatively different usage scenarios and resource consumption in nginx. Both are served by same uwsgi application.

Group 1: Regular HTML pages up to cca 128k in size, default
Group 2: Large XML "files" up to many MB in size (created by the app on the fly)

The requirement is for both to be gzipped, and the XMLs should be cached 1hr as well.
What I don't quite understand is the meaning of the number of buffers and how to optimally set their number and sizes according to expected traffic and content size, both for uwsgi and gzip.
Also, for the below configuration I get error from nginx:
"uwsgi_busy_buffers_size" must be less than the size of all "uwsgi_buffers" minus one buffer in /etc/nginx/nginx.conf:113

uwsgi_busy_buffers_size appears to be undocumented.
location ~ /path/to/(.+)\.xml {
    uwsgi_hide_header       "Set-Cookie";
    uwsgi_ignore_headers    "Set-Cookie";
    uwsgi_buffers           2 8M;
    uwsgi_cache             xmlfiles;
    uwsgi_cache_key         $request_uri;
    uwsgi_cache_valid       200 1h;
    uwsgi_cache_valid       04 5m;
    uwsgi_cache_use_stale   error;
    expires                 +1h;

    gzip                    on;
    gzip_types              application/xml;
    gzip_buffers            2 8M;

    try_files               $uri @backend;
}

location / {
    try_files       $uri @backend;
    gzip            on;
    gzip_buffers    8 256k;
    uwsgi_buffers   8 256k;

    try_files       $uri @backend;
}

location @backend {
    include      uwsgi_params;
    uwsgi_pass   unix:///tmp/uwsgi-app.sock;
}

Please help. nginx 1.0.14
Answer: uwsgi_busy_buffers_size has the same meaning as fastcgi_busy_buffers_size, but for uwsgi.

Limits the total size of buffers that can be busy sending a response to the client while the response is not yet fully read. In the mean time, the rest of the buffers can be used for reading a response and, if needed, buffering part of a response to a temporary file. By default, size is limited by two buffers set by the uwsgi_buffer_size and uwsgi_buffers directives.

p.s. default values of the buffers related directives in nginx have optimal values for most cases. So, you should tune them only if you are fully understand what you're doing and why.
Comment: Thanks for your answer. My problem is a) I don't know what is the effect of buffer count, and b) I want to avoid hitting disc for uwsgi response and gzipping content if the said content is bigger than the default buffer size.
Answer: Just as a side-note, the maximum allowed size for each gzip buffer is 8k.
To get the same overall size you would need to use gzip_buffers 2048 8k;.
I'm not sure if the same limitation applies to uwsgi_buffers.
In general, it's better to have more small buffers available than a few huge buffers.


Why can 'linear' regressions include non-linear transformations of the independent variables?

Question: So the definition of linear regression is that the response variable is a linear function of the estimators. If we consider univariate regression (for ease of visualization), we have
$$
y = \beta_1x + \beta_0
$$
But we could also have
$$
y = \beta_1x^2 + \beta_0 \\
y = \beta_1 \exp(\log(x^3))
$$
which also satisfies the condition that the response variable is a linear function of the ESTIMATORs.
I find this terminology to be a bit confusing as I would expect "linear" regression to be restricted to strict lines in univariate regression.
When linear regression is introduced in courses, the examples are always straight lines, and I think some instructors even introduce linear regression as fitting a LINEAR line to a set of data, but that's not true.
So isn't it rather confusing that it's called "linear" regression? I feel like "linear" regression connotes that the fit will be a straight line (in the univariate case).
Comment: Contrast $y = \beta_0 + \beta_1 x$ (linear in parameters) with $y = b_0 + x^{\beta_1}$ or $y = \beta_0 + \beta_{1}^{x}$ (nonlinear in parameters).
Comment: The sense in which regressing $y$ on nonlinear transformations of $x$ *still* is just a matter of lines, planes, and linear subspaces generally is explained and illustrated at https://stats.stackexchange.com/a/354256/919.
Comment: I've personally thought (I'm not a statistician), that if we don't change the parameter, and by using a substitution of variables can make it linear and solve it with least squares, then it is linear.  A rational would be a problem but in this if $z = x^2$ then $y\left(z\right)$ is linear in z.
Comment: @EngrStudent Right, that's how I was thinking about it just now. If we consider something like $y = sin(x)$. We could define the transformation $z = sin(x)$. Then $y=z$ is linear in the connotated sense of the word.
Comment: I like how MathematicalMonk addresses this: https://youtube.com/watch?v=rVviNyIR-fI. "It's not just lines & planes."
Comment: Why? It's the difference between the empirical data and an underlying pattern or theoretical model. In other words, your predictors can be nonlinear in any sense but the theoretical model fit to that data will remain linear.
Answer: Check out these threads:
How to tell the difference between linear and non-linear regression models?
Why must linear regressions only generate linear functions that resemble "lines or planes" (*Introduction to Statistical Learning* question)?
"Linear" is talking about the calculation that happens to the unknown coefficients $\beta$. Any regression of the form
$$y \approx \beta^Tf(x)$$
is reasonably called linear as long as $f$ and $x$ are known. This is because a lot of the same theory and computation applies when you go to estimate $\beta$, regardless of what values $f$ and $x$ take.
EDIT: When might you know $f$?

Spline regression
RBF's or Nardaraya-Watson as in PRML Section 6.3 (This is like an interaction term on steroids; it's completely nonparametric. It's still linear in $\beta$.)
When using a wavelet basis

How do you know if you should be using a wavelet basis or a spline? Sorry, that's a whole different question.
Comment: @LBogaardt The latter should still be within the definition of linear regression because $y$ is still a linear function of $\beta$. You can introduce a new variable $z = x_ix_j$. It's just $z$ will have a strong degree of multicollinearity with $x_i$ and $x_j$.
Comment: When you're given an input variable, in this case, $f(x)$, which situations would you ever know what $f(x)$ is?
Comment: "linear" means no interaction terms.
Comment: @LBogaardt When I think of interaction, I think of interaction between $\beta_i$ and $\beta_j$ for $i \neq j$. But $\beta_i^2$ would no longer be linear.
Comment: As far as I know, $\beta_i x_i + \beta_j x_j$ is called 'linear', so is $\beta_i log(x_i) + \beta_j x_j^5$, but $\beta_{ij} x_i x_j$ is not.


How can I hide the details within my .NET executable?

Question: If I have a simple CRUD app executable in .NET, what prevents a user from loading it into RedGate Reflector and viewing all the contents, including db connection strings, passwords, etc?
Can I protect against this in any way?
Comment: Storing connection strings and passwords in an executable is asking for trouble.
Comment: Rewrite you app in C++ and reflector won't be able to load it. ;)
Comment: You need to redesign so that all this information is NOT in the executable PERIOD.  C++ doesn't protect you, any idiot with a hex editor can view your strings just fine through that.  Look at stored procedures, web services, or any sort of good remote authentication scheme to do this right.
Comment: This question has been asked over and over again on SO. Please search for obfuscation.
Comment: Sid, is it asking for trouble in general, or with regard to .NET executables only?
Comment: PK, I just started a new VC++ app after reading your responses.  Thanks everyone.
Comment: @PK not very helpfull and you'd have the same problem with a c++ binary you'd just need a different tool. The problem is not the platform but embedding security information without encrypting it. Which .NET does support in app.config
Answer: No, the best you can do is obfuscate the assembly to make it harder to read and understand but other than that there is not way to stop someone from using Reflector or ILDASM to view the IL in your assembly.
Remember that the CLR needs to be able to read this assembly as well so if the CLR can read the assembly, so can anyone else.
Answer: Obfuscation is never the right answer. Maybe you're going about this the wrong way.
There are several ways I can think of so that the connection string is either not available or not important.
You could put your database behind a web service so that the connection string to the database would only be known to the web service.  Of course, you'd need another way to restrict access to the web service, such as using login credentials.
Or, you could give each user their own SQL login name/password.  That way, they would know their own userid/password but it would be easy to "turn it off" from the database.  This also gives you much better control over each person's access to the database itself... like what tables/views they have access to, and what type of access.
Answer: You can make it hard to do so (obfuscation, string encryption, etc.) but it will never be impossible to reverse-engineer as long as the user has access to the executable.
Answer:
including db connection strings, passwords

Define security in the database engine: limit users, limit machines from which users can connect, limit what users can do, and/or specify that users may interact with the database only via defined "stored procedures".
Answer: It depends what you want to protect.
If it's DB connection string with passwords, then don't store them, Example: set up IIS App Pool to run a limited service account to connect to the database and the trusted security. Assuming decent database security, knowledge of servername etc is useless.
Answer: You can put your passwords and connection strings in an hashtable or xml then encrypt your data, make an zip file with a password hidden in text file hidden with stenography behind an image. All your unzip and reading make in memory after making a read make a release, flush of memory, call the collector, after using your connection strings and fill clean your variables. Keep only the data you need in memory, be careful in using ram.
Comment: Nice ideas. Next time you answer try setting out your answer to make it a bit easier to read.
Answer: Obfuscation would be an option. But I guess you cannot hide e.g. password strings completely. That's really unsafe.