Why is NSURLSession slower than cURL when downloading many files?

Question: I've been using cURL to download about 1700+ files -- which total to about ~290MB -- in my iOS app. It takes about 5-7 minutes on my Internet connection to download all of them using cURL. But since not everyone has fast internet connection (especially when on the go), I decided to allow the files to be downloaded in the background, so that the user can do other things while waiting for the download to finish. This is where NSURLSession comes in.
Using NSURLSession, it takes about 20+ minutes on my Internet connection to download all of them while the app is in foreground. I don't mind it being slow when the app is in background, because I understand that it is up to the OS to schedule the downloads. But it's a problem when it's slow even when it's in foreground. Is this the expected behaviour? Is it because of the quantity of the files?
In case I'm not using NSURLSession correctly, here's a snippet of how I'm using it:
// Initialization

NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:@"<my-identifier>"];
sessionConfiguration.HTTPMaximumConnectionsPerHost = 40;

backgroundSession = [NSURLSession sessionWithConfiguration:sessionConfiguration
                                                  delegate:self
                                             delegateQueue:nil];

// ...

// Creating the tasks and starting the download
for (int i = 0; i < 20 && queuedRequests.count > 0; i++) {
    NSDictionary *requestInfo = [queuedRequests lastObject];
    NSURLSessionDownloadTask *downloadTask = [backgroundSession downloadTaskWithURL:[NSURL URLWithString:requestInfo[@"url"]]];
    ongoingRequests[@(downloadTask.taskIdentifier)] = requestInfo;
    [downloadTask resume];
    [queuedRequests removeLastObject];
    NSLog(@"Begin download file %d/%d: %@", allRequests.count - queuedRequests.count, allRequests.count, requestInfo[@"url"]);
}

// ...

// Somewhere in (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)location

// After each download task is completed, grab a file to download from
// queuedRequests, and create another task

if (queuedRequests.count > 0) {
    requestInfo = [queuedRequests lastObject];
    NSURLSessionDownloadTask *newDownloadTask = [backgroundSession downloadTaskWithURL:[NSURL URLWithString:requestInfo[@"url"]]];
    ongoingRequests[@(newDownloadTask.taskIdentifier)] = requestInfo;

    [newDownloadTask resume];
    [queuedRequests removeLastObject];
    NSLog(@"Begin download file %d/%d: %@", allRequests.count - queuedRequests.count, allRequests.count, requestInfo[@"url"]);
}

I've also tried using multiple NSURLSession, but it's still slow. The reason I tried that is because when using cURL, I create multiple threads (around 20), and each thread will download a single file at a time.
It's also not possible for me to reduce the number of files by zipping it, because I need the app to be able to download individual files since I will update them from time to time. Basically, when the app starts, it will check if there are any files that have been updated, and only download those files. Since the files are stored in S3, and S3 doesn't have zipping service, I could not zip them into a single file on the fly.
Comment: 2. When using background session, you don't need this "queue up the first 20 process". Just queue them all up. By metering them out like that, you're going to slow down the background process because it's going to be constantly re-firing up your app if suspended/terminated. Foreground session require controlling the degree of concurrency to avoid timeouts, but not for background sessions. (Then again, if I was controlling degree of concurrency in foreground session, I think operation queue is easier solution in that case. But that's perhaps academic if you're using background session.)
Comment: @HasyimiBahrudin I just benchmarked the scheduling 97 downloads, and it took 2.9 seconds on simulator and 1.7 seconds on device, so I guess I could imagine that scheduling 1700 could take a minute or so (assuming the scale was linear). Interestingly, though, I'm not sure I'd be too worried about that delay because I saw the initial requests already finishing before the the scheduling was done, so as long as you (a) don't block the main thread scheduling these background requests; and (b) make sure to wrap it in a `beginBackgroundTask`, you should be fine. Better than 20 at a time, IMHO.
Comment: Try replacing `backgroundSessionConfigurationWithIdentifier:` with `defaultSessionConfiguration` and compare the results. I don't think iOS really cares if your app is running or not when downloading on a session configured for background downloads. It will always do those downloads on a separate process which throttles speed.
Comment: 1. I agree with Filip: Temporarily try using non-background session and I think you might see performance difference. My anecdotal testing suggested that background session (even fired from foreground) was slower, though I haven't tested this recently.
Comment: @FilipRadelic You're right. It's as fast as using `cURL` now. I'll try creating a separate `NSURLSession` to download when the app is switched to background. I'll post an answer if this works.
Comment: @Rob I did try queueing them all up. But it seems that the queueing all 1700+ files at once takes minutes, and the download will only start after queueing is finished.
Comment: @Rob Ah, you're right. And actually the scheduling only takes seconds (like around 20 - 30 seconds.) It took minutes because I was removing object from an array in a loop. Thanks! :)
Comment: @Rob It turns out that there is a problem when I schedule them all at one go. After a minute or so, all of the tasks will timeout. This does not happen when I only queue 20 tasks.
Comment: As I tried to describe in my point #2, up above, that's true for foreground sessions. Not background sessions, though.
Comment: @Rob, I also experienced scheduling lots of _background_ downloads tasks, and it's definitely *not* linear. More the number of background tasks are, longer it will take. If the app enter background before the end of the scheduling, the background download will never finish! I think Hasyimi solution to split the tasks in chucks of N is a good one, because it ensure the app to schedule and download everything
Comment: @Martin - Hey, if that works for you, that's fine. I'd rather schedule everything (and just give the app enough time to finish the scheduling with
`beginBackgroundTask`), but to each his own. FYI, if you follow this N-at-a-time approach, be aware that the subsequent batches will always have `discretionary` set to `true`. But, then, again, if you're downloading that much data, you probably should use discretionary connection, anyway.
Answer: As mentioned by Filip and Rob in the comments, the slowness is because when NSURLSession is initialized with backgroundSessionConfigurationWithIdentifier:, the download tasks will be executed in the background regardless if the app is in the foreground. So I solved this issue by having 2 instances of NSURLSession: one for foreground download, and one for background download:
NSURLSessionConfiguration *foregroundSessionConfig = [NSURLSessionConfiguration defaultSessionConfiguration];
foregroundSessionConfig.HTTPMaximumConnectionsPerHost = 40;

foregroundSession = [NSURLSession sessionWithConfiguration:foregroundSessionConfig
                                                  delegate:self
                                             delegateQueue:nil];
[foregroundSession retain];

NSURLSessionConfiguration *backgroundSessionConfig = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:@"com.terato.darknessfallen.BackgroundDownload"];
backgroundSessionConfig.HTTPMaximumConnectionsPerHost = 40;

backgroundSession = [NSURLSession sessionWithConfiguration:backgroundSessionConfig
                                                  delegate:self
                                             delegateQueue:nil];
[backgroundSession retain];

When the app is switched to background, I simply call cancelByProducingResumeData: on each of the download tasks that's still running, and then pass it to downloadTaskWithResumeData::
- (void)switchToBackground
{
    if (state == kDownloadManagerStateForeground) {
        [foregroundSession getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *downloadTasks) {
            for (NSURLSessionDownloadTask *downloadTask in downloadTasks) {
                [downloadTask cancelByProducingResumeData:^(NSData *resumeData) {
                    NSURLSessionDownloadTask *downloadTask = [backgroundSession downloadTaskWithResumeData:resumeData];
                    [downloadTask resume];
                }];
            }
        }];

        state = kDownloadManagerStateBackground;
    }
}

Likewise, when the app is switched to foreground, I do the same but switched foregroundSession with backgroundSession:
- (void)switchToForeground
{
    if (state == kDownloadManagerStateBackground) {
        [backgroundSession getTasksWithCompletionHandler:^(NSArray *dataTasks, NSArray *uploadTasks, NSArray *downloadTasks) {
            for (NSURLSessionDownloadTask *downloadTask in downloadTasks) {
                [downloadTask cancelByProducingResumeData:^(NSData *resumeData) {
                    NSURLSessionDownloadTask *downloadTask = [foregroundSession downloadTaskWithResumeData:resumeData];
                    [downloadTask resume];
                }];
            }
        }];

        state = kDownloadManagerStateForeground;
    }
}

Also, don't forget to call beginBackgroundTaskWithExpirationHandler: before calling switchToBackground when the app is switched to background. This is to ensure that the method is allowed to complete while in background. Otherwise, it will only be called once the app enters foreground again.
Comment: @Martin - So, just request extra time, if needed with [`beginBackgroundTask(withName:expirationHandler:)`](https://developer.apple.com/library/ios/documentation/UIKit/Reference/UIApplication_Class/index.html#//apple_ref/occ/instm/UIApplication/beginBackgroundTaskWithName:expirationHandler:). That way you have 3 minutes after the app enters background mode to do the scheduling, not just 5 seconds.
Comment: you call this twice, `foregroundSessionConfig.HTTPMaximumConnectionsPerHost = 40;` I think the second time you meant to use the object `backgroundSessionConfig` instead?
Comment: 2015 and still in manual reference counting?! :) Btw, very good question and answer, thanks.
Comment: Is it shorter to schedule with `downloadTaskWithResumeData:` than `downloadTaskWithURL:` ? Because if no, and if you really have thousand of files, you'll never finish the foreground-to-background migration withing the 5 seconds that `- (void)applicationDidEnterBackground:(UIApplication *)application` gives to you
Comment: thanks @Rob, I'm currently experimenting background tasks, this method seems to be a pretty good solution.
Comment: This answer is amazingly amazing! Thank you!
Comment: Is this only available for downloads? What about uploads?
Comment: @CristianoCoelho i try to use sessionWithConfiguration instead of backgroundSessionConfigurationWithIdentifier, the speed of both download,upload have significantly when use in foreground. I haven't tried implementing all the code above but the issue about the slow download/upload might be it


Replace Text using Apache POI in Header/Footer not working when a dot is inside the placeholder

Question: I use templ4docx/Apache POI (2.0.3/3.17).
There you can set a VariablePatten like this:
private static final String START_PATTERN = "#{";
private static final String END_PATTERN = "}";
...
targetDocx.setVariablePattern(new VariablePattern(START_PATTERN, END_PATTERN));

When i use a placeholder with dots, it´s not working inside Header/Footer. In the Body with dots it works. And Images works too with placeholder and dots inside!
Example in Word-Template:
#{Person.Name} // works in Body NOT in Header/Footer!
#{Person.Name} // works in Body and Header/Footer!
#{Person} // works in Body and Header/Footer!
#{Image.Name} // works in Body and Header/Footer! Here i use ImageVariable instead of Textvariable.

I debug the code an saw the "run.setText()" is called with the right Text but in the final document it´s not.
@Override
public void insert(Insert insert, Variable variable) {
    if (!(insert instanceof TextInsert)) {
        return;
    }
    if (!(variable instanceof TextVariable)) {
        return;
    }

    TextInsert textInsert = (TextInsert) insert;
    TextVariable textVariable = (TextVariable) variable;
    for (XWPFRun run : textInsert.getParagraph().getRuns()) {
        String text = run.getText(0);
        if (StringUtils.contains(text, textInsert.getKey().getKey())) {
            text = StringUtils.replace(text, textVariable.getKey(), textVariable.getValue());
            if (text.contains("\n")) {
                String[] textLines = text.split("\n");
                run.setText(textLines[0], 0);
                for (int i = 1; i < textLines.length; i++) {
                    run.addBreak();
                    run.setText(textLines[i]);
                }
            } else {
                run.setText(text, 0);
            }
        }
    }
}

Any ideas why it didn´t work with Placeholder "#{Person.Name}" as a TextVariable in Header/Footer? But it works with "#{PersonName}" and ImageVariable "#{Images.Logo1}"???
Comment: You need to provide the code you're using that's "not working". There's too little information provided here to even begin guessing what the problem might be. Use the [Edit] link below the question.
Answer: It looks like Word sometimes separates the placeholders, so you'll only find parts of the placeholder in the runs.
In the class "TextInsertStrategy" i check in the loop of the runs for split placeholders and treat it accordingly. With that I could solve the problem.


How do I make a chrome app write to a file

Question: I just started learning about chrome applications and I'm fairly new to the javascript aspect of it while I do have some experience with the HTML side of it. How do I create an application that has a check box (The check box will be html) and then have a button (which all also be in html) and this button will write to a file on my desktop whether the check box is checked or unchecked? i.e. it will write either true or false?
Comment: Your question is too broad. You are not asking a question which can be answered easily because "How do I create an application...?" questions imply that you want someone to write your code for you. Please write some code, and if it doesn't work then come back and post another question.
Comment: I sort of do want some kind of generalized code since I'm really clueless on how a chrome apps really are. But since my question is kind of broad ill just ask for starters, how do I get a button that i create in the html to do a simple action such as open a web page maybe because inline code such as "" is not allowed? And i dont know how that would work with javascript...
Answer: Saving to a file is not as as easy as it sounds. I suggest checking the Chrome App examples (found here), loading the extensions and seeing which one does what you want. THEN you go and check the code. The example that would work best for you would be the mini-code-edit one, which has both loading a file and saving to a file.
Still, if you won't do that, this example code should work:
<code>//Selector for your "Save" button
document.querySelector("#saveButton").onclick = saveToFile;
//Save variables
var fileEntry;
var hasWriteAccess;
//Save methods
function saveToFile()
{
    if (fileEntry && hasWriteAccess) {
    writeEditorToFile(fileEntry);
    } else {
        chrome.fileSystem.chooseEntry({ type: 'saveFile', suggestedName: "suggestedFileName.txt"}, onChosenFileToSave);
    }
}

function setFile(theFileEntry, isWritable) {
  fileEntry = theFileEntry;
  hasWriteAccess = isWritable;
}

function writeEditorToFile(theFileEntry) {
  theFileEntry.createWriter(function(fileWriter) {
    fileWriter.onerror = function(e) {
      console.log("Write failed: " + e.toString());
    };

    var blob = new Blob(["You put whatever you want written to your file here."]);
    fileWriter.truncate(blob.size);
    fileWriter.onwriteend = function() {
      fileWriter.onwriteend = function(e) {
        //handleDocumentChange(theFileEntry.fullPath);
        console.log("Write completed.");
      };

      fileWriter.write(blob);
    }
  }, errorHandler);
}

var onChosenFileToSave = function(theFileEntry) {
  setFile(theFileEntry, true);
  writeEditorToFile(theFileEntry);
};
</code>
This was taken from the example I mentioned. Also, remember to add "fileSystem": ["write"] to your manifest.


Is there any difference between lm and glm for the gaussian family of glm?

Question: Specifically, I want to know if there is a difference between lm(y ~ x1 + x2) and glm(y ~ x1 + x2, family=gaussian). I think that this particular case of glm is equal to lm. Am I wrong?
Comment: Yes and no. As a statistical model, no. As a fitted object in R, yes; different returned objects, different algorithm used.
Comment: It seems to me there is a statistical question here, as well as an R coding one.
Answer: While for the specific form of model mentioned in the body of the question (i.e. lm(y ~ x1 + x2) vs glm(y ~ x1 + x2, family=gaussian)), regression and GLMs are the same model, the title question asks something slightly more general:

Is there any difference between lm and glm for the gaussian family of glm?

To which the answer is "Yes!".
The reason that they can be different is because you can also specify a link function in the GLM. This allows you to fit particular forms of nonlinear relationship between $y$ (or rather its conditional mean) and the $x$-variables; while you can do this in nls as well, there's no need for starting values, sometimes the convergence is better (also the syntax is a bit easier).
Compare, for example, these models (you have R so I assume you can run these yourself):
x1=c(56.1, 26.8, 23.9, 46.8, 34.8, 42.1, 22.9, 55.5, 56.1, 46.9, 26.7, 33.9,
37.0, 57.6, 27.2, 25.7, 37.0, 44.4, 44.7, 67.2, 48.7, 20.4, 45.2, 22.4, 23.2,
39.9, 51.3, 24.1, 56.3, 58.9, 62.2, 37.7, 36.0, 63.9, 62.5, 44.1, 46.9, 45.4,
23.7, 36.5, 56.1, 69.6, 40.3, 26.2, 67.1, 33.8, 29.9, 25.7, 40.0, 27.5)

x2=c(12.29, 11.42, 13.59, 8.64, 12.77, 9.9, 13.2, 7.34, 10.67, 18.8, 9.84, 16.72,
10.32, 13.67, 7.65, 9.44, 14.52, 8.24, 14.14, 17.2, 16.21, 6.01, 14.23, 15.63,
10.83, 13.39, 10.5, 10.01, 13.56, 11.26, 4.8, 9.59, 11.87, 11, 12.02, 10.9, 9.5,
10.63, 19.03, 16.71, 15.11, 7.22, 12.6, 15.35, 8.77, 9.81, 9.49, 15.82, 10.94, 6.53)

y = c(1.54, 0.81, 1.39, 1.09, 1.3, 1.16, 0.95, 1.29, 1.35, 1.86, 1.1, 0.96,
1.03, 1.8, 0.7, 0.88, 1.24, 0.94, 1.41, 2.13, 1.63, 0.78, 1.55, 1.5, 0.96,
1.21, 1.4, 0.66, 1.55, 1.37, 1.19, 0.88, 0.97, 1.56, 1.51, 1.09, 1.23, 1.2,
1.62, 1.52, 1.64, 1.77, 0.97, 1.12, 1.48, 0.83, 1.06, 1.1, 1.21, 0.75)

lm(y ~ x1 + x2)
glm(y ~ x1 + x2, family=gaussian)
glm(y ~ x1 + x2, family=gaussian(link="log"))
nls(y ~ exp(b0+b1*x1+b2*x2), start=list(b0=-1,b1=0.01,b2=0.1))

Note that the first pair are the same model ($y_i \sim N(\beta_0+\beta_1 x_{1i}+\beta_2 x_{2i},\sigma^2)\,$), and the second pair are the same model ($y_i \sim N(\exp(\beta_0+\beta_1 x_{1i}+\beta_2 x_{2i}),\sigma^2)\,$ and the fits are essentially the same within each pair.
So - in relation to the title question - you can fit a substantially wider variety of Gaussian models with a GLM than with regression.
Comment: +1. One the computational side of things I would also think that an GLM algorithm would use some IRWLS variant (in most cases) while an LM would relay on some closed-form solution variant.
Comment: It doesn't respond to seeing "outliers" (except via the likelihood as described above); reweighting is due to the effect of the variance function and the shift in the local linear approximation.
Comment: @ChrisChiasson: +1 to Glen_b's comment. As mentioned this is not related to the robustness of the algorithm in the presence of outliers. You might want to explore different families (eg. suitably scaled $t$-distributions, or a Huber loss; check `MASS::rlm` for more on that) - sorry just got online after a couple days off..
Comment: You could achieve the kind of robustness I think you intend in a number of ways. However, with glms and regression type models, you have to beware not just of outliers in the y-direction but of *influential* outliers, which can make themselves not look out of place..
Comment: @usεr11852 - I would have thought it was E-M, but they might be the same thing in this case.
Comment: @usεr11852, does this imply that if outliers are in the data set that GLM will be less affected by their presence?
Comment: @EngrStudent Usually stats packages use Fisher scoring for GLMs (which boils down to a form of iterated least squares approximations - essentially as usεr11852 suggests).
Comment: @ChrisChiasson Not from the algorithm used for fitting, no. Some GLMs are more robust to y-outliers (e.g. a Gamma GLM is more robust to a high outlier than LS is), but some not -- it depends on the particular family and kind of outlier, not the fitting algorithm.
Comment: @Glen_b I guess I was referring ti user11852's mention of IRwLS... assuming the reweighting process scales down the weight of outliers due to extremely large variance/deviation
Comment: So would it be fair to say that I should (must?) use a robust function to estimate the variation/deviance to make the glm be robust to outliers...and that otherwise the glm would not be robust? (I use a different system than R)
Answer: From @Repmat's answer, the model summary are the same, but the C.I.'s of the regression coefficients from confint are slightly different between lm and glm.
> confint(reg1, level=0.95)
               2.5 %    97.5 %
(Intercept) 2.474742 11.526174
x1          1.971466  2.014002
x2          2.958422  3.023291
> confint(reg2, level=0.95)
Waiting for profiling to be done...
               2.5 %    97.5 %
(Intercept) 2.480236 11.520680
x1          1.971492  2.013976
x2          2.958461  3.023251

$t$-distribution is used in lm while normal distribution is used in glm when constructing the intervals.
> beta <- summary(reg1)$coefficients[, 1]
    > beta_se <- summary(reg1)$coefficients[, 2]
> cbind(`2.5%` = beta - qt(0.975, n - 3) * beta_se,
        `97.5%` = beta + qt(0.975, n - 3) * beta_se) #t
                2.5%     97.5%
(Intercept) 2.474742 11.526174
x1          1.971466  2.014002
x2          2.958422  3.023291
> cbind(`2.5%` = beta - qnorm(0.975)*beta_se,
        `97.5%` = beta + qnorm(0.975)*beta_se) #normal
                2.5%     97.5%
(Intercept) 2.480236 11.520680
x1          1.971492  2.013976
x2          2.958461  3.023251

Answer: Short answer, they are exactly the same:
# Simulate data:
set.seed(42)
n <- 1000

x1 <- rnorm(n, mean = 150, sd = 3)
x2 <- rnorm(n, mean = 100, sd = 2)
u  <- rnorm(n)
y  <- 5 + 2*x1 + 3*x2 + u

# Estimate with OLS:
reg1 <- lm(y ~ x1 + x2)
# Estimate with GLS
reg2 <- glm(y ~ x1 + x2, family=gaussian)

# Compare:
require(texreg)
screenreg(l = list(reg1, reg2))

=========================================
                Model 1      Model 2
-----------------------------------------
(Intercept)        6.37 **       6.37 **
                  (2.20)        (2.20)
x1                 1.99 ***      1.99 ***
                  (0.01)        (0.01)
x2                 3.00 ***      3.00 ***
                  (0.02)        (0.02)
-----------------------------------------
R^2                0.99
Adj. R^2           0.99
Num. obs.          1000          1000
RMSE               1.00
AIC                           2837.66
BIC                           2857.29
Log Likelihood               -1414.83
Deviance                       991.82
=========================================
*** p < 0.001, ** p < 0.01, * p < 0.05

Longer answer; The glm function fits the model by MLE, however, because of the assumption you made about the link function (in this case normal), you end up with the OLS estimates.
Comment: +1, a typo in the last sentence. The normal assumption is about the error distribution, not about the link function. In your example, the default link function is "identity". A more complete form for `glm` is `glm(y ~ x1 + x2, family = gaussian(link = "identity"))`.


$\frac{\partial ||\mathbf{X}^T -\mathbf{A} ||_F^2}{\partial \mathbf{X}}$ is zero?

Question: Let $\mathbf{X} \in \mathbb{R}^{n \times m}$, $\mathbf{A} \in \mathbb{R}^{m \times n}$, $(.)^T$ denotes the transpose operator and $||.||_F$ is the matrix Frobenius norm
What is the value of $\frac{\partial ||\mathbf{X}^T  -\mathbf{A} ||_F^2}{\partial \mathbf{X}}$ ?
Is it zero?
Please, any help?
Comment: Consider the case when $n=1$ and $a=0$ (for example)... Is it true then? No. In this case you get $2x$ since this is the derivative of $x^2$ and the transpose operator does nothing when $n=1$.
Comment: derivative of matrix w.r.t matrix doesnt exist
Comment: what you said means you didnt read what i say: you have a scalar w.r.t matrix .. i said derivative of matrix w.r.t matrix doesnt exist.
Comment: Yes $\mathbf{A}$ is of size $m \times n$ . Sorry error with my latex. I wrote my question so fast. I edited it
Comment: @ElBazzi why cant derivative of matrix with respect to matrix exist?
Comment: @ElBazzi ok, understood. it is somewhat similar to the jacobian. The derivative is not explicitly with respect to matrix, but with each element of the matrix, right?
Comment: @vidyarthi : The notation chosen by the OP is perhaps not very suggestive, but she surely means this : https://en.wikipedia.org/wiki/Fr%C3%A9chet_derivative. In a standard analysis course, you show that this Frechet derivative can be computed by evaluating the partial derivatives and forming the Jacobian.
Comment: I added my answer. I solved it.
Comment: Celina, I've added an answer to your earlier question on the same topic here http://math.stackexchange.com/questions/1987919/transposition-problems-inside-the-gradient-of-squared-l2-norm/1992848#1992848 which will close this one too.
Comment: Yes thank you:)
Answer: The answer is $2(\mathbf{X} - \mathbf{A}^T)$.
You have $||\mathbf{X}^T - \mathbf{A}||_F^2 = trace((\mathbf{X}^T - \mathbf{A})(\mathbf{X}^T - \mathbf{A})^T) = trace((\mathbf{X}^T - \mathbf{A}) (\mathbf{X} - \mathbf{A}^T)) = trace(\mathbf{X}^T \mathbf{X} - \mathbf{X}^T \mathbf{A}^T - \mathbf{AX} + \mathbf{A}\mathbf{A}^T) = trace(\mathbf{X}^T \mathbf{X}) - trace(\mathbf{X}^T\mathbf{A}^T) - trace(\mathbf{AX}) + trace(\mathbf{A}\mathbf{A}^T)
=trace(\mathbf{X\mathbf{X}^T}) - trace((\mathbf{AX})^T) - trace(\mathbf{AX}) + trace(\mathbf{A}\mathbf{A}^T) = trace(\mathbf{X\mathbf{X}^T}) - 2trace(\mathbf{AX}) + trace(\mathbf{A}\mathbf{A}^T)$.
Keep in mind that $trace(\mathbf{AB}) = trace(\mathbf{BA}))$, where $\mathbf{B}$ is a matrix, and $trace(\mathbf{A}^T) = trace(\mathbf{A})$.
Then we have:
$\frac{\partial (trace(\mathbf{X}\mathbf{X}^T))}{\partial \mathbf{X}} = 2\mathbf{X}$.
$\frac{\partial (trace(\mathbf{AX}))}{\partial \mathbf{X}} = \frac{\partial (trace(\mathbf{XA}))}{\partial \mathbf{X}}  = \mathbf{A}^T$.
$\frac{\partial (trace(\mathbf{A}\mathbf{A}^T))}{\partial \mathbf{X}} = 0$
Therefore, $\frac{\partial ||\mathbf{X}^T - \mathbf{A}||_F^2}{\partial \mathbf{X}} = 2(\mathbf{X} - \mathbf{A}^T)$.
Comment: Your result is correct, but using the Frobenius (:) inner product would be more succinct $$\eqalign{Y&=X-A^T\cr f&=\|Y\|^2_F&=Y:Y\cr df&=2Y:dY&=2Y:dX\cr\frac{\partial f}{\partial X}&=2Y}$$
Answer: Let $X$ and $A$ be two matrices of same dimensions, then
$$\Vert X - A \Vert_F^2 = \text{trace}\big( X^T X - X^TA  - A^TX + A^TA\big)$$
And using
$$\frac{\partial}{\partial X} \text{trace} (X^TAX) = X^T(A + A^T)$$
and
$$\frac{\partial}{\partial X} \text{trace} (X^TA) = A$$
we get
$$\frac{\partial \Vert X - A \Vert^2}{\partial X} = 2X^T - A - A^T$$
You ask if it is zero. Let's equate it to zero, we get
$$X = \frac{1}{2}(A + A^T)$$
Comment: As @vidyarthi has commented, $A$ is rectangular,so  the sum $(A+A^T)$ does not exist. The problem with this answer, is that it incorporates $trace(X^TAX)$ -- which appears nowhere in the expansion of the norm.
Comment: It is what it is.. If $X$ is equal to half the value of $A$ summed up to the transposition of $A$.
Comment: The value of $\frac{\partial ||\mathbf{X}  -\mathbf{A} ||_F^2}{\partial \mathbf{X}}$ is simply $2(\mathbf{X} - \mathbf{A})$. But for me, I have $\mathbf{X}^T$. So I don't know  the value of $\frac{\partial ||\mathbf{X}^T  -\mathbf{A} ||_F^2}{\partial \mathbf{X}}$.
Comment: @ElBazzi But, A being of order$m \times n$, cannot be added to its transpose, right?
Comment: @Celina By symmetry, isnt your derivative simply $2(\mathbf{X}^T -\mathbf{A})$
Comment: No in fact it will be $2(\mathbf{X} - \mathbf{A}^T)$.
Comment: I added my answer. Thank you @ElBazzi for your time. I appreciate your help:)


Haskell Tokens in expression `A._`

Question: How does Haskell interpret the expression A._?
E.g. in main = print $ A._.
Haskell throws the following error error:
source_file.hs:1:16:
    Found hole '_' with type: r0
    Where: 'r0' is an ambiguous type variable
    Relevant bindings include
      main :: IO ()
        (bound at source_file.hs:1:1)
    In the second argument of '($)', namely 'A._'
    In the expression: print $ A._
    In an equation for 'main': main = print $ A._

However, according to Haskell's Syntax Reference, A._ should be interpreted as consym (A), varsym (.) and reservedid (_), since _ is not a valid varid (varids cannot be reservedids).
Haskell throws the same error for main = print $ A._t:
source_file.hs:1:16:
    Found hole '_t' with type: r0
    Where: 'r0' is an ambiguous type variable
    Relevant bindings include
      main :: IO ()
        (bound at source_file.hs:1:1)
    In the second argument of '($)', namely 'A._t'
    In the expression: print $ A._t
    In an equation for 'main': main = print $ A._t

In this case however, the token sequence should be qvarid (A._t) as A matches modid and _t matches varid.
Can anybody help me out and explain what is going on?
Comment: What version of GHC are you using?
Comment: could you include the full contents of `source_file.hs`.
Comment: The source file does not contain anything else. I am currently trying to implement an haskell parser in typescript and for that I need to understand the behavior in this case.
Comment: Are these the only errors you are getting?
Comment: I have been using this [online tool](http://rextester.com/l/haskell_online_compiler) which is `The Glorious Glasgow Haskell Compilation System, version 7.10.3`.
The mentioned error is the only one.
Comment: This seems to be a bug in ghc:
`data A = B | C A deriving Show; main = print ((C.let x = (\arg -> B) in x) B)`
does not compile (it throws `parse error on input '='`), but `data A = B | C A deriving Show; main = print ((C . let x = (\arg -> B) in x) B)` does.
According to the syntax reference, both should compile, as `C.let` does not match `qvarid` (`let` is a `reservedid`).
Answer: These are typed holes, which are part of a GHC extension enabled by default. From this link:

Typed holes are a feature of GHC that allows special placeholders written with a leading underscore (e.g., "_", "_foo", "_bar"), to be used as expressions. During compilation these holes will generate an error message that describes which type is expected at the hole's location, information about the origin of any free type variables, and a list of local bindings that might help fill the hole with actual code. Typed holes are always enabled in GHC.

That means that A._ gets parsed as an application of composition, (.), to the constructor A and the typed hole expression _. The same holds for A._t.
The purpose of typed holes is mostly debugging: it lets you know what type GHC infers an expression in that position should have and reports that in an error message.
Comment: It isn't attempting to specifically match the type of `_` against a function type though. That makes me think it's interpreting `A` as a module name.
Comment: Are you sure `A` is interpreted as type constructor, I am getting errors about a missing import `A` - it is hard to figure this without the full source code.
Comment: at [begriffs.com](https://begriffs.com/posts/2015-02-07-haskell-type-holes.html) there is a video which shows the power and usage of type(d) holes in (AGDA and) Haskell
Comment: @epsilonhalbe I actually expected errors about a module `A` - my answer is just based on OPs error message. I reckon this probably comes down to a GHC version difference.
Comment: `I reckon this probably comes down to a GHC version difference.` those are  the things I _never_ think of - good idea.
Comment: What do you mean with `special placeholders`? I thought `_` is treated as a lower case letter, thus `_foo` should be a valid variable identifier, shouldn't it? Only a single `_` behaves differently, as `_` is a reserved id and varid's cannot be reservedid's (=> `_foo` is not a reserved id).

GHC does not seem to have a problem with

`main = print $ _a; _a = 32`

- it outputs 32 without any compiler errors/warnings.
However, just `main = print $ _a` yields the same error as `main = print $ _` as mentioned before.
Comment: @Henning It depends on whether `_a` is bound or unbound. If it is unbound, it is treated as a "named" [type hole](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#typed-holes).

Haskell Tokens in expression `A._`


DKIM: To each their own or one to rule them all?

Question: I have been handed over responsibility for an email gateway serving multiple domains and got into a bit of a philosophical debate with a colleague.
In our setup the gateway is the instance adding DKIM signatures to all outgoing emails and I am told that we need a different DKIM key pair per domain, I don't object this request, but no-one seems to be able to explain why.
Here's my understanding of the DKIM process:

Private DKIM key (one for all or one per domain) is kept on the gateway

Public DKIM key is published in the domains DNS records

For each outgoing email the gateway will encrypt a random string with the private key and add it to the email header, alongside with the clear text string and the name (selector) of the DKIM entry in DNS.

The receiving email server…
a, takes sending domain and DKIM selector from email header
b, looks up the public key stored in DNS of sending domain
c, uses the public DKIM key to decrypt the encrypted string in the email header

If the decrypted string matches the clear-text one, DKIM verification is successful

If the above is correct, why would I need domain individual DKIM keys? Why can't all domains hosted on the same server share the same public/private DKIM keys? In fact if DKIM is supposed to prove the validity of the sending server, wouldn't it even be more accurate to use the same key pair for all domains given that it is the same server for all of them?
From a technical point of view I think both concepts will work, with domain specific DKIM keys representing a little more work as the number of domains grows.
I'd be happy to go with individual key pairs, but I'd like to understand why.
Any benefits/downsides of going either way?
Anyone got an opinion?
Comment: Didn't get any additional input aside from what's here. We eventually went with individual key pairs
Comment: Have you come to a conclusion for this question? I also would like to know the benefits/downside of the domains each having their own DKIM entry in the DNS record vs having a "one for all" entry.
Answer: I see value in the system being capable of using different DKIM keys for different domains in case the DKIM keys have to be revoked for one domain without affecting the other domains. This can be desirable if a signed message is sent to many new recipients as part of a replay/spam attack (see the sections 8.6 and 8.7 of the RFC). However, you can also revoke the DKIM key of a single domain when all domains start out with the same key – as long as your system is able to handle this. I see no benefit (but also no downside) in starting with/enforcing different DKIM keys per domain.
(One could make an argument for increased privacy, but the IP address of your server leaks through other message headers anyway (unless your setup is designed to avoid this). This means that one can tell that the same server handles the outgoing messages of several domains even if these domains use different DKIM keys.)


how to use BETWEEN mysql function in codeigniter to get data between two dates

Question: <code> $this->db->SELECT('*')
          ->WHERE('VendorId',5)
          ->WHERE("Date BETWEEN CAST(2014-08-04 AS Date) AND CAST(2014-08-04 AS Date)")
          ->FROM('report1');

       $query = $this->db->get();
       return  $query->result();
</code>
I am getting a blank page with that query.Please any help.I am using Codeigniter
Answer: i think you are missing ' to wrap date value
<code>WHERE("Date BETWEEN CAST('2014-08-04' AS Date) ...
</code>
check this .


SQL Improvement in PostgreSQL

Question: I have two tables (Order, OrderStatus) as below. Order.OrderCode and OrderStatus.Status are indexed.
Order table:
Id(PK),OrderCode
1,"AA-001"
2,"AA-001"
3,"AA-002"
4,"AA-002"
5,"AA-003"

OrderStatus table:
Id(PK),OrderId(FK),Status,CreatedAt
1,1,"Open",2021-05-01 13:00:00
2,1,"Close",2021-05-01 13:05:00
3,2,"Open",2021-05-01 13:10:01
4,3,"Open",2021-05-01 13:10:02
5,3,"Close",2021-05-01 13:11:00
6,4,"Open",2021-05-01 13:11:01
7,4,"Close",2021-05-01 13:11:05
8,5,"Open",2021-05-01 13:12:00

I have a query to return the total Order count for any OrderCode group which has a latest Open status.
For example OrderCode AA-001 and OrderCode AA-003 have Order (ID:2,5) who have latest Open status. The query result would be 3.
OrderCode,Count
"AA-001",2
"AA-003",1

This is the query I have now. The inner query in the IN condition is basically to find Order record whose latest Status is Open. Is there any other way to improve this query? Thanks.
SELECT COUNT(1)
FROM Order as ord
WHERE ord.OrderCode in (
   SELECT distinct(or.OrderCode)
   from Order as or
   left JOIN (
      SELECT DISTINCT ON (OrderId) *
      FROM OrderStatus
      ORDER BY OrderId, CreatedAt DESC
   ) orsts ON orsts.OrderId = or.Id
   where orsts.Status = 'Open'
)

Comment: Unrelated, but: `count(1)` is actually slower than `count(*)` if you expected that to do some magic performance improvement ( `count(1)` is never faster than `count(*)` in any database). Also: `distinct` is not a function. It always applies to all columns in the select list. Enclosing a column in parentheses won't change anything
Comment: Do you want a result per order_code? So `AA-001, 2` and `AA-002, 2`? Or a total count of all orders that have at least one "latest" status in `open`? Please **[edit]** your question and add the expected result based on your sample data.
Comment: Latter case. I want a total count of all orders that have at least one latest status as Open.
Comment: I don't understand why `AA-001` has a count of 2. The latest status of order_id = 1 is `Close` not `Open`. Only for order_id = 2 the latest status is `Open`
Comment: `AA-001` has count of 2 because one of its Order (ID:2) is Open. Even though Order.ID:1 has status Close, I need it to be part of the result.
Answer: You seem to be describing:
SELECT COUNT(DISTINCT o.OrderCode)
FROM Order o JOIN
     (SELECT DISTINCT ON (os.OrderId) *
      FROM OrderStatus os
      ORDER BY os.OrderId, os.CreatedAt DESC
     ) orsts
     ON orsts.OrderId = o.Id
WHERE orsts.Status = 'Open';

For performance, you want an index on OrderStatus(OrderId, CreatedAt DESC).

SQL Improvement in PostgreSQL


Access Gravity Forms fields on previous page (before pre-submit)

Question: Here's the sitch: I'm using a multi-page Gravity Form in conjunction with an external API.
On the first page of the form, the user supplies a phone number. On a subsequent page, I need to send the external API the phone number in order to retrieve the user's current settings, which then must be prepopulated in other fields.
I know how to use gform_post_paging, but since $entry hasn't been created yet I can't use it to pull fields from previous pages.
gform_pre_submission/gform_after_submission don't help me because I need to make the API call prior to users reaching the end of the form.
I've also tried handling this via jQuery, by pulling and storing the value of the phone field on page advancement (e.g. var phoneNum = jQuery('#input_2_25').attr('value');), but I get "undefined" no matter what when using field ids. (It works fine, in the same location, looking for other ids on the site, so the issue isn't with jQuery.) I'm guessing this means that between pages, the form elements don't exist in the view?
How can I accomplish this goal (i.e., retrieving a value from a previous page in a multi-page Gravity Form prior to the user reaching the end of the form)?
Answer: All of the data is stored in the $_POST variable on each page submission. Gravity Forms has a helper function for accessing $POST data. The format for the variable name will be 'input{fieldId}'.
$value = rgpost( 'input_1' ); // replace "1" with your field ID

Comment: Many thanks. Didn't realize that got updated prior to the full form submit.
Comment: Follow-up issue: I've got the data I need, but can't figure out a way to get the information into fields on subsequent pages. gform_field_value seems only to work on initial form load, and I've also tried getting the dang values into a query string to dynamically populate that way, but since this is all being done in functions.php I can't seem to find a reliable way to refresh the page. Any thoughts?
Comment: Here is a snippet that does the trick: https://gist.github.com/spivurno/cee0e1c372bf988019e1

I have a plugin that provides the ability to use merge tags in the default value field so that you can populate previous field values ([example](http://grab.by/M1y0). It's called [GP Preview Submission](http://gravitywiz.com/documentation/gp-preview-submission/).


Android - Conexão com PostgreSQL

Question: Estou tentando conectar o Android diretamente com PostgreSQL.
Incluir no pacote do projeto: postgresql-9.1-903.jdbc4.jar e estou usando PostgreSQL 9.1.
Porém não consigo realizar a conexão com sucesso.
Modifiquei o código e ele me retorna:

"Erro: Driver!"

Estou usando JDBC: postgresql-9.1-903.jdbc4.jar.

Modifiquei o código e está dando erro:

Conexão: Erro Database!

Segue o código:
// Conectar
public String conectarDB() {

    // Variáveis
    String driver = "org.postgresql.Driver";
    String dbURL = "jdbc:postgresql://localhost:5432/database";
    String user = "postgres";
    String pass = "123456";

    try{

        //Carrega o driver
        Class.forName(driver);
        Connection conn = DriverManager.getConnection(dbURL, user, pass);
        return "Conexão: Ok!";

    } catch(ClassNotFoundException e) {
        e.printStackTrace();
        return "Conexão: Erro Driver!";

    } catch (SQLException e) {
        e.printStackTrace();
        return "Conexão: Erro Database!";
    }

}

Comment: Ocorre algum erro? tem log?
Comment: Como faço para vê o log e lhe enviar?
Comment: Adicione isto ao **catch(Exception e) { e.printStackTrace(); return false;}** e verifique a aba inferior **Android Monitror**
Comment: Não estou utilizando o Android Studio é muito pesado para máquina que estou usando. Mesmo sem PostgreSQL não é exibo erro algum nem mesmo com  } catch(Exception e) { e.printStackTrace();  return false;
  }
Answer: Há necessidade de carregar o Driver antes de se conectar, segundo esta documentação.
Tente da seguinte forma:
 public boolean conectarDB() {

        try{
            //Carrega o driver
            Class.forName("org.postgresql.Driver");
            String dbURL = "jdbc:postgresql://localhost:5432/database";
            String user = "postgres";
            String pass = "123456";
            Connection conn = DriverManager.getConnection(dbURL, user, pass);

            if (conn != null) {
                return true;
            } else {
                return true;
            }

        } catch(Exception e){
            e.printStackTrace();
            return false;
        }

    }

Comment: Já incluir. Infelizmente o erro continua. Obrigado!

Android - Conexão com PostgreSQL


Double sum subtraction with one missing term

Question: If I have some sum and subtract another why is the following relation true?
$$\sum_{h}^{n}  \sum_{i}^{n} f(h,i) - \sum_{h \neq \theta}^{n}  \sum_{i \neq \theta}^{n} f(h,i) =  \sum_{i}^{n} f(\theta,i) $$
Comment: Your notation is not very clear, which I think may be contributing to the confusion. What is meant by $\sum_h^n$? Does this mean $\sum_{h=1}^n$? What is mean by $\sum_{h \neq \theta}^n$?
Comment: Sorry. The first question yes. And the second means summing over everything but 1 term which we call $\theta $.
Answer: Let $[n] = \{1, 2, \ldots, n\}, \theta \in [n]$ and $N_\theta = [n] \setminus \{\theta\}$.
I am assuming

$\sum_h^n$ is really $\sum_{h=1}^n$ and
$\sum_{h \ne \theta}^n$ is really $\sum_{h \in N_\theta}$

Then,

the first sum on the left includes all indices $(h,i)$ where $h,i \in [n]$
the second sum on the left includes all indices $(h,i)$ where $h,i \in N_\theta$

The difference includes all terms where $h=\theta$ or $i = \theta$, i.e.
$$
\sum_{h\ne \theta} f(h, \theta) + \sum_{i\ne \theta} f(\theta, i) + f(\theta,\theta)
= \sum_{h\ne \theta} f(h, \theta) + \sum_i^n f(\theta, i)
$$

A slightly modified version of your expression would be true:
$$\sum_{h}^{n}  \sum_{i}^{n} f(h,i) - \sum_{h \neq \theta}^{n}  \sum_{i}^n f(h,i) =  \sum_{i}^{n} f(\theta,i) $$


TypeError: Cannot read property 'image' of null when Firebase is removed from app

Question: My react app runs on Firebase Realtime databases, which I want to remove. But, whenever I do that, I get the following error
TypeError: Cannot read property 'image' of null
Fish.render
src/components/Fish.js:22
  19 |   this.props.addToOrder(this.props.index);
  20 | };
  21 |
> 22 | render() {
  23 |   const { image, name, price, desc, status } = this.props.details;
  24 |   const isAvailable = status === "available";
  25 |   return (

Here's Fish.js

import React from "react";
import { formatPrice } from "../helpers.js";

class Fish extends React.Component {
  handleClick = () => {
    this.props.addToOrder(this.props.index);
  };

  render() {
    const { image, name, price, desc, status } = this.props.details;
    const isAvailable = status === "available";
    return (

          {name}
          {formatPrice(price)}

{desc}

        <button onClick={this.handleClick} disabled={!isAvailable}>
          {isAvailable ? "Add to Order" : "Sold out !!"}
        </button>

    );
  }
}
export default Fish;

The error pops in when I delete a fish inside the store I have made, with firebase it gets deleted in App.js file. Without it, this error comes. Please help.

import React from "react";
import Header from "./Header.js";
import Order from "./Order.js";
import Inventory from "./Inventory.js";
import sampleFishes from "../sample-fishes.js";
import Fish from "./Fish.js";
// import base from "../base.js";

class App extends React.Component {
  state = {
    fishes: {},
    order: {}
  };

  componentDidMount() {
    const localStorageRef = localStorage.getItem("/cotd");
    if (localStorageRef) {
      this.setState({ order: JSON.parse(localStorageRef) });
    }

    // this.ref = base.syncState(`/cotd/fishes`, {
    //   context: this,
    //   state: "fishes"
    // });
  }

  componentDidUpdate() {
    localStorage.setItem("/cotd", JSON.stringify(this.state.order));
  }

  componentWillUnmount() {
    // base.removeBinding(this.ref);
  }

  addFish = fish => {
    // Taking a copy of the fishes
    const fishes = { ...this.state.fishes };
    // Adding out new fish to fishes variable
    fishes[`fish${Date.now()}`] = fish;
    // New fishes object to state
    this.setState({ fishes });
  };

  updateFish = (key, updateFish) => {
    const fishes = { ...this.state.fishes };
    fishes[key] = updateFish;
    this.setState({ fishes });
  };

  deleteFish = key => {
    const fishes = { ...this.state.fishes };
    fishes[key] = null;
    this.setState({ fishes });
  };

  //  Feature Additon - Reset the order
  resetOrder = key => {
    var order = { ...this.state.order };
    delete order[key];
    this.setState({ order });
  };

  loadSampleFishes = () => {
    this.setState({ fishes: sampleFishes });
  };

  addToOrder = key => {
    const order = { ...this.state.order };
    order[key] = order[key] + 1 || 1;
    this.setState({ order });
  };

  render() {
    return (

            {Object.keys(this.state.fishes).map(key => (
              <Fish
                key={key}
                index={key}
                details={this.state.fishes[key]}
                addToOrder={this.addToOrder}
              />
            ))}

        <Order
          fishes={this.state.fishes}
          order={this.state.order}
          resetOrder={this.resetOrder}
        />
        <Inventory
          addFish={this.addFish}
          updateFish={this.updateFish}
          loadSampleFishes={this.loadSampleFishes}
          deleteFish={this.deleteFish}
          fishes={this.state.fishes}
        />

    );
  }
}

export default App;

The application uses Firebase to presist the state.
Answer: It happens because before your variable image is  null so when it can't render the component:
What you should do is put the initial value to image instead of null
const image = this.props.details.image? this.props.details.image : '';

Hope it helps!!!
Comment: Hello Tony, though the logic of your answer seems sound enough. It doesn't help in fixing the problem. The error is still there. Anything else you can think of or need to look at the entire codebase then you can ping me here. Thanks for trying. Much Appreciated.
Answer: When u deleted a fish u set the  value of that key to null in the local state , that is passed as a prop to fish component as null. Now your child component only gets the null value for details and {image,description,name}= this.props.details.
All restructured variables recieve null.
Comment: Yes, I understand the problem. Any solution you can suggest for the issue?


Dynamically load the menu items created from the links.task.yml file

Question: I need to programmatically load the links that are generated from  my_module.links.task.yml on another page.
My exact use case is that I have a set of links that are shown when viewing my node page. The links will be shown and hidden based on the current user's access and the access control is handled through the options in the my_module.routing.yml file.
I need to create a separate page that displays node data in a table and I need to show these links in a drop-down option (similar to how the edit button functions when viewing content as an admin).
Is there a way to dynamically get the links so I can then display them in that drop-down?
Answer: You can load all module defined menu links (i.e. links defined in *.menu.links.yml with the MenuLinkManager service's getDefinitions() method and then parse the list for only links defined by your module.
$all_module_link_definitions = \Drupal::service('plugin.manager.menu.link')
  ->getDefinitions();

// Next parse the list for only links defined by my_module.
$my_module_link_definitions = [];
foreach ($all_module_link_definitions as $plugin_id => $plugin_definition) {
  if ($plugin_definition['provider'] === 'MY_MODULE') {
    $my_module_link_definitions[$plugin_id] => $plugin_definition;
  }
}

Similarly, you can load all module defined local tasks with the LocalTaskManager service's getDefintions() method and then parse the list for only tasks defined by your module.
$all_module_task_definitions = \Drupal::service('plugin.manager.menu.local_task')
  ->getDefinitions();

// Next parse the list for only tasks defined by my_module.
$my_module_task_definitions = [];
foreach ($all_module_task_definitions as $plugin_id => $plugin_definition) {
  if ($plugin_definition['provider'] === 'MY_MODULE') {
    $my_module_task_definitions[$plugin_id] => $plugin_definition;
  }
}


Explain the statemets related to separable spaces.

Question:
Separable spaces are "simpler" than nonseparable ones-KREYSZIG
Separable space $X$ is not "too big"  in the sense that we can approach each element of space $X$ through a sequence of elements of a countable set-PONNUSAMY

I'm reading the article of Dense subsets and separability from different texts.I got the above statements which I'm not getting.
1.What notions are involved in non-separable spaces so that they are complex than Separable ones?
2.What is the meaning  of "too big" in the second statement?
Answer: A separable space is a space with a countable dense subset. So for a non-separable space, for any countable subset, its closure will not be the entire space.
For example $l_{\infty}$ is non-separable because the open balls of radius $\frac{1}{2}$ centred at the elements with only 0 and 1 entries form an uncountable collection and any dense set would have to contain an element of each of these balls. This in particular proves that $R$ cannot continuously surject onto $l_{\infty}$. Intuitively this makes $l_{\infty}$ look rather large.
Having said all these, remember that these are just intuitions in a broad sense and things that are meant to sell the idea of non-separable to you as a new and mostly inexperienced reader. After all, $\mathbb{R}$ and $l_{\infty}$ have the same cardinality so $l_{\infty}$ isn't really larger than $\mathbb{R}$ in this sense. An experienced mathematician doesn't really use these intuitions in proofs or reasonings.
Answer: Re: 2.
Not all topologies can be described in terms of sequences. The definition of a topology $T$ on a set $X$ is short and  not very restrictive. An important class of topological spaces is the class of sequential spaces, in which the closure of a set can be described in terms of convergent sequences. This class includes all metric spaces.
If $X$ is a sequential space and $D$ is  countable dense subset of $X,$ let $E$ be the set of all sequences of members of $D.$ The cardinal $|E|$ of $E$ is at most $2^{\aleph_0}=|\Bbb R|.$ Let $F$ be the subset of $E$ that contains only the convergent sequences. For any sequence $f\in F$ let $\psi(f)$ be the limit of the sequence $f.$ Then, since $D$ is dense in $X,$ we have $X=\{\psi (f):f\in F\}$. Therefore $|X|=|\{\psi(f):f\in F\}|\le |F|\le |E|\le |\Bbb R|.$ So $X$ is not "too big" in terms of its cardinality.
On the other hand if the topology $T$ on $X$ is $T=\{X,\emptyset\}$ then any non-empty $D\subset X$ is dense, so $X$ is separable. But $X$ can be any set so $|X|$ can be any cardinal. And if $X$ has at least $2$ members then $X$ is not a sequential space.
BTW. A topological definition of "$(x_n)_{n\in \Bbb N}$ converges to $x$" is: $\{x\}=\cap_{n\in \Bbb N}\overline {S_n},$ where $S_n=\{x_j:j\ge n\}.$


Como utilizar argc e argv em um terminal windows?

Question: Estou criando uma pilha para verificação de expressão válida, porém deve-se utilizar nesta int main( int argc, char **argv ) para entrada. Compilei e não deu erro algum, porém não consigo criar um arquivo executável através do prompt. E já forcei a criação de um através do devc++, mas ao fazer o teste no terminal o programa para de funcionar. Já tentei colocar o argumento entre aspas simples mas o programa também é morto dessa forma.
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #define TNOME 100

    struct nodoPilha {
        char nome [TNOME];
        struct nodoPilha *proximoPtr;
    };

    typedef struct nodoPilha NodoPilha;
    typedef NodoPilha *NodoPilhaPtr;

    void imprimePilha( NodoPilhaPtr atualPtr );
    int ehVazia( NodoPilhaPtr headPtr );
    void pop( NodoPilhaPtr *headPtr );
    void push( NodoPilhaPtr *headPtr, char nome[TNOME] );

    int main( int argc, char **argv ){
        NodoPilhaPtr headPtr = NULL;
        int h;
        char *aux;
        int i = 1;
        while (i < argc){
            i++;
            h = ehVazia(headPtr);
            if (h){
                push (&headPtr, argv[i]);
                aux = argv [i];
            }
            else {
                if (aux != argv [i]){
                    pop (&headPtr);
                    char aux;
                }
                else {
                    push (&headPtr, argv [i]);
                }
            }
        }
        if (h){
            printf ("Valida");
        }
        else {
            printf ("Nao Valida");
        }

    }

    void push( NodoPilhaPtr *headPtr, char nome[TNOME] ){
        char * novalinha = strchr(nome, '\n');
        if (novalinha)
            *novalinha = '\0';

        NodoPilhaPtr newPtr;
        newPtr=malloc(sizeof(NodoPilha));

        if ( newPtr != NULL ) {
            newPtr->proximoPtr = *headPtr;
            strcpy( newPtr->nome,nome);

            *headPtr = newPtr;

        }
        else {
            printf( "%c nao foi inserido. Memoria nao foi disponibilizada.\n", nome );
        }
    }
    void pop( NodoPilhaPtr *headPtr ) {
        NodoPilhaPtr tempPtr;

        tempPtr = *headPtr;
        *headPtr = ( *headPtr )->proximoPtr;
        free( tempPtr );
    }
    int ehVazia( NodoPilhaPtr headPtr ){
        return headPtr == NULL;
    }
    void imprimePilha( NodoPilhaPtr atualPtr ){
        NodoPilhaPtr inicioPtr;
        inicioPtr=atualPtr;
        if ( atualPtr == NULL ) {
            puts( "Pilha esta vazia.\n" );
        }
        else {
            puts( "A pilha eh:" );
            while ( atualPtr != NULL ) {
                printf( "%s --> ", atualPtr->nome );
                atualPtr = atualPtr->proximoPtr;
            }
            puts( "NULL\n" );

        }
    }

Comment: Como você está fazendo a chamada? Ocorre algum erro?
Answer: O argc (argument count) é um inteiro e possui o número de argumentos com os quais a função main() foi chamada na linha de comando.
O argv (argument values) é um vetor de strings. Cada string deste vetor é um dos parâmetros da linha de comando. É para saber quantos elementos temos em argv que temos argc.
Exemplo: O programa a seguir faz uso dos parâmentros argv e argc. O programa
recebe da linha de comando o dia, mês e ano correntes, e imprimi a data em formato apropriado.
Veja o exemplo, supondo que o executável se chame data:
data 26 03 88

O sistema terá a seguinte saída:
26 de Março de 1.988

#include <stdio.h>
int main(int argc, char *argv[])
{
    int mes;
    char *nomemes [] = {"Janeiro", "Fevereiro", "Março", "Abril", "Maio", "Junho",
                        "Julho", "Agosto", "Setembro", "Outubro", "Novembro", "Dezembro"};
    if(argc == 4)
    {
        mes = atoi(argv[2]);

        if (mes<1 || mes>12)
            printf("Erro!\nMes invalido!");
        else
            printf("\n%s de %s de 20%s", argv[1],

        nomemes[mes- 1], argv[3]);
    } else
        printf("Erro!\nUso: data dia mes ano, todos inteiros");
    }

Fonte: www.univasf.edu.br/~marcelo.linder/arquivos_pc/aulas/aula19.pdf
Answer: Você parece que quer criar um verificador de expressões aritméticas, certo? O código de fato compila, e compilando gera um executável. O problema é que esse executável, ao receber uma expressão válida como a + b, gera um erro de proteção.
Me parece que o erro é gerado pelo incremento da variável i na linha 29: quando o programa entra no loop da linha 28, i já é 1, e imediatamente incrementando, você faz com que o código ignore o primeiro argumento, fazendo com que o programa enxergue só + b; além disso, da última vez que ele entrar o loop, o i será incrementado para ficar igual a argc e o sistema tentará empilhar argv[argc] que, segundo o padrão C, é NULL; ao tentarmos acessá-lo tomamos o erro.
Além disso, o código ainda não efetivamente faz verificação nenhuma, só empilha o primeiro argumento, o desempilha quando vê o segundo, empilha o terceiro, e por aí vai, até encontrar o fim dos argumentos e então retorna "Válida" ou "Não Válida" de acordo com se o programa recebeu um número ímpar ou par de argumentos.
Comment: O problema realmente estava na linha 28, o código eu terminei e fiz a modificação.. obg


Using panels and panelizer with i18n

Question: I'm building a multilingual site with 4 different languages, using panels and panelizer, but am struggling to see how to have different panels for different languages.
The site has to be content-manageable by the client, so I need to provide panels that can be translated by them.
I've tried 2 routes - first of all using the i18n_panels submodule in panels. The readme.txt file suggests that I can create a custom pane which can be translated, which would be great, but it isn't very clear how to do this. The example doesn't work.
I've also tried adding a pane for each language and using visibility rules based on the node language. This doesn't work either.
Does anyone have an example of a custom pane that is multilingual using i18n_panels, or another way of achieving the same effect?
Answer: I couldn't get the i18n_panels module to work, but the Multilingual panels module works a treat.
https://www.drupal.org/project/mlpanels

Using panels and panelizer with i18n


Why Cant add a class to an element by jquery

Question: In the page.tpl.php I have some elements like

I have linked js file to the page and it is correctly works since all functions operate based on click attributes. but I would like to mudify some elements in page.tpl.php page. for example in my JS file I have a code like this:
$(".play").attr("id", "myID");

I could not make it working. basically it does not add myID to .play.
Any idea?
Comment: Did you wrap your javascript as documented here: [Managing JavaScript in Drupal 7](https://www.drupal.org/node/756722#using-jquery)
Answer: Going on little information provided its probably because you cannot access jQuery with the $ by default in drupal.  This is to prevent other library conflicts (see jQuery.noConflict())  If this is the case you will probably be getting an undefined function error in your console.  To fix this you can either use jQuery or wrap it in a function like this:
(function($){
  $(".play").attr("id", "myID");
})(jQuery);

or can do this:
jQuery(".play").attr("id", "myID");

By default the version of jQuery that is used on drupal is 1.5.  You may consider using jQuery Update module for a newer version.
"Many JavaScript libraries use $ as a function or variable name, just as jQuery does. In jQuery's case, $ is just an alias for jQuery, so all functionality is available without using $. If you need to use another JavaScript library alongside jQuery, return control of $ back to the other library with a call to $.noConflict(). Old references of $ are saved during jQuery initialization; noConflict() simply restores them."


Why is Testing Error Spiking late in the training process?

Question: I'm training a FCN on 550K datapoints (90/10 train-test split) and tracking training error, testing error, and actual MAE (un-z-scored true error project cares about) over each epoch. Below is plots for each:

The testing error + MAE jump from decent to incredibly off at around Epoch 27. Training loss seems to be generally decreasing.
Parameters used:
<code>    train_set = Dataset(train_data,train_labels) # Dataset class just holds data/labels together
    train_sampler = RandomSampler(train_set)

    args = Params(batch_size=861, epochs=200, lr=0.01, momentum=0.5, seed=1, cuda=True, log_interval=5)
    train_loader = DataLoader(train_set, batch_size=args.batch_size, shuffle=None, sampler=train_sampler)
</code>
Also using z-scoring for regularization, early stopping, mse_loss on training/testing loss. Model is 7 layer FCN:
<code>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(52, 100)
        self.fc2 = nn.Linear(100, 100)
        self.fc3 = nn.Linear(100, 100)
        self.fc4 = nn.Linear(100, 100)
        self.fc5 = nn.Linear(100, 100)
        self.fc6 = nn.Linear(100, 50)
        self.fc7 = nn.Linear(50, 1)

        self.bn1 = nn.BatchNorm1d(100)
        self.bn2 = nn.BatchNorm1d(100)
        self.bn3 = nn.BatchNorm1d(100)
        self.bn4 = nn.BatchNorm1d(100)
        self.bn5 = nn.BatchNorm1d(100)
        self.bn6 = nn.BatchNorm1d(50)

    def forward(self, x):
        x = self.bn1(Func.relu(self.fc1(x)))
        x = self.bn2(Func.relu(self.fc2(x)))
        x = self.bn3(Func.relu(self.fc3(x)))
        x = self.bn4(Func.relu(self.fc4(x)))
        x = self.bn5(Func.relu(self.fc5(x)))
        x = self.bn6(Func.relu(self.fc6(x)))
        x = self.fc7(x)
        return x
</code>
First thought is overfitting but training loss doesn't seem to be too dramatic. In addition, output on test set is almost all (95%) the same value.
Any ideas what would cause such incredible spikes later in the training process?
Comment: What definition of overfitting are you using? Do your observations of wildly increasing holdout loss fit that definition? Why or why not? (As an aside, you may wish to learn more about the `nn.Sequential` class in pytorch.)
Comment: @SycoraxsaysReinstateMonica by overfitting I simply mean that the model memorizes the training set to a point that testing loss suffers. I don't see that happening because typically you would see training loss start to taper off and testing loss gradually/rapidly increase simultaneously. In this case is seems to single epochs of huge test loss late in training.

Thanks for referencing `nn.Sequential`! Have used it previously
Comment: I didn't notice before that the scale of the test axis is measured in $10^{11}$. This suggests to me that for some reason there's a division-by-0 somewhere, possibly because the batch norm denominator has vanished. It's possible to get a unit with 0 variance: it suggests a dead relu for the test data but not the train data.
Answer: It makes sense that once 27 epochs have got completed some features are extracted which suddenly generalizes the model better which make the test loss suddenly to spike up.
Comment: I do understand that,but this is the one kind of reasoning which is more convincing as neural network is a black box.
Comment: Thanks for the response! I would understand if test loss spiked some amount as new features are extracted-- even an order of magnitude would be fine in some cases. But the loss is spiking from near zero the epoch before ,~0.002, to 1.6e11, and back down to near 0. That's a spike 10 trillion times larger as the previous epic. It's inconceivable that a spike of that magnitude is due to feature extraction at epoch 27


Conditional entropy of sum of random variables

Question: How can be proven that for random variables $A$ and $B$, and $C = A + B$,
$$H(C\mid A) = H(B\mid A).$$
Also, would it be possible to determine if $H(C)$ would be greater than $H(A)$?
Comment: Differential or Shannon entropy, or some other type of entropy?
Comment: Seems roughly equivalent to showing $H(A,A+B)=H(A,B)$.
Comment: Is the fact that the operation is addition important? Should this statement also hold: $H(A,AB)=H(A,B)$, under suitable conditions? Or could it more generally be $H(A,f(A,B))=H(A,B)$?
Answer: I guess $A$ and $B$ are discrete random variables. Since $C$ is a function of $A$ and $B$ one has $H(C \mid A) \leq H(A,B \mid A)$ (because this is even true that  $H(C \mid A=a) \leq H(A,B \mid A=a)$ for every $a$). Moreover $H(A,B \mid A) = H(B \mid A)$ because it is easy to check that $H(A,B \mid A=a) = H(B \mid A=a)$ for every $a$. Finally $H(C \mid A) \leq H(B \mid A)$ and one similarly gets the converse inequality.
For your second question: no, take $B=-A$ then $H(C)=0$. I have not tried to check whether this is true under the independence assumption of $A$ and $B$.


URLSearchParams.set() without URIEncoding

Question: URLSearchParams.set(key, value) URIEncodes the values its given, producing ugly non-specified urls.
The following test is based off this list of url friendly characters

const url = new URL("http://www.example.com/path");
const test = "abc123+-_$#%?@,"
url.searchParams.set("foo", test);

console.log(`What foo should be: ${test}`);
console.log(`What foo is: ${url.search}`)

Is there a way to use URLSearchParams from URL.searchParams to update a search param to the value given and not an encoding of it?
Comment: The reason a UX developer would want to have readable URLs is so users can read them.  And while you are correct that the `&` symbol should be avoided, `#` `?` and `=` can be used without issue.   Give it a try!
Comment: Using URLS as a primary user interface is indeed silly, but treating them as non-user facing is also not a great experience.  I've tested use of `#` and had no issues.  It only registers as a hash if it's at the end.  Try it out.
Comment: Why would you want to do that? If `test` was something like `bar&baz=bla` doing it the way you want would create both a `bar` and a `baz` param. Also note that `#` has to be converted no matter what, or it will be considered as the URL's hash, and if you need to decode this character, the others would also need to be encoded or you won't be able to tell apart the original sequence `%23` from the original `#` that got encoded to `%23`.
Comment: And why a UX developper would use an URLSearchParam to display something to their users? If you want to show them the content of the POST params show just that, the URL is for the server to read. And `#` is the one that's not ok, look at your answer's result.
Comment: No `#` is the hash marker anywhere. Your own examples in your own answer to your own question demonstrate that: the output of the first snippet is `What foo is: ?foo=abc123+-_$%23%?@,`, the `#` character has been converted to `%23`.  Because the `.search` setter srill does encode this char, because it would be defining the hash otherwise. If you want to show something "friendly" to your users don't show the URL, show each components on their own, decoded.
Answer: If one wants all their searchParams to be unencoded, they can just set url.search to an unencoded version of itself:

const url = new URL("http://www.example.com/path");
const test = "abc123+-_$#%?@,"
url.searchParams.set("foo", test)
url.search = decodeURIComponent(url.search)

console.log(`What foo should be: ${test}`);
console.log(`What foo is: ${url.search}`)

If one only wants the specified key to be unencoded, it takes some extra

const testUrl = new URL("http://www.example.com/path");
testUrl.searchParams.set("bar", "++ ++");

const testVal = "abc123+-_$#%?@,";
setValueUnencoded(testUrl, "foo", testVal);

console.log(`What foo should be: ${testVal}`);
console.log(`What foo is: ${testUrl.search}`)

function setValueUnencoded(url, key, value) {
  url.searchParams.set(key, value);
  const entries = Array.from(url.searchParams.entries());
  url.search = "";
  entries.forEach(item => {
    if (item[0] === key) { return; }
    url.searchParams.set(item[0], item[1]);
  });
  url.search += `${key}=${value}`;
}
Comment: The `.set` method replaces spaces with "+" instead of " " for some reason tho
Comment: Yeah if you put a space after the + in test variable in your first snippet, it does not work.
But a space after the + in your second snippet in testVal it works.
How to make spaces encoded with %20 ?
[SO-encoding-spaces](https://stackoverflow.com/q/1634271/1065953)


$\operatorname{Var}[\sum_{i=1}^n (X_i-\mu_0)^2]=\operatorname{Var}[ \sum_{i=1}^n(X_i-\overline{X})^2]$?

Question: Consider independent $X_1,\ldots, X_n\sim\mathcal{N}(\mu_0,\sigma^2)$ with a known $\mu_0\in\mathbb{R}$ and unknown $\sigma^2\in(0,\infty)$. I already know that
$$\DeclareMathOperator{\Var}{Var}\frac{1}{n}\sum_{i=1}^n (X_i-\mu_0)^2$$
is variance minimising and that
$$E\Big[\frac{1}{n}\sum_{i=1}^n (X_i-\mu_0)^2\Big]=E\Big[\frac{1}{n-1} \sum_{i=1}^n(X_i-\overline{X})^2\Big]=\sigma^2$$
What I want to show is that
$$\frac{1}{n^2}\Var[\sum_{i=1}^n (X_i-\mu_0)^2]=\Var[\frac{1}{n}\sum_{i=1}^n (X_i-\mu_0)^2]<\Var[\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X})^2]=\frac{1}{(n-1)^2}\Var[\sum_{i=1}^n(X_i-\overline{X})^2]$$
Since the first moments are the same it would be enough, to know, that
$$E\Big[\Big(\sum_{i=1}^n (X_i-\mu_0)^2\Big)^2\Big]=E\Big[\Big(\sum_{i=1}^n(X_i-\overline{X})^2\Big)^2\Big]$$
Is this true? There might be an easy argument, but I do not find it.
Thanks in advance!
Answer: Recall that if $X_1,X_2,...,X_n$ iid $N(\mu, \sigma ^ 2)$, then
$$ \DeclareMathOperator{\Var}{Var}
\frac{\sum ( X_i - \mu ) ^ 2}{ \sigma ^ 2} \sim \chi^2_n, \quad \frac{\sum ( X_i - \bar{X}_n ) ^ 2}{ \sigma ^ 2} \sim \chi^2_{n-1},
$$
and
$$
\Var(\chi^2_n)=2n, \quad \Var(\chi^2_{n-1})=2(n-1).
$$
Comment: Where $\overline{X}_n=\frac{1}{n}\sum_{i=1}^nX_i$?
Comment: @user408858 Right
Comment: So I get $Var[\sum_{i=1}^n (X_i-\mu_0)^2]= 2n\sigma^4$ and $Var[ \sum_{i=1}^n(X_i-\overline{X})^2]=2(n-1)\sigma^4$, right?
Comment: Yup, that is right


TypeError: Cannot read property 'instance' of undefined

Question: Here is the method I am trying to test:
showSnackbar(): void {
    if (this.modifiedReferences.length) {
      const snackbar = this.snackbarService.open({
        message: '',
        type: 'success',
        duration: 10000,
        actionText: 'SHOW DETAILS'
      });
      snackbar.instance.actionClicked.subscribe(() => {
        this.dialogService.modal(ModifiedComponent);
      });
    }
}

and here is the test:
  it('should call the showSnackbar method', () => {
    const showSnackbarSpy: jasmine.Spy = spyOn(TestBed.inject(SnackbarService), 'open');

    component.showSnackbar();

    expect(showSnackbarSpy).toHaveBeenCalled();
  });

When I try to test it displays zn error saying that it can't read property 'instance' of undefined.
Answer: for me, adding .compileComponents() after my TestBed.configureTestingModule() resolved this issue.


Products list not seen while editing, using form model binding in laravel 5.1

Question: In my ecommerce project, I have Product and Carousel Model.
Product.php
<?php

namespace App;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\SoftDeletes;

class Product extends Model
{
    use SoftDeletes;

    protected $dates = ['deleted_at'];

    protected $fillable = [
        'code', 'name', 'description', 'special_note', 'sort', 'display', 'weight', 'enquiry'
    ];

    public function carousels()
    {
        return $this->belongsToMany('App\Carousel')->withTimestamps();
    }
}

Carousel.php
<?php

namespace App;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\SoftDeletes;

class Carousel extends Model
{
    use SoftDeletes;

    protected $dates = ['deleted_at'];

    protected $fillable = ['name', 'display', 'sort'];

    public function products()
    {
        return $this->belongsToMany('App\Product')->withTimestamps();
    }

    public function getProductListAttribute()
    {
        return $this->products->lists('id');
    }
}

Here's the controller:
public function create()
{
    $products = Product::lists('name', 'id');

    return view('admin.carousels.create', compact('products'));
}

public function store(Request $request)
{
    $carousel = Carousel::create($request->all());

    $carousel->products()->attach($request->input('product_list'));

    return redirect()->back();
}

public function edit($id)
{
    $carousel = Carousel::findOrFail($id);

    $products = Product::lists('name', 'id');

    return view('admin.carousels.edit', compact('carousel', 'products'));
}

public function update(Request $request)
{
    dd($request->all());
}

The carousel form:

        {!! Form::label('name', 'Name:') !!}
        {!! Form::text('name', null, ['class' => 'form-control input-sm', 'id' => 'name']) !!}

        {!! Form::label('sort', 'Sort:') !!}
        {!! Form::text('sort', null, ['class' => 'form-control input-sm', 'id' => 'sort']) !!}

        {!! Form::label('display', 'Display:') !!}
        {!! Form::select('display', ['Disabled' => 'Disabled', 'Enabled' => 'Enabled'], null, ['class' => 'form-control input-sm', 'id' => 'display']) !!}

        {!! Form::label('product_list', 'Products:') !!}
        {!! Form::select('product_list[]', $products, null, ['class' => 'form-control input-sm', 'multiple']) !!}

        {!! Form::submit($submitButtonText, ['class' => 'btn btn-primary btn-block m_top_15', 'id' => $submitButtonId]) !!}

Now, the problem is that, when I use form model binding to edit the carousel form, I get no product displayed in the select box, though I can see all the products to select from. The selected products are not seen for the particular carousel.
What is the mistake that I have made ? Kindly help me out.
Thanks.
P.S: I am using Laravel 5.1 and came across this issue for the first time. Earlier I used to do this in Laravel 5 without any issue.
Comment: Were you able to resolve this? Wondering if my answer regarding the changes to `list()` was the solution.
Answer: This may be related to how lists() is handled in 5.1. From the upgrade guide -

The lists Method
The lists method now returns a Collection instance instead of a plain
  array for Eloquent queries. If you would like to convert the
  Collection into a plain array, use the all method:
User::lists('id')->all(); Be aware that the Query Builder lists method
  still returns an array.


Configuring Postfix to not use a daemon

Question: I realize this is wildly offtopic but this is my last hope. I would like to only send mail from my computer, both locally and to remote servers, so I don't think I need postfix to run a daemon, but I can't find anything that will tell me how to configure it. Would anyone happen to know?
Comment: Can you explain why you do not want to run it as a daemon? It would help pick the correct answer.
Comment: "Why you do not want to run it as a daemon?" What about when you want to run it as a Docker process which doesnt quit?
Answer: Generally it's still useful to have an SMTP server running as a daemon - just make sure it only listens on the loopback (127.0.0.1) interface so that it doesn't accept mail from the outside world.
By running as a daemon, your Postfix installation can continue to manage your outgoing mail queue optimally (for those instances where an outgoing e-mail couldn't be delivered immediately).
Also, some software prefers to make a direct SMTP connection to port 25 on a machine somewhere rather than invoking /usr/sbin/sendmail and piping the mail to that program.  Running as a daemon is then preferable to having your SMTP service started on demand via inetd.
Comment: FYI, I had Postfix running on a 486 with 16MB of RAM at one point and it only used a small fraction of that, just a few hundered K.
Comment: Hmm, so you'd advise me to use the default setup? My machine only has an Atom processor and I don't want to bog it down with unused daemons, would postfix consume many resources running?
Comment: no, when it's not being used it should use negligible resources.  Mostly it'll use RAM in any case, not CPU.
Comment: If you've got a netbook or something with an Atom, you've probably got at least two or three orders of magnitude more memory than Postfix will use.  Don't worry about it.
Answer: Just comment out the line starting with 'smtp' in /etc/postfix/master.cf.
Answer: You may stop postfix service and remove it from startup services.
If you are using redhat you may use
chkconfig --level 2,3,4,5 postfix off

Configuring Postfix to not use a daemon


Add absence rows based on presence data

Question: I have presence data  in R for individuals at 4 different sites on different days. I would like to create rows for every individual ID that was NOT seen at each site on each day, with an additional presence/absence column.
This is a subset of my presence data:
ID    SP SITE  DATE SMP
975    1    1 41579 FJB
997    1    1 41579 FAW
114    1    1 41579 FJW
926    2    1 41609 FJB
101    2    1 41609 FJB
108    2    1 41609 FAB
129    3    2 41710 FUB
131    3    2 41710 MAW
132    3    2 41710 FAW

This is what I would like to create:
ID    SP SITE  DATE SMP Present?
975    1    1 41579 FJB   Yes
997    1    1 41579 FAW   Yes
114    1    1 41579 FJW   Yes
926    1    1 41579 FJB   No
101    1    1 41579 FJB   No
108    1    1 41579 FAB   No
129    1    1 41579 FUB   No
131    1    1 41579 MAW   No
132    1    1 41579 FAW   No
975    2    1 41609 FJB   No
997    2    1 41609 FAW   No
114    2    1 41609 FJW   No
926    2    1 41609 FJB   Yes
101    2    1 41609 FJB   Yes
108    2    1 41609 FAB   Yes
129    2    1 41609 FUB   No
131    2    1 41609 MAW   No
132    2    1 41609 FAW   No
975    3    2 41710 FJB   No
997    3    2 41710 FAW   No
114    3    2 41710 FJW   No
926    3    2 41710 FJB   No
101    3    2 41710 FJB   No
108    3    2 41710 FAB   No
129    3    2 41710 FUB   Yes
131    3    2 41710 MAW   Yes
132    3    2 41710 FAW   Yes

I hope someone may be able to help!
Answer: First, you need to prepare a data set that contains all possible combinations of ID, SP, SITE, DATE and SMP and I call it attendance_list.
library(tidyverse)
txt <- "ID    SP SITE  DATE SMP
975    1    1 41579 FJB
997    1    1 41579 FAW
114    1    1 41579 FJW
926    2    1 41609 FJB
101    2    1 41609 FJB
108    2    1 41609 FAB
129    3    2 41710 FUB
131    3    2 41710 MAW
132    3    2 41710 FAW"
presence_data <- txt %>%
  gsub(" +", " ", x=.) %>%
  read.delim(text=., sep=" ")

# Attendance list
## It looks like each ID has only one SMP
id_smp <- presence_data %>%
  select(ID, SMP) %>%
  distinct()
## Working site and date
sp_site_date <- presence_data %>%
  select(SP, SITE, DATE) %>%
  distinct()
attendance_list <- merge(id_smp, sp_site_date)

Then, you can left join the presence_data with a 'Yes' column to attendance_list, and replace 'NA' (Data in attendance_list but not in presence data) by 'No'.
attendance_list <- attendance_list %>%
  left_join(presence_data %>% mutate(PRESENT="Yes"), by=c("ID", "SMP", "SP", "SITE", "DATE")) %>%
  mutate(PRESENT = ifelse(is.na(PRESENT), "No", PRESENT))

Output:
> attendance_list
#     ID SMP SP SITE  DATE PRESENT
# 1  975 FJB  1    1 41579     Yes
# 2  997 FAW  1    1 41579     Yes
# 3  114 FJW  1    1 41579     Yes
# 4  926 FJB  1    1 41579      No
# 5  101 FJB  1    1 41579      No
# 6  108 FAB  1    1 41579      No
# 7  129 FUB  1    1 41579      No
# 8  131 MAW  1    1 41579      No
# 9  132 FAW  1    1 41579      No
# 10 975 FJB  2    1 41609      No
# 11 997 FAW  2    1 41609      No
# 12 114 FJW  2    1 41609      No
# 13 926 FJB  2    1 41609     Yes
# 14 101 FJB  2    1 41609     Yes
# 15 108 FAB  2    1 41609     Yes
# 16 129 FUB  2    1 41609      No
# 17 131 MAW  2    1 41609      No
# 18 132 FAW  2    1 41609      No
# 19 975 FJB  3    2 41710      No
# 20 997 FAW  3    2 41710      No
# 21 114 FJW  3    2 41710      No
# 22 926 FJB  3    2 41710      No
# 23 101 FJB  3    2 41710      No
# 24 108 FAB  3    2 41710      No
# 25 129 FUB  3    2 41710     Yes
# 26 131 MAW  3    2 41710     Yes
# 27 132 FAW  3    2 41710     Yes

Add absence rows based on presence data


Java Programming Error: Actual and formal argument lists differ in length

Question: All I am trying to do here is have the user inputted integer from the "displayMenu" method control the switch statement above in the main method. But I get the following error in this picture when I compile:
This is the error message I get when I compile
The error basically states that my actual and informal arguments differ in length.
Here is the relevant code:
public static void main(String[] args) {

    int userChoice = displayMenu();

    switch(userChoice) {
        case 1:
            break;
        case 2:
            break;
        case 3:
            break;
        case 4:
            break;
        case 5:
            break;
    }
}

public static int displayMenu(int userSelection) {
    Scanner keyboard = new Scanner(System.in);

    userSelection = keyboard.nextInt();
    return userSelection;
}

What does this error mean?
Comment: Use an IDE like IntelliJ or Eclipse, it will highlight such mistakes.
Answer: The error tells you exactly what is wrong: you're calling displayMenu without a parameter, and it requires one, an int to be precise.
Answer: You should remove the parameter from the displayMenu method.
Use
public static int displayMenu()

instead of
public static int displayMenu(int userSelection)


How to redirect the user to an external website with POST data?

Question: I want to implement an offsite payment method and everything is going well except the fact that I can't find how to redirect the user to the offsite payment gateway after submitting the cart checkout review
When the user submits the checkout review, the orderSubmit function is called. Inside this function I create the array with all the parameters required in the request, but I don't know how to send this request
public function orderSubmit(OrderInterface $order) {
    $fields = array(
      'version' => $this->configuration['version'],
      'TPE' => $this->configuration['tpe'],
      'date' => date('d/m/Y:H:i:s', time()),
      'montant' => round($order->getTotal(), 2).\Drupal::state()->get('uc_currency_code', 'USD'),
      'reference' => $order->id(),
      'texte-libre' => md5(uniqid(rand(), TRUE)),
      'mail' => $order->getEmail(),
      'lgue' => 'FR',
      'societe' => $this->configuration['company'],
      'url_retour' => Url::fromRoute('uc_cmic.cancel', array('absolute' => TRUE)),
      'url_retour_ok' => Url::fromRoute('uc_cmic.success', array('absolute' => TRUE)),
      'url_retour_err' => Url::fromRoute('uc_cmic.return', array('absolute' => TRUE)),
      'MAC' => "",
      'options' => ""
    );
    $fields = uc_cmcic_complete_request($this->configuration['security_key'], $fields); // This function fills the 'MAC' key by using the security key
    $host = UC_CMCIC_URL_CM_TEST;

//TODO send the request and redirect the user to the offsite payment gateway here
  }

What I tried to do and didn't work :
$response = \Drupal::httpClient()->request('POST', $host, [
  'form_params' => $fields,
]);

I tried some other variations of this too (like returning the $response after), which didn't work either
Question : What can I do to redirect the user to the offsite payment gateway while sending POST data in the request ?
Answer: You are defining an orderSubmit() method, which implies you are trying to write a payment method plugin that implements PaymentMethodPluginInterface.
That interface is not designed for offsite payment methods. Instead, you should be implementing OffsitePaymentMethodPluginInterface which has a buildRedirectForm() method exactly for the purpose of redirecting to the other site to complete payment.
The example for this is PayPalPaymentsStandard plugin which is part of the uc_paypal module bundled with Ubercart.
Answer: In D8 for redirection you need to use RedirectResponse Object & thats best way forward. However for improved security the same RedirectResponse object does not allow External URL redirections. For this You may want to use TrustedRedirectResponse. Sample below with assumption that you are trying to post data to the payment getway which can be added as a query string on the URL.
use Drupal\Core\Routing\TrustedRedirectResponse;

public function myController() {
  return new TrustedRedirectResponse($host.'?xyz=abc);
}

PREFERRED
However the above suggestion does has security implications as the data is exposed on the URL. Below solution is a minor tweak from your code.
$response = \Drupal::httpClient()->post($host, [
        'form_params' => $fields,
        'headers' => [
          'Content-type' => 'application/x-www-form-urlencoded',
        ],
    ]);

You can try below if you would like to capture response (If its an XML / JSON)
$response = \Drupal::httpClient()->post($host, [
        'form_params' => $fields,
        'headers' => [
          'Content-type' => 'application/x-www-form-urlencoded',
        ],
    ])->getBody()->getContents();

Comment: How would you do this _and_ redirect to the $host at the same time? That is, how to do you set the `action` to an external URL as well as implement a `post` method?
Comment: This answer does not help and will not work because it ignores the context that this redirect is part of an Ubercart payment method.


Hammer.js pan event only for touch devices and not for desktop computer Click+Drag

Question: I use this standard code for Pan / Pinch (1) with Hammer.js :
var mc = new Hammer(document.body);

mc.add(new Hammer.Pinch({ threshold: 0 })).recognizeWith(mc.get('pan'));

// let the pan gesture support all directions. this will block the vertical scrolling on a touch-device while on the element
mc.get('pan').set({ direction: Hammer.DIRECTION_ALL });
mc.on("panleft panright panup pandown tap press pinchstart pinchmove", function(ev) {
    $('#myElement').text(ev.type +" gesture detected. ev.scale=" + ev.scale + "  ev.deltaX,Y=" + ev.deltaX +'  ' + ev.deltaY);
});

How to get the hammer.js pan events only for touch devices and not for desktop computer Click+Drag? (because I have already working code for desktop computer click+drag, and I don't want to rewrite this!)
Note (1) : By the way, is this code good?, why is the recognizeWith really needed? :
Answer: You may force Hammer.js to use the input class that you want by setting inputClass option:
var hammer = Hammer(element, {
  inputClass: Hammer.TouchInput
})

You may want also to support pointer events when available:
var hammer = Hammer(element, {
  inputClass: Hammer.SUPPORT_POINTER_EVENTS ? Hammer.PointerEventInput : Hammer.TouchInput
})

Hammer.js has following inputClasses built in:

PointerEventInput
TouchInput
MouseInput
TouchMouseInput
Answer: You can check 'pointerType' property value in event (ev) object.
It should be "mouse" for desktop.


Baseball force play

Question: Bases loaded, nobody out. Ground ball to pitcher, throws home for out. Catcher throws to first and the runner on first never ran and is standing on first when the first baseman catches the ball. What is the call on the runner? The batter was called out because he didn't run either.
Answer: This depends on the actions of the first baseman. The batter is not out until the first baseman touches first base or he gets tagged.
Case 1: The first baseman is standing on the base when he catches the ball, or catches the ball and then touches first base without tagging the runner. The batter is now out and this removes the force play on the runner on first. He no longer has to run to second. If he is standing on first base he is safe, if he is off the base he needs to be tagged to be out.
Case 2: The first baseman tags the runner on first before touching first base. The runner on first is forced to run to second by the batter, so when he is tagged he is out. The first baseman can then touch first base to retire the batter and complete the triple play.
Comment: Nevermind, you are correct on both. BTW, here's an article that seems to explain this if you want backup. http://m.providencejournal.com/sports/red-sox/content/20130623-confusing-play-at-first-base-proves-costly-for-red-sox.ece
Comment: I think [Rule 7.03](http://www.baseball-almanac.com/rule7.shtml) covers what happens if the runner never leaves the base before the batter reaches it in case 2. Allowing the runner to be tagged even while on the base protects the defense from losing an easily turned double play if the runner simply refuses to run.
Comment: Agreed with case 1, I'd like to see backup on case 2. My understanding is that the runner isn't forced to run to 2nd until the batter touches 1st.

Baseball force play