Example of using FindFirstFIleEx() with specific search criteria

Question: I asked about finding in subdirs with criteria.  First answer was use FindFirstFileEx().  It seems the function is no good for this purpose or I'm using it wrong.
So can someone explain how I would go about searching in a folder, and all it's subfolders for files that match (to give some sample criteria) .doc;.txt;*.wri;  and are newer than 2009-01-01?
Please give a specific code example for those criteria so I know how to use it.
If it isn't possible, is there an alternative for doing this not-at-all-obscure task???   I am becoming quite baffled that so far there aren't well known/obvious tools/ways to do this.
Answer: From MSDN:
If you refer to the code fragment in that page:
#include <windows.h>
#include <tchar.h>
#include <stdio.h>

void _tmain(int argc, TCHAR *argv[])
{
   WIN32_FIND_DATA FindFileData;
   HANDLE hFind;

   if( argc != 2 )
   {
      _tprintf(TEXT("Usage: %s [target_file]\n"), argv[0]);
      return;
   }

   _tprintf (TEXT("Target file is %s\n"), argv[1]);
   hFind = FindFirstFileEx(argv[1], FindExInfoStandard, &FindFileData,
             FindExSearchNameMatch, NULL, 0);
   if (hFind == INVALID_HANDLE_VALUE)
   {
      printf ("FindFirstFileEx failed (%d)\n", GetLastError());
      return;
   }
   else
   {
      _tprintf (TEXT("The first file found is %s\n"),
                FindFileData.cFileName);
      FindClose(hFind);
   }
}

You'll see that you can call FindFirstFileEx, where argv1 is a string (LPCSTR) pattern to look for, and &FindFileData is a data structure that contains file info of the found data.. hFind is the handle you use on subsequent calls with FindNextFile.. I think you can also add more search parameters by using the fourth and sixth parameter to FindFirstFileEx.
Good luck!
EDIT: BTW, I think you can check a file or dir's attributes by using GetFileAttributes() .. Just pass the filename found in FileFindData.. (filename can refer to a file's name or a directory name I think)
EDIT: MrVimes, here's what you could do (in pseudocode)
find the first file (match with *)

Check the file find data if it is ".", ".." (these are not really directories or files)

if check passed, check file find data if it has the attributes you are looking for (i.e. check filename, file attributes, even file creation time can be checked in the file find data, and what not) and do whatever with it

if check passed, do whatever you need to do with the file

if check failed, either call findnextfile or end, up to you

Something like that..
Answer: I think you use FindFirstFile to find all files and ignore the ones whose WIN32_FIND_DATA values don't match your search criteria.
Comment: right,  that's pretty much exactly what I already do with the findfirst function.  So I guess I won't achieve anything by using a different function.  I was hoping for something that was designed to do the filtering for me (therefore would be more efficient).  I guess I'll stick.  thanks anyway :)
Comment: I don't think this is right.. FindFirstFile returns when it finds the first file that matches your critera I think..
Comment: this is a bit difficult to explain.  if I specify criteria in the first param (such as "c:\asampledir\*.doc") then it won't find directories, therefore it won't allow the surrounding code to search in subdirectories, since the surrounding code waits for the file to be a dir and then runs recursively
Comment: That's not correct, if you have a directory name "c:\asampledir\sampledocdir.doc\somefile.doc", and you search for "c:\asampledir\*.doc" it will match the directory.
Comment: Since the FindFirst* variants match only one filename mask at a time, you can use PathMatchSpecEx(findData.cFileName, L"*.doc;*.txt;*.wri", PMSF_MULTIPLE) to know which ones to ignore.
Answer: Well you could use it to search for *.doc, *.txt and *.wri by passing those values as the name to search for:
FindFirstFileEx("*.doc", FindExInfoStandard, &fileData, FindExSearchNameMatch, NULL, 0);

To search by date is a little more complicated, but not overly so:
SYSTEMTIME createTime;
SYSTEMTIME searchDate;
FILETIME compareTime;
HANDLE searchHandle;

searchDate.wYear = 2009;
searchDate.wMonth= 1;
searchDate.wDay = 1;

SystemTimeToFileTime(searchDate, &compareTime);

searchHandle FindFirstFileEx("*", FindExInfoStandard, &fileData, FindExSearchNameMatch, NULL, 0);

if(searchHandle != INVALID_HANDLE_VALUE)
{
While(searchHandle != ERROR_NO_MORE_FILES)
{
FileTimeToSystemTime(fileData.ftCreationTime, &createTime);

if((ULARGE_INTEGER)compareTime < (ULARGE_INTEGER)createTime)
  printf("%s matches date criteria", fileData.cFileName);

FindNextFile(searchHandle, &fileData);
}
}

Answer: You need to do two searches.  The first is just to find the subdirs, and you do that without any file spec.  The second search for the files uses the file spec.


Why New Zealand vs India continued for second day instead of starting from fresh?

Question: In world cup 2019 Semi Final, New Zealand vs India continued for second day due
to the rain. During 2002 ICC Champions Trophy Final, Match cancelled for first day and again played fresh from second day.
Why New Zealand vs India continued for second day instead of starting from fresh?
Answer: The obvious benefit of continuing the match into the second day is that it increases the chance of getting a result, given that there is no guarantee that a full day's play is possible on the second day.
Indeed, the 2002 ICC Champions Trophy demonstrated the risk of starting anew. Despite 52 overs being completed on the first day, and 58 on the reserve day, no result was achieved.
This was criticized by experienced former players at the time. Farokh Engineer stated

I think it's absolutely daft. It doesn't make any sense, They should
  look at that rule straight away and change it. The obvious thing is to
  continue from where they left off.

and Sanjay Manrekar opined

The International Cricket Council should learn a lesson from this
  result ahead of the 2003 World Cup. We had more than 100 overs but
  still no result. If the match is continued rather than replayed, it
  gives you more of a chance to finish the game

Maybe there are other contributing reasons, such as tiredness of a team having to field twice, the unfairness of a team in the ascendancy having to start again, and the odd feeling of important contributions being voided (what if you scored a century, or took a hat-trick?), but the huge benefit is the improved chance of a result being achieved.
Answer: The regulations covering ODI cricket allow for a reserve day, in the event the match can't be completed on the first day. The umpires are required to try to complete the game on the scheduled day, by reducing overs etc
If this isn't possible, and a reserve day is available, the regs state that the match will continue from the point where it was stopped.
See the regs at https://icc-static-files.s3.amazonaws.com/ICC/document/2018/09/27/c9264750-2778-44df-949f-d1a8dfbdc402/04-Mens-ODI-Playing-Conditions-2018.pdf - section 13.6 and Appendix G are the relevant sections.


K-fold cross validation implementation python

Question: I am trying to implement the k-fold cross-validation algorithm in python.
I know SKLearn provides an implementation but still...
This is my code as of right now.
from sklearn import metrics
import numpy as np

class Cross_Validation:

@staticmethod
def partition(vector, fold, k):
    size = vector.shape[0]
    start = (size/k)*fold
    end = (size/k)*(fold+1)
    validation = vector[start:end]
    if str(type(vector)) == "<class 'scipy.sparse.csr.csr_matrix'>":
        indices = range(start, end)
        mask = np.ones(vector.shape[0], dtype=bool)
        mask[indices] = False
        training = vector[mask]
    elif str(type(vector)) == "<type 'numpy.ndarray'>":
        training = np.concatenate((vector[:start], vector[end:]))
    return training, validation

@staticmethod
def Cross_Validation(learner, k, examples, labels):
    train_folds_score = []
    validation_folds_score = []
    for fold in range(0, k):
        training_set, validation_set = Cross_Validation.partition(examples, fold, k)
        training_labels, validation_labels = Cross_Validation.partition(labels, fold, k)
        learner.fit(training_set, training_labels)
        training_predicted = learner.predict(training_set)
        validation_predicted = learner.predict(validation_set)
        train_folds_score.append(metrics.accuracy_score(training_labels, training_predicted))
        validation_folds_score.append(metrics.accuracy_score(validation_labels, validation_predicted))
    return train_folds_score, validation_folds_score

The learner parameter is a classifier from SKlearn library, k is the number of folds, examples is a sparse matrix produced by the CountVectorizer (again SKlearn) that is the representation of the bag of words.
For example:
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from Cross_Validation import Cross_Validation as cv

vectorizer = CountVectorizer(stop_words='english', lowercase=True, min_df=2, analyzer="word")
data = vectorizer.fit_transform("""textual data""")
clfMNB = MultinomialNB(alpha=.0001)
score = cv.Cross_Validation(clfMNB, 10, data, labels)
print "Train score" + str(score[0])
print "Test score" + str(score[1])

I'm assuming there is some logic error somewhere since the scores are 95% on the training set (as expected) but practically 0 on the test test, but I can't find it.
I hope I was clear.
Thanks in advance.
________________________________EDIT___________________________________
This is the code that loads the text into the vector that can be passed to the vectorizer. It also returns the label vector.
from nltk.tokenize import word_tokenize
from Categories_Data import categories
import numpy as np
import codecs
import glob
import os
import re

class Data_Preprocessor:

def tokenize(self, text):
    tokens = word_tokenize(text)
    alpha = [t for t in tokens if unicode(t).isalpha()]
    return alpha

def header_not_fully_removed(self, text):
    if ":" in text.splitlines()[0]:
        return len(text.splitlines()[0].split(":")[0].split()) == 1
    else:
        return False

def strip_newsgroup_header(self, text):
    _before, _blankline, after = text.partition('\n\n')
    if len(after) > 0 and self.header_not_fully_removed(after):
        after = self.strip_newsgroup_header(after)
    return after

def strip_newsgroup_quoting(self, text):
    _QUOTE_RE = re.compile(r'(writes in|writes:|wrote:|says:|said:'r'|^In article|^Quoted from|^\||^>)')
    good_lines = [line for line in text.split('\n')
        if not _QUOTE_RE.search(line)]
    return '\n'.join(good_lines)

def strip_newsgroup_footer(self, text):
    lines = text.strip().split('\n')
    for line_num in range(len(lines) - 1, -1, -1):
        line = lines[line_num]
        if line.strip().strip('-') == '':
            break
    if line_num > 0:
        return '\n'.join(lines[:line_num])
    else:
        return text

def raw_to_vector(self, path, to_be_stripped=["header", "footer", "quoting"], noise_threshold=-1):
    base_dir = os.getcwd()
    train_data = []
    label_data = []
    for category in categories:
        os.chdir(base_dir)
        os.chdir(path+"/"+category[0])
        for filename in glob.glob("*"):
            with codecs.open(filename, 'r', encoding='utf-8', errors='replace') as target:
                data = target.read()
                if "quoting" in to_be_stripped:
                    data = self.strip_newsgroup_quoting(data)
                if "header" in to_be_stripped:
                    data = self.strip_newsgroup_header(data)
                if "footer" in to_be_stripped:
                    data = self.strip_newsgroup_footer(data)
                if len(data) > noise_threshold:
                    train_data.append(data)
                    label_data.append(category[1])
    os.chdir(base_dir)
    return np.array(train_data), np.array(label_data)

This is what "from Categories_Data import categories" imports...
categories = [
    ('alt.atheism',0),
    ('comp.graphics',1),
    ('comp.os.ms-windows.misc',2),
    ('comp.sys.ibm.pc.hardware',3),
    ('comp.sys.mac.hardware',4),
    ('comp.windows.x',5),
    ('misc.forsale',6),
    ('rec.autos',7),
    ('rec.motorcycles',8),
    ('rec.sport.baseball',9),
    ('rec.sport.hockey',10),
    ('sci.crypt',11),
    ('sci.electronics',12),
    ('sci.med',13),
    ('sci.space',14),
    ('soc.religion.christian',15),
    ('talk.politics.guns',16),
    ('talk.politics.mideast',17),
    ('talk.politics.misc',18),
    ('talk.religion.misc',19)
 ]

Comment: Sorry, but I think it's somewhat of a waste of time to implement something that is available so easily in sklearn. The only point might be for pedagogical purposes - if you're trying to learn to code yourself, or ran into some language point you can't figure out. In each of these cases, what would be the point of throwing this wall of code at someone, and have them debug it for you? At best you'd have another k-fold working implementation, and there already is one like that...
Comment: Well it is of course only for the purpose of understanding what I'm doing wrong. Since it's been a couple of days and I can't figure it out I asked if there is perhaps an obvious logic error or something I don't know about scipy ecc... I do not know any other way of explaining the problem to someone else without the code (especially beacuse I do not know what the problem is)
Comment: Is it possible for you to upload some dataset we can test this on and also import all the relevant scikit packages?
Comment: I edited the question. You can find the dataset here http://qwone.com/~jason/20Newsgroups/ . I use the original (first) one. Thanks
Comment: Great. Let's have a look :)
Comment: Do you also have the code to get the data into the `CountVectorizer`?
Comment: I added it at the end...
Comment: There is still a lot of issues with the code. which makes it not so easy to reproduce your problem. I am not sure what package this `from Categories_Data import categories` is from. Can one hard code the categories and is the label just the name of the categories?
Comment: Yeah it's just in another .py file that's why I import it. For simplicity https://github.com/LorenzoNorcini/Learning_Curves
Answer: The reason why your validation score is low is subtle.
The issue is how you have partitioned the dataset. Remember, when doing cross-validation you should randomly split the dataset. It is the randomness that you are missing.
Your data is loaded category by category, which means in your input dataset, class labels and examples follow one after the other. By not doing the random split, you have completely removed a class which your model never sees during the training phase and hence you get a bad result on your test/validation phase.
You can solve this by doing a random shuffle. So, do this:
from sklearn.utils import shuffle

processor = Data_Preprocessor()
td, tl = processor.raw_to_vector(path="C:/Users/Pankaj/Downloads/ng/")
vectorizer = CountVectorizer(stop_words='english', lowercase=True, min_df=2, analyzer="word")
data = vectorizer.fit_transform(td)
# Shuffle the data and labels
data, tl = shuffle(data, tl, random_state=0)
clfMNB = MultinomialNB(alpha=.0001)
score = Cross_Validation.Cross_Validation(clfMNB, 10, data, tl)

print("Train score" + str(score[0]))
print("Test score" + str(score[1]))

Comment: @LorenzoNorcini My pleasure. I also do not agree with the comments that it is a pointless exercise.
Comment: Cross validation data should not always be randomly split. Time series data have different way for splitting the data, like sliding window.
Comment: @PankajDaga, it must've taken so much time to go through the code and figure out the thing about shuffling. I'm glad that people like you are around to help. :)
Comment: That's exactly it! Thank you very much!


Probability that this stamp is one that you haven't seen before

Question: Suppose you have 4 stamps with probabilities 0.2, 0.4, 0.3, and 0.1, respectively. Each time you pick a stamp is independent of every other pick. Suppose you pick your 10th stamp. What is the probability this stamp is one you haven't seen before?
How should I approach these kinds of problems? Should I do 1-prob(I've seen this stamp before)?
Answer: Classify according to the card that is revealed:
What is the probability it is the first stamp?
the probability it hasn't appeared before is $0.2^9$ since the probability a stamp other than the first one appears in each pick each time is $0.8$ and this happens $9$ times. and the probability it is picked in the tenth pick is $0.2$ so the probability for the first stamp is $0.8^9\times 0.2$
Using this method the probabilities for each stamp are:
first stamp $0.8^9\times 0.2$
second stamp $0.6^9\times 0.4$
third stamp $0.7^9\times 0.3$
fourth stamp $0.9^9\times 0.1$
So the desired probability is the sum of these
Comment: Hi Jorge; the third stamp contains a typo.
Answer: A start: Divide into cases, the $10$-th stamp is stamp A, and we have not seen it, it i stamp B and we haven't seen it, and so on.
The probability it is stamp A and we have not seen it is $(0.2)(0.8)^9$.


Post hoc comparisons following nparLD

Question: I would like to compute post hoc tests to follow up on results from the nparLD package.
In my sample dataset, I have two conditions, "drugA" and "drugB". There are 6 animals A-F and the weight of each animal has been measured 3 times under the influence of each drug.
<code>id <- rep(c("A","B","C","D","E","F"),6)
drug <- c(rep(c("drugA"), 18), rep(c("drugB"), 18))
time <- rep(rep(1:3, each = 6),2)
value <- c(rnorm(6, 1, 0.4), rnorm(6, 3, 0.5), rnorm(6, 6, 0.8), rnorm(6, 1.1, 0.4), rnorm(6, 0.8, 0.2), rnorm(6, 1, 0.6))
df <- data.frame(id,drug, time, value)

df$id <- as.factor(df$id)
df$drug <- as.factor(df$drug)
df$time <- as.factor(df$time)

library(nparLD)
stats <- nparLD(data = df, formula = value ~ drug*time, subject = "id")
stats$Wald.test
stats$ANOVA.test
</code>
What do you recommend for post hoc testing? Individual comparisons, where I compare subsets of the data using nparLD? Or is there a more elegant way of doing this?
Also, the results of the Wald and ANOVA-type tests differ, especially regarding the factor 'time'. What do you recommend?
Answer: Edgar Brunner has recommended using the "nparcomp" R package to further probe results from nparLD. You can see more info in the paper: nparcomp: An R Software Package for Nonparametric Multiple Comparisons and Simultaneous Confidence Intervals, authors: Frank Konietschke, Marius Placzek, Frank Schaarschmidt, Ludwig A. Hothorn (2015), Journal of Statistical Software, doi:10.18637/jss.v064.i09.

Post hoc comparisons following nparLD


Calculate double integral with specific region

Question: I need to calculate
$$\iint\frac{dx\,dy}{\sqrt{x^2 + y^2}}$$
$$x^2 + y^2 < 2y$$
I tried to solve this via polar coordinates
$$0 \leq r < 2\sin(\phi)$$
$$0 \leq \phi \leq 2\pi$$
So, our integral becomes
$$\int_0^{2\pi} d\phi \int_0^{2\sin(\phi)} \, dr $$
which seems to be wrong because of $$r < 2\sin(\phi)$$
Comment: You probably only need to integrate from $-\pi/2$ to $\pi/2,$ since there is no portion of your region in the left two quadrants.
Comment: @ThomasAndrews that would be true for $r = 2 \cos\theta$. Here there is no portion of the region in bottom two quadrants.
Comment: $$x^2+y^2<2y$$ is equivalent to $$x^2+(y-1)^2<1$$
Answer: $$ \iint\frac{dx\,dy}{\sqrt{x^2 + y^2}}$$
Region is $x^2+y^2 \leq 2y$.
So if you are using $x = r \cos \theta, y = r \sin\theta$,
$$r \leq 2 \sin\theta, 0 \leq \theta \leq \pi.$$
This is a circle centered at $(0, 1)$ with radius $1$. As you are measuring radius from the origin, $r = 2 \sin\theta$ on the circle and note that the complete circle forms for $0 \leq \theta \leq \pi$.
So integral becomes $ \ \displaystyle \int_0^\pi \int_0^{2 \sin\theta}\,dr \ d\theta $


Where to find the html.tpl.php $head variable content generation

Question: I'm looking for a way to edit the $head variable of the html.tpl.php.
Where exactly is this variable composed and set?
Answer: The most common way to add information to the  is using  drupal_add_html_head().
If the data you want to alter was added using that function, you can alter it by implementing hook_html_head_alter() within a custom module. Using the alter hook requires that you know the $key of the data that was added during drupal_add_html_head().
If you want to alter CSS or JS, use hook_css_alter() and hook_js_alter() respectively.


How to group by timestamp and other columns in Polars

Question: I have a dataframe with many columns, only interested in timestamp, country, province, and population.
I would like to determine the mean population over time by province. I can calculate this using the following in Polars: df.groupby(['timestamp','country', 'prov']).agg(pl.col('population').mean())
However I would like to incorporate rolling timewindows. I have found the following, but I am unsure if I can incorporate both into one expression. df.groupby_rolling('timestamp', period = '1y')
I am looking to find the mean by year, perhaps rolling window is not the correct terminology
Any suggestions?
Comment: Any reproducible example? Also, see the `by` argument for `groupby_rolling`, e.g. `df.groupby_rolling('timestamp', by=['country', 'prov'], period='1y')`
Comment: Then maybe `group_by_dynamic` is what you are looking for? `df.group_by_dynamic("timestamp", every="1y", by=["country", "prov"])`
Comment: @Wayoshi it seems to be working with your suggestion. I was hoping to get a mean per 1 year timestamp, however I don't believe rolling is the way to go. I think I will need to chunk it up before aggregating - thanks for your help!
Comment: @keraion That is exactly the function that I am looking for, thank you!
Answer: For grouping data into interval based sizes, take a look at group_by_dynamic. The every parameter will define the interval to group the data, and by will add the additional grouping levels for the intervals.
The group_by_dynamic requires that your data is sorted on the index column (e.g. timestamp).

index_column: Column used to group based on the time window. Often of type Date/Datetime. This column must be sorted in ascending order (or, if by is specified, then it must be sorted in ascending order within each group).`

from datetime import datetime
import polars as pl

df = pl.DataFrame(
    {
        "timestamp": [datetime(2022, n + 1, 1) for n in range(12)] * 6
        + [datetime(2023, n + 1, 1) for n in range(12)] * 6,
        "country": ["USA", "CAN", "MEX"] * 48,
        "prov": ["Florida", "Ontario", "Oaxaca", "California", "Quebec", "Jalisco"]
        * 24,
        "population": range(500, 644, 1),
    }
)

print(
    df.sort("timestamp")
    .group_by_dynamic("timestamp", every="1y", by=["country", "prov"])
    .agg(pl.col("population").mean())
)

Results:
shape: (12, 4)
┌─────────┬─────────┬─────────────────────┬────────────┐
│ country ┆ prov    ┆ timestamp           ┆ population │
│ ---     ┆ ---     ┆ ---                 ┆ ---        │
│ str     ┆ str     ┆ datetime[μs]        ┆ f64        │
╞═════════╪═════════╪═════════════════════╪════════════╡
│ USA     ┆ Florida ┆ 2022-01-01 00:00:00 ┆ 533.0      │
│ USA     ┆ Florida ┆ 2023-01-01 00:00:00 ┆ 605.0      │
│ CAN     ┆ Ontario ┆ 2022-01-01 00:00:00 ┆ 534.0      │
│ CAN     ┆ Ontario ┆ 2023-01-01 00:00:00 ┆ 606.0      │
│ …       ┆ …       ┆ …                   ┆ …          │
│ CAN     ┆ Quebec  ┆ 2022-01-01 00:00:00 ┆ 537.0      │
│ CAN     ┆ Quebec  ┆ 2023-01-01 00:00:00 ┆ 609.0      │
│ MEX     ┆ Jalisco ┆ 2022-01-01 00:00:00 ┆ 538.0      │
│ MEX     ┆ Jalisco ┆ 2023-01-01 00:00:00 ┆ 610.0      │
└─────────┴─────────┴─────────────────────┴────────────┘


Statistically significant equality of sample sizes (does 50 equal to 53)***?

Question: Significance tests are used for a variety of reasons and in many research scenarios. One of them is checking the equality of sample sizes.
There's a book for psychologists whose authors recommend chi-squared test to check if two samples are of the same size. First sample size is 50 and the other one is 53. Then they use chi-squared test to find out if 50 is statistically equal to 53. If it is, then a researcher can state equal sample sizes (f.e. for t-tests or ANOVA, etc).
Q: Isn't it a really bad way to use significance tests?
Comment: Is this test really usefull ? You can do T-tests or ANOVA with different sample sizes, it's a bit more complex, but using R, it's alright.
It does seem to be a poor use of testing.
Comment: @kjetilbhalvorsen I'm taking your question as a recognition for my English ;) The book is all in Polish. However, do you think that's bad advise?
Comment: Still, can you reference the book (and page number)? I speak Polish and I'm really curoius to see how authors justify their recommendations...
Comment: "Drogowskaz statystyczny" (2007), p.183.
Comment: It makes no sense whatsoever. It makes sense to test for the prevalence of groups in the population, not to test in-sample sizes. In-sample, if one group is sized 50 and the other 53, then you already know, with 100% certainty, that both samples are of different sizes.
Comment: @Pohoua You're right, I just had to put my question into some context.
Answer: It does not look like good advice, but not knowing the authors' intention what to say? But, more important, it is not needed.  Anova or t-tests with unequal sample sizes is not a problem (it might be inefficient, so in the planning phase try to avoid it.)
See for instance Are unequal groups a problem for one-way ANOVA?  and many similar posts you can find by searching this site!
Answer: OK, I got relevant excerpt of the book from OP. It says that ANOVA needs equal smaple sizes across groups and chi-squared test may be used to check this assumption.
Of course, ANOVA doesn't need equal samples, which is nicely explained in link provided by kjetil b halvorsen.
But even if equal sample sizes were needed, testing this assumption makes no sense. It is because statistical test are trying to use information provided in sample from some population to say something about whole population. T-test, for example, tries to locate population mean given a sample from this population. Population mean is unknown and that is why we need testing procedure to tell something about it with, say, 95% certaininty. Notice now, that sample mean is very different from population mean. Sample mean is perfectly known so we just know where it is (with 100% certainity). So we know for sure if it differs from, say, 153.32. Problem is that sample mean is not very interesting quantity, it's population mean that is interesting.
If equal sample sizes were needed we would be, contrary to common use of statistical tests, interested in some quantity (number of elements in this case) in sample, not in population. So we do not need any testing procedure. We can just count observations in our sample and can be 100% sure if it differs from what authors of the book call "ANOVA assumptions".


How should I use hook_cache_flush() or hook_rebuild() in a custom module to execute JavaScript code when the cache is cleared?

Question: I'm trying to create a module that executes some JavaScript code when the cache is flushed (via administrator menu or otherwise). I believe I can implement either hook_cache_flush() or
hook_rebuild(), but I'm not quite sure how. I'm not necessarily looking for this to be done for me, but if I could be given a starting point and direction to head in, that'd be great.
Answer: Neither hook_cache_flush() nor hook_rebuild() implementation can directly add JavaScript code to a page, simply for the fact they don't get any render array as argument, nor they return a render array to the function/method invoking them.
You could add a form submission handler to the system_performance_settings form for when the Clear all caches button on that form is clicked. This won't work when the cache is cleared using other methods, for example by a module that calls drupal_flush_all_caches() in its code.
// Put the following line on the top of the file containing this code.
use Drupal\Core\Form\FormStateInterface;

function mymodule_form_system_performance_settings_alter(&$form, FormStateInterface $form_state) {
  if (isset($form['clear_cache']['clear'])) {
    if ($form_state->getTemporaryValue('mymodule_cache_cleared')) {
      $form['#attached']['library'][] = 'mymodule/cache.rebuild';
    }
    $form['clear_cache']['clear']['#submit'][] = 'mymodule_system_performance_settings_submit';
  }
}

function mymodule_system_performance_settings_submit(array &$form, FormStateInterface $form_state) {
  $form_state->setRebuild();
  $form_state->setTemporaryValue('mymodule_cache_cleared', TRUE);
}

For a solution that works in all the cases drupal_flush_all_caches() is called, and adds the JavaScript code to the first page requested after the cache is cleared, I would implement hook_cache_flush() to set a value that is then checked in hook_page_attachments().
function mymodule_cache_flush() {
  \Drupal::state()->set('mymodule_cache_cleared', TRUE);
}

function mymodule_page_attachments(array &$attachments) {
  $state = \Drupal::state();
  if ($state->get('mymodule_cache_cleared')) {
    // Delete the state value to avoid the library is added to
    // every page after the cache is cleared.
    $state->delete('mymodule_cache_cleared');

    $attachments['#attached']['library'][] = 'mymodule/cache.rebuild';
  }
}

To add the JavaScript code to the first X pages requested after the cache clear, I would use the following hook implementations. (The following code attaches the library to the first three page requests.)
function mymodule_cache_flush() {
  \Drupal::state()->set('mymodule_cache_cleared_count', 3);
}

function mymodule_page_attachments(array &$attachments) {
  $state = \Drupal::state();
  if ($count = $state->get('mymodule_cache_cleared_count')) {
    $attachments['#attached']['library'][] = 'mymodule/cache.rebuild';
    $state->set('mymodule_cache_cleared_count', $count - 1);
  }
}

I used \Drupal::state()->get() and \Drupal::state()->set() because:

The value set with drupal_static() won't be preserved between page requests. The documentation page clearly says:

All functions requiring a static variable to persist or cache data within a single page request are encouraged to use this function unless it is absolutely certain that the static variable will not need to be reset during the page request.

Setting a session value, each user would get a different value. This means the JavaScript code would be added to the next page requested after the cache is cleared only when that page is requested from the same user who was logged in when the cache has been cleared.

Add JavaScript to Your Theme or Module shows how a JavaScript library is defined from a module.
Comment: @Clive, that's a good idea setting a session value, like you could set a message for the the next full page load. To read and clear this session value you can't use the page preprocess hook, though, pages are cached. You can use a response event subscriber after the page is retrieved from cache or an uncacheable block with cache max-age 0, which is always cached as placeholder and runs on every page request.
Comment: I wanted to use one of the functions I listed so that the code would execute every time one of those functions was called. That way, yes, the button in performance settings would do it, but also if someone had the admin menu module installed, and cleared their cache that way, it would still execute. Isn't there a way to execute JS when a certain hook is used, regardless of its arguments or what it returns?
Comment: The only hook that would be able to alter the render array for a page would be invoked for every page and it won't be able to understand whether the cache is cleared.
Comment: I know which hook you're talking about and you're correct. As for the JS, I won't ask for how to do what I'm particularly looking to do, but if I wanted to add something simple like an alert just to test my library, could you give me a hint on how to do that? I read through the documentation you linked and didn't find a solution that worked for me.
Comment: You could always set a static var in the hook and pick it up again in `hook_preprocess_page()` (see [`drupal_static()`](https://api.drupal.org/api/drupal/core%21includes%21bootstrap.inc/function/drupal_static/9.3.x) for the pattern to use). But it won't work in all contexts; for example when you clear cache from the admin menu, it redirects back to the page it came from, so you don't have an opportunity to attach libraries to the page. You could get around that by using session or temporary storage, but it's more effort and could get messy if a page redirect fails for example
Comment: @Joseph I added a solution more, since what reported in the comments is not completely correct.
Comment: @4uk4 It's a good idea if the OP needs a different value for each user. If the OP needs to set a value when the cache is cleared and use that value to add JavaScript code to the next required page, independently from the user who accesses that page, the session cannot be used.
Comment: OK, but having the cache clear in a post request directly redirecting to a normal page load (then of course with the same session cookie) is the most likely use case of drupal_flush_all_caches() according to the docs. If there is a browser involved. Otherwise javascript doens't make much sense.
Comment: @4uk4 I don't understand either what the purpose of running JavaScript code when Drupal cache is cleared is. I would think the OP is trying to change something at browser level, but this is only a wild guess. That's why I avoided to say anything about whether using JavaScript is a good or bad idea.


How i can to calculate the harmonic partial series: $\sum_{k=1}^n\frac{\left(H_k\right)^2-H^{(2)}_k}{k}$?

Question: I have trying to calculate $$S_n=\sum_{k=1}^n\frac{\left(H_k\right)^2-H^{(2)}_k}{k} $$
I'm trying to apply Abel's Summation, i.e.: $$ \sum_{k=m}^n a_kb_k=A_nb_n-A_{m-1}b_m-\sum_{k=m}^{n-1}A_k\left(b_{k+1}-b_k\right)\tag{1}$$
where $\displaystyle A_k=\sum_{i=1}^k a_i$. Taking $m=1$, $\displaystyle a_k=\frac1k\to A_k=H_k$ and $\displaystyle b_k=\left(H_k\right)^2-H^{(2)}_k$ in $(1)$, i got that:
\begin{align*}
S_n&=H_n\left\{ \left(H_k\right)^2-H^{(2)}_k\right\}-\sum_{k=1}^{n-1}H_k\left\{ \left(H_{k+1}\right)^2-H^{(2)}_{k+1}-\left(H_{k}\right)^2+H^{(2)}_{k}\right\} \\
&=H_n\left\{ \left(H_k\right)^2-H^{(2)}_k\right\}-\sum_{k=1}^{n-1}H_k\left\{ \left(H_{k+1}+H_k\right)\left(H_{k+1}-H_k\right)-H^{(2)}_{k+1}+H^{(2)}_{k}\right\}\tag{2}\\
&=H_n\left\{ \left(H_k\right)^2-H^{(2)}_k\right\}-\sum_{k=1}^{n-1}H_k\left\{ \left(2H_{k}+\frac1{k+1}\right)\left(\frac1{k+1}\right)-\frac1{(k+1)^2}\right\}\\
&=H_n\left\{ \left(H_k\right)^2-H^{(2)}_k\right\}-2\sum_{k=1}^{n-1}\frac{\left(H_k\right)^2}{k+1}\color{red}{-\sum_{k=1}^{n-1}\frac{H_k}{(k+1)^2}}\color{red}{+\sum_{k=1}^{n-1}\frac{H_k}{(k+1)^2}}\\
&=H_n\left\{ \left(H_k\right)^2-H^{(2)}_k\right\}-2\sum_{k=1}^{n }\frac{\left(H_{k-1}\right)^2}{k}
\end{align*}
Notice that, in $(2)$ i used the facts $\displaystyle H_{k+1}=H_k+\frac{1}{k+1}$ and $\displaystyle H_{k+1}^{(2)}=H_k^{(2)}+\frac{1}{(k+1)^2}$. My difficulty here is to evaluate the last sum above. Thank you in advance for any tip or solution and i accept any approach for the original problem.
Comment: Just a question : so do you want an answer to your last sum, or do you want an answer to the original problem, which could be different from your approach and your last sum, for example?
Comment: @TeresaLisbon I can perfectly consider both possibilities. But, out of curiosity, I would like to proceed with my approach.
Comment: @TeresaLisbon I apologize for the misunderstanding, I really confused my words. But I edited the final sentence of my post. Thank you for the signage!
Comment: You are welcome, and I hope you get a good answer.
Comment: Sure, thanks a lot. I was just asking , because recently I'd run into a situation where someone had asked a question but not told people that their emphasis was on their own approach, so someone else who had a different approach was a bit disappointed. I just wanted to make sure you emphasised your approach. Great! +1. (Note : I checked your question for duplicates on the site using Approach0 and didn't find anything. It is still possible that it may be a subpart of a larger computation that has been done on site).
Answer: First, I wondered if the sign between the two harmonic numbers in the denominator is correct. Supposing this is the case, then we have

$$\sum_{k=1}^n\frac{H_k^2-H_k^{(2)}}{k}=\frac{1}{3}(H_n^3-3 H_nH_n^{(2)}-4 H_n^{(3)})+2\sum_{k=1}^n\frac{H_k}{k^2},\tag1$$

where for the last sum there is no known simpler form (e.g., to express it in terms of harmonic numbers).
During the calculations, I used that
$$\sum_{k=1}^n\frac{H_k^2+H_k^{(2)}}{k}=\frac{1}{3}(H_n^3+3 H_nH_n^{(2)}+2 H_n^{(3)}),$$
which appears in (Almost) Impossible Integrals, Sums, and Series (for a proof, see pages $61$-$62$).
Also, by Abel's summation we immediately we have that
$$\sum_{k=1}^n\frac{H_k}{k^2}+\sum_{k=1}^n\frac{H_k^{(2)}}{k}=H_n H_n^{(2)}+H_n^{(3)},$$
which is useful to get the result in $(1)$.


Zend Framework, Form - Form renders with just one field

Question: I am using ZF Form to generate forms.
I wrote a class
class Form_Client extends Zend_Form
{
    public function init()
    {
        $this->setMethod('post');

        $name = $this->createElement('text', 'email',array('label'=>'Name','size'=>'50'));
        $name->setErrorMessages(array('Field is required'));
        $name->setRequired(TRUE);
        $this->addElement($name);

        $contact_person = $this->createElement('text', 'email',array('label'=>'Contact person','size'=>'50'));
        $contact_person->setErrorMessages(array('Field is required'));
        $contact_person->setRequired(TRUE);
        $this->addElement($contact_person);

        // add element: submit button
        $submit = $this->createElement('submit', 'submit', array('label' => 'Save'));
        //$submit->setDecorators(array('ViewHelper'));
        $this->addElement($submit);

        $btn = $this->createElement('button', 'cancel', array('label' => 'Cancel'));
        $btn->setAttribs(array('onClick'=>'window.location="/system/login"','style'=>''));
        //$btn->setDecorators(array('ViewHelper'));
        $this->addElement($btn);
    }
}

In my controller I've got
public function editAction()
{
    $frmClient = new Form_Client();
    $frmClient->setAction('edit');

    /*submit*/
    if ($this->_request->isPost()) {
            if ($frmClient->isValid($_POST)) {
                $data = $frmClient->getValues();
            }
    }

    $this->view->form = $frmClient;
    $this->_helper->viewRenderer('form');
}

And in my View there's
<?php echo $this->form; ?>

The problem is that form renders with just the first input fields.
Do you know what's wrong with that?
Thanks,
Jacob
Answer: Both of your text fields are called "email" - Each field must have a unique name.
Answer: Currently, both form elements have the same name. Try something like:
[..]
$name = $this->createElement('text', 'name',array('label'=>'Name','size'=>'50'));
[..]
$contact_person = $this->createElement('text', 'contact_person',array('label'=>'Contact person','size'=>'50'));
[..]


public and non-public IP addresses in a VPN

Question: We have a remote web server that talks to a SQL Server database (located in a different city) via a VPN. That's working fine.  The SQL database is on 192.168.1.xxx on the local LAN, and in addition to those remote web apps, many local desktop applications also talk to it over the LAN.
After some bad weather brought large trees down and cut electric power for the better part of a week, management has asked me to move the SQL database off the LAN and up to the remote server, which is housed in a fancy datacenter with backup generators.
So I have all of these desktop applications that would need to go out over a new VPN to reach the remote SQL database.
The remote web server has a hardware firewall with only three ports opened to the world at present, port 80, a port for mail, and a port for Remote Desktop Connection.
Is there a way to set up a VPN so that the desktop applications here on the LAN can see the database up there on the remote server, the web apps on the remote server can see the database on the same remote server, but the port that SQL Server is listening on is not opened to the world in the firewall, while still using TCP as the SQL Server protocol?
Answer: Sounds like what you should do is create an IPSec tunnel between webserver location and desktop application location.
http://www.firewall.cx/networking-topics/protocols/870-ipsec-modes.html
Comment: I can see how the IPSec tunnel would allow the remote web-server to communicate with the database on our LAN (that's what we have now), or allow our desktop applications to communicate with the database if it is moved up to the remote server.  But if we move the database up to the web-server, can the web apps on that server communicate with the database on that server without our opening up the SQL Server's port to the world? Can the SQL Server port be opened in the firewall with a restriction that says "requests must originate on this server" and/or "requests must come in over the VPN"?
Comment: As far as I understand (haven't used IPSec very much), what you are doing, when using IPSec in tunnel mode, is to create router-router tunnel between the two sites (lets say site A and B), and the traffic is transparent between the two.

This is actually, what possibly is already done there, since your users are able to connect to offsite SQL server to its LAN address 192.168.1.X

In that case you just need to move webserver to SQL server site, give it a new address and update any clients/user computers/servers who used the old address, to have the new one.
Comment: Site A: internal network 192.168.1.0/24
SQL server address 192.168.1.50
Web server address (after moving to this site): 192.168.1.60

Site B: internal network 192.168.20.0/24
user computers 192.168.20.1, etc.

From site B perspective they can directly see 192.168.1.* as if it were part of their local LAN (if the routers are configured properly).

In this setup, you only need to open port in firewall, if you need to have access to webserver from outside the site A and site B.


Como pegar o atributo src de uma lista de imagens e criar um array utilizando apenas javascript?

Question: O objetivo é um slider-show, porem quero criar um array dinâmico apenas listando algumas imagens dentro de uma div  utilizando javascript puro, por exemplo:

Entretanto não sei como percorrer as imagens desta div, pegar e salvar o valor do atributo src em um array.
Answer: você pode utilizar um querySelectorAll para selecionar todas as imagens, então um map para montar o array.

//selecionar todos os elementos   dentro de uma
 com à classe ".slider"
var elements = document.querySelectorAll("div.slider > img");

//realizando um mapeamento dos elementos , criando um array com os valores da propriedade "src" destes elementos.
var imgs = [].map.call(elements, function (element, indice) {
  return element.src;
});

console.log(imgs);

Answer: Usando somente javscript puro, dá pra pegar todo as imagens com querySelectorAll
e depois varrendo o array e passando o src para outro vetor

//pegando todas as imagens dentro da div com classe .slider
var slider = document.querySelectorAll(".slider img");

var imagens_slider = [];

//varrendo array e pegando imagens do src para um novo array
for(cont=0; cont < slider.length; cont++){

  imagens_slider.push(slider[cont].src);

}

console.log(imagens_slider)


Absolute vs. Relative Difference in Survival Time - Is this possible?

Question: So there's a fairly well characterized difference between relative risk and absolute risk for conventional cohort studies, and for many questions, the absolute risk is arguably more appropriate.
Is there an analogous way to measure absolute difference for accelerated failure time models? I suspect there isn't, because these models assume proportional survival times, so the relative measure will be constant, but the absolute measure will vary over time, but I wanted to make sure before I abandoned this line of reasoning.
Would it be possible to do it at a fixed point in time, say the median - and do you think this would be of any value?
Comment: Just to clarify my understanding: absolute risk of disease is a risk of experiencing an event over a specified interval conditional upon not dieing from other causes (ranges from 0 to 1), whereas the relative risk is a comparison of risks under two different settings (ranges from 0 to $\infty$). Further, the criticism of relative risk is a tendency to exaggerate the deleterious effects of exposures for extremely rare diseases, e.g. a 100-fold risk ratio for a less than 1/1,000,000 incidence is practically negligible. Is this correct?
Answer: As you said the PH assumption prevents evaluating an absolute measure of difference in survival.
If you are interested a method to evaluate adjusted differences in survival percentiles, such as the median, has been recently developed and proposed. I'm reporting below a couple of references. Do not hesitate to contact me if you need further information.
Bottai M, Zhang J. Laplace regression with censored data. Biom J. Aug 2010;52(4):487-503.
Orsini N, Wolk A, Bottai M. Evaluating percentiles of survival. Epidemiology. Sep 2012;23(5):770-771.
Answer: Risk ratios and risk differences are two different measures of association for time-to-event outcomes as observed in cohort studies. Risk ratios are the most commonly presented type of outcome because they are easily estimated from Poisson, logistic, or Cox proportional hazards models. Risk differences, or the difference in absolute risk, while less common, boasts several advantages over the risk ratio in terms of interpretability. For instance, if a 50 year old non-smoking woman's 10 year CVD death risk is 1 (10 events per 1,000 person years) whereas a counterfactually matched 50 year old smoking woman has a 10 year CVD death risk of 2% (20 events per 1,000 person years) the risk difference is 1% but the risk ratio or relative risk is 2 or 100%; people often conflate percentages in this case as a fraction of events to person years $\times 100$ (absolute risk difference) or as a proportional difference (RR1/RR2: unitless).
Absolute risk can definitely be measured even in the presence of censoring. With no censoring or competing risks, logistic regression using the identity link is an additive risk model; the coefficients measure absolute risk difference on the natural scale. Similarly, poisson GLMs with the identity link for the counts using the offset of person-years exposure lead to similar estimates of rate differences on the natural scale. For survival models with censoring, parametric models lead to predictions of risk which can be combined to predict risk differences. For instance, with exponential survival models, the log-rate leads to predicted survival curves at several survival times and the model has efficient CIs. Fully stratified Kaplan Meier curves also provide estimates of risk differences but the calculation of the CIs remains an area of research.
Lastly, survival curves and by extension risk differences can be obtained from Cox models using the smoothed estimates of the baseline hazard function with either the Breslow step estimator or the Schoenfeld residuals.


$|z| = e^x, arg(z) = y$?

Question: In introducing complex analysis, my instructor made the following assertion:

Let $w = x + iy$ and suppose $e^w = z$ for $z \in \mathbb{C}$. We have
$z = e^w = e^x(\cos(y) + i\sin(y))$,
so $|z| = e^x, arg(z) = y$

I'm confused as to how we went from
$z = e^w = e^x(\cos(y) + i\sin(y))$
to
$|z| = e^x, arg(z) = y$.
Since this is an introduction to complex analysis, I have no understand of complex analysis beyond the most basic concepts. I would greatly appreciate it if people could please take the time to explain this and help me understand it.
Comment: Strictly speaking, we can only conclude that $y \equiv \arg z \pmod {2 \pi}$. We can get around this by making a choice of branch of $\arg$ and restricting the choice of $w$ accordingly.
Comment: @Travis Indeed. In this case, according to my instructor, we have $arg(z) \in (-\pi, \pi]$, which is in accordance with our chosen branch: $\{ x + iy \mid x \in \mathbb{R}, -\pi < y \le \pi \}$.
Answer: Note that
$$
|z| = |e^x(\cos y + i \sin y)| = |e^x||\cos y + i \sin y| = e^x(\cos^2y + \sin^2 y )^{1/2} = e^x
$$
As for the argument, remember to calculate both the imaginary and real part of $z$ and form the ratio
$$
\Re(z) = e^x\cos y ~~~\mbox{and}~~~\Im(z) = e^x\sin y
$$
Therefore
$$
\tan\arg(z) = \frac{\Im(z)}{\Re(z)}=\frac{e^x\sin y}{e^x\cos y} = \tan y  ~~~\therefore~~~ \arg(z) = y
$$
Comment: Thanks for the response. I should have remembered that the absolute value of the product is equal to the product of the absolute values. Thanks again!

$|z| = e^x, arg(z) = y$?


Correlation of latent variables: Sum-scores vs. SEM correlation

Question: I use a set of about 20 attitudinal items and confirmatory factor analysis (CFA). Loadings and model for are sufficient. In the next step, I want to test for correlations between these latent factors. I calculate factor scores based on the CFA. And here comes my question: If I calculate correlations between these factor scores, correlations are quite high (up to .7). If I calculate sum scores (adding up the items) and correlate these scores I get only a medium correlation. How does this come? I am aware that the latent factor in the CFA are weighted while the sum scores are not. However, I do not think that this can be the reason for the different correlations.
General framework: N > 8000, Likert scale, attitude items.
This is a similar topic, but does not on the possible differences between the two methods.
Correlational study or ordinal data using 5-point Likert scale
edited to make the question more clear
Comment: In the CFA model I added a covariance between the latent factors to check, if these constructs are independent. From the theory it is not reasonable to suggest that these latent factors are uncorrelated. You are right, in this case I can use another rotation. But this does not change the results a lot.
Comment: `Confirmatory factor analysis suggests a high (standardized) correlation (0.59, -0.45) for 3 latent factors`. How to understand this passage? Moreover, you extracted factors in EFA as orthogonal. Why are allowing them to correlate afterwards?
Answer: I assume that you are thinking of a simple structure in which each of the 20 items loads on exactly 1 factor. Suppose items 1-10 load on factor 1, and 11-20 load on factor 2. Then you could average items 1-10, average items 11-20 for each individual and calculate their correlation. Alternatively, you can estimate factor scores for the factors and obtain an estimate of the correlation that way. If you do a CFA, allowing the correlation between factors to be free, the software will estimate that parameter for you. Is this your question?
So yes ... these two statistics will be different. You can think of each item as being a noisy estimator of factor 1 or factor 2 (as appropriate). Taking the average will reduce the noise, but you still have noisy observations. Adding noise to a pair of variables reduces their correlation, so the first statistic will be biased downwards as an estimate of the correlation you seek. This is true even if the factor loadings are the same.
In confirmatory factor analysis, you estimate the various components of the model (uniqueness variances, loadings, factor covariances) through maximum likelihood (or some other method), so you end up actually estimating the parameter of interest (the factor correlation).
As a bonus, you can still get the covariance of the factors in a more complex model, where items load on more than 1 factor. Sum scores would totally not work in that case, but the covariance of the factors will emerge from the optimization.


Web Moderna: SPA/REST

Question: Falando na web moderna, temos conceitos novos como SPA/Rest e mantenho algumas dúvidas.
O correto ao ingressar na web é focar/estudar essas conceitos? Já que surgiram como melhoria de alguns conceitos antigos.
É viável sempre ou quando poder utilizar SPA/Rest? Sei que com uma rest em mãos posso utilizar ela para servir várias outras aplicações, como apps mobile, fora a estabilidade e etc.
Em questão de segurança, é seguro utilizar uma SPA? Em uma palestra sobre SPA, que a validação se foca no front, porem eu Nao deixaria de fazer a validação no servidor, acho que Nao tem nem sentido fazer isso.
Falo em sistemas de todos os contextos, tamanhos, objetivos e etc.
Sei que muitas dessas coisas nao são tão novas assim, estando presentes na web a bastante tempo, porem ainda são propostos muitos conceitos "antigos".
Comment: Validação client-side faz sentido pra diminuir a carga do servidor e para tornar a experiência do usuário mais ágil (validação client-side é sempre mais rápida). Claro que não substitui a validação no servidor, mas validar instantaneamente melhora a experiência do usuário sem onerar o servidor.
Comment: Então no caso poderia continuar usando nós dois lados, mas com um adiantamento do cliente, apesar que acho inseguro, por ser possível manipular o javascript.
Comment: Não é inseguro porque a validação final é no servidor.
Comment: Foi mal, acabei me expressando errado, eu quis dizer que continuaria achando inseguro se for totalmente no cliente e não no servidor. Se for nós dois lado, eu concordo.
Answer: O correto ao ingressar na web é focar/estudar esses conceitos?
Sim. É sempre bom estar atualizado com novas tecnologias. SPA significa Single Page Application, é um novo modelo de desenvolvimento de aplicações Web e mobile que vem ganhando destaque em grandes empresas como Microsoft, Google, Twitter, Facebook, etc. Ele basicamente significa codificar menos no server-side e mais no client-side. Ou seja, a aplicação estará contida toda ou quase toda no cliente (dentro do navegador Web). É praticamente uma aplicação Desktop rodando sob o navegador.

É viável sempre ou quando poder utilizar SPA/Rest?
Tudo tem vantagens e desvantagens.

Vantagem:

Balanceamento da responsabilidade da execução entre cliente e servidor
Menos código do servidor, e mais responsabilidade no cliente;
Melhorar a experiência ao usuário (UX) criando interface com usabilidade moderna e de fácil entendimento do usuário;
Menor consumo de banda, pois as cargas de dados são feitas por demanda e por AJAX.

Desvantagem

Muito código JavaScript para gerenciar no navegador, o carregamento da página inicial pode ser muito lenta (uma vez que você tem que carregar e executar uma tonelada de JS). Se você usar um servidor web JavaScript como Node.js, você pode resolver parcialmente as duas últimas questões, tornando o server-side mesma página.

Em questão de segurança, é seguro utilizar uma SPA?
De acordo com esse site, não. Por que estes tipos de aplicações são chamadas de "clientes não confiáveis", já que o nosso código do lado do servidor não tem controle sobre o ambiente em que executar. Mesmo aplicações web regulares têm esses problemas. As pessoas podem facilmente alterar ou injetar código JavaScript em uma página através do console desenvolvedor. Aplicativos móveis, tais como aqueles no Android e iOS, pode ser compilado e inspecionados. Como tal, você não gostaria de incorporar informações sensíveis, como chaves secretas ou senhas nestes tipos de clientes.

Referências:

Não exponha seu REST
O que é SPA
What is SPA
ASP.NET - Single-Page Applications: Build Modern, Responsive Web Apps with ASP.NET
Maravilhas da Web Moderna
Comment: Não estou dizendo que sua resposta esta errada, essa parte ''teórica'' eu já sabia, ate alguns dos sites que deu de referencia eu havia entrado, mas o que se matem são as duvidas que coloquei aqui na pergunta.
Comment: @Marcial, editei a resposta enfatizando suas perguntas.
Comment: foi mal se caso anteriormente eu pareci "arrogante" ou algo parecido, e acabei não pedindo se pudesse complementar com educação, já irei ler novamente, grato. @Taisbevalle
Comment: @Marcial, não tem problema, achei que você queria saber um pouco da teoria também, com o seu comentário pude melhorar a resposta =)
Comment: É realmente, vai depender da situação e do que o cliente deseja, requisitos, se o foco é mais na usabilidade, arquitetura disponível e etc. Eu venho do back, e não sou fan da sintaxe do javascript(typescript *-*), mas como não posso contar com alguém como front, vou ter que me tornar "full-stack" ou pelo menos quase isso kkk'. Sobre Rest, mesmo que não tenha comentado, seguindo a mesma situação das SPA, não se encaixando em todas situações, é de grande relevância poder servir varias aplicações e ate distribuir seus serviços e etc, Dês de já, obrigado @Taisbeballe
Comment: @Marcial, de nada =)

Web Moderna: SPA/REST


Suppose $(X_n)_{n\geq 0}$ is a Markov Chain, would it be the case that $(Y_n)_{n\geq 0}:=X_{n+1}-X_n$ also a Markov Chain?

Question: Suppose $(X_n)_{n\geq 0}$ is a Markov Chain, would it be the case that $(Y_n)_{n\geq 0}:=X_{n+1}-X_n$ is also a Markov Chain? To avoid any potential confusion, what I meant was $Y_0=X_1-X_0$, $Y_1=X_2-X_1,\cdots .$
I am really not sure how to prove this or disprove this. My intuition tells me that since $X_{n+1}$ only depends on $X_n$ (loosely speaking) and thus $X_{n+1}-X_n$ should also just only depends on $X_n-X_{n-1}$ (loosely speaking). However I could not convert this intuition to a proper, rigorous argument.
Could someone please help me with this rather simple-looking problem? Many many thanks in advance!
Comment: Hint: think about the gambler's ruin problem where the gambler stops at certain money thresholds. Does knowing whether the gambler won his last game tell you whether he plays the next one?
Comment: @Ian Ah Understood! Thank you so much Ian!
Comment: Since that helped you, I'll just copy it as an answer so you can accept it.
Answer: Hint: think about the gambler's ruin problem where the gambler stops at certain money thresholds. Does knowing whether the gambler won his last game tell you whether he plays the next one?


Can I pass an "at" parameter for the x-axis locations of bars, to an R barplot?

Question: Hi in base graphics in R, I would like to overlay a bar plot on a chart which has irregularly spaced points on a curve. The bars should represent the change in these points since the previous sample. But I would like the bars to align to the points in the horizontal direction, so there should basically be a lot of whitespace, and wherever there's a point in the line chart, a bar should also appear at the same x-axis location, showing the change. The bottom of bars will start either on the x-axis (i.e. at y=0), or possibly underneath as a separate chart.
How would I go about doing this in R?
I know I can do this with bloxplots, as you can see in the chart in question below, where the boxes align to the various points (which are, fyi, south african government bonds). Now how do I add the bar plot?
Answer: barplot() is just a wrapper for rect(), so you could add the bars yourself. This could be a start:
x    <- sort(sample(1:100, 10, replace=FALSE)) # x-coordinates
y    <- log(x)                                 # y-coordinates
yD   <- c(0, 2*diff(y))                        # twice the change between steps
barW <- 1                                      # width of bars

plot(x, y, ylim=c(0, log(100)), pch=16)
rect(xleft=x-barW, ybottom=0, xright=x+barW, ytop=yD, col=gray(0.5))

Your second idea could be realized by splitting the device region with par(fig).
par(fig=c(0, 1, 0.30, 1))                    # upper device region
plot(x, y, ylim=c(0, log(100)), pch=16)
par(fig=c(0, 1, 0, 0.45), bty="n", new=TRUE) # lower device region
plot(x, y, type="n", ylim=c(0, max(yD)))     # empty plot to get correct axes
rect(xleft=x-barW, ybottom=0, xright=x+barW, ytop=yD, col=gray(0.5))

Comment: Awesome thanks Caracal. Off I go and do it that way.


Add html markup to Drupal Webform

Question: I want to extend the simple text the Webform module allows for radio buttons to include HTML markup.
The Drupal Options Markup module purports to add markup to the text for radio buttons. It does, but it also seems to be very buggy and causes missing index errors in the Webform module.
Hand coding seems to be a better option. Digging into hook_form_alter, I can see where the array is built.
Here is the question: How do add markup to the plain text of the #options array. From the forms api reference, the t function is used. Can I insert HTML into that?
   '#title' => t('Selected'),
   '#options' => array(
      0 => t('No'),
     1 => t('Yes'),

The result I want can be seen in the Illustration from webform options markup.
Answer: You can't add html to the t() function.
A quick (and maybe dirty way) is to just do something like this:
   '#title' => t('Selected'),
   '#options' => array(
      0 => t('No') . 'Markup',
     1 => t('Yes') . '

Markup

',

Comment: Thank you for the suggestion. But I want to add the markup to the options, not the title. (see the graphic in my original post for illustration).
Answer: You can add html to the t() function but i do not like it and i think it is not recommended.
quoting @thumbson

The multiple checkboxes/radios field types do not support
  field_prefix or field_suffix, though #prefix or #suffix should be possible. However, given that he's trying to add HTML to each option
  rather than the field, I don't think this will work.

Maybe this can help to achieve what you want #field_prefix:

Description: Text or code that is placed directly in front of the
  textfield. This can be used to prefix a textfield with a constant
  string.

And #field_suffix

Description: Text or code that is placed directly after a textfield.
  This can be used to add a unit to a textfield.
Comment: The multiple checkboxes/radios field types do not support #field_prefix or #field_suffix, though #prefix or #suffix should be possible. However, given that he's trying to add HTML to each option rather than the field, I don't think this will work.
Comment: Yes, i just noticed that after posted, i edit the post.

Add html markup to Drupal Webform


Statistical usage of WPF / Winforms / ClickOnce /

Question: I'm after some sort of statistic of usage of WPF in programming application.
Is there any reference online or stats, such as the browser usage stats, for WPF vs Winforms, ClickOnce vs MSI, C# vs. VB, etc.?
Also is there such thing per year? (ie. tracking the evolution in the usage of WPF)
I've googled but no luck so far.
Thanks.
Answer: It's not exactly scientific, but you could use the StackOverflow feed data to get stats on how many questions are asked regarding each of these technologies.
Other than that, I think this is a really tough one since many of the apps are inside firewalls. Maybe one of the big research firms has stats, but even their stats would be estimates (IMHO).
Answer: Another non-scientific idea is to type winforms or wpf in the nuget package manager and see how many downloads the first ten nuget packages have.


A question about Exponents

Question: I've been reading about Exponents, and I was wondering if there is a shorter way to do this same calculation, below:
24 = 2 * 2  *  2 * 2 = 16
I keep seeing what seems to me a pattern in this, and other examples. The pattern I keep seeing (maybe it's a coincidence?), is that they always seem to only multiply the base number by the base number, until it reaches the same value that you would reach if you just multiplied the exponent by itself, just once.
What I mean is, can we not just do it this way instead?
24 = 4 * 4 = 16
I.e. Instead of multiplying the base by the base a billion times, we just multiply the exponent by itself, once.
And how do we calculate the power of using just the Windows calculator?
Comment: A rule which only works in one (or just a handful) of cases is not a very good rule to use.
Comment: I agree. But I have no idea about exponents so I thought I would ask.
Comment: Yes, but what I'm getting at is that if you have a conjecture based on one observation only, you might try a few other random combinations to see if it holds, rather than proceeding with the conjecture :)
Answer: What you saw is a bit of a coincidence since $2^2=4$.
What was used here is this: $$2^{4}=2^{2\cdot2}=(2^{2})^{2}=4^{2}=16$$
Answer: The pattern is a only coincidence that occurs when the base is 2.
Take an example when the base is 3:
$3^4 = 3*3*3*3 = 81$
$3^4$ is not $4*4 = 16$
Comment: If the exponent is a large number, as in  $2^{500}$, you can save a lot of multiplication by calculating $2^{250}$ and then multiplying that once by itself.  To calculate $2^{250}$ you can calculate $2^{125}$ and multiply it once by itself.  To calculate $2^{125}$ you can calculate $2^{62}$, multiply it by itself to get $2^{124}$ and the multiply by 2 to get $2^{125}$. Doing it this way takes a lot less arithmetic than multiplying by two 499 times.
Comment: Thank you very much, @Cheeku! So, in every case, all that you need to do, is to multiply the base by the number of the exponent. But what if the exponent is a large number, like 500? Is there a shorter way to calculate it?
Comment: You approximate them with something known as the concept of logarithms and anti-logarithms
Comment: Thank you. I will look them up. I will accept the answer when the timer runs out.
Answer: Your result is a coincidence. The reason it works for the example is that $2^4=(2^{2\cdot2})=(2^2)^2=4^2$. It is because $a^{bc}=(a^b)^c$.  If speed in exponentiation is what you want, the standard divide and conquer algorithm will work. It goes like this: $a^{2b}=(a^b)^2$,
$a^{2b+1}=a(a^b)^2$
For example
$2^{10}=(2^5)^2$
$2^5=2(2^2)^2$
$2^2=4$
$2^5=2.4.4=32$
$2^10=32.32=1024$
Note that this reduces the no. of multiplications. It may not be of much help to you as it is already implemented in all calculators, and is tedious to do by hand.
Answer: You certainly all know about the coincidence
$$\frac{\not \!6 4}{1 \!\!\not \!6} = 4.$$
Answer: That is simply a coincidence. It is because 2^4 = 2*2*2*2 = (2*2)*(2*2) = 4*4. Therefore in this case it works because 2^2 = 4 but, for example, 5^4 != 4*4 but 5^4 = 25*25 since 5^2 = 25.

A question about Exponents


For Loop in print statement is giving generator funtion as output

Question: This is my program I can't understand what is wrong with this program.
I want to Print the pattern shown in the given sample output
for i in range(1,int(input())):
    print(i for x in list(range(0,i)))

Sample Input:
5
Sample output:
1
22
333
4444

Output Given By the program:
<generator object <genexpr> at 0x7feb4598cdb0>
<generator object <genexpr> at 0x7feb4598cdb0>
<generator object <genexpr> at 0x7feb4598cdb0>
<generator object <genexpr> at 0x7feb4598cdb0>

Comment: I believe you need to wrap the loop with `[` and `]`, ie `print([i for x in list(range(0,i))])`
Comment: `print("".join(str(i) for x in list(range(0,i))))` ?
Answer: for i in range(1,int(input())):
    print([i for x in range(0,i)])

Generators in python are defined with (), make sure you use the brackets, to make a list comprehension.


Deconvolutions/Transpose_Convolutions with tensorflow

Question: I am attempting to use tf.nn.conv3d_transpose, however, I am getting an error indicating that my filter and output shape is not compatible.

I have a tensor of size [1,16,16,4,192]
I am attempting to use a filter of [1,1,1,192,192]
I believe that the output shape would be [1,16,16,4,192]
I am using "same" padding and a stride of 1.

Eventually, I want to have an output shape of [1,32,32,7,"does not matter"], but I am attempting to get a simple case to work first.
Since these tensors are compatible in a regular convolution, I believed that the opposite, a deconvolution, would also be possible.
Why is it not possible to perform a deconvolution on these tensors. Could I get an example of a valid filter size and output shape for a deconvolution on a tensor of shape [1,16,16,4,192]
Thank you.
Answer:

I have a tensor of size [1,16,16,4,192]
I am attempting to use a filter of [1,1,1,192,192]
I believe that the output shape would be [1,16,16,4,192]
I am using "same" padding and a stride of 1.

Yes the output shape will be [1,16,16,4,192]
Here is a simple example showing that the dimensions are compatible:
import tensorflow as tf

i = tf.Variable(tf.constant(1., shape=[1, 16, 16, 4, 192]))

w = tf.Variable(tf.constant(1., shape=[1, 1, 1, 192, 192]))

o = tf.nn.conv3d_transpose(i, w, [1, 16, 16, 4, 192], strides=[1, 1, 1, 1, 1])

print(o.get_shape())

There must be some other problem in your implementation than the dimensions.
Comment: @DevinHaslam I would definitely not use 3298 channels in the first place. Why did you want to use so many? I never did hired tutoring and I don't really have time for it, sorry.
Comment: @DevinHaslam the number of features is the same as the number of channels. But residual connections (as defined in https://arxiv.org/pdf/1512.03385.pdf) are not concatenations, but a sum of the skipped and not skipped channels. However the channels of orange lines don't match in inception B and C. The channel size in the beginning and end of an orange line must be the same to make their addition possible. Either there are wrong numbers in the pictures or the orange lines are not residual connections.
Comment: Thank you for your help. This information helped me solve my problem almost instantly. My problem was that I was calling the conv3d_transpose like this: o = tf.nn.conv3d_transpose(i, [1, 1, 1, 192, 192], [1, 16, 16, 4, 192], strides=[1, 1, 1, 1, 1]). Very dumb problem, but I am new to tensorflow and never would have recognized this issue myself. I can't thank you enough.
Comment: Using the implementation that you provided here, I have run into some problems. Sometimes I have a filter size that has a large number of channels. For example [16,16,7,3298], also described here: https://stackoverflow.com/questions/46955515/initializing-a-large-tf-variable-produces-an-error. Some people have suggested that I split up the filter size ("w" in your implementation) into smaller tensors. Does that make sense to do here, or does it not make sense to have a filter with 3298 channels in the first place? Also do you ever do hired tutoring/help?
Comment: This is the architecture that I am replicating: i.stack.imgur.com/1qLP2.png As you can see there are many layers and several indications of concatenation (the orange arrows are shortcuts for residual learning). The reason that there are a large amount of channels is due to the concatenation I am doing. I use tf.concat, I concatenate along the axis of the channels. Does this sound correct? Is there another way to concatenate that wouldnt make my channels so large? Thanks!
Comment: In the image I provided, the number of feature maps can be seen in parentheses. I have constructed my architecture with the assumption that feature maps are the same as channels. Is this a correct assumption?


How were the parameters for a "quality" start defined?

Question: A "quality start" for a pitcher in baseball is defined as giving up no more than three runs in six or more innings of work. For frames of reference, a starter needs to complete at least five full innings (a majority out of nine) to be eligible for a win, and three runs in six innings would be a 4.50 ERA. So how did this definition come about?
I've worked with a sliding scale, where a "quality start equivalent" (QSE) which would require at least five innings of work, and the number of innings minus the number of runs cannot be less than three. That is to say, five innings and two runs would qualify, as would seven innings and four runs. Has anyone in the sport used, or proposed, a similar "equivalent?" (On this scale, the Pittsburgh Pirates, who have "only 16 quality starts in 26 games, would have 20 QSE's. Other teams would benefit from this adjustment, of course but not to the same degree.)
Another thing that puzzles me is that a quality start is "retracted" if a starter goes six innings with no more than three runs, then pitches the seventh (and possibly an eighth) and gives up a fourth run. The way I look at it is that the pitcher earned a quality start for "three in six," then functioned as a "middle reliever," for the seventh inning. Put another way, the pitcher gave his manager the option of using only one inning of "middle" relief, even though the manager didn't use this option. Or don't the scorers see it that way?
Comment: Even though a player may ear a quality start, it is such an outdated stat.  Very few if any baseball people will consider a pitcher's quality starts for some of the reasons you mentioned.  4 runs through a 9 inning complete game is a better ERA than 3 through 6 innings but not a "quality start".  There are much more important and telling sabermetrics that front offices use to evaluate a pitchers performance.  Also its not retracted because the start wasn't complete.
Comment: @digger3: It seems unfair that six innings three runs is a quality start, and eight innings four runs isn't. But the answer is more telling: if the "unwritten" rule is that most quality starts feature fewer than three runs, then four runs in eight isn't so great.
Comment: That stat in the answer is a great stat.  But like I said the whole quality start thing is outdated anyway.
Answer:
In baseball, a quality start is a statistic for a starting pitcher defined as a game in which the pitcher completes at least six innings and permits no more than three earned runs.
The quality start was developed by sportswriter John Lowe in 1985 while writing for the Philadelphia Inquirer.[1] ESPN.com terms a loss suffered by a pitcher in a quality start as a tough loss and a win earned by a pitcher in a non-quality start a cheap win.[2]
Nolan Ryan has used the term "High Quality Start" for games where the pitcher goes seven innings or more and allows three earned runs or less.

Basically, as this quote states, the quality start statistic was brought about to point out when a pitcher might have lost, but still pitched a quality game, in which he could have, or even should have won.
The Quality Start states that it is completed when a pitcher pitches "AT LEAST" 6 innings, while giving up no more than 3 runs.  So the reason it is retracted if he pitches a 7th or 8th inning and gives up a 4th run is because the rule does not simply state if he pitches 6 innings and gives up no more than 3 runs in those 6 innings, he gets a quality start, anything after that is gravy.  It simply states that to qualify for a quality start, you must pitch at least 6 innings, meaning you still can't give up a 4th run after the 6 innings have been completed.  [3]

An early criticism of the statistic, made by Moss Klein, writing in The Sporting News, is that a pitcher could conceivably meet the minimum requirements for a quality start and record a 4.50 ERA, seen as undesirable at the time. Bill James addressed this in his 1987 Baseball Abstract, saying the hypothetical example (a pitcher going exactly 6 innings and allowing exactly 3 runs) was extremely rare amongst starts recorded as quality starts, and that he doubted any pitchers had an ERA over 3.20 in their quality starts. This was later confirmed through computer analysis of all quality starts recorded from 1984 to 1991, which found that the average ERA in quality starts during that time period was 1.91.

Quality Start
Comment: My understanding was that at its "outer limit" (only six innings and a 4.50 ERA), a quality start gave a team a 50-50 chance of victory against a "random" opponent. Anything better, of course, would make the team a favorite.


explain "open" by examples

Question: I am reading the beginning pages of Lee's Introduction to smooth manifolds. It reads " set Xis open", "inverse map F is open in X", etc.
I am confused about the concept of "open". For example, a function $F: \mathbb{R} \rightarrow \{1\} $, i.e., $f(x)=1$. Is the inverse map $F^{-1}$ open? and why?
Another example is bounded periodic function, say,  $F: \mathbb{R} \rightarrow [-1,1] $, i.e., $\cos(x)$. Is the inverse map $\cos^{-1}$ open? and why?
It's great if one explains to me the motivation bedhond the "open" definition. Thanks!
Comment: *Open set* and *open function* are two different (though not completely unconnected) notions of *open*. Do you at least know their definitions?
Comment: Well, the first definition appears in the book is "open set", and the definition says "$X$ and $\Phi$ are open" ..., the definition is a dead loop, that is the exact place I get confused about "open set".
Comment: Do you have previous experience with general topology?
Comment: I don't think you should read differential geometry without a having basic understanding of general topology. My advice is to go through Munkres's text on topology.
Answer: The open sets of a space $X$ are precisely the subsets of $X$ that are elements of the given topology $\mathcal{T}_X$ on $X$.
A topology $\mathcal{T}_X$ on $X$ is a family of subsets of $X$, that is is a subset of the powerset $\mathcal{P}(X)$, that satisfies the axioms of a topology, namely

$X,\varnothing \in \mathcal{T}$
closed under any union $\bigcup U_i$ of elements $U_i$ of $X$
closed under finitely many intersections $\bigcap\limits_i^n U_i$ of elements $U_i$ of $X$

This is first and foremost the axiomatic introduction to what an open set is supposed to be.
Now for $\mathbb{R}^n$ the open sets are precisely what we would expect of an open set to be. For the sake of simplicity consider $\mathbb{R}$ with the standard topology $\mathcal{T}$. The open sets are the open intervalls $(a,b) \subset \mathbb{R}$.
Now one reason why open sets are such a big deal is because the structure preserving maps (i.e. morphisms) in the category of topological spaces are precisely the continous maps.
Let $X, Y$ be topological spaces with their respective topologies $\mathcal{T}_X, \mathcal{T}_Y$. A map $f\colon X\to Y$ is continous if the preimage $f^{-1}(U)$ of any open set $U \in \mathcal{T}_Y$ is open in $\mathcal{T}_X$.
You mentioned something about open inverse functions, i assume that you were most likely be talking about preimages $F^{-1}(U)$ of open sets $U$ under a given function $F$ to be open.
However, there is indeed a notion for open maps. Namely, a function $g\colon X\to Y$ between topological spaces $X,Y$ is open if the image $g(V) \subset Y$ of any open set $V\in X$ is open in $Y$.
Your example of the inverse of the constant map $F\colon \mathbb{R} \to \{1\}$ hardly qualifies since the inverse $F^{-1}\colon \{1\} \to \mathbb{R}$ is not well defined.

explain "open" by examples


How to configure a LSI MegaRAID controller to host non-RAID backup disks which allow hot swap

Question: I'm running a server with a LSI MegaRAID controller that has 4 slots in the backplane for 4 SATA HDDs. Two of those slots are used for HDDs which are configured as RAID1 which is used as the "main storage" for the server.
Now I'm planning to make my backup procedure more robust. Currently I use 3 portable USB SSD drives where I connect one at a time to the server and let the backup software store the backup data on the SSD. I regularly flip the conntected SSD and store the other ones in a secure place. This setup works mostly well, except that the USB connection sometimes causes problems.
My idea is now to get two more caddies for the HDD slots of the server and to mount SATA SSD drives into those. In the end, I have 3 caddies which one SSD in each that are supposed to serve as backup media. In the same way as with the previous USB drives, I will the plug them round robin (one at a time) into the server (slot 0) and make the backup software store its data on the currently plugged in backup disk.
That is the theory. How should I configure the three SATA backup HDDs in the LSI/MegaRAID controller? One of the most important points is that I do not want to reconfigure/repare the setup every time I pull out and flip in a backup disk. My ideal setup would be that I can plug in any of those 3 disks into slot 0 of the backplane and the controller just mounts the drive in the same way as a USB disk would do, that I just plug into the bus.
Any ideas or recommendations would be great!
Thanks!
Answer: You can run disks in a non-RAID configuration on a MegaRAID using the JBOD option.
However, since a RAID controller expects to always see specific disks and requires importing 'foreign' disks, a rotation scheme regularly causes drive errors (when removing) and careful insertion of (re)imported previous drives - both can seriously impact server monitoring and even resilience.
I'd seriously recommend not running that rotation scheme on the MegaRAID but on a separate, hot-plug capable SAS or SATA HBA. A simpler HBA doesn't (or shouldn't) have problems with rotating drives.
If you can't rededicate a drive cage to the new HBA I'd use an external case. If you use internal drive slots do mark them well to avoid accidental removal of production drives.
Comment: Ok, thanks. I will give it a try... - and to let everyone know: It works. The MegaRAID seems to have no problem if one SATA port of the backplane is used by a different controller. The only thing is, that the LED indicators of the drive do not work any more (for the SSD on the different controller!); but that is, according the manual of the mainboard correct. So, I now have a setup where I can swap my backup SDDs easily on the fly without messing up with my RAID configuration. Thanks for your help!
Comment: Thanks for your feedback. The idea of using a separate "dumb" SATA controller for this purpose sounds feasible. Thanks!
Comment: My SuperMicro AS-1013S-MTR board has 4 SATA connectors which are all not used currently. Instead, the LSI MegaRAID controller card is connected with the 4 bays of the backplane. I checked the manual of the BPN-SAS3-815TQ backplane but I'm sure if I can just rip off one of the SATA cables coming from the MegaRAID controller and instead use a SATA cable to directly connect this bay to one of motherboards own SATA connectors, since besides the 4 SATA cables, there is also an extra cable called "sideband" which connects the backplane and the MegaRAID. Can this cause problems (mixed controllers)?
Comment: Can't be 100% without more information, but when each bay is connected separately you can reconnect it to the SATA HBA - just make sure they're hot plug. The sideband connector likely won't hurt, but do make sure it doesn't signal fake drive swaps (swaps should be signaled in band).


How do I add a boolean column to an SQL result set based on comparison with a second table

Question: Essentially I have the first part of the question answered and here is the SQL statement I have so far...
SELECT DbaRolePrivs.GRANTEE,
       DbaRolePrivs.GRANTED_ROLE,
       ApplicationRoleDefinition.ROLE,
       ApplicationRoledefinition.description_tx,
       CASE
         WHEN GRANTED_ROLE = ROLE AND GRANTEE = USERNAME THEN
           'T'
         ELSE
           'F'
       END AS checkMark
FROM DBA_ROLE_PRIVS DbaRolePrivs, APPLICATION_ROLE_DEFINITION ApplicationRoleDefinition

Here is the logic I am trying to perform in plain english.
"Given a list of all roles and their descriptions, mark each role as TRUE or FALSE based on if the current selected USERNAME has been granted that role based on their own GrantedRoles list."
Right now it is of course returning a double list since the "case when" section compares each of the granted roles with the total list of roles.  (I say duplicate because the test user only has two roles, in the current SQL query it will return a row per granted role for every possible role.
I've barely worked with SQL so I'm more used to C++, Objective-C, etc.  Really all I need in the "case when" section is to check if the current ROLE exists at all in the GRANTED_ROLE column.  I've seen the EXISTS function but that seems to be meant solely for result sets such as "WHERE EXISTS."
Thanks in advance, I know it's probably simple for anyone whose worked with databases often, I've only just started.
Comment: `'T'` and `'F'` aren't truly booleans...
Comment: @Matt Ball: They are as good as you can get in Standard SQL-92. Arguably SQL-99's `BOOLEAN` type isn't a true Boolean either: with SQL's three-valued logic, all bets are off.
Answer: You need to outer join and then test whether the join was successful or not. If the outer join fails a row is still returned but all the DbaRolePrivs columns will be null
SELECT  DbaRolePrivs.GRANTEE,
        DbaRolePrivs.GRANTED_ROLE,
        ApplicationRoleDefinition.ROLE,
        ApplicationRoledefinition.description_tx,
        NVL2( DbaRolePrivs.GRANTED_ROLE, 'T', 'F' ) AS checkMark
FROM    DBA_ROLE_PRIVS DbaRolePrivs
    ,   APPLICATION_ROLE_DEFINITION ApplicationRoleDefinition
WHERE   ApplicationRoleDefinition.ROLE = DbaRolePrivs.GRANTED_ROLE (+)
AND     USERNAME                       = DbaRolePrivs.GRANTEE      (+)


Update entityManager()->getViewBuilder from 5 year old site?

Question: I have a Drupal 8 module which hasn't been updated for 5 years.
I m trying to update to current version of Drupal.
The error I get is
Error: Call to undefined method Drupal::entityManager()

The code causing the error is below.
      $viewbuilder = \Drupal::entityManager()->getViewBuilder('paragraph');
      $data = $viewbuilder->view($paragraph);
      $array['#cache']['tags'][] = 'node:' . $node->id();

When I search one suggestions is to load through drupal service like below
\Drupal::service('entity_field.manager')->getViewBuilder('paragraph');

This gives a similar error, as does using EntityTypeManager
\Drupal::EntityTypeManager()->getViewBuilder('paragraph');

How do I update this code?
Thanks for any help
Comment: Welcome to Drupal Answers! From the question, I understand that every time you call a `\Drupal` method, you get an error saying that method does not exist. Is that correct? Otherwise, what you do you mean with *a similar error*?
Answer: View builders are now retrieved from the service entity_type.manager, which can be retrieved globally using \Drupal::entityTypeManager():
\Drupal::service('entity_type.manager')->getViewBuilder('paragraph');

Or:
\Drupal::entityTypeManager()->getViewBuilder('paragraph');

Answer: In Drupal 10, the code to get an entity view builder is used, for example, in BlockViewBuilderTest::testBasicRendering().
$builder = \Drupal::entityTypeManager()->getViewBuilder('block');
$output = $builder->view($entity, 'block');

The same code is used in Drupal 9 and Drupal 8.9.
Older code used Drupal::entityManager(), but as the documentation page says, that method has been deprecated and removed in Drupal 9. \Drupal::entityTypeManager() should be instead used in most cases.
If you get an error about \Drupal::entityTypeManager() being a non existing method, then the core/lib/Drupal.php file is probably corrupted.
The entity_field.manager is a completely different service and it is not the service you need to use. The class implementing it (EntityFieldManager) does not define any getViewBuilder() method.