c# - Regular Expression to split by delimiter and text qualifier

Question: i need to split a text file, with values separated by comma and with text qualifier like ¨|¨
I was trying to use these function:
    public string[] Split(string expression, string delimiter,
                string qualifier, bool ignoreCase)
    {

        string _Statement = String.Format
            ("{0}(?=(?:[^{1}]*{1}[^{1}]*{1})*(?![^{1}]*{1}))",
                            Regex.Escape(delimiter), Regex.Escape(qualifier));

        RegexOptions _Options = RegexOptions.Compiled | RegexOptions.Multiline;
        if (ignoreCase) _Options = _Options | RegexOptions.IgnoreCase;

        Regex _Expression = new Regex(_Statement, _Options);
        return _Expression.Split(expression);
    }

to process a text file with rows like this one:
¨|¨column 1¨|¨,¨|¨column 2¨|¨,¨|¨column 3¨|¨,¨|¨column 4¨|¨
But my regex expression is not working...
Any ideas that could help me to make this work?
Thanks in advance
Answer: You can do this without a Regex, just split the string by ¨|¨ then each item by a space to get the individual key/value  e.g.
foreach (var item in str.Split(new[] { "¨|¨" }, StringSplitOptions.RemoveEmptyEntries))
{
    var tokens = item.Split(' ');
    Console.WriteLine(tokens[0]);
    Console.WriteLine(tokens[1]);
}

Answer: Not really sure why you need Regex for something like this, string.Split can give you the output you need like:
string str = "¨|¨column 1¨|¨,¨|¨column 2¨|¨,¨|¨column 3¨|¨,¨|¨column 4¨|¨";
string[] splitArray = str.Split(new[] { "¨|¨,", "¨|¨" }
                                        , StringSplitOptions.RemoveEmptyEntries);

For output:
foreach (var item in splitArray)
{
    Console.WriteLine(item);
}

Output:
column 1
column 2
column 3
column 4

Comment: @JenishZinzuvadiya, yes it shouldn't, but where did you get this pattern from, it is not in the question...
Comment: @JenishZinzuvadiya, ok, but the answer is for the current question/current pattern.
Comment: This will not work when textqualifier is in content of the string like this  "TEST",""TEST"","TEST"
Comment: I have that pattern in my csv file and I want to split that?
Answer: In .net, we can do this! :)
I just pushed through it and feel like sharing.
This is a pretty full regex solution to splitting a delimited file row:
    private bool RowMe(string strColumnDelimiter, string strTextQualifier, string strInput, out string[] strSplitOutput, out string strResultMessage)
    {

        string[] retVal = null;
        bool blnResult = false;
        strResultMessage = "";

        //---- We need to escape at least some of the most common
        //              special characters for both delimiter & qualifier ----

        switch (strColumnDelimiter) {

            case "|":
                strColumnDelimiter = "\\|";
                break;

            case "\\":
                strColumnDelimiter = "\\\\";
                break;

        }

        switch (strTextQualifier)
        {

            case "\"":
                strTextQualifier = "\\\"";
                break;

        }

        //---- Let's have our delimited row splitter regex! ----
                                string strPattern = String.Concat(
                                            "^"

                                                ,"(?:"

                                                    ,"("
                                                        , "[^\\S" + strColumnDelimiter + strTextQualifier + "]*"        // allow leading whitespace, not counting our delimiter & qualifier

                                                        ,"(?:"
                                                            ,"(?:[^" + strColumnDelimiter + strTextQualifier +"]*)"         // any amount of characters not colum-delimiter or text-qualifier
                                                            ,"|"
                                                            , "(?:" + strTextQualifier + "(?:(?:[^" + strTextQualifier + "])|(?:" + strTextQualifier + strTextQualifier + "))*" + strTextQualifier + ")"        // any amount of characters not text-qualifier OR doubled-text-qualifier inside leading & trailing text-qualifier (allow even colum-delimiter inside text qualifier)
                                                            ,"|"
                                                            ,"(?:(?:[^" + strColumnDelimiter + strTextQualifier + "]{1})(?:[^" + strColumnDelimiter + "]*)(?:[^" + strColumnDelimiter + strTextQualifier + "]{1}))"     // any amount of characters not column-delimiter inside other leading & trailing characters not column-delimiter or text-qualifier (allow text-qualifier inside value if it is not leading or trailing)
                                                        ,")"

                                                        , "[^\\S" + strColumnDelimiter + strTextQualifier + "]*"        // allow trailing whitespace, not counting our delimiter & qualifier
                                                    ,")"

                                                , "){0,1}"

                                                            //-- note how this second section is almost the same as the first but with a leading delimiter...
                                                            //                  the first column must not have a leading delimiter, and any subsequent ones must
                                                , "(?:"
                                                    ,"(?:"
                                                        , strColumnDelimiter        // << :)
                                                        ,"(?:"

                                                            , "("
                                                                , "[^\\S" + strColumnDelimiter + strTextQualifier + "]*"        // allow leading whitespace, not counting our delimiter & qualifier

                                                                , "(?:"
                                                                    , "(?:[^" + strColumnDelimiter + strTextQualifier + "]*)"           // any amount of characters not colum-delimiter or text-qualifier
                                                                    , "|"
                                                                    , "(?:" + strTextQualifier + "(?:(?:[^" + strTextQualifier + "])|(?:" + strTextQualifier + strTextQualifier + "))*" + strTextQualifier + ")"        // any amount of characters not text-qualifier OR doubled-text-qualifier inside leading & trailing text-qualifier (allow even colum-delimiter inside text qualifier)
                                                                    , "|"
                                                                    , "(?:(?:[^" + strColumnDelimiter + strTextQualifier + "]{1})(?:[^" + strColumnDelimiter + "]*)(?:[^" + strColumnDelimiter + strTextQualifier + "]{1}))"        // any amount of characters not column-delimiter inside other leading & trailing characters not column-delimiter or text-qualifier (allow text-qualifier inside value if it is not leading or trailing)
                                                                , ")"

                                                                , "[^\\S" + strColumnDelimiter + strTextQualifier + "]*"        // allow trailing whitespace, not counting our delimiter & qualifier
                                                            , ")"

                                                        ,")"
                                                    ,")"
                                                , "){0,}"

                                            ,"$"
                                        );
                                        );

        //---- And do the regex Match-ing ! ----
        System.Text.RegularExpressions.Regex objRegex = new System.Text.RegularExpressions.Regex(strPattern);
        System.Text.RegularExpressions.MatchCollection objMyMatches = objRegex.Matches(strInput);

        //---- So what did we get? ----
        if (objMyMatches.Count != 1) {
            blnResult = false;
            strResultMessage = "--NO-- no overall match";
        }
        else if (objMyMatches[0].Groups.Count != 3) {
            blnResult = false;
            strResultMessage = "--NO-- pattern not correct";
            throw new ApplicationException("ERROR SPLITTING FLAT FILE ROW!  The hardcoded regular expression appears to be broken.  This should not happen!!!  What's up??");
        }
        else {

            int cnt = (1 + objMyMatches[0].Groups[2].Captures.Count);

            retVal = new string[cnt];

            retVal[0] = objMyMatches[0].Groups[1].Captures[0].Value;

            for (int i = 0; i < objMyMatches[0].Groups[2].Captures.Count; i++) {
                retVal[i+1] = objMyMatches[0].Groups[2].Captures[i].Value;
            }

            blnResult = true;
            strResultMessage = "SUCCESS";
        }

        strSplitOutput = retVal;

        return blnResult;

    }


Normal distributions obey central limit theorem

Question: Let $X_1,\dots,X_n$ be independent random variables, each normally distributed as $X_k\sim N(m_k;\sigma^2_k)$. Let $S_n = \sum_{k=1}^n X_k - m_k$ and $T_n = \frac{S_n}{\sqrt{\operatorname{Var}(S_n)}}$.
We wish to show that $\lim_{n\to\infty} T_n \sim N(0;1)$.
I've laboriously proven that $T_n \sim N(m; \sigma^2)$ where $m=\sum_k m_k$ and $\sigma^2=\sum_k \sigma^2_k$. So now I'm wondering if I'm supposed to assume that the $\lim_{n\to\infty}m=0$ and similarly $\sigma^2\to 1$, which I don't really see a strong justification for.
Is this what I'm supposed to do? What's the justification for it?
(Note this is exercise 14.31.7 in Apostol's Calculus II. Looking online it seems like some other places don't define CLT as it approaches standard normal, just normal distribution of some mean and variance. So this could be an unusual definition he's using.)
Comment: Your laborious proof that $T_n\sim N(m;\sigma^2)$ must be wrong unless $m= 0$ and $\sigma^2 = 1$. Just by the definition of $T_n$ you always have $$\sqrt{Var[S_n]} E[T_n] = E[S_n] = \sum_{k=1}^n (E[X_k] - m_k) = 0$$ and $$Var[T_n] = \frac{Var[S_n]}{Var[S_n]} = 1$$ But a $N(m;\sigma^2)$-distributed random variable has mean $m$ and variance $\sigma^2$.
Comment: Did you maybe calculate the distribution of $X_1 + \dots + X_n$ instead of the one for $T_n$? $X_1 + \dots + X_n \sim N(m;\sigma^2)$ should be right. In this case you would only need to show that if $Y \sim N(m;\sigma^2)$, then $Y-m \sim N(0;\sigma^2)$ and $(Y-m)/\sigma \sim N(0;1)$.
Comment: @SamL: That is indeed what I did, thanks!
Answer: You don't need these assumptions. In fact, $E[S_n]=0$ and $\operatorname{Var}(T_n)=1$. Furthermore, we know that a sum of two independent random variable $Y_1$ and $Y_2$ normally distributed, respectively $Y_1\sim \mathcal N(m_1,\sigma_1^2)$, $Y_2\sim \mathcal N(m_2,\sigma_2^2)$ then $Y_1+Y_2\sim\mathcal N(m_1+m_2,\sigma_1^2+\sigma^2_2)$. So by induction we deduce that $T_n\sim\mathcal N(0,1)$ for each $n$. Convergence in law follows now from the definition.
Answer: Each $X_k-m_k$ is normal and has mean 0 and $S_n$ is $N(0, Var(S_n))$ so  $\frac{S_n}{\sqrt{Var(S_n)}}$ is $N(0,1)$.


Json into Array

Question: I am reading a JSON file into one string and one array. I already have a string where the JSON is saved, let's call it myString. Here is the JSON file:

As you can see, the file contains three styles, from "styleCount": "3". My goal is to now create three string variables for each style, similar to the following pseudo variables:
String name_style1 should contain: "Sommer-Fashion"
String name_style2 should contain: "Dream-Style"
String name_style3 should contain: "Perfect-Look"

Then I need an array of strings for each style with the SKU numbers:
private String[] sku_style1 = new String[6];
sku_style1[0] = "392714";
sku_style1[1] = "395895";
sku_style1[2] = "392450";
sku_style1[3] = "371706";
sku_style1[4] = "383748";
sku_style1[5] = "385275";

And also for the other styles:
private String[] sku_style2 = new String[6];
private String[] sku_style3 = new String[6];

Is there a function of Java which helps with simply adding elements from a JSON file (or in my case a string: myString) into a string and an array?
Any help is appreciated.
Comment: Have you done any research on parsing JSON using/with Java or on any of the 3rd party libraries like GSON, etc.?
Comment: hey guys, code examples would be appreciated...just sending me links doesn't help me too much, since I am not that experienced. Thanks a lot!
Answer: Google GSON! No functions native to Java really help much, but Google GSON has helped me numerous times with issues much harder than this. I think you'll find it very helpful. Here is a link!
https://mvnrepository.com/artifact/com.google.code.gson/gson
EDIT
This link is for the repository for the jar downloads!
EDIT 2
Gson gson = new Gson();
Staff obj = new Staff();

// 1. Java object to JSON, and save into a file
gson.toJson(obj, new FileWriter("D:\\file.json"));

// 2. Java object to JSON, and assign to a String
String jsonInString = gson.toJson(obj);

Comment: do you have a practical example on how to solve this with the suggested solution, like an example line of code, thanks!
Comment: I would gladly, and I know you said "No more links", but this page is nothing but examples on using Google GSON. It's the same page I used when I used it for the first time and it really helped me format what I needed quickly! https://www.mkyong.com/java/how-do-convert-java-object-to-from-json-format-gson-api/
Comment: I have also added a block of code to my answer. +1 if it helps!
Comment: Awesome, thank you! Do you know how I can reformat the imported json to the requested String/array? Thanks a lot for your help again!
Comment: Once you've reached the end of the code block that I posted, you should have the entire JSON as a string. From there, simple String manipulation functinos that you write can search the String for specific tags/words/formats. However, while I haven't used GSON in a while, I believe that there are methods in which you pull parts of the json as a string to minimize how much you have to manipulate
Comment: I dont know how familiar you are with API since you said that you are new, but with most public jar's, documentation is usually written to explain all methods that you have access to. Here is the link to GSON's methods and other help! https://sites.google.com/site/gson/gson-user-guide
Answer: There are JSON parson libraries that serve this exact purpose.
JSONArray
JSONString
Read more here.
Comment: do you have a practical exampl how to do it in my case?

Json into Array


drupal_add_js weight

Question: I've written a javascript that for some reason whenever I run it, it removes any hover events.  Since I am running my site with nice menus I would like to run my script as high up in the $script array as possible.
I added this line to the template.php:
drupal_add_js('sites/itascacg.com/themes/custom/icg/js/fixtitles.js',array('type' => 'file', 'weight' => -19));

since according to the documentation I've read jQuery itself is at weight -20, but no matter what weight I give my script i won't float any higher on the list.
does anybody have anybody have any ideas how to push my script up, or at worst push all the other scripts down?
P.S. I'm using drupal 7
Answer: In what function in template.php are you adding this? A preprocess function I hope.
Anyway, you have to set the 'group' option as outlined on the drupal_add_js page. eg.
drupal_add_js('sites/itascacg.com/themes/custom/icg/js/fixtitles.js',array('type' => 'file', 'weight' => -19, 'group' => JS_LIBRARY));

Answer: drupal_add_js states

weight: A number defining the order in which the JavaScript is added
to the page relative to other JavaScript with the same 'scope',
'group', and 'every_page' value.
The exact ordering of JavaScript is
as follows:

First by scope, with 'header' first, 'footer' last, and
any other scopes provided by a custom theme coming in between, as
determined by the theme.
Then by group.
Then by the 'every_page' flag,
with TRUE coming before FALSE.
Then by weight.
Then by the order in
which the JavaScript was added. For example, all else being the same,
JavaScript added by a call to drupal_add_js() that happened later in
the page request gets added to the page after one for which
drupal_add_js() happened earlier in the page request.

Weight is at number fourth. Please update your scope, group and every_page parameters.
Secondly try a very smaller number for weight i.e -100000. It may work for you.

drupal_add_js weight


Job para gravar dados de uma query em um arquivo XML

Question: Preciso gravar o resultado de uma consulta em um XML através de um job que vai rodar diariamente agendado no SQL Server.
A procedure já trás o resultado no formato do XML com as tags, em uma variável.
Só preciso jogar o conteúdo dessa variável no XML.
Qual a melhor forma de gerar esse XML?
Como eu defino o caminho que o XML será gravado e como eu defino o nome do arquivo?
Valeu!
Answer: Bom, basicamente é usando o comando xp_cmdshell:
DECLARE @command VARCHAR(8000) = 'echo ' + @xml + ' > c:\meus\diretorios\teste.txt';

EXEC xp_cmdshell @command;


Mixed effects model for prediction

Question: I have a dataset where I collected cortisol samples 3 times a day, for 3 days at 2 timepoints. I am interested in looking at changes in cortisol right after awakening, at 0 min after awakening, 30 min and 60 min. However, I know that participants actually woke up earlier and took the cortisol samples much later than they were supposed to. So now I want to use the data I have to predict what the estimated cortisol values would have been if samples were taken precisely at 0, 30 and 60 min after awakening. Below is an generated example of what my data looks like:
    head(data)
    ID Trimester       Date Day sample         wakeup_time    sample_time       Cortisol time_since_wake
    1 1002        T0 01/01/2022   0      0 2022-01-01 05:40:00 2022-01-01 08:00:00 0.71343686             140
    2 1002        T0 01/01/2022   0     30 2022-01-01 05:40:00 2022-01-01 08:30:00 0.34559956             170
    3 1002        T0 01/01/2022   0     60 2022-01-01 05:40:00 2022-01-01 09:00:00 0.42300318             200
    4 1003        T0 01/01/2022   0      0 2022-01-01 03:00:00 2022-01-01 08:00:00 0.08194017             300
    5 1003        T0 01/01/2022   0     30 2022-01-01 03:00:00 2022-01-01 08:30:00 0.40100879             330
    6 1003        T0 01/01/2022   0     60 2022-01-01 03:00:00 2022-01-01 09:00:00 0.02570335             360

Using the wakeup time and sample time, I calculated the time difference. So the time_since_wake variable actually tells us when the sample was taken in relation to awakening. So in first row you can see that participant 1002 on day 0 took sample 0 at 140 min after awakening. I want to use this data and create a mixed effects prediction model, so that I can predict estimated values of cortisol samples at 0, 30 and 60 min.
For this purpose, I am implementing the below model and using the following code to predict:
model = lmer(AUCg ~ bs(time_since_wake, df = 3) + (1|ID/Trimester) +
                     (1|ID/Trimester:Day), data = data)

> summary(model)
Linear mixed model fit by REML ['lmerMod']
Formula: AUCg ~ bs(time_since_wake, df = 3) + (1 | ID/Trimester) +
           (1 |      ID/Trimester:Day)
   Data: data

REML criterion at convergence: 1.8

Scaled residuals:
    Min      1Q  Median      3Q     Max
-1.9225 -0.5841  0.0369  0.5540  2.3695

Random effects:
 Groups           Name        Variance Std.Dev.
 Trimester.Day.ID (Intercept) 0.00000  0.0000
 Trimester.ID     (Intercept) 0.00000  0.0000
 ID               (Intercept) 0.00000  0.0000
 ID.1             (Intercept) 0.00000  0.0000
 Residual                     0.05767  0.2401
Number of obs: 36, groups:  Trimester:Day:ID, 12; Trimester:ID, 4; ID, 2

Fixed effects:
                             Estimate Std. Error t value
(Intercept)                   0.44780    0.19719   2.271
bs(time_since_wake, df = 3)1  0.19440    0.48305   0.402
bs(time_since_wake, df = 3)2  0.06442    0.32564   0.198
bs(time_since_wake, df = 3)3 -0.34822    0.30261  -1.151

Correlation of Fixed Effects:
            (Intr) b(__,d=3)1 b(__,d=3)2
bs(__,d=3)1 -0.898
bs(__,d=3)2 -0.117 -0.264
bs(__,d=3)3 -0.765  0.809     -0.318
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see help('isSingular')

predicted_data = expand.grid(ID = unique(data$ID),
Trimester =unique(data$Trimester),
    Day = unique(data$Day), time_since_wake = c(0, 30, 60))

    predictions = predict(model, newdata = predicted_data)

    predicted_data$cortisol_predicted = predictions

    head(predicted_data)
    ID Trimester Day time_since_wake cortisol_predicted
    1 1002        T0   0               0          0.3940981
    2 1003        T0   0               0          0.3940981
    3 1002        T1   0               0          0.3940981
    4 1003        T1   0               0          0.3940981
    5 1002        T0   1               0          0.3940981
    6 1003        T0   1               0          0.3940981

But as you can see in this predicted_data dataframe, the value for cortisol_predicted is repeated. I noticed that it generates unique predictions for estimated cortisol value at 0, 30 and 60 min time_since_wake, but it is the same across participants, trimesters and days. In my understanding of the model, I believed that including the random effects that I have specified in the model should allow to factor the day-level, trimester-level and individual-level variations. But why is this not reflected in the predictions? I expected that I would get a unique predicted cortisol value for each combination of ID, Trimester, Day and time_since_wake. What am I doing incorrectly? I would appreciate any and all help.
Comment: Can you add your model summary to the question text?  Does it have non-zero variances for the random effects?
Comment: @GeorgeSavva the question text has been edited to include the model summary. The random effects do not have non-zero variances with this randomly-generated example dataset.
Comment: What are the variances in your real data?  If the random effect variances are zero then there will not be different predictions at different levels of the predictors.  You could also check your `ranef()`s from the model.
Comment: @GeorgeSavva Thanks so much for that hint. Being fairly new to mixed effects modeling to create such prediction models, I missed out on this bit. I tried the same model with a different random dataset with greater variability in response, which yielded non-zero variances for random effects and then the prediction works as expected. Thanks!
Answer: As alluded to in the comments, with the simulated data, the model converges to a singular fit:
boundary (singular) fit:

This can occur for several reasons but in your case it appears that you didn't simulate any variance in the random effects. Hence the estimate for the random effects would be close to zero (near or on the boundary of the parameter space).
If your real data does have variation in the grouping factors then hopefully the model will converge normally and you can then use the predict function.

Mixed effects model for prediction


Question regarding queries/personalization

Question: I am new to web development and I am aware that this is a fairly basic question.
I have a poll that creates a custom link after the poll is created with the user's email. The poll data is stored in an AWS DynamoDB table. I want to be able to customize the HTML of the website to reflect personalized data for the user using Javascript variables from data in the DynamoDB table.
How can this be done? I think it has something to do with URL queries but I honestly don't know what to do.
I've looked everywhere online. Help would be very much appreciated!
Comment: Could you share the custom link?
Comment: @DavidF http://www.crownpiccustom.com.s3-website-us-west-2.amazonaws.com/
Comment: ok, as you using Node.js for your project? If yes, this link will help you. https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/dynamodb-example-query-scan.html
Comment: let me know your questions.
Comment: You can get the custom link of datatable of dynamoDB like this. https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/dynamodb-example-table-read-write.html
Answer: The simple answer is create a lambda function with an exposed URL that securely identifies the user somehow and then, using that user information, queries your dynamodb table and returns a response with the relevant information.
The complex answer requires a lot of research into each of these mentioned services: creating a lambda; researching secure authentication and hooking up to cognito; setting up a dynamodb table; and adding permissions to your lambda so that it has the appropriate role/permissions to access both your cognito service and your dynamodb table.
I'd start with just getting a lambda set up with a URL and static return value, and getting your HTML page to work with that response.
Answer: For this to work you will need to update your websites JavaScript code to retrieve the user information from DynamoDB.
You will need to be able to obtain enough information from user to gain their information from your table.
Query Params
For example, if your table has a partition key of user_id then you would need your link to look like this:
https://www.mywebsite.com?user_id=123
Obtain the user_id from the query param in the URL:
https://stackabuse.com/get-query-string-values-in-javascript/
Retrieve user details
Do a GetItem from DynamoDB
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/dynamodb-example-table-read-write.html
Then show the data on the webpage using variables in JS


What do I do when my long fingernails for classical guitar interfere with playing piano?

Question: I'm learning classical/flamenco guitar techniques about 5 months ago, and now just started learning piano:
I get some problems when i play on keyboard(noise and nails hurts, especially the thumb).. I really love those two instrument and wants to master them..
How can i play piano without cutting my right-hand-nails?
Any suggestion?
Comment: Secondary query: are fingernails required for guitar picking? Us bass-players typically play with the fingertips rather than nails.
Comment: @CarlWitthoft - long fingernails for guitar playing help with articulation and projection among other things.  Classical/flamenco guitar technique is much different than typical guitar technique.  The alternative is to use fingerpicks similar to a banjo player, but as other people have pointed out here, they are uncomfortable and not as intuitive as using one's nails.
Comment: I'm a pianist who also plays guitar and compulsively bites my fingernails. Now I understand why I've been having trouble picking up classical guitar...
Comment: @fluffy as I say below, you can initially learn a lot of classical guitar technique without right-hand nails. The problems with biting your nails (aside from the obvious, non-guitar related ones…!) are that, as they grow even a little, they catch on the strings (as they are a little ragged and not smooth) - this makes a really bad sound! If your nails remain really short, and don't touch the strings when you play, it doesn't really matter whether you bite them, or have them beautifully manicured.
Answer: Valentin is right, keeping your nails a bit shorter is probably one of the most useful things to do.  But, let me add a few things.
I am primarily a guitarist, and I play the piano, but wouldn't consider myself a pianist; so, much of my advice is from a guitarist's perspective.
Even though most of the classical guitarists I know (including me!) use nails, you can learn to play without nails.  In fact, most of the guitar teachers I know (including me), initially teach beginner classical guitarists to play without nails.  However, for a wide range of tone, and plenty of volume and articulation, using nails generally seems best.
If you do use nails, there are still a variety of approaches to right-hand technique.  Some players play with nails alone, which requires slightly longer nails; some players initially touch the string with the flesh of the finger or thumb and the nail at the same time. Personally I favour the latter approach for several reasons: feeling the string on the flesh of finger or thumb helps me feel more in control of each stroke; the tone seems less brittle; and of course, I can keep my nails shorter!
It is also worth noting that a warmer, richer tone can be achieved by leading with the left-hand side of your finger nails (this will always happen with the thumb anyway). In order to do this, the nails are shaped to be curved, but shorter on the left-hand side (or right if you are left-handed!) This allows the nail to create a 'ramp' which releases the string more gradually. This kind of nail shaping will particularly help your piano playing when using your thumb, as it is the left-hand side of the thumb which touches the keys.
From a piano point of view, just a very slight 'flattening' of the fingers will allow you to avoid touching the keys with your nails.  But, as any pianist/piano teacher will point out, your fingers should still be fairly curved, rather than straight, with your wrist neither to low or too arched.
In the end, I compromise a little: I keep right-hand nails for guitar, but keep them quite short (although this is no big deal as I prefer this anyway, particularly as it helps to keep them in good condition); and I very slightly flatten my hand/fingers for piano playing, but certainly not to the extent that my fingers are straight. (If you don't have piano lessons, it might be a good idea to discuss this with a piano teacher.)
There is a fantastic book about why and how to shape nails for classical guitar playing called Tone Production on the Classical Guitar by J Taylor. It is now out of print, but can be found at scribd.com.
The techniques associated with flamenco playing can place more demands on your nails, and a brighter tone, which longer nails can give, may seem more attractive, but again having shorter nails should help you to keep them in good condition.
I'm sure I don't need to add that you should always keep the nails very short on your fretting hand…!
Answer: I have the exact same problem. In the end, I just kept my fingernails a bit shorter and learned to live that on some occasions my nails would interfere with the piano playing... You could get rid of the long nails and play with finger picks, but I find that highly uncomfortable.
Answer: I'm not a guitar player myself, but I met one once who used a type of finger pick that basically worked by extending the fingernail, rather than like a traditional fingerpick, that puts a substitute nail somewhere near your fingertip. He loved it, and a lot of other guitar players at the gathering decided to order their own. Since I play violin, I couldn't say what the subtle differences vs your own nails might be, but from what I'm told its pretty similar to using your own nails.  I think the brand was http://www.alaskapik.com/.
Answer: There is really no solution for having long fingernails on the left hand. You can only keep them as long as is confortable with placing your finger upright on the string.


increase in post request 404 errors and 499 status codes using nginx as reverse proxy against IIS

Question: I'm after some assistance as I've run into a couple of issues implementing nginx as a reverse proxy backing onto windows servers.  I've used the config that works elsewhere with no issues but I'm seeing issues with more failed requests (404s) and nginx specific error codes 499.
We are seeing around 200 post 404 errors in the nginx logs when using the setup and only around 100 of the same errors when running against IIS.  The files are there in most cases and the same (get requests anyway) work when hit directly on the windows server.
My first through was that there was something wrong with my setup using the post requests through nginx but it's only a small percentage so don't think there's anything fundamentally wrong with nginx post setup.
The 499 status codes, client closed connection seems a little weird.  I can see around 150-300 499 requests for 20k requests.  It may be that these were happening anyway and we are just not seeing them being reported in IIS.
Here's my nginx site config:
server {
        listen 80;
        server_name www.mydomain.com;

        access_log /var/log/nginx/www.mydomain.com/mydomain.com.log main_ext;

        location / {
                proxy_pass http://mydomain;
                health_check;
                }

                }
upstream myupsteam {
        zone myupsteam 64k;
        sticky cookie srv_id expires=1h domain=.mydomain.com path=/;
        server IP;
}

Really need to start pushing the traffic through these servers as the IIS servers are struggling with the load but just need to get to the bottom of the errors before we can sign the switch over off.
I've tried changing the proxy timeout settings, proxy bind address onto private NICs.
Anyone any ideas?
Answer:
This was meant to be a commentary, but since I don't have enough reputation on ServerFault, I had to write this as an answer. Please forgive any inconsistencies.

So far what I have experienced with NGINX HTTP 499 errors (friendly known as Client Connection Closed or Failed to load: Connection reset by peer) was caused by wrong permissions under /var/lib/nginx.
Since I have switched to a containerized web server environment using Docker, we picked NGINX to serve as a reverse proxy for our containers without having to map host ports in every container.
During the recycling process of the main proxy container in our environment, the permissions at /var/lib/nginx get messed up and that's the temporary folder that NGINX uses to store/retrieve the requests sent through the proxy_pass directive.
Properly chowning recursively through that directory to the user:group mapped to the nginx process and, when necessary, fixing permissions to 775 was the solution to the mysteriously closing connections while loading our applications.
We also needed to proxy HTTP headers in order to have our applications properly recognize (and not drop/abort the connection) that they had been handed legitimately proxied requests and also providing our apps with enough information to properly give information back.
From our staging /etc/nginx/snippets/proxy-headers.conf:

proxy_set_header    Host               $host;
  proxy_set_header    X-Real-IP          $remote_addr;
  proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
  proxy_set_header    X-Client-Verify    SUCCESS;
  proxy_set_header    X-Client-DN        $ssl_client_s_dn;
  proxy_set_header    X-SSL-Subject      $ssl_client_s_dn;
  proxy_set_header    X-SSL-Issuer       $ssl_client_i_dn;
  proxy_set_header    X-Forwarded-Proto  $scheme;
  proxy_set_header    X-Forwarded-Port   $server_port;
  proxy_set_header  Upgrade            $http_upgrade;
  proxy_set_header  Connection         "Upgrade";
  proxy_pass_header   Set-Cookie;
  proxy_http_version                   1.1;
  proxy_read_timeout                       900s;

Your needs might (and probably will) diverge. Adjust and use those as a starting point if desired. We have also tried to improve our virtual hosts configurations using snippets to spread changes across all configurations that needed to share the same headers, SSL parameters (such as allowed ciphers, for example.), etc...
In order to use snippets like that, simple include it into your virtual host, considering the snippet and path above:

include /etc/nginx/snippets/proxy-headers.conf;

PS: The above proxy-header.conf snippet was from an SSL-enabled virtual host. The SSL headers are definitively not needed if you are talking in plain HTTP


Calculating the Magnitude of Change in Hazard/Odds Ratios for Different Fractional Units of Change

Question: I have a Logistic and Cox PH model built. My covariates are Age Since Start, Incentive and Starting Balance. All covariates are numerical. The dependent variable is whether or not the account made a prepayment on their monthly bill -- A 1 for prepayment and 0 for no prepayment. After running the model and exponentiating the coefficients I have my odds ratios:

Age: .991
Incentive: 1.044
Log(Starting Balance): .883

For my Cox model after exponentiating the coefficients I have hazard rates:

Incentive: 1.09
Log(Starting Balance): .86

The incentive covariate is referring to an account's incentive to refinance which is expressed as a percentage. So given my hazard results, which I interpret as a 1% increase in Incentive increase the chances of prepayment by 9%.
Recently I was asked to find the magnitude of change in the odds/hazard ratios given, say, a .25% increase in the Incentive or a 25% increase in Starting Balance. I've tried searching for ways to do this but only came up with one similar question. How do I go about calculating this magnitude of change?
Thanks
Answer: This is easy if you want to calculate a change in log-odds or log-hazard. For a logistic model, a regression coefficient is the estimated change in log-odds per unit change in the predictor. For a Cox model, it's the change in log hazard. You don't say enough about the units of your predictors to provide specifics here. For example: is Age in months or years? Is Incentive in percentage points, basis points, or something else? Is the log used for StartingBalance the natural log, base-10 log, or something else? As you know what units you modeled in, getting corresponding changes in log-odds or log-hazard for any change in predictor values is straightforward.
What's tricky is looking for changes in odds or hazard, not on the log scale. How much a change in log-odds or log-hazard means in terms of a change in odds or hazard depends on the original value of odds or hazard from which you want to calculate the difference.
To get a change in odds or hazard, you thus need to specify both a reference condition and a new condition. Those specified conditions should include values for all predictors in the model for both conditions, including the intercept. With those, you calculate the log-odds or log-hazard for each condition. Get the odds/hazard for each condition by exponentiating, and only then calculate the difference in odds or hazard between the conditions.


Record created by class in django models not getting recorded in mysql table

Question: I created a table class in models.py in my django app. When I did the migration (into mysql), everything worked ok. I checked the mysql database and there are tables from django. One of the table is derived from the class that I created in models.py in django. I checked the structure of this mysql table, and it matches the attributes in my class.
Now I activate the django shell:
$python manage.py shell

I imported the class and created an instance.
>>> from tsts.models import Qstns
>>> Qstns.objects.all()

[Qstns is my class which represents the mysql table)
I get an empty set above, as there are no records.
Next I created an instance of this class Qstns, hoping that one record would be appended to my linked mysql table.
>>> q=Qstns(question='What is the capital of Australia?', subject='GK', topic='Countires', a='Sydney',b='Canberra',c='Melbourne',d='Brisbane',e='Adelaide',ans='b')

>>> q.save

These execute without any message.
Next I checked the id
>>>q.id

I do not get anything. It is expected to give me: 1. It looks like database table has not received this record.
>>>q.ans

It correctly gives me: 'b'
I logged into mysql and checked the linked table. It has not received the record.
Am I missing something here?
Thanks in advance for your reply.
sp
Comment: Use object.save() not object.save. Save is a class method not class property.
Answer: Everything is fine except one thing:
Use this:
q.save()

instead of:
q.save

Hope this will solve your problem.


Taking out absolute value on the solution to integral equation

Question: I have this equation:$$y=2+\int_2^x (t-ty(t))dt$$
After solving it I got the answer $-\ln|1-y|=\frac {x^2} 2-2$ although the book has the same answer without the absolute value in the logarithm, why is this possible?
E:
Attempt:
$$y=2+\int_2^x (t-ty(t))dt \implies \frac {dy}{dx}=x-xy(x)=x(1-y(x))$$
$$\frac {dy}{1-y}=xdx \implies -ln|1-y|=\frac {x^2}2+c$$
Comment: I'm interested to know how you found $\int(t-y(t))dt$
Comment: Take the derivative with respect to x and you get $dy/dx=x-xy$, separate and integrate.
Comment: Was $2+\int (t-ty(t))dt$ the original question at the beginning? I'm just a bit unclear about $\int ty(t)dt$ Presumably, you have to integrate by parts for this one but we only know y is a function of t.
Comment: I edited my post showing my work, does that make it any clearer?
Answer: The rationale for determining the unique solution is embedded in the general solution
$$-\log |1-y|=\frac12(x^2-4)\tag 1$$
Note that to arrive at $(1)$, we already imposed the condition $y(2)=2$.  Therefore, we are seeking solutions for which $y$ actually attains the value of $y=2>1$.
To resolve this further, let's solve $(1)$ for $y(x)$, disregarding the requisite condition.  We find that
$$y(x)=
\begin{cases}
1+e^{-(x^2-4)/2},&y>1 \tag 2\\\\
1-e^{-(x^2-4)/2},&y<1
\end{cases}$$
is multi-valued.  Upon inspection, we see that for $x=2$, only the first equation of $(2)$ satisfies either the condition $y(2)=2$ or the initial integral equation.  That is, by assumption, the second solution must have values less than $1$ and can therefore, never attain the value $2$.
Therefore, the unique solution to the integral equation is
$$y(x)=1+e^{-(x^2-4)/2}$$
for which $y>1$.
Finally, we can write the equivalent solution as
$$-\log(y-1)=\frac12x^2-2$$
without need for the absolute value sign.


ALB Connection Draining is always reaching the "Deregistration Delay"

Question: I'm using ECS along with ALB to expose my containers to the internet. When I'm updating a container image (I'm using CloudFormation to update the tasks and services), the target group set the connections to the old containers to Draining. The issue is that this step always take a full 5 minutes which is the same as my Deregistration Delay attribute.
According to the doc, this shouldn't happen:

Deregistration Delay Elastic Load Balancing stops sending requests to
  targets that are deregistering. By default, Elastic Load Balancing
  waits 300 seconds before completing the deregistration process [...].
If a deregistering target has no in-flight requests and no active
  connections, Elastic Load Balancing immediately completes the
  deregistration process, without waiting for the deregistration delay
  to elapse.

The obvious explanation would be that I have a persistent connection to the container. However, I'm experiencing the issue on a test service that only myself is aware of.
Comment: I don't know if this was there when you looked at it, but the docs now say "However, even though target deregistration is complete, the status of the target will be displayed as draining until the deregistration delay elapses."
Comment: I wonder if "no active connections" is only possible with an unhealthy/unresponsive target.   It would seem wrong to count health checking connections in "active,"  but balancers do seem to keep them open and reuse them.  Do you see connections via `netstat`?
Comment: @Michael-sqlbot good call, I'll try to ssh on a draining container and run a network diagnostic on it.
Answer: The documentation has meanwhile been updated:

If a deregistering target has no in-flight requests and no active connections, Elastic Load Balancing immediately completes the deregistration process, without waiting for the deregistration delay to elapse. However, even though target deregistration is complete, the status of the target will be displayed as draining until the deregistration delay elapses.
Comment: This is a royal pain... an idle service should be able to stop sooner
Comment: if the status of the target doesn't change, how can you ever know that the de-registration has completed early?


All Subsets Regression with Missing Values

Question: Consider a dataset of 81 observations with 4 measured independent variables and one dependent variable.  In this dataset, some of of the observations are missing values for some of the independent variables.  There's no pattern to the missing values, but I'm left with only 50 rows of complete data.
In R it can be generated thusly:
set.seed(1001)
X1 = c(-1,0,1)
df = expand.grid(X1=X1,X2=X1,X3=X1,X4=X1)
df$Y = df$X1+3*df$X2-0.5*df$X3+rnorm(81,0,1)
df$X1[sample(1:81,10,replace = FALSE)] = NA
df$X2[sample(1:81,10,replace = FALSE)] = NA
df$X3[sample(1:81,10,replace = FALSE)] = NA
df$X4[sample(1:81,10,replace = FALSE)] = NA

any(apply(is.na(df[,1:4]),1,all))
> FALSE

sum(apply(!is.na(df[,1:4]),1,all))
> 50

If I were to fit a regression model to this data I would use only the 50 complete data points.  However, I wish to identify the 'best' model and so I plan to use all subsets regression.  That is, I will take each variable and regress it against the response, keeping the best.  Then I will take sets of two variables and perform multiple regression, keeping the best, etc...
When I am calculating each of the regression equations, should I:

Use only the 50 complete data points?
Use the set of complete observations as determined by my current subset?

It seems to me that if I'm going fishing I may as well use as many observations as are available to me - after all, if only X1 and X2 are meaningfully correlated with the response then I don't care (anymore) about X3 or X4.  However, most software implementations opt for (1) but I don't know if this is because it is computationally easier to extract the subset of observations which are complete and then iterate of the columns or if it is statistically more appropriate to use the same set of observations for each comparison.
EDIT
Additional information.  Please assume that if required, any of the variables could be consistently and reliably generated.  It's just that up to this point, they've been inconsistently recorded.  I could, if desired, design and execute an experiment on all four variables, I would just rather not go through that hassle.
This brings up another point - I really don't care about prediction, I care about explanation.  My 'best' model may be the one which minimizes VIFs.
I understand that this approach may not result in me selecting the true 'best' model - I really am only interested in the question I posed, namely, when I am taking this approach, should I use only the complete cases for the whole dataset or should I use the complete cases for the subset being fit.
Answer: You should rethink your approach to this problem in two ways.
First, although best-subset selection can give you the best predictors for the particular data sample that you have in hand, it typically will not give you the best predictors for future samples from the same population. After you have a complete data set (se below), try repeating your best-subset selection on multiple bootstrap samples of the data and see how variable the "best" subsets can be. In your particular case, with only 4 predictors and 81 cases, there probably is no harm in just keeping them all, particularly if your interest is in predictions based on your model. Predictive models are often best if you include all relevant predictors, even those that aren't "statistically significant." See this superb answer and many others under the feature-selection tag on this site for further discussion.
Second, with large amounts of missing data you need to consider carefully how you are using your model. If you are going to use it for outcome predictions from future observations of new sets of independent variables, you probably should consider and focus on the predictor/independent variables that are most likely to be available in the future. For exploring your present data, removing all except complete-data cases poses a risk of bias. Under reasonable assumptions, you might be better off filling in the missing data by imputation. See the missing-data tag on this site. There are well established ways to approach imputation of missing data, where you use the available data to do your best to predict what the missing values might have been. Multiple imputations, adding some variability into the imputation scheme to generate several different copies of the data set with different values for the originally missing values, is particularly good as it takes into account the variability introduced by the imputation process. See the multiple-imputation tag on this site for more information.
Comment: thank you for your answer. I'm actually only interested in an answer to the question I asked, not an alternate solution to the larger problem of how to do the modeling or handle missing data. I'm aware of all the points you've made, I'm just curious from a statistical perspective, which approach to all subsets regression is more appropriate - not if all subsets regression is appropriate.


Add filter expressions to paginated Boto3 dynamodb scan

Question: I am attempting to filter a paginated scan request to a dynamodb table.
While a traditional scan filter would use something like the following:
response = table.scan(
        FilterExpression=Attr("Date").eq(date)
    )

Attempting to pass this to the paginator scan however:
dynamodb = boto3.client('dynamodb')
dynamores= boto3.resource('dynamodb')
table = dynamores.Table('table')

pagination_config = {"MaxItems": 2, "PageSize": 2}

paginator = dynamodb.get_paginator('scan')
response_iterator = paginator.paginate(
                TableName=table.table_name,
                PaginationConfig=pagination_config,
                FilterExpression=Attr("Email").contains(user) &
                    Attr("Category").contains(types)
            )

Results in a parameter validation error
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter FilterExpression, value: <boto3.dynamodb.conditions.And object at 0x7f550f4b1940>, type: <class 'boto3.dynamodb.conditions.And'>, valid types: <class 'str'>

I have tried filtering on different keys without success and the response remains the same, even when filtering on boolean or integer columns. Is there a way to filter the scan using the operation_parameters argument?
I have also attempted using JMES filtering without success as the followiing:
response_iterator = paginator.paginate(
                TableName=table.table_name,
                PaginationConfig=pagination_config
            )

filtered_iterator = response_iterator.search("Contents[?contains(Category, 'File Status')]")

Yields no results despite there being an entry matching this criteria in the table.
Comment: Try passing it as a single string specifying the full expression.
Comment: Could you please provide a simple code example as I am not quite following?
Comment: Not at 1:50am :) Maybe tomorrow
Answer: You are treating the paginator like the Resource client which it is not. You need to be more low level like this example:
https://github.com/aws-samples/aws-dynamodb-examples/blob/master/DynamoDB-SDK-Examples/python/WorkingWithScans/scan_with_paginator.py
Comment: Thank you, this was the optional ScanFilter argument I required.
Answer: Two possible solutions identified that I will leave here for anyone attempting to filter / search on paginated DynamoDB responses as the Boto3 documentation is frankly useless.
Option 1:
Drill down into the response object using JMES on the dynamodb specific response object:
response object = {
   "Items":[
      {
         "Date":{
            "N":"54675846"
         },
         "Message":{
            "S":"Message02"
         },
         "_id":{
            "S":"4567"
         },
         "Category":{
            "S":"Validation Error"
         },
         "Email":{
            "S":"firstname.lastname@example.com"
         }
      }
]}

filtered_iterator = response_iterator.search("Items[?Category.S == 'Validation Error'].{_id: _id, Category: Category, Date: Date, IsRead: IsRead, Message: Message, Email: Email}")

You can then iterate over the response object pages as usual with a standard for loop.
Option 2:
Use list comprehension on the paginated response object and filter at that point. Note that this will be the more expensive option as it will still scan and return the max items within your paginator configutration:
for page in response_iterator:
page["Items"] = [x for x in page["Items"] if '{search_term}' in x['Category']['S']]

Option 3:
By far the simplest solution is that identified by Lee Harrigan.


Tensorflow data pipeline with from_generator is not producing data fast enough

Question: I am using tf.data.Dataset to build pipeline for training and I cannot seem to get the speed. I am using from_generator (see below) in pipeline and I am suspecting python global lock is part of the issue. I am struggling to figure out a design that can speed it up. I tried different variations of num_parallel_calls, cycle_length, block_length in the interleaves but none of them seem to help. Appreciate any insights.
Tensorflow 2.0, Default Eager Mode.
Pipeline - Read TFRecords and feed data to training
I have two generator functions that I am overlaying to spread the work load.

generator_step1 - loads the tf_record file, uses map to parse data
generator_step2 - processes the data and yields batches of data

Pipeline Psuedo Code
dataset = tf.data.Dataset.from_tensor_slices(tf_records_files_list)

dataset = dataset.interleave(lambda x: tf.data.Dataset.from_generator(generator_step1,
                                output_types=(tf.string, tf.float32), args=(x,batch_size,)),
                                cycle_length=2,
                                block_length=1)

dataset = dataset.interleave(lambda arg1, arg2: tf.data.Dataset.from_generator(generator_step2,
                                output_types=(tf.float32, tf.int16), args=(arg1, arg2,)),
                                cycle_length=2,
                                block_length=1)

dataset = dataset.prefetch(5)

This runs on a powerful machine and I don't see I/O or CPU being the bottleneck. Any thoughts on how to improve or optimize this further? Appreciate any insights.
Comment: If you have enough RAM you can use `ds.cache` to load the data into the RAM or cache it to a file.
Comment: I have enough RAM but not nearly enough to cache all the data. I have 128GB RAM and the data is hundreds of GBs so that is not an option. I was hoping the pipeline can be designed with minimal lag but I am not able to get it to deliver consistent speed.
Answer: Ok, I think you design is not optimal. First you create a tf.dataset and then you create another tf.dataset for each batch, which produces a lot of overhead. Instead of interleave I reccomend to use .map. Therefore, you have to rewrite the generators as a function that returns one sample at a time. Afterwards you shoudl do the batching by tf.data. Furthermore, you should use tensorflow functions (e.g. tf.image.decode_iamge) in you mapped functions.
dataset = tf.data.Dataset.from_tensor_slices(tf_records_files_list)

dataset = dataset.map(load_func, num_parralel_calls=24)

dataset = dataset.map(preprocess_func, num_parralel_calls=24)

dataset = dataset.batch(batch_size).prefetch(5)

This should do the job.
Comment: Thank you very much. I will try this. In my mapped functions, I need to trim/process large arrays to create a batch. It is not image data but rather multi-dimensional discrete data. Once I move them to map, I will try to see if there are any  tf functions that I can leverage. Let me know if you have any pointers. Thanks for your help.
Comment: Please described your data more in detail it give an example implementation. Furthermore, don't forget to mark the answer correct if helped you.
Comment: I tried to create a sample implementation but it became too complex. The trouble I am having is using map on function that can generate multiple batches. Our tfrecords have multiple samples in them so in our processing func, we create batches of data based on the training criteria. One TFRecord file can create multiple batches and we were using generators to do that. But with map, we cannot use generators. Any thoughts on how to look at it?

TFFile has multiple samples (each sample is 2048x100x500 )
load_fn loads the TFRecord files
process_fn - basic ops on arrays, create multiple batches


Oracle database connection in drupal 7

Question: I want to retrieve a list of products from an external oracle database and show it in my drupal site as a catalog.
But the problem is that, how can I make the connection to oracle db from a drupal 7 site.
Please any suggestion on how to perform this operation is highly welcome.
Thanks!
Answer: I recommend separate this from Drupal. I mean if you just want a list of products so write an external application to prepare a xml format of this list and feed it to Drupal.
You need install PHP PDO_OCI to make connection from PHP application into Oracle.
Drupal has an Oracle Driver but I recommend external application instead
Comment: Thanks for your suggestion, not did it only help me to solve my problem but by using xml file I have taken away my Dbase online.Something I knew that was not secure to connect the production dbase online but was trying to find out an efficient solution...
Comment: So you need web service which you have authentication rules

Oracle database connection in drupal 7


Programmatically adding and removing users to a user reference field?

Question: I would love to see some code samples for this task. I have found this example already http://www.midwesternmac.com/blogs/jeff-geerling/programmatically-adding-or, but it is unclear why that example uses node language for array key, and based on the comment the user removal could be better as well.
Answer: It should be pretty simple:

Load the node you are trying to modify using node_load().
Look for the user referencing field.
Modify the user referencing field.
Save the node using node_save().

I wrote those steps before looking at the example you linked, but after looking at it, it pretty much follows those steps. To answer your question regarding the language: The reason for that is that you are not supposed to hardcode the language key directly. If I am not mistaken, in Drupal 7 this key is something like 'und', but when you use a language on a node this key changes dynamically to something like 'es' or 'en'. If you hardcode the language key, then your code will be translation incompatible. Even if you don't use translation, it is still consider a good Drupal programming practice not to hardcode that language key ;)
If you follow the code that it is given in your link you will be fine.


Predict crowded areas based on time series data points of latitude and longitude

Question: I have time series location data (latitude/longitude) from users who send their location every minute. Basically the history of different users moving on a map.
<code>+---------+-------------------------+---------------------+
| user_id |     point(lat,lon)      |        date         |
+---------+-------------------------+---------------------+
|     111 | 26.453121,44.297412     | 2020-07-24 13:17:00 |
|     222 | 26.453155,44.297489     | 2020-07-24 13:17:00 |
|     111 | 26.453132,44.297455     | 2020-07-24 13:18:00 |
|     222 | 26.453191,44.297468     | 2020-07-24 13:18:00 |
|     ... |                         |                     |
+---------+-------------------------+---------------------+
</code>
Based on this location data I need to predict how many users will be in a certain location.
For example given the point (lat=26.453121,lon=44.297412) (which in the image below is the center of the green circle) and a radius of 30 meters predict how many points (users) will be inside the perimeter based on past data.
In the image below I have 3 points (users) inside.

The challenging part is that the center of the area (circle) and the radius are variable parameters and can get any value.
My questions:
Is it feasible to have the center of the circle and the radius as variable parameters? If not then how would someone approach a problem like this?
I am considering making the radius a fixed value but the fact that the center of the circle can be any point on a map is still giving me headaches.
Answer: This is a very interesting problem. I would propose the following approach:

Build a regression model that predicts the lat/long coordinates of a user given a history of lat/long coordinates e.g., a Kalman filter.
Once you have your regression built, write a function that takes a lat/long coordinate and a radius, and checks if any users are predicted to be within the circle defined by the provided coordinates and radius given the users last location.

I think an "even-better-if" solution would be to fit a model that can simulate "random walks" for users based upon their past behavior. This would allow you to explore questions around how likely it is for users to be in certain locations at certain times.
Answer: If the radius would vary it would lead to too many problems:

There's an infinite number of solutions, your algorithm needs to consider a radius as small as 1 centimeter and as big as the whole earth.
There's also an infinite number of positions where you could center the circles ("+1 cm right").
Let's take a 1-centimeter radius, it could be either empty or maximally crowded, but such a result doesn't make much sense.
The whole earth radius would always have the same density of people. Again, not useful.

So this leads to problems with defining what "crowded" means and makes the search space unnecessary big.
I'd  split the map to a fixed grid of equally sized squares and calculate it per each square. For intermediate squares, you can always interpolate. It also makes defining "crowdy" easier as in the number of people per area you have at least the area fixed.
Computationally, this makes the search space smaller as the grid has a finite size and doesn't consider many intermediate positions. It would also be sparse, so this can be potentially used for some improvements in terms of algorithms and data structures, to improve computation time and space to be used.
It also reveals trivial solution, as you can just COUNT people and GROUP grid. The solution can be easily expanded by grouping by hours, days of the week, months, etc to consider seasonality. Of course, there are many non-trivial solutions as well, but the trivial ones offer a good, cheap start.


Ocultar valores duplicados SQL

Question: Bom, estou fazendo um projeto pequeno de processo de aprovação. Preciso consultar duas tabelas com dados que podem ser iguais, mas preciso que esse dados iguais não apareçam, no caso fiquem ocultos.
Exemplo:

TABELA1         TABELA2
 Col1            Col2
   1               1
   2               4
   3               5

Precisava que ele retorne um SELECT sem exibir os valores duplicados.
Comment: Favor colocar a resposta esperada para o exemplo acima de forma a dirimir as dúvidas dos que querem ajuda-lo.
Comment: Alguma resposta ajudou a resolver o problema e pode sanar dúvidas similares de outros usuários? Caso positivo não esqueça de marcar a resposta como aceita. Pra fazer isso é só clicar no ✓ do lado esquerdo da mesma (abaixo do indicador de up e down votes).
Answer: Utilize o UNION:
SELECT t1.campo1 FROM tabela1 t1
UNION
SELECT t2.campo2 FROM tabela2 t2

Answer: O importante é entender o que ocorre na consulta. O UNION faz a união de dois resultados retirando as duplicidades.
Lembro que o UNION vai levar em consideração todas as colunas de retorno da consulta.
Assim, a consulta via UNION vai trazer um campo apenas de resultado com Ids distintos.
Já o comando citado acima (representado abaixo) vai fazer uma projeção entre as duas tabelas e somente vai relacionar os itens da tabela 1 que não conteém id = na tabela 2:
SELECT * FROM #t1 tabela1 CROSS JOIN #t2 tabela2 WHERE (tabela1.id <> tabela2.id);

O retorno da consulta acima, assumindo seus dados de exemplo seria algo como:
   id tb1  id tb2
    1        4
    1        5
    2        1
    2        4
    2        5
    3        1
    3        4
    3        5

O id 1 da tabela1 somente projetou com os id's diferentes.
Answer: Se entendi corretamente sua pergunta a expressão SQL seria:
SELECT * FROM tabela1 CROSS JOIN tabela2 WHERE (tabela1.col1 <> tabela2.col2);

Ocultar valores duplicados SQL


Como fazer a média de uma lista de tuplas no python?

Question: Eu estou travado em uma atividade da faculdade que pede para fazer a média das listas de tuplas.
Eu gerei as listas de tuplas através desse código:
lista1 = []
lista2 = []
listaTupla = []
for i in range(5):
    var1 = int(input())
    lista1.append(var1)
for j in range(5):
    var2 = int(input())
    lista2.append(var2)

for k,l in zip(lista1, lista2):
    tupla = (k,l)
    listaTupla.append(tupla)

print(listaTupla)

O resultado final tem que ser esse.
listaDeTuplas=[(14, 93), (23, 84), (0, 5), (4, 43), (5, 1)]
listaDaMedia=[53.5, 53.5, 2.5, 23.5, 3.0]

Como posso fazer isso sem importar nada?
Answer: No exemplo apresentado na pergunta a função embutida zip() está sendo usada de maneira equivocada:
for k,l in zip(lista1, lista2):
    tupla = (k,l)
    listaTupla.append(tupla)

A função zip() por si só já itera sobre vários iteráveis em paralelo produzindo tuplas com um item de cada um, sendo desnecessário reiterar seu retorno para produzir uma lista de tuplas. O código acima pode ser simplesmente substituído por:
listaTupla = list(zip(lista1, lista2))

Quanto ao problema que gerou sua pergunta, calcular uma lista de médias de uma lista de tuplas, pode ser resolvido de pelo menos três formas.
Uma solução é utilizar a declaração for para iterar pela lista de tuplas e com o auxilio da função embutida sum() a cada iteração somar os elementos de cada tupla dividindo pela quantidade de elementos da  tupla, obtido com o auxilio da função embutida len(), adicionando o resultado a lista de médias:
lista1 = [14,23,0,4,5]                  #Simplificação do fragmento criando uma lista a partir entrada dum usuário.
lista2 = [93,84,5,43,1]                 #Simplificação do fragmento criando uma lista a partir entrada dum usuário.
listaDaMedia = []                       #Declara a lista que receberá as médias calculadas pelo algoritimo.

listaTupla = list(zip(lista1, lista2))  #Cria uma lista de tuplas geradas a partir da entrada do usuário.

#Para cada tupla t na lista de tuplas...
for t in listaTupla:
  listaDaMedia.append(sum(t)/len(t))    #...soma os elementos da tupla t, divide pelo número de elementos e adiciona ao final da média de médias.

print(listaTupla)
print(listaDaMedia)
#[(14, 93), (23, 84), (0, 5), (4, 43), (5, 1)]
#[53.5, 53.5, 2.5, 23.5, 3.0]

Outra forma seria utilizar list comprehension para criar a lista de médias:
lista1 = [14,23,0,4,5]
lista2 = [93,84,5,43,1]

listaTupla = list(zip(lista1, lista2))

#Cria uma lista cujo para cada tupla t na lista de tuplas o elemento resultante será a soma dos elementos da tupla t dividido pelo número de elementos de t.
listaDaMedia=[sum(t)/len(t) for t in listaTupla]

print(listaTupla)
print(listaDaMedia)
#[(14, 93), (23, 84), (0, 5), (4, 43), (5, 1)]
#[53.5, 53.5, 2.5, 23.5, 3.0]

Ainda é possível fazer uso da função embutida map() para aplicar uma expressão lambda calculando a média de cada tupla:
lista1 = [14,23,0,4,5]
lista2 = [93,84,5,43,1]

listaTupla = list(zip(lista1, lista2))

#Cria uma lista cujo a cada elemento na lista de tuplas será aplicada uma função que a recebe a tupla como como argumento pelo parâmetro t e calcula soma os elementos de t e divide pelo número de elementos de t, onde o retorno da função e o elemento resultante.
listaDaMedia= list(map(lambda t: sum(t)/len(t), listaTupla))

print(listaTupla)
print(listaDaMedia)
#[(14, 93), (23, 84), (0, 5), (4, 43), (5, 1)]
#[53.5, 53.5, 2.5, 23.5, 3.0]

Comment: Apenas respondi pois a outra resposta(já devidamente removida) além de não atender ao problema proposto ensinava o uso errado da linguagem.


How to add a user rank in their profile?

Question: I want to have rank for every user. A simple -> amin rank is 18 with 300 points.
Any suggestions?
Comment: possible duplicate of [How to assign permissions based on user points?](http://drupal.stackexchange.com/questions/20102/how-to-assign-permissions-based-on-user-points)
Answer: Use a combination of the Rules, Userpoints and User badges module, as further detailed below ...
Add a new badge
Have a look at this Rules example (in Rules export format):
{ "rules_grant_a_new_badge" : {
    "LABEL" : "Grant a new badge",
    "PLUGIN" : "reaction rule",
    "OWNER" : "rules",
    "REQUIRES" : [ "userpoints_rules", "rules", "user_badges", "rules_conditional" ],
    "ON" : { "userpoints_event_points_awarded_after" : [] },
    "DO" : [
      { "userpoints_rules_get_current_points" : {
          "USING" : { "user" : [ "site:current-user" ], "tid" : "all" },
          "PROVIDE" : { "loaded_points" : { "total_points" : "Number of points in all categories together" } }
        }
      },
      { "CONDITIONAL" : [
          {
            "IF" : { "NOT data_is" : { "data" : [ "total-points" ], "op" : "\u003C", "value" : "7" } },
            "DO" : [
              { "add_badge_by_name" : { "badge" : "1", "user" : [ "userpoints-transaction:user" ] } }
            ]
          }
        ]
      }
    ]
  }
}

It does retrieve, as the very first Rules Action (not Rules Condition!) the current amount of user points of a user. If the amount is at least 7, it will grant the user badge.
In this example I'm using the Conditional Rules module, though you could also rework the Rules Action to performing Rules Components (in which you add the appropriate Rules Conditions), so that you don't need this extra module (for more details on that, refer to the answer to "What's the alternative to using the Conditional Rules module?"). However, by using the Conditional Rules module, you can consolidate the logic for various user badges in a single rule, e.g bronze for 7 points, silver for 77 points, gold for 777 points.
The clue to make this rule work is the very first Rules Action "Load user points of a user". That's the alternative (work around?) for something like a missing Rules Condition to "check the current user points for a user".
Remove a new badge
If you want "a user's old badge to be replaced by a new one", then you should also add a 2nd Rules Action to the rule shown above. The action to be used is remove_badge_by_name (instead of add_badge_by_name).
Only grant a new badge once
To avoid "displaying a message each time when user getting more than 7 user points" (as in your question), just add a Rules Condition like "NOT User has badge". This is one of the Rules Conditions provided by the User badges module.
Display the badge
To actually "display the badge with the username at the top", you could use what's written in the Community Documentation about User Badges, i.e:

To display user badges in your theme use:
<?php
  if (module_exists('user_badges')) {
    print user_badges_for_uid($uid);
  }
?>

Note: $uid means 'a user id' here, not the literal use of $uid. Most people will use the uid of the node ($node->uid) or comment ($comment->uid) author.
In Drupal (and theming), $node is a PHP object that represents the node, and $comment represents the comment. They are not always both available in every template file, so check the comments at the top of the file to see what is available. You may have access to other uids (user ids), and in these cases you can replace $uid with that value.


A Java Runtime Environment (JRE) or Java Development Kit (JDK) must be available in order to run Dbeaver. No Java virtual machine

Question: Java Virtual Machine not found while installing dbeaver tool in the system. What will do to recover from it ?.
Answer: If you installed zip archive or tar.gz archive of the dbeaver then

Go to dbeaver directory (Directory created after extraction of archive)
open dbeaver.ini in a text editor
Type the following at the top of that file
-vm
copy & paste your java path here (up to bin)

eg:
-vm
/home/admin/jdk1.8.0_131/bin

Save the file
Done
Answer: Windows 11, Dbeaver Community Version 23.3.1, after opted out to add Java at installation, in <dbeaver_installation_folder>/dbeaver.ini file I've added the first 2 lines:
-vm
C:\\Users\\<user_folder>\\.jdks\\openjdk-21\\bin
-vmargs
-XX:+IgnoreUnrecognizedVMOptions
-Dosgi.requiredJavaVersion=17
-Dfile.encoding=UTF-8
--add-modules=ALL-SYSTEM
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.nio.charset=ALL-UNNAMED
--add-opens=java.base/java.text=ALL-UNNAMED
--add-opens=java.base/java.time=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/jdk.internal.vm=ALL-UNNAMED
--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--add-opens=java.base/sun.security.ssl=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.security.util=ALL-UNNAMED
--add-opens=java.security.jgss/sun.security.jgss=ALL-UNNAMED
--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED
--add-opens=java.desktop/java.awt=ALL-UNNAMED
--add-opens=java.desktop/java.awt.peer=ALL-UNNAMED
--add-opens=java.sql/java.sql=ALL-UNNAMED
-Xms64m
-Xmx1024m
-Ddbeaver.distribution.type=exe

Please modify the second line with your path to the java bin directory.
Answer: I have the same issue in windows 10 when I try to update DBeaver. I manage to solve it by downloading the Windows 64 bit (installer) in the official website and install it.
Answer: I solved the problem by uninstalling dbaver and install again. If you have still problem with it try to solution what I did.


Proof for $\cos(\alpha)^2 + \cos(\beta)^2 + \cos(\gamma)^2 = 1$ in Euclidean space

Question: What is the proof for this formula:
$$
\cos(\alpha)^2 + \cos(\beta)^2 + \cos(\gamma)^2 = 1,
$$
where $\alpha$, $\beta$ and $\gamma$ are the angles between a vector and the base of a right-handed orthonormal base in Euclidean space? I can't seem to find a solid proof.
Answer: Hint: Use the inner/dot  product interpretation of these angles. Then the statement becomes immediate.
Comment: Except it is dot product not cross product.
Comment: You mean, $\cos(\alpha) = \frac{\mathbf{V}_1 \cdot \mathbf{V}_2}{|\mathbf{V}_1|*|\mathbf{V}_2|}$ where $\alpha$ is the angle between V1 and V2?
Comment: yes, corrected it


MS SQL Server configuration backup strategy

Question: my company developed complex configuration(databases, jobs, maintenance plans, replications) on MS SQL server which is located in customer office.
I want to setup some backup strategy for this entire solution(not only databases).
Can you please help me with some advices how to start doing this(source control?entire partition backup?,...)?
Thanks
Answer: Start with documenting everything. Get a good high level overview and then dig into all the details.
Now deceide how quickly you need to have things back up and running. Do you need a hot site? Cold site? Do you just need the data and config or do you need full hardware redundency?
For most of my sql db's there are server images, registry backups and regular sql backups, the sql .bak file is then distributed to an off site server and then backed up to tape overnight.
The details of exactly what you do will depend on what kind of security you require and how quickly you need to restore the systems.
In any case, PLAN TO TEST YOUR BACKUP PLAN. A disaster is a really bad time to find out it didn't work.
Answer: Your backup strategy should be built off of business requirements and SLA (service level agreements). The number one question you have to ask is "how much can I loose, and still function as a business?". This can also be controlled by business/industry standards (HIPPA, PCI, etc) depending on what type of business you support.
Take that question for each aspect of your system and figure out if it got swept away by weather, fire, or theft could you or your customer continue business without it. If they could not then your next step would be to ask, how much data can you loose. With configuration information that should be at most documented somewhere in the event you can't just restore it somehow. You would also need to take into account restore time, the turn around time to get the system back up and running can dictate the backup method you choose.
Just my opinion...
Answer: Your question is quite vague and open ended, but for a starting point you might want to look at Windows Server Backup (here's a handy step-by-step guide from Technet).
If you run this alongside your SQL Server backup strategy, you should have a quite comprehensive solution, but whatever you go for, you definitely want to make sure your recovery plan works.
Comment: What does Windows Backup have to do with MSSQL metadata ?
Comment: It was the comment of "I want to setup some backup strategy for this entire solution (not only databases)" which suggested to me that the OP was after a complete server backup tool, rather than just a DB or SQL specific backup tool.
Comment: His comment referred to "complex configuration(databases, **jobs, maintenance plans, replications**)"; the topic and tag clearly limit the question to SQL Server.


Leaflet map: map container is already initialized

Question: I am a begineer and have a problem reloading a map with already set coordinates. I used fetch to pull data from an APi, then I put the data in the map. It shows an error: Map is already initialized!
const fetchPosts = async () => {
  fetch("https://itk-exam-api.herokuapp.com/api/offices")
    .then((response) => response.json())
    .then((posts) => {
      console.log(posts);
      return getPosition(posts);
    });
};

const getPosition = (posts) => {
  posts.forEach((post) => {
    latitude = post.latitude;
    longitude = post.longitude;
    const coords = [latitude, longitude];
    console.log (coords);

    const mymap = L.map('map').setView(coords, 13);

    L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
      attribution: '© OpenStreetMap contributors'
    }).addTo(mymap);

    L.marker(coords).addTo(mymap)
      .bindPopup('A pretty CSS3 popup.
 Easily customizable.')
      .openPopup();
  });
}

fetchPosts();

Answer: L.map('map') initializes a Leaflet map using the given element id. You cannot call this twice for the same element.

const attribution = document.querySelector("#attribution").innerHTML;
const mymap = L.map("map").setView([0, 0], 0);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { attribution })
  .addTo(mymap);

L.map("mymap"); // <- error, the map is already initalized
#map { height: 70px; }
<link crossorigin rel="stylesheet" href="https://firstname.lastname@example.com/dist/leaflet.css" />

<template id="attribution">
  ©

    OpenStreetMap

  contributors
</template>

Instead of initializing the Leaflet map as part of the getPosition() function you could reference the map via a variable that is in scope, or pass the map through the parameters.

// initialize the map a single time
const attribution = document.querySelector("#attribution").innerHTML;
const mymap = L.map("map").setView([0, 0], 0);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { attribution })
  .addTo(mymap);

async function fetchPosts() {
  const response = await fetch("https://itk-exam-api.herokuapp.com/api/offices");
  const posts = await response.json();
  getPosition(posts);
}

function getPosition(posts) {
  posts.forEach((post) => {
    const coords = [post.latitude, post.longitude];

    // Reference the Leaflet map that is already initialized (mymap),
    // instead of trying to re-initialize a new Leaflet map on the same
    // container.

    L.marker(coords).addTo(mymap)
      .bindPopup('A pretty CSS3 popup.
 Easily customizable.')
      .openPopup();
  });
}

fetchPosts()
  .catch(console.error);
#map { height: 180px; }
<link crossorigin rel="stylesheet" href="https://firstname.lastname@example.com/dist/leaflet.css" />

<template id="attribution">
  ©

    OpenStreetMap

  contributors
</template>


Fundamental Theorem of Calculus Confusion regarding atan

Question: According to this site,
$$ \int \frac{1}{a^2 \cos^2(x) + b^2 \sin^2(x)} \,dx =\frac{1}{ab} \arctan\left(\frac{b}{a} \tan(x)\right)$$
Thus,
$$ \int_0^{\pi} \frac{1}{a^2 \cos^2(x) + b^2 \sin^2(x)} \, dx =\frac{1}{ab} \arctan\left(\frac{b}{a} \tan(\pi)\right)-\frac{1}{ab} \arctan\left(\frac{b}{a} \tan(0)\right)=0-0=0$$
But in fact value of integral is not zero.
What am I doing wrong?
Comment: My first thought is that the arctangent function is somewhat problematic.  It is an inverse only of a restriction of the tangent function. $\qquad$
Comment: I believe it is because $\arctan(0)$ is not always $0$, e.g. it could also be $\pi$. In that sense $\arctan$ is a multi-valued function.
Comment: Let $a=b=1$. Then $\frac{1}{ab} \arctan\left(\frac{b}{a} \tan(\pi)\right)
=\frac{1}{ab} \arctan 0 = 0 \ne \pi$. Your equation only works on $(-\pi, \pi)$
Comment: See [this graph](https://www.desmos.com/calculator/1ms6oibnug). Your function is not continuous, that's why it works only on $(-\pi/2;\pi/2)$
Comment: Some related questions: http://math.stackexchange.com/questions/1296351/calculating-int-0-pi-frac1ab-sin2x-dx, http://math.stackexchange.com/questions/1356523/what-are-the-restrictions-on-using-substitution-in-integration
Answer: First, let us assume that $a, b > 0$ without loss of generality.  Then in this regime, $f(x) = (a^2 \cos^2 x + b^2 \sin^2 x)^{-1}$ is clearly a well-defined function everywhere, and is periodic with period $\pi$.  However, the antiderivative is not unique, and the particular choice of antiderivative $$F(x) = \frac{1}{ab} \tan^{-1} \left( \frac{b}{a} \tan x \right)$$ may not be continuous over a given interval.  For example, if we choose the branch of $\tan^{-1}$ that restricts the range to lie in $(-\pi/2, \pi/2)$, then you get this picture for $a = 2$, $b = 0.5$:

As you can see, there is a jump discontinuity in the antiderivative at $x = \pi/2$ that could be rectified by choosing a different branch of the inverse tangent; e.g., we could choose $\tan^{-1}$ to have the range $[0,\pi)$.  This fixes the discontinuity at $\pi/2$ so that a definite integral containing this point will not result in an incorrect evaluation.

In general, it is important when dealing with branches of multivalued functions to choose the branch appropriate to the interval or domain of integration, so as to avoid these problems.

An example of how you could specify an antiderivative that is continuous for all real $x$ provided that we state that the range of $\tan^{-1}$ is $(-\pi/2,\pi/2)$, is by writing it as $$F(x) = \begin{cases} \displaystyle \frac{1}{ab} \left( \tan^{-1} \left(\frac{b}{a} \tan x \right) + \pi \left(\lfloor 2x/\pi \rfloor - \lfloor x/\pi \rfloor\right) \right), & 2x/\pi \not \in \{\pm 1, \pm 3, \ldots\} \\ \displaystyle \frac{x}{ab}, & 2x/\pi \in \{\pm 1, \pm 3, \ldots\}. \end{cases}$$
But I think this is a rather complicated way to express the antiderivative.  Instead, it might be better to state that the branch of the inverse tangent that is chosen depends on the argument of $x$; i.e., if we define $y = \tan^{-1} c \tan x$ for some $0 < c < \infty$, then $y$ is chosen such that $x, y \in ((2m-1)\pi/2, (2m+1)\pi/2)$ for a suitable integer $m$.
Comment: You've completely misunderstood the issue. When I wrote an alternative form of $F(x)$, that form uses a definition of inverse tangent such that $\tan^{-1} x \in (-\pi/2, \pi/2)$.  Plot it yourself and you will see it is continuous.  If you choose to define $\tan^{-1} x$ in a way that its range is on $[0,\pi)$, then that formula doesn't work.  The formula doesn't determine the choice of branch:  it is **your** choice of branch that determines what a formula looks like.
Comment: What is the value of antiderivative then? Where should a shift occur?
Comment: So, what is antiderivative in this particular case? If we take integer m then 2m-1 is never 0. How we then choose interval (0,pi)?
Comment: That's the question. What is the formula for inverse tangent, whose range is [0, pi)?
Answer: Note that, in the link, $u=\tan x$ which is increasing in $[0,\pi/2)$. In this integral, $x\in[0,\pi]$ and hence you can't use the result directly. In fact
$$ \int_0^\pi \frac{1}{a^2 \cos^2(x) + b^2 \sin^2(x)} \,dx=2\int_0^{\pi/2} \frac{1}{a^2 \cos^2(x) + b^2 \sin^2(x)} \,dx.$$
Use the result in $[0,\pi/2]$ and you will get the result.
Comment: So, my result is wrong because tan is infinite at pi/2?
Comment: Recall that in your result you will actually have a term like this:$$\frac{1}{ab}\arctan(\frac{b}{a}\tan(\frac{\pi}{2}))=\frac{1}{ab}\arctan( \infty )=\frac{1}{ab}\frac{\pi}{2}$$
Comment: @AndrewFount, because $y=\tan x$ is not increasing in $[0,\pi]$.
Comment: @Mufasa, you are right.
Comment: @Andrew Fount -- that is essentially correct.  The tan function goes to infnity and then jumps to - infinity and that introduces a discontinuity.
Answer: Some Background on Integration by Substitution
It is important to understand the assumptions one makes in Integration by Substitution.
Let $I$ denote the integral given by $I=\int_a^b f(x)\,dx$.
Condition $(i)$:
 We assume that $f$ is continuous on $[a,b]$.
Condition $(ii)$:
The inverse function $u^{-1}$, of $u(x)$ is continuously differentiable on $[u(a),u(b)]$.
Under Conditions $1$ and $2$, we have

$$\begin{align}I=&\int_a^b f(x)\,dx\\\\&=\int_{u(a)}^{u(b)}f(u^{-1}(t))\frac{du^{-1}(t)}{dt}\,dt \tag 1\end{align}$$

Note that the relationship expressed in $(1)$ can be obtained formally through the substitution $u(x)=t$, under Assumptions $(i)$ and $(ii)$.

Application to Evaluating $\int_0^\pi \frac{1}{a^2\sin^2(x)+b^2\cos^2(x)}\,dx$
To apply the substitution in $(1)$ correctly to evaluate the integral of interest, one must ensure that both $(i)$ and $(ii)$ are satisfied.
Since $f(x)=\frac{1}{a^2\sin^2(x)+b^2\cos^2(x)}$ is continuous for $x\in [a,b]$ and Condition $1$ is satisfied.
The substitution used in the reference cited in the OP is $\tan(x)=t$.  Note that for $x\in [0,\pi]$ the inverse function for the tangent function is given by
$$\tan^{-1}(t)=\begin{cases}\arctan(t)&, t>0\\\\\pi+\arctan(t)&,t<0 \tag 2\end{cases}$$
where the $-\pi/2 < \arctan(x)<\pi/2$ is the principal branch of the inverse tangent function.  The inverse tangent is not even continuous at $0$ and we cannot apply, therefore, the substitution over the interval $x\in [0,\pi]$.
Instead, we can write the integral as the sum
$$\begin{align}
\int_0^\pi f(x)\,dx&=\lim_{\epsilon \to 0^+}\left(\int_0^{\pi/2-\epsilon}f(x)\,dx+\int_{\pi/2+\epsilon}^\pi f(x)\,dx\right)
\end{align}$$
where $f(x)=\frac{1}{a^2\sin^2(x)+b^2\cos^2(x)}$.
The inverse tangent function $\tan^{-1}(x)$ as expressed in $(2)$ is continuously differentiable on each separate interval $x>0$ and $x<0$ and so, Condition $2$ is satisfied on each separate interval.
Then, we can write
$$\begin{align}\lim_{\epsilon \to 0^+}\int_0^{\pi/2-\epsilon}f(x)\,dx&=\int_0^{\infty}f(\arctan(t))\frac{1}{1+t^2}\,dt\\\\
\lim_{\epsilon \to 0^+}\int_{\pi/2+\epsilon}^\pi f(x)\,dx&=\int_{-\infty}^0f(\pi+\arctan(t))\frac{1}{1+t^2}\,dt \tag 3
\end{align}$$
Since $f(x)$ is even and $\pi$-periodic, then the second equality in $(3)$ is identical to the first.  Therefore, we have
$$\begin{align}
\int_0^\pi f(x)\,dx&=2\int_0^\infty \frac{f(\arctan(t))}{1+t^2}\,dt\\\\
&=2\int_0^\infty \frac{1}{a^2t^2+b^2}\,dt\\\\
&=\left. \frac{2\arctan(at/b)}{ab}\right|_{0}^{\pi/2}\\\\
&=\frac{\pi}{ab}
\end{align}$$
Answer: This very integral, with a little manipulation, is referenced in the Jeffrey paper so the nice form in that paper can be attained by observing that $\tan^{-1}(\tan x)$ has the same discontinuities as the original integral, so we may hope to cancel them out by setting
$$\frac1{ab}\tan^{-1}\left(\frac{b}a\tan x\right)=\frac1{ab}\left\{x+\tan^{-1}\left(\frac{b}a\tan x\right)-\tan^{-1}(\tan x)\right\}$$ Now,
$$\begin{align}\tan^{-1}\left(\frac{b}a\tan x\right)-\tan^{-1}(\tan x) & =\tan^{-1}\left(\frac{\frac{b}a\tan x-\tan x}{1+\frac{b}a\tan^2x}\right) \\ & =\tan^{-1}\left(\frac{(b-a)\sin2x}{(a+b)-(b-a)\cos2x}\right) \end{align}$$ after a little algebra, so
$$\int\frac1{a^2\cos^2x+b^2\sin^2x}dx=\frac1{ab}\left\{x+\tan^{-1}\left(\frac{(b-a)\sin2x}{(a+b)-(b-a)\cos2x}\right)\right\}+C$$ removes all discontinuities and even checks by differentiation.


Windows Hiding Folders

Question: Has anyone noticed that after certain actions, a Windows folder will just go hidden?  Like its been entirely removed from the taxonomy of the file system, but is still accessible if you type in the exact path?
Does anyone know how to remedy this?
Comment: Is this on a server or...as worded, this might get migrated to SuperUser. Are you talking about a share or a local folder? What version(s) of Windows?
Comment: And after *what* actions? I notice they go hidden if I set the hidden flag :-)
Comment: Its on a Server yes.  Its only invisible locally on the server too.  And its if I browse to the root folder, its not visible at all from any microsoft tools, but if you use a non-microsoft tool, its visible.
Answer: Turns out the system and hidden flags were set
This is remedied by entering the command
<code>attrib FOLDER -h -s
</code>

Windows Hiding Folders


Verificar nome do módulo via VBA

Question: Olá, preciso verificar o nome de um módulo para altera-lo em posterior.
O nome do módulo se altera bastante, dependendo de cada documento,e não da pra rodar todos os nomes possíveis, pois demoraria muito.
Pra isso preciso de um código que leia os nomes dos módulos no meu projeto VBA.
O que realizei até então foi isso:
Sub Mudar_cód()

    ObjVbProj = Workbooks(ActiveWorkbook.Name).VBProject.VBComponents("MFPC0000").CodeModule

    If ObjVbProj = "MFPC0000" Then

        MsgBox "SIM"

    End If

End Sub

Answer: Veja se a função abaixo te ajuda, pois lista todas os módulos por planilha:
Sub BuscaModulos()
Dim modName As String
Dim wb As Workbook
Dim l As Long

 Set wb = ThisWorkbook

For l = 1 To wb.VBProject.VBComponents.Count
    With wb.VBProject.VBComponents(l)
        modName = modName & vbCr & .Name
    End With
Next

 MsgBox "Resultado:" & vbCr & modName
 Set wb = Nothing

End Sub

Comment: Perfeito! Eu só vou encaixar no código no meu código, mas era exatamente isso.

Verificar nome do módulo via VBA


Cannot load ctools automodal associated menu call from another module

Question: I have several modules that make extensive use of the Ctools Automodal modal to activate custom-driven modals. These Link references live in their own module, example_item_modals:
/**
 * Implements hook_menu()
 */
function example_item_modals_menu() {
  $items['product/product1'] = array(
    'page callback' => 'product_order_modal_1',
    "access arguments" => array('interact with orders'),
    'type' => MENU_LOCAL_TASK,
    'modal' => TRUE
  );

  $items['product/product2'] = array(
    'page callback' => 'product_order_modal_2',
    "access arguments" => array('interact with orders'),
    'type' => MENU_LOCAL_TASK,
    'modal' => TRUE
  );

  return $items;
}

We have another module that loads in a User Profile (developed by another third party) and the requirements call for loading some of the products here, and interacting with the modal. Actually displaying the links are done using a tpl.php call to those menu calls with an l().

    <?php echo l("View Product", "product/product1"); ?>

So when we eventually load the template call in this custom user profile module, this link should load and have the necessary logic to fire the modals. Except, its not. Here, it treats it like a backpage link. Is there something I am missing for this link to function as a modal? Do I need to explicitly include the parent module the links are defined with, and if so, what API call accomplishes that? (I looked at module_load_include() but i'm importing settings thrown in the .module, not a .inc file).
Answer: I think you need specific class for your link to be preprocessed as a modal link (but i don't know the 'automodal' feature so maybe im wrong)
Answer:

    <?php echo l("View Product", "product/product1", array('attributes' => array('class' => array('ctools-use-modal')))); ?>

Comment: To make this a more useful answer, try adding some text indicating what this does and why this fixes their problem


How to make Big beat drum from scratch?

Question: As a beginner producer, I am having trouble making drums for my records.
I usually go to YouTube to learn how to create drum beats, but when I came across Big Beat, there was no detailed tutorial for the drum part since most of the tutorials used samples, and I do not like samples. For Big Beat, I use Ableton Live and I'm inspired by Fluke, Crystal Method, and Prodigy.
My question concerns a drum pattern made with Ableton's in-built drum machine, and how I can make it sound dreamy and wide.
Comment: What are trying to accomplish? The big beat sound *is* sampled loops, so doing it from scratch would still involve samples.
Comment: Please edit to tell more about what kind of work you're doing—I guess it would help to know what the tutorials you've been watching tell you to do. Are you looking for musical guidance about the actual rhythms and feel? For technical recording details like micing and processing? For guidance in musical postprocessing concerns like quantization or aligning loops?
Comment: I think it's a fair question. A lot of people don't realize that the famous recordings they like were (often) really made by sampling old vinyls etc. The most straight-forward way to get the same results is to use the same method: sample from records. For some genres the method almost defines the genre - you sample and chop the Amen break, etc. and that's how you get a chopped-drum-loops sound. If you don't chop, it doesn't sound like that genre. (In _some_ imaginary genre, not necessarily talking about big beat) But I can understand wanting to really recreate everything from scratch.
Comment: Are you talking about synthesizing the individual drum sounds, perhaps by physical modelling? Or just the creation of the loop/pattern? Or just what?
Comment: @KarlKnechtel  I meant creating drum patterns or loops Yes
Answer: According to the Wikipedia page about Big Beat, the drum beats in Big Beat are "heavy loops from 1960s and 1970s funk, soul, jazz, and rock songs." So if you want to make such drum loops without sampling existing records or using existing loop libraries, you'll have to learn how to create drum tracks that sound like they're coming from 1960s and 1970s funk, soul, jazz, and rock songs.
To emulate drum (or other) tracks from a certain period, find out how records were historically recorded and produced during the period of interest, and try to recreate the same sounds. I suppose recording an actual drummer playing actual drums in a proper room, with proper mics etc. is not an option, and you want to do something "in the box". You could use drum libraries which have the kinds of drum sounds you want, and use mixing techniques like what was used in the target era. Maybe use emulations of "vintage" compressors, tape recorders, mixing consoles etc., but IMO the most important thing is to know the original production techniques, not to have the right plugin. I haven't looked, but there may be tutorials on how to replicate 60s and 70s drum sounds. It's very much possible to do - I've heard quite convincing make-believe recreations of old recordings. But there's a learning curve.
Comment: Perhaps study the output of [Propellerheads](https://en.wikipedia.org/wiki/Propellerheads) as part of the big beat movement, who quite famously made their own drum loops in the studio rather than sample them from others. Their 2 biggest tunes were 'History Repeating' & 'On Her Majesty's Secret Service' [both with heavy James Bond connotations] Their drums don't have that true 60's vibe, but they still fit within big beat.
Comment: @Tetsujin Completely unrelated random comment, but I'm waiting to subscribe to your upcoming Youtube channel where you tell stories about working for Yamaha, the 80s and 90s, synths and MIDI etc.!
Comment: Whilst I fully appreciate the sentiment, I don't think I would ever make a presenter. My only recent YouTube activity was the skit Eminem video I put up earlier this year [which I think I linked in chat]. I doubt I'll ever get round to making that type of 'documentary' content. Sorry ;))

How to make Big beat drum from scratch?