Low classification accuracy, what to do next?

Question: So, I'm a newbie in ML field and I try to do some classification. My goal is to predict the outcome of a sport event. I've gathered some historical data and now try to train a classifier.
I got around 1200 samples, 0.2 of them I split off for test purposes, others I put into grid search (cross-validation included) with different classifiers. I've tried SVM with linear, rbf and polynominal kernels and Random Forests to the moment.
Unfortunately, I can not get accuracy significantly larger than 0.5 (the same as random choice of class). Does it mean I just can't predict outcome of such a complex event? Or I can get at least 0.7-0.8 accuracy? If it's feasible, then what should I look into next?

Get more data? (I can enlarge dataset up to 5 times)
Try different classifiers? (Logistic regression, kNN, etc)
Reevaluate my feature set? Are there any ML-tools to analyze, which features make sense and which don't? Maybe, I should reduce my feature set (currently I have 12 features)?
Comment: What sport is this and what do you consider a "correct" classification? If you're simply trying to predict a win/loss outcome in virtually any major sport it's almost inconceivable that even the simplest of classifiers wouldn't predict better than 0.5. If you are, say, trying to predict win/loss against a spread or some other handicapped outcome, then much better than 0.5 may be difficult.
Comment: @fspirit: That depends on the sport and the inequity in ability between the participants, for one thing. Just knowing *who* is participating in each contest can often be a strong predictor. [Here](http://stats.stackexchange.com/a/16148/2970) and [here](http://stats.stackexchange.com/q/30976/2970) are a couple of related posts.
Comment: What is your training accuracy? And how many samples you have in each class?
Comment: @Leo Training accuracy is around 0.5. Classes are evenly distributed, I have classes 0 and 1.
Comment: @cardinal Yes, I try to predict win/loss outcome, no handicaps. Is it feasible to reach, say, 0.8 accuracy on test set?
Comment: @cardinal Thanks for the links. Though, it will be hard to evaluate  a contribution of an individual player, cause a) they don't change teams b) in each team there are 20 of them.
Comment: @fspirit: That first link may have been taken too literally. :) A simple model is the Bradley-Terry model. That involves only having a single ternary (-1/0/1) predictor for each participant (each whole team, in this case) and can be fit with logistic regression. It's not a bad "baseline" model for comparison and there are many variants and ways it can be extended.
Answer: First of all, if your classifier doesn't do better than a random choice, there is a risk that there simply is no connection between features and class. A good question to ask yourself in such a position, is weather you or a domain expert could infer the class (with an accuracy greater than a random classifier) based on given features. If no, then getting more data rows or changing the classifier won't help. What you need to do is get more data using different features.
IF on the other hand you think the information needed to infer the class is already in the labels, you should check whether your classifier suffers from a high bias or high variance problem.
To do this, graph the validation error and training set error, as a function of training examples.
If the lines seem to converge to the same value and are close at the end, then your classifier has high bias and adding more data won't help. A good idea in this case is to either change the classifier for a one that has higher variance, or simply lower the regularization parameter of your current one.
If on the other hand the lines are quite far apart, and you have a low training set error but high validation error, then your classifier has too high variance. In this case getting more data is very likely to help. If after getting more data the variance will still be too high, you can increase the regularization parameter.
This are the general rules I would use when faced with a problem like yours.
Cheers.
Comment: Did you mean validation set error and TEST set error? Otherwise, I'm confused. I dont even know train set error, cause I use validation set error to choose model and them check selected model on test set.
Comment: No, I mean training set error where it is written. The training error is the number of misclassified examples in the training set divided by training set size. Similarly test set error is number of misclassified examples in test set divided by training set size. Also you may want to check Coursera's Machine Learning Class, (https://class.coursera.org/ml-2012-002/lecture/index), especially videos for "Advice for applying Machine Learning". Those advice are quite relevant to your situation.
Comment: I've competed the course, when it was ran for the first time. As for the training set error, I now output it too, for  SVM its quite high - 0.5, but for random forests its 0.
Answer: I would suggest taking a step back and doing some exploratory data analysis prior to attempting classification.
It is worth examining your features on an individual basis to see if there is any relationship with the outcome of interest - it may that the features you have do not have any association with the class labels. How do you know if the features you have will be any use?
You could start with doing hypothesis testing or correlation analysis to test for relationships. Generating class specific histograms for features (i.e. plotting histograms of the data for each class, for a given feature on the same axis) can also be a good way to show if a feature discriminates well between the two classes.
It is important to remember though not to let the results of your exploratory analysis influence your choices for classification. Choosing features for classification based on a prior exploratory analysis on the same data, can lead to overfitting and biased performance estimates (see discussion here) but an exploratory analysis will at least give you an idea of whether the task you are trying to do is even possible.
Comment: @BGreene - your third paragraph is a tough one for me.  If exploratory analysis shows predictor x1 to be highly correlated with the outcome, wouldn't it defeat the purpose of checking that correlation if one didn't use x1 as at least a candidate predictor in a multivariate model?
Comment: I'll try to draw the histograms and see what they will look like.
Comment: @rolando2 - I'm not suggesting that you don't include the feature as a candidate as part of a feature selection routine but you should not choose features based on such an exploratory analysis as this will overfit. However for the purposes of evaluating the generalized performance of a classifier model, feature selection should be done within the model selection routine (i.e. within each fold of cross validation). What I am suggesting is that exploratory analysis and classification should be treated as separate activities - each tells you  different things about your data
Comment: Also remember a feature might be useful in a multivariate model, while not being useful alone!
Answer: Why not follow the principle "look at plots of the data first"? One thing you can do is a 2-D scatterplot of the two class conditional densities for two covariates.  If you look at these and see practically no separation, that could indicate a lack of predictability; you can do this with all the pairs of covariates.  That gives you some ideas about the ability to use these covariates to predict. If you see some hope that these variables can separate a little then start thinking about linear discriminants, quadratic discriminants, kernel discrimination, regularization, tree classification, SVM etc.
Comment: Sorry, um, is covariate == feature?
Answer: It's good that you separated your data into the training data and test data.
Did your training error go down when you trained? If not, then you may have a bug in your training algorithm. You expect the error on your test set to be greater than the error on your training set, so if you have an unacceptably high error on your training set there is little hope of success.
Getting rid of features can avoid some types of overfitting. However, it should not improve the error on your training set. A low error on your training set and a high error on your test set might be an indication that you overfit using an overly flexible feature set. However, it is safer to check this through cross-validation than on your test set. Once you select your feature set based on your test set, it is no longer valid as a test set.
Comment: I use separate train, validation and test sets. I select hyper-params based on validation set error and then apply selected model to the test set. I doubt there is a bug in training algorithm, because I use off-the-shelf lib.
Comment: You had to connect that library to your data somehow. Always check that you are training correctly. If you are getting a training error rate of $50\%$, this either means your features are terrible, or else you are not training correctly.
Comment: In the "features are terrible" possibility, I include the case that there is no solution possible. However, I doubt that very much. There is no sport I know where there aren't ways to see that one competitor is a favorite over another. It is even possible in rock-paper-scissors.


Proper way of coding

Question: I am new in developing android applications. I want to create an android application(using eclipse) which needs a login page where the user needs to put his/her username and password and when the username and password is correct the application will open the main page. I already made a layout for the login page. My question is, is it proper to put the function codes of login and main page in the same package?
Answer: Ideally you would define a Login Activity which handles all the Login action, save it in preferences etc and then have it start the MAIN activity when the user is logged in.
Comment: We are all here to help, but the best way to learn still is: Read the Examples, get the documentation, learn how to use the docs, then come back here for problems.. and most importantly use google.
Comment: thank you. I've read your profile and found out that your an Android Engineer. I would just like to ask if I can ask you questions about creating android applications. I'm new to this and I am currently studying in creating android applications.
Answer: yes if they are meant to ship together that'll be easiest
Answer: Well, there is no problem in putting the login code and code for the main activity in the same package.
However, to provide the login functionality, you should ask the user login credentials the first time. On successful authentication, you should save it either in your database or in Preferences (preferable). I myself save the authentication details in a preference file. You can even encrypt the saved details if you are too much conscious.
So the next time when the user tries to run your app requiring authentication, it should first check as to whether the credentials are saved in database/Preferences.. if yes, then use those details for login, else perform fresh authentication.
Hope this helps.
Regards,
Mahendra Liya.

Proper way of coding


Set permissions of the installation directory

Question: I would like to change the permissions on the install directory (specified in CMAKE_INSTALL_PREFIX), but I don't find how to do that.
I'm able to set the permission of all the installed files, but not for the automatically generated directories.
For example, if my CMAKE_INSTALL_PREFIX is /softs/myappli/64/gcc/4.7.2 I would like to be able to set the permission of /softs/myappli to "700" when I do my "make install".
Answer: to do so, you may use install command w/ the following signature:
install([[SCRIPT <file>] [CODE ]] [...])`

and write a code to change permissions at install time. I.e. smth like this:
execute_process(COMMAND /bin/chmod ARGS 0700 "${CMAKE_INSTALL_PREFIX}")

put this to a chmod.cmake or to a string and then call install above.
Comment: The argument `ARGS` in the [`execute_process`](https://cmake.org/cmake/help/v3.0/command/execute_process.html) call seems to be superfluous. My instance started working after I removed it.
Comment: It seems a good solution. The only problem is there is no rules on the order of calling install commands. So I created a sub-directory "post-install" at the end of my main CMakeLists.txt to call this install command.
Comment: install(CODE ...) is an alternative solution for those, who don't want to add another `.cmake` file


How To add array of DatagridView control on panel vb.net?

Question: I have to add the Datagridview control at run time 1 below another on panel. how could i achieve this so that i'll be able to add controls location 1 below another, and how could i handle the click event, paint event and ect. for all datagridview.
Comment: If you have a question with `vb.net` don't use `c#` tag.
Comment: Hi syed mohsin, i got this link on net http://www.vbforums.com/showthread.php?662230-RESOLVED-How-to-create-a-DataGridView-control-array-at-runtime. so i have to add datagridview 1 after another on panel.
Comment: is your question how to handle events from two different datagridviews?
Answer: Just do something like this:
Dim lastCtrl As Control
Panel1.Add(AllDataGrids(0))

For i As Integer = 1 to AllDataGrids.Count - 1
    Dim dgv As DataGridView = AllDataGrids(i)

    lastCtrl = Panel1.Controls(Panel1.Controls.Count - 1)
    Panel1.Add(dgv)

    dgv.Top = lastCtrl.Top + LastCtrl.Height + 5
Next

The idea is to add each control after the last one, so in the loop you get the last one in the way you want, and then add the new control on the bottom. In order to have the events you have to add them manually for each DataGridView inside the loop with:
AddHandler dgv.CellClick, AddressOf CellClickMethod

Comment: Hi SysDragon, i did some change in your code and got it working..! Thanks.
Comment: Hi SysDragon, I added the 10 datagridview 1 after another. but it taking time for rendering so it look blurring, while scrolling among them.I've added the DoubleBuffered to each datagridview but no effect is there any solution for it..?
Comment: You can ask another question about rendering options.
Comment: Okay.I asked the new question,Thanks.


Mini motor de busca

Question: Eu estou a fazer um mini sistema de busca por urls que estão armazenados em um ficheiro.txt, um por linha, o meu problema é a questão da performance pois tenho 5000 urls armazenados (só) e ele já fica lento na procura, no lado servidor tenho:
if(isset($_GET['search'])) {
    $urls = array();
    if(trim($_GET['search']) != '') {
        $urls = search_in_urls($_GET['search']);
    }
    echo json_encode($urls);
}
function search_in_urls($search) {
    $file = fopen("urls.txt", "r");
    $urls = array();
    while(!feof($file)){
        $url = fgets($file);
        if (strpos($url, $search) !== false) {
            $urls[] = $url;
        }
    }
    return $urls;
}

Vi esta resposta aceite no SOen e gostaria de implementa-la, mas como devolver todas as linhas onde seja reconhecido um match com o meu $search?
A solução de procura que vi e que gostaria (a não ser que haja melhores opções) de implementar foi:
if(exec('grep '.escapeshellarg($search).' ./urls.txt')) {
    // do stuff
}

Ou seja, existe alguma forma de não percorrer todas as linhas à procura de um match (conjunto de chars igual aquele que escrevi) e devolver essa linha correspondente por inteiro? Ou que pelo menos para que esta busca fique bem mais rápida?
Comment: O grep que você postou faz exatamente isso. Não consigo saber o resultado que você vai receber em questão de performance. Segundo a documentação http://php.net/manual/en/function.exec.php você deve passar uma segunda variável "output" na qual será inserida as linhas ele conseguiu encontrar um match.
Comment: @ViniciusZaramella Estávamos com a mesma dúvida, acho que só faltou ele passar um output mesmo. Veja minha resposta.
Comment: Você terá que utilizar algum mecanismo de indexação. Isso é apenas para fins de estudo ou é para um sistema real? Pois se for para um sistema real sugiro utilizar ferramentas prontas como elasticsearch.
Comment: É apenas por brincadeira/curiosidade, também sei que a performance seria melhor se tivesse os `urls` armazenados em uma base de dados, seria assim que faria num projeto real. Obgado @ViniciusZaramella
Comment: Boa sorte: http://homepages.dcc.ufmg.br/~nivio/cursos/ri16/transp/indexing.pdf
Comment: Nem há comando shell que possa ser usado? que devolva a(s) linha(s) toda(s) onde haja match? @taiar. Dessa maneira podia injetar o comando no `exec(...`
Answer: Se entendi bem a pergunta, você quer isto:
$matches = array();
if(exec('grep '.escapeshellarg($search).' ./urls.txt', $matches)) {
    // $matches contém as linhas encontradas
}

Basta passar ao exec a referência a uma array onde a saída será guardada. E a saída do grep são justamente as linhas do arquivo onde houve matches da expressão regular.
Comment: Mas isso devolve a(s) linha(s) por inteiro? . Obgado, já vou testar
Comment: Sim, por inteiro. É como o `grep` funciona.
Comment: Realmente o problema era para onde iam os resultados. Obgado mais uma vez

Mini motor de busca


php exec() not showing expected output

Question: I am trying to run cpp executable on php, but its not working. Please help.
 var_dump(shell_exec("D:\\c_cpp_programs\\string.exe"));
 $output = system('D:\\c_cpp_programs\\string.exe', $retval);
 var_dump($output);
 passthru ('D:\\c_cpp_programs\\string.exe');
 if(file_exists('D:/c_cpp_programs/string.exe'))
    echo "File exist";
 exec('D:/c_cpp_programs/string.exe',$array,$error);
 var_dump($array);
 var_dump($error);

below is the output I get,
C:\wamp64\www\chatclub\application\views\test.php:2:null
C:\wamp64\www\chatclub\application\views\test.php:4:string '' (length=0)
File exist
C:\wamp64\www\chatclub\application\views\test.php:9:
array (size=0)
  empty
C:\wamp64\www\chatclub\application\views\test.php:10:int -1073741515

cpp program is,
#include<iostream>
using namespace std;

int main(){
    cout<<"Hello"<<endl;
    return 0;
}

Gives me the output on PHP Interactive shell though,
C:\Users\Shoyeb>php -a
Interactive shell

php > var_dump(shell_exec("D:\\c_cpp_programs\\string.exe"));
string(5) "Hello"
php > var_dump(shell_exec("D:\\c_cpp_programs\\string.exe"));
string(6) "Hello
" <----- this output is with <<endl
php > var_dump(shell_exec("D:\\c_cpp_programs\\string.exe"));
string(5) "Hello"
php >

Comment: Just consult the manual, which is full of examples per function. That would have also led you into trying [`proc_open()`](https://www.php.net/proc_open). As for [`exec()`](https://www.php.net/manual/en/function.exec.php) the manual clearly says trailing whitespaces per line aren't captured.
Answer: Finally I got the output when I started wamp with admin privileges,
C:\wamp64\www\chatclub\application\views\test.php:2:string 'Hello
' (length=6)
Hello
C:\wamp64\www\chatclub\application\views\test.php:4:string 'Hello' (length=5)
Hello File exist
C:\wamp64\www\chatclub\application\views\test.php:9:
array (size=1)
  0 => string 'Hello' (length=5)
C:\wamp64\www\chatclub\application\views\test.php:10:int 0

Answer: You can use:
exec("D:/c_cpp_programs/string.exe", $out);

Note that $out will hold the output.
Remember thatshell_exec returns all of the output stream as a string, but exec returns the last line of the output.
I'm not cpp expert but <<endl may output a empty string. I'm not sure.
Comment: I removed <<endl from cpp program, still it wont show Hello on php.

php exec() not showing expected output


Presenting a View Controller after login

Question: I recently started a new project and after I successfully set up a login page, I came into a problem.
If I type in valid credentials in the text fields, my app should do the following:
if (user) {
    [self.delegate loginViewControllerDidLogin:self];
    ...

The "if user" part means that everything was okay. After a few hours of searching I got to know that my [self.delegate loginViewControllerDidLogin:self]; should be defined in the AppDelegate.m file. Here I wrote the following code:
- (void)loginViewControllerDidLogin:(LoginViewController *)controller {
    UIStoryboard *storyboard = [UIStoryboard storyboardWithName:@"MainStoryboard" bundle:nil];
    UIViewController * ViewController = [[UIViewController alloc] init];

    [self.window addSubview:ViewController.view];
    [self.window.rootViewController presentViewController:ViewController animated:YES completion:nil];
}

My problem is that I cannot define the ViewController on my story board, so I cannot connect the code with the ViewController itself.
So basically I would like the app to open a new ViewController after logging in.
Comment: Why can't you define the ViewController on your storyboard?
Comment: I don't know how to... tried it in the ViewController.h, but even after I imported the it to the AppDelegates.m file it just couldn't recognize it.
Answer: On your storyboard select the view controller you want to display and fill out the "Storyboard ID":

Then instantiate it like this:
  UIViewController * ViewController = [storyboard instantiateViewControllerWithIdentifier:@"YourVCStoryboardID"];


How to Send Additional Data to Server When using Uploadify when Clicking Upload Button in ASP.net

Question: I want use Uploadify jQuery plugin to upload files in ASP.net. But when a user selects a file and clicks the 'Upload' button I want send some additional parameters to the server. I have written this code:
     Start Upload
     | <a href="javascript:$('#<%=fuBrandIcon.ClientID%>').fileUploadClearQueue()">Clear

            BrandName

                <input id="BrandName" type="text" />

            BrandAbbr

                  <input id="txtBrandAbbr" type="text" />

               Icon

                <asp:FileUpload ID="fuBrandIcon" runat="server" ClientIDMode="Static" />

I want to send the file selected by the user as well as BrandName and BrandAbbr to the server when the user clicks to bb the link. But this code sends empty BrandName and BrandAbbr values.
Edit: i write this code but i get error uploadifySettings in undifine

Please help me. Thanks
Comment: What is the purpose of using both uploadify and the .net file upload control?
Answer: You can send additional data through formData. return to the Documentation
Update:
Anyway the presented way in the documentation didn't work well with me. I used this code and it's work
   $('#uploadify_1').uploadifySettings('property_something', 'value_something');

Comment: @ dotWasim: thanks for help me, when  write this code in server request.Params["BrandName"] i get null


sudo service mongodb restart gives "unrecognized service error" in ubuntu 14.0.4

Question: I just installed mongoDB on ubuntu 14.0.4.
I tried to start the shell but I'm getting a connection refused error.
me@medev:/etc/init.d$ mongo
MongoDB shell version: 2.6.5
connecting to: test
2014-11-10T15:06:28.084-0500 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
2014-11-10T15:06:28.085-0500 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed

So I decided to try to restart the service but that's failing too.  I get the following error message:
me@medev:/etc/init.d$ sudo service mongodb restart
mongodb: unrecognized service
me@medev:/etc/init.d$

This is what I have in my /var/log/mongodb/mongod.log - http://pastebin.com/MrHt8tce
what i've tried so far:
I found another post here: can't start mongodb as sudo
which made a comment about remove the mongo lock file.
I deleted the lock file and then retried my command but it still fails as you can see below:
me@medev:/var/lib/mongodb$ sudo rm mongod.lock
me@medev:/var/lib/mongodb$ ls
journal  local.0  local.ns  _tmp

me@medev:/var/lib/mongodb$ sudo service mongodb start
mongodb: unrecognized service

But I can start it using /etc/init.d as you can see below:
me@medev:/var/lib/mongodb$ sudo /etc/init.d/mongod start
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service mongod start

Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the start(8) utility, e.g. start mongod
mongod start/running, process 27469
me@medev:/var/lib/mongodb$ ls
journal  local.0  local.ns  mongod.lock

me@medev:/var/lib/mongodb$ mongo
MongoDB shell version: 2.6.5
connecting to: test
> db
test
>

Any ideas on why I can't start it using the service command would be appreciated.  From what I've read, i should be using sudo service mongodb
Comment: The problem is that the upstart scripts for mongodb are kind of broken when it comes down to _stopping_ mongod, as there are problems for identifying the PID correctly. I have noticed that problem, too and will submit a patch for it. Meanwhile, please restart your mongod instance with `sudo killall mongod && sleep 3 && sudo service mongod start`.
Comment: @MarkusWMahlberg you'd better add this as an answer to the question because this was the only thing which solved my problem! As of July of 2015 it doesn't seem to be fixed yet!
Comment: I'm on Ubuntu 16.04 and got it to work by following the instructions here - https://www.techrepublic.com/article/how-to-install-mongodb-community-edition-on-ubuntu-linux/

This post says - "If you are using Ubuntu 16.04, you may run into an issue where you see the error mongodb: unrecognized service due to the switch from upstart to systemd."
Answer: Try this:
Write mongodb instead of mongod
sudo service mongodb status

Comment: Weird - this worked for me as well even though the official documentation says to use 'mongod'.  Anybody know why?
Comment: You man need:
`$ sudo mongod --repair --dbpath /var/lib/mongodb/`
Comment: Version or platform specific ? I installed mongodb 3.2.3 with the org package on an Ubuntu 14, and my service is indeed called `mongod`
Comment: Mongo really needs to clean up their documentation etc if they are to compete.
Answer: I got the same error one day You should use this:
1.Get the status of your mongo service:
 /etc/init.d/mongod status

or
 sudo service mongod status

2.If it's not started repair it like this:
sudo rm /var/lib/mongodb/mongod.lock
mongod --repair
sudo service mongodb start

And  check again if the service is started again(1)
Comment: Non of those works
It says `bash: /etc/init.d/mongod: No such file or directory`
or `Unrecognised Service`
Comment: on my server it's called mongodb instead of mongod for some reason.
Comment: If you don't have a file in /etc/init.d/mongod (or /etc/init.d/mbongod, then you need to create one. See http://stackoverflow.com/a/22336712/249672
Answer: For me the solution was to replace
service mongod start

with
start mongod

Comment: This option worked for me in MongoDB 4. https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
Answer: You need to make sure the file (ex. /etc/init.d/mongodb) has execute permissions.
chmod +x /etc/init.d/mongodb

Comment: I get "No such file or directory". Which Ubuntu version are you running?
Answer: For debian, from the 10gen repo, between 2.4.x and 2.6.x, they renamed the init script /etc/init.d/mongodb to /etc/init.d/mongod, and the default config file from /etc/mongodb.conf to /etc/mongod.conf, and the PID and lock files from "mongodb" to "mongod" too. This made upgrading a pain, and I don't see it mentioned in their docs anywhere. Anyway, the solution is to remove the old "mongodb" versions:
update-rc.d -f mongodb remove
rm /etc/init.d/mongodb
rm /var/run/mongodb.pid
diff -ur /etc/mongodb.conf /etc/mongod.conf

Now, look and see what config changes you need to keep, and put them in mongod.conf.
Then:
rm /etc/mongodb.conf

Now you can:
service mongod restart

Answer: I installed mongo server on Debian Jessie using manual from official site.
It didn't started after recommended command  sudo service mongod restart with the same error - mongodb: unrecognized service.
After looking into installed package contents, I noticed that it contains only Systemd service unit, but no SystemV init script:
# dpkg -L mongodb-org-server
/.
/usr
/usr/bin
/usr/bin/mongod
/usr/share
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/mongodb-org-server
/usr/share/doc
/usr/share/doc/mongodb-org-server
/usr/share/doc/mongodb-org-server/LICENSE-Community.txt
/usr/share/doc/mongodb-org-server/README
/usr/share/doc/mongodb-org-server/copyright
/usr/share/doc/mongodb-org-server/changelog.gz
/usr/share/doc/mongodb-org-server/GNU-AGPL-3.0.gz
/usr/share/doc/mongodb-org-server/THIRD-PARTY-NOTICES.gz
/usr/share/doc/mongodb-org-server/MPL-2.gz
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/mongod.1.gz
/etc
/etc/mongod.conf
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/mongod.service

But my system was running on SysV init:
# stat /proc/1/exe
File: '/proc/1/exe' -> '/sbin/init'

So, there are 2 options now:

(Continue on SysV) Write sysV init script manually as @khylo mentioned above
(Switch to SystemD) and run systemctl start mongod
Answer: For me nothing have helped, I've ended up with a solution:
create /lib/systemd/system/mongod.service file with content
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual

[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf

[Install]
WantedBy=multi-user.target

then start/stop commands should work
$ sudo service mongod start

For reference - I have Ubuntu 14.04 LTS, MongoDB 3.2.9 installed from
deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.2 multiverse

Answer: If you're running Ubuntu in WSL (Windows Subsystem for Linux), you will have issues because WSL does not currently support systemd.
The link below explains how to run MongoDB without systemd, and even how to add a script for using the service command with WSL.
https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-database#mongodb-init-system-differences

tutorials may start MongoDB using the operating system's built-in init system. You might see the command sudo systemctl status mongodb used in tutorials or articles. Currently WSL does not have support for systemd (a service management system in Linux).
You shouldn't notice a difference, but if a tutorial recommends using sudo systemctl, instead use: sudo /etc/init.d/. For example, sudo systemctl status docker, for WSL would be sudo /etc/init.d/docker status ...or you can also use sudo service docker status.
Answer: I think you may have installed the version of mongodb for the wrong system distro.
Take a look at how to install mongodb for ubuntu and debian:
http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/
http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
I had a similar problem, and what happened was that I was installing the ubuntu packages in debian
Answer: Original Source - https://www.techrepublic.com/article/how-to-install-mongodb-community-edition-on-ubuntu-linux/

If you're on Ubuntu 16.04 and face the unrecognized service error, these instructions will fix it for you:-

Open a terminal window.
Issue the command sudo apt-key adv —keyserver hkp://keyserver.ubuntu.com:80 —recv EA312927
Issue the command sudo touch /etc/apt/sources.list.d/mongodb-org.list
Issue the command sudo gedit /etc/apt/sources.list.d/mongodb-org.list
Copy and paste one of the following lines from below (depending upon your release) into the open file.
For 12.04: deb http://repo.mongodb.org/apt/ubuntu precise/mongodb-org/3.6 multiverse
For 14.04: deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.6 multiverse
For 16.04: deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse

Make sure to edit the version number with the appropriate latest version and save the file.
Installation
Open a terminal window and issue command sudo apt-get update && sudo apt-get install -y mongodb-org
Let the installation complete.
Running MongoDB To start the database, issue the command sudo service mongodb start. You should now be able to issue the command to see that MongoDB is running: systemctl status mongodb
Ubuntu 16.04 solution
If you are using Ubuntu 16.04, you may run into an issue where you see the error mongodb: unrecognized service due to the switch from upstart to systemd. To get around this, you have to follow these steps.

If you added the /etc/apt/sources.list.d/mongodb-org.list, remove it with the command sudo rm /etc/apt/sources.list.d/mongodb-org.list
Update apt with the command sudo apt-get update
Install the official MongoDB version from the standard repositories with the command sudo apt-get install mongodb in order to get the service set up properly
Remove what you just installed with the command sudo apt-get remove mongodb && sudo apt-get autoremove

Now follow steps 1 through 5 listed above to install MongoDB; this should re-install the latest version of MongoDB with the systemd services already in place. When you issue the command systemctl status mongodb you should see that the server is active.

I mostly copy pasted the above (with minor modifications and typo fixes) from here - https://www.techrepublic.com/article/how-to-install-mongodb-community-edition-on-ubuntu-linux/
Answer: This is a simple solution that worked for me with the same problem (I think):

mv /var/lib/mongodb /var/lib/mongodb_backup
mkdir /var/lib/mongodb
chmod 700 /var/lib/mongodb
chown mongodb:daemon /var/lib/mongodb
systemctl restart mongodb or service mongod restart

Answer: You can use mongod command instead of mongodb, if you find any issue regarding dbpath in mongo you can use my answer in the link below.
https://stackoverflow.com/a/53057695/8247133


Youtube API SeekTo() and Stop() on specific timepoint

Question: I want to play specific points in a Youtube video. Is there a possibility to stop a video on a certain timepoint with the YouTube javascipt API or maybe with another video website as Vimeo?
I can now say (pseudocode)
PlayFragment(start, stop){
    player.seekTo(40);
    player.stop();
}

But I want to stop at for example point 60, to show only the fragment of 20 seconds.
Is there any possibility for this. Or maybe using onStateChange or another trigger when video is on certain timepoint?
Answer: There looks like there is, check out this link for 'YouTube JavaScript API Reference'
https://developers.google.com/youtube/js_api_reference
PlayFragment(start, stop){
player.seekTo(40);

while(player.getCurrentTime() !> 60)
{
     sleep(1000);
}

player.stop();
}

function sleep(milliseconds) {
  var start = new Date().getTime();
  for (var i = 0; i < 1e7; i++) {
    if ((new Date().getTime() - start) > milliseconds){
      break;
    }
  }
}

Comment: That looks like some kind of busyloop cpu wise, may want to use setInterval or bind to an event...
Answer: Typically I'd set an interval, then check "has the timestamp passed 60 since last time? if so, stop"
There's also a "start and end" option you can put in the URL itself which may be useful...


VB6: Surely this simple Hex addition is wrong?

Question: I'm getting odd results in some VB6 code which I've narrowed to this:
Debug.Print Hex(&hEDB80000 + &h8300)

Shows EDB78300
That can't by right can it? Surely it should be EDB88300?
Am I going mad?
Answer: Don't forget how negative numbers are expressed in binary, and that VB6 and VB.NET interpret numbers like &h8300 differently.
Because &hEDB80000 doesn't fit in 16-bits, VB interprets it as a long (32-bits).  Because the high bit is set, VB6 knows it's negative.
Let's undo the two's complement (in a 32-bit world) to figure out the decimal value
(~&hEDB80000 + 1) = &h1247FFFF + 1 = &h12480000 = 306708480

since the sign bit was set, that's -306708480
Because &h8300 fits in 16-bits, VB interprets it as an integer (16-bits).  Because the high bit is set, VB6 knows that it's negative.
Let's undo the two's complement (in a 16-bit world)
(~&h8300 + 1) = &h7DFF + 1 = &h7D00 = 32000

since the sign bit was set, that's -32000.  When the addition happens, both values are considered to be longs (32-bits).
(-306708480) + (-32000) = -306740480

Let's put that back into two's complement hex
~(306740480 - 1) = ~(&h12487D00 - 1) = ~(&h12487CFF) = &hEDB78300

So &hEDB78300 is the correct answer.

Notes:
I personally thing the confusion happens because of the following:
&h0004000 is interpreted as  16384 // Fits in 16-bits, sign bit is not set
&h0008000 is interpreted as -32768 // Fits in 16-bits, sign bit is set
&h0010000 is interpreted as  65536 // Requires 32-bits, sign bit is not set

as mentioned in the other post, you can get around this by explicitly marking values as longs
&h0004000& is interpreted as  16384
&h0008000& is interpreted as  32768
&h0010000& is interpreted as  65536

Comment: Can you elaborate on that?  I'm not seeing how the result of anything negative plus a small positive number would end with the same small number.  (Switching to base10...  -1000 + 97 = 903, not something ending with 97.)  His output ends with "8300", which is the number he added.
Comment: I've done the math in a hex calc and I get his expected result.  I must not be seeing the forest for the trees.
Comment: @JMD: VB6 does things kind of weird, and interprets 0x8300 as a negative number, so it comes out strange
Comment: @JMD: because the version ofg this answer you were reading was inaccurate.  Daniel has since made an attempt at correcting it but seems to be overcomplicating it.
Comment: @daniel:  There is nothing weird or strange about it.  Its not 0x8300 VB doesn't know what that is, its &h8300 which is an interger value of -32000, whats strange about that?
Comment: @Anthony: I think it feels strange if you're coming back to VB6 from VB.NET where &H8300 is interpreted as 33536, _not_ -32000
Answer: Fundementally because VB6 sees &h8300 as an integer having the value -32000.  To get the results you were expecting you would need to explictly mark it as a Long:-
Debug.Print Hex(&hEDB80000 + &h8300&)

What your were doing was adding a Long to an Interger.  To do that VB6 first extends the Integer to a Long, since &h8300 represents a negative number the Long it is converted to ends up with the value &hFFFF8300.  Armed with that value you can see that the result returned by VB6 is correct.
FF + B8 = B7 with carry bit set
FF + ED + carry bit = ED

Comment: +1 for being succinct.  (also, I think you're missing an h on &EDB80000)


Why would forfeits in football be scored as 3-0?

Question: Why would forfeits in football be scored as 3-0 to the non-forfeiting team?
Is there any reason for this score?
Comment: Related older post: [Is a match forfeit always scored as 3-0 regardless of situation?](http://sports.stackexchange.com/questions/3993/is-a-match-forfeit-always-scored-as-3-0-regardless-of-situation)
Answer: If the final standings depend on goal difference, goals scored, goals conceded, or a score between two specific teams, then a score needs to be recorded.
Clearly it would be a benefit to the team forfeiting to be granted any goals, as this helps their goals-for. On the other hand, if a team would only get goals by cheating, they shouldn't be given any. Thus, the forfeiting team gets a score of zero.
The team that wins shouldn't also be given zero, since a bad team could just enforce a nil-all draw (in terms of goals scored/conceded) with anybody better, which does more damage to ranking than anything else might. If they're one of those good teams, a single goal isn't fair either, but neither is say, twenty (since it causes a similar problem).
So it has to be something sensible, something realistic.
Now three goals is pretty decent, and affects the forfeiting team's goals-against more than just playing the match (ideally, anyway), acting as a deterrent.
You'll find some competitions will give five-nil scores instead of three-nil, or will use a formula to estimate a "more accurate and fair" winning score. Three is an overall guess at the average expected score if the forfeited match were actually played, as it

removes extra thought
removes additional oversight requirements
is easy and simple to remember and apply as a rule.


Why sample complexity must be polynomial for PAC learning?

Question: I'm reading up on Probably Approximately Correct (PAC) learning and most sources require that the sample complexity must be polynomial in $\frac{1}{\epsilon}$ and $\frac{1}{\delta}$, where ${\epsilon}$ and ${\delta}$ are error rate and failure rate, respectively.
I have also found lecture notes that states there are two limitations to PAC:

Polynomial sample complexity, which is also called information    theoretic constraint, governs if there is enough information in the sample to distinguish a hypothesis $h$ that approximate $f$
Polynomial    time complexity, also called computational complexity, which tells if    there is an efficient algorithm that can process the
sample and    produce a good hypothesis $h$

$f$ is a concept from some concept class $C$.
While it is somewhat usefull, I don't understand where this requirement comes from.
Answer: It's part of the original motivation of "A Theory of the Learnable" Valliant (1984).
The idea is that people appear to be able to learn concepts from a reasonable number of examples, so that learning from a reasonable number of examples should be a process that can be automated. It's certainly plausible that learning more complicated classes will take longer, and learning with fewer errors will take longer. And it's traditional in computer science to defined 'reasonable' in terms of polynomial growth.
So, if you consider PAC-learning as a model for how learning could really happen, it makes sense to have polynomial complexity and error bounds. And, in addition, it turns out to be a 'nice' class with other interesting properties.


How to pass multiple parameters to the Web API controller methods in angular4

Question: Spring Rest Api :
<code>@RequestMapping(value={"/save-userlist"}, method=RequestMethod.POST)
    public ResponseEntity<?> saveUserList(@RequestBody UserListDTO userListDTO, @RequestBody List<User> users, @RequestParam Integer userId) {
        ResponseEntity<?> response= null;
        try{
            logger.debug(userListDTO, );
                        logger.debug(users, );
                     return  ResponseEntity.status(HttpStatus.OK).body("success");
        }
        catch(Exception e){
            logger.debug(e.getMessage());
        }
        return  response;
    }
</code>
can anyone please suggest me how to pass multiple parameters to the Web-API from angular4
Comment: Did you try with this: @RequestParam Map requestParams ?
Answer: Please correct me if i'm wrong but you can only pass 1 request body. Your request body should be 1 object which contains all your variables in json format, this will automatically be serialized to i.e. the UserListDRO object.
See this example:

Method parameter user is marked with @RequestBody annotation. Thanks
  to this annotation, Spring will try to bind the request body [which
  can be JSON/XML/Other] to user object[ Means crating a new user object
  with the details found in the request body like user name,age etc..],
  based on Content-Type header in Http request.


Does anyone know how to answer this stats q.?

Question: Let $X$ and $Y$ be random variables with $E(Y) = 3$, $\operatorname{Var}(Y) = 4$, $E(X) = 5$, $\operatorname{Var}(X) = 6$, and $\operatorname{Cov}(X,Y) = 0$. Suppose we do a linear regression of $X$ on $Y$, which gives us the resulting linear model:
$Y = a + b X.$
What are the values of $a$ and $b$?
Answer: We can't answer this question without additional information since it's possible that $X$ and $Y$ are not independent. If we assume they are gaussian then we can say that they are independent and that the population parameters would be $a=3$ and $b = 0$ since that yields the estimator with the lowest mean square error.


Commuting matrix with rotation matrix?

Question: If we suppose that:
$AR=RA$, for any rotation matrix $R$.
Then can we say that $A$ is invariant under rotation?
If so, why?
I tried to do it myself, but I can't seem to get there.
I know what invariant under rotations means, I don't know how I get from what I assumed to invariant under rotation. Or if it is even possible..
Comment: EDIT. "Invariant under rotation ", they're just words; scrap this idea! Show that "$AR=RA$, for any $R\in O^+(n)$" implies that $A$ is a scalar matrix, when $n\geq 3$. It is easy to show  that the result is false when $n=2$.
Comment: Seems to me a legit question. Anybody care to explain the downvotes?
Comment: Invariant under rotation would mean that $A$ remains same after rotation, so that would be $AR=RA=A$ for all rotations $R$.
Comment: I didn't down vote the question, but the probable reason for it is that you didn't give any indication that you've tried to do the problem yourself.  For all anyone can tell, you're just asking for somebody to do your homework for you.  Give some context; what have you tried, where are you having trouble?
Comment: @loupblanc Could you provide the proof? I cannot seem to find it and it drives me mad because I'm sure it is something straightforward.
Comment: @student , cf. my post below.
Answer: Let $A\in M_n(\mathbb{R})$ be s.t., for every $R\in O^+(n)$, $AR=RA$.
$\textbf{Proposition.}$ If $n=2$, then $A$ is in the form $\lambda S$ where $S\in O^+(2)$.
If $n=3$, then $A$ is a scalar matrix.
$\textbf{Proof.}$ $\bullet$ If $n=2$, then $A$ commute with the matrices $\begin{pmatrix}\cos(u)&-\sin(u)\\\sin(u)&\cos(u)\end{pmatrix}$, that is equivalent to $A$ commute with $U=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$, that is equivalent to $A\in span(I_2,U)$.
$\bullet$ If $n=3$, let $v$ be a unitary vector. Since $A$ commute with the matrix $Rot(v,\pi/2)$, there is $\lambda$ s.t. $Av=\lambda v$. Thus, any non-zero vector is an eigenvector of $A$ and consequently, $A$ is a scalar matrix.

Commuting matrix with rotation matrix?


VMware Workstation guest OS (Windows) resets date/time to host on restart

Question: Everytime I restart my windows guest machine, it resets the date/time to the host date/time.
How can I change it permanently?
UPDATE: Time sync is already disabled in VMware tools
Comment: http://www.vmware.com/pdf/vmware_timekeeping.pdf
Comment: Can't you just change the timezone?
Answer: Base on your VMware Workstation version you have to use "0" or "false".
For Vmware Workstation 10 its working with these settings:
tools.syncTime = "FALSE"
time.synchronize.continue = "FALSE"
time.synchronize.restore = "FALSE"
time.synchronize.resume.disk = "FALSE"
time.synchronize.shrink = "FALSE"
time.synchronize.tools.startup = "FALSE"
time.synchronize.tools.enable = "FALSE"
time.synchronize.resume.host = "FALSE"

You can use also rtc.startTime = "1416441600" to let the machine start with specific date and time - in this example its 20.11.2014. That number are seconds starting from 1.1.1970
Info from: http://kb.vmware.com/kb/1189
Comment: These other answers do not work for Vmware Workstation 10. When its set as "0" , its ignored. You have to use "false" instead.
Comment: Could you explain how this is different and better from the two similar answers from '10 and '09?
Answer: I had the same issue.
No matter what I tried in the .VMX file or options to set, it always seems to sync the time.
To solve it, I booted in to safe mode and actually disabled the VMware tools service. I cannot remember now if it was the whole tools or if there was just one that was for time sync.
Then edited the time.
This worked and survived a reboot.
Answer: It turns out it was the windows synchronization service. I disabled automatic time update with the time servers and now it stays changed.
UPDATE: Turns out I was wrong, it DID change back after a while. Disabling VMware service does work!
Answer: Assuming you have the VMware tools installed in the guest, you might try
tools.syncTime = "FALSE"

in your .vmx file.
A related question.
Comment: It's already in the file (by default)
Comment: Ah, I see.  You may be out of luck.  I suspect VMware is setting the emulated hardware clock based on your host clock when it starts the VM.
Answer: Maybe you guest windows virtual machine is joined to the same domain as your host machine and syncing to the server who has the  PDC emulator role.
Answer: You should not have to disable the VMWare service, it is useful for other reasons. Try setting all the time sync related settings:
tools.syncTime = 0
time.synchronize.continue = 0
time.synchronize.restore = 0
time.synchronize.resume.disk = 0
time.synchronize.shrink = 0
time.synchronize.tools.startup = 0

I was experiencing the same problem you are seeing, and this fixed it for me.
Answer: Actually the .VMX didn't contain any of these parameters except tool.syncTime = "False".
Just try to edit/add these parameters to the .VMX file
tools.syncTime = 0
time.synchronize.continue = 0
time.synchronize.restore = 0
time.synchronize.resume.disk = 0
time.synchronize.shrink = 0
time.synchronize.tools.startup = 0
After setting up a desired time in guest OS and after reboot, it came up properly with the desired time.
Thanks guys.


How to use hook_views to add calculation to a field?

Question: In a custom module (assoc), I have a custom Content Entity (Person) with these fields:
$fields['lastname'] = BaseFieldDefinition::create('string')
$fields['firstname'] = BaseFieldDefinition::create('string')
$fields['email'] = BaseFieldDefinition::create('email')
$fields['cellphone'] = BaseFieldDefinition::create('telephone')
$fields['balance'] = BaseFieldDefinition::create('integer')

I use Views to display a table of the persons. Here is the corresponding Views preview:

Before displaying the output, I'd like to add some value to the field 'balance':
balance = balance (of content entity) + somecredit - somedebit
'somecredit' and 'somedebit' being calculated by a query to the database.
What is the hook_views I should use ?
And how do I do?
Answer: Finally, after some (!!) digging, I did it in assoc_views_post_execute
function assoc_views_post_execute(ViewExecutable $view)
{
  switch ($view->id()) {
    case 'assoc_persons':
      if ($view->current_display == 'page_1') {
        $database = \Drupal::database();
        foreach ($view->result as $key => $value) {
          $person = $value->_entity->get('id')->value;
          $solde = $value->_entity->get('balance')->value;

          some code to calculate credit with database query
          $credit = $result[0]->credit;

          some code to calculate debit with database query
          $debit = $result[0]->debit;

          $solde = $solde + $credit - $debit;
          $value->_entity->set('balance', $solde);
        }
      }
      break;
    default:

  }
}


Sin and cosine calculations are only calculate on acute angles of triangles?

Question: So sine and cosine calculations are only calculate on the acute non-right angles of triangles. Is that correct?
This from math2.org:
Definition 1
Given any angle q (0 £ q £ 90°), we can find the sine or cosine of that angle by constructing a right triangle with one vertex of angle q. The sine is equal to the length of the side opposite to q, divided by the length of the triangle's hypotenuse. The cosine is equal to the length of the side adjacent to q, divided by the length of the triangle's hypotenuse. In this way, we can find the sine or cosine of any q in the range 0 £ q £ 90°.
Definition 2
Draw a unit circle, in that a circle of radius 1, centered at the origin of a 2-dimensional coordinate system. Given an angle q, locate the point on the circle that is located at an angle q from the origin. (According to standard convention, angles are measured counter-clockwise from the positive horizontal axis.) The sin(q) can be defined as the y-coordinate of this point. The cos(q) can be defined as the x-coordinate of this point. In this way, we can find the sine or cosine of any real value of q (q Î Â).
So my questions:

What are the sin and cosine exactly? Do you only calculate the sine and cosine of right triangles? What do you if the triangle is not right? Are there still relationships between the sides given an angle?
When you calculate sin(85 degrees), what exactly are we calculating? Are we calculating the opposite side/hypotenuse of a right triangle?
Comment: Definition 2 is just a generalization of definition 1 beyond the 90 degree range. I don't see any discrepancy.
The answer to the question in the first line: obviously no. Sines and consines are calculated with arguments
in the entire complex plane via Euler's formula, cos(x) = Re(exp(i*x)), sin(x) = Im(exp(i*x)), and in this
way can be extended to arguments which are matrices (using the Taylor expansion of the exp) and probably much further.
Answer: Sine and cosine are useful in calculus, and in analyzing vibrations, and in waves, including sound waves and radio waves.  Scientists and engineers have to know all about them.
There are connections between sines, cosines and every triangle:
SINE RULE: If a triangle has sides of length $a,b,c$ and angles of size $A,B,C$, so that $A$ is the angle opposite side $a$, and so on, then $$\frac{\sin A}a=\frac{\sin B}b=\frac{\sin C}c$$
COSINE RULE:
$$c^2=a^2+b^2-2ab\cos C$$
For example, the cosine of a right angle is zero, so the cosine rule for a right-angled triangle is $c^2=a^2+b^2$, which you may know as Pythagoras' Theorem.
Comment: What does sinA equal to in the sine rule definition? How do you know which side is the hypotenuse if the sides are unknown?
Comment: Also if you calculate the cosine of the only right angle of a right triangle... why is the adj/hypotenuse = 0? Doesn't it depend on the lengths of the two non-hypotenuse sides?
Comment: If $A=65^{\circ}$, then you could draw a right-angled triangle, one angle $65^{\circ}$, measure the ratio.  The ratio will be $0.9065....$.  If another angle is $B=75^{\circ}$, then $\sin B=0.9659....$  The third angle will then be $C=40^{\circ}$ because $A+B+C=180^{\circ}$, and $\sin40=0.6428...$.  The sine rule tells you the sides of the triangle are in the ratio $a=0.9065R,b=0.9659R,c=0.6428R$
Comment: Take the right-angled triangle, and then make the adjacent side shorter and shorter.  You find the other angle gets closer to $90^{\circ}$.  You almost get two right angles.  When the cosine is very close to zero, the other angle is very close to $90^{\circ}$.
Comment: Oh so you never actually measure the right angle itself when dealing with sin and cosine. It always deals with the acute angles and not the actual right angle itself?
Comment: Yes, that's right.
Comment: But then in the law of sines... you're comparing the sin of every angle to the length of the opposite side. So it does seem like in some instances you're taking the sin or cosine of the right angle of a triangle. Imagine a 45-45-90 triangle. At some point you're going to take the sine of the right triangle when making a law of sines ration comparison. However, the sine of that right angle doesn't seem to be equal to 1 because what exactly is the opposite side? More so... what is the cosine of that right angle? Which side is the adjacent side? Do you see my point?


Bind por posição em SQL Server utilizando Node JS (MSSQL)

Question: Gostaria de saber se é possível realizar o bind dos parâmetros por posição ao invés do nome utilizando mssql em NodeJS.
Ex.:
//Por nome
return pool.request()
.input('input_parameter1', sql.Int, 10)
.input('input_parameter2', sql.Int, 20)
.query('select * from mytable where id = @input_parameter1 and valor = @input_parameter2')

O que preciso seria algo do tipo:
//Por posição
return pool.request()
.query('select * from mytable where id = ? and valor = ?', [10, 20],
function(err, result){
console.log("Resultado: " + result);
});

Comment: Tenha cuidado em alterar a forma de consulta, pois no primeiro código o "mssql" implementa uma proteção contra SQLi. Veja aqui: https://github.com/patriksimek/node-mssql#sql-injection
Comment: @MarceloJunior se souberes uma alternativa de resolver isto melhor que a minha coloca uma resposta, eu não conheço a biblioteca por isso pode haver maneiras melhores.
Comment: Obrigado pela dica @MarceloJunior, não tinha pensado nisso....
Comment: @Sergio também não conheço, procurei alternativas para ajudar na resposta e não achei nada que já tenha sido implementado.
Comment: @LucasSouza Por nada :)
Answer: Podes usar dessa maneira com o mysql (link). A ideia é como mostras no código, usar marcadores ? na query e depois passar uma array com as posições cujo valor deve ser usado em vez do marcador.
Segundo o exemplo da documentação:
connection.query('UPDATE users SET foo=?, bar=?, baz=? WHERE id=?', ['a', 'b', 'c', userId], (error, results, fields) => {
  if (error) throw error;
  // ...
});

Para implementar isso no mssql podias fazer o teu próprio wrapper:
const superQuery = (conn, query, values) => {
  let req = pool.request();
  let value = values.shift();
  let counter = 1;

  while (typeof value !== 'undefined') {
    let sep = 'input_parameter' + (counter++);
    req = req.input(sep, value);
    query = query.replace('?', '@' + sep);
    value = values.shift()
  }
  return req.query(query);
}

E depois usares assim:
superQuery(
    connection,
    'UPDATE users SET foo=?, bar=?, baz=? WHERE id=?',
    ['a', 'b', 'c', userId]
).then(res => {
    ...etc
});

Nota: o mssql tem um conceito de protótipos como segundo argumento opcional de . input(). Seria simples incorporar isso no meu wrapper para não se perder esse nivel de segurança. Porem a array passada em vez de simples seria um objeto com o prototipo também.
Comment: Funcionou como vc falou e mantive a segurança que o mssql utiliza..
Comment: Então, com o mysql eu vi que tem como.. Só que meu banco é o mssql, sabe se tem como usar por posição nele, como no mysql? Os marcadores podem alterar, isso não tem problema, se for o caso, tipo @ID... Só que queria por posição do que por nome na hora de fazer o bind...
Comment: @LucasSouza desculpa, não reparei no `mssql`! Tentei redimir-me com uma sugestão de solução :)
Comment: Obrigado! Estou fazendo o wrapper conforme comentou para manter a segurança
Comment: @LucasSouza ótimo!


How to set a column as an index in dateframe in pandas

Question: I have a dict like this:
   dict = {'open': 245.04, 'high': 245.769, 'low': 244.7, 'close': 245.51, 'volume': 1489478, 'datetime': 1584451800000}

I want to convert it into a dateframe data and set datetime as an index.
So I tried:
data=pd.DataFrame.from_dict(dict,orient='index')

And then I received:
                     0
open      2.450400e+02
high      2.457690e+02
low       2.447000e+02
close     2.455100e+02
volume    1.489478e+06
datetime  1.584452e+12

How to set datetime as an index?
Comment: What you mean index, can you show us the output ?
Answer: IIUC notice do not name the dict with dict , since python have the variable  called dict
df=pd.DataFrame.from_dict(d,orient='index').T.set_index('datetime')
                open     high    low   close     volume
datetime
1.584452e+12  245.04  245.769  244.7  245.51  1489478.0

Comment: Thank you for your answer,can you tell me what is T?
Comment: @williamfaith https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.T.html


Use ruby one liners from ruby code

Question: I read about Dave Thomas Ruby one liners
Its says
  # print section of file between two regular expressions, /foo/ and /bar/
      $  ruby -ne '@found=true if $_ =~ /foo/; next unless @found; puts $_; exit if $_ =~ /bar/' < file.txt

May I know how i can use this is my Ruby code and not from command line?
Answer: According to ruby CLI reference,
-n              assume 'while gets(); ... end' loop around your script
-e 'command'    one line of script. Several -e's allowed. Omit [programfile]

So, just copy the code snippet to a ruby file enclosed in gets() loop
foobar.rb
while gets()
   @found=true if $_ =~ /foo/
   next unless @found
   puts $_
   exit if $_ =~ /bar/
end

And execute the file using
ruby foobar.rb < file.txt

You can also replace IO redirect by reading the file programmatically
file = File.new("file.txt", "r")
while (line = file.gets)
   @found=true if line =~ /foo/
   next unless @found
   puts line
   exit if line =~ /bar/
end

Use ruby one liners from ruby code


Inertia and Decomposition Field in $\mathbb{Q}(\omega=exp(2\pi i/m)$, where $m=p^{k}n,(p,n)=1$.

Question: let $\omega=exp(2\pi i/m),m=p^{k}n,(p,n)=1$.
We know that the Galois group $\mathbb{Q}(\omega)$ over $\mathbb{Q}$ is isomorphic to $(\mathbb{Z}_{m})^\times$, which is naturally isomorphic to $\mathbb{Z}_{p^k}^{\times}\times(\mathbb{Z}_{n})^{\times}$. My question is: how does one describe the decomposition group, $D$ and inertia group $E$ with respect to $p$ in terms of $(\mathbb{Z}_{p^k})^\times$ and $(\mathbb{Z}_{n})^\times$?
Here is my attempt: Let $Q$ be any prime lying over $p$ in $\mathbb{Q}(\omega)$. $p$ splits completely in $\mathbb{Q}(\omega^{p^k})$. Hence, $\mathbb{Q}(\omega^{p^k})$ lies in the fixed field of the decomposition group $L_{D}$, which is a subfield of $L_{E}$ (the fixed field of the inertia group). Therefore, we have $[L_{E}:\mathbb{Q}]\geq [\mathbb{Q}(\omega^{p^{k}}):\mathbb{Q}]=\phi(n)$. But we know that $[L:L_{E}]=\phi(p^{k})$, which implies $L_{E}=\mathbb{Q}(\omega^{p^{k}})$. Hence, $E$ is isomorphic to $\mathbb{Z}_{p^k}$ (need to use Galois theory).
The part that i am having trouble with is how does one describe $D$? I know that $D/E$ will be a subgroup of $\mathbb{Z}_{n}^\times$, and also cyclic, but i am not sure how to describe it. Thanks.
Answer: With $\Phi_m(x)$ the minimal polynomial of $\omega_m$ you'll have $x^{p^k}-1 = (x-1)^{p^k} \bmod p$ so $\omega_{p^k} = 1 \bmod p,\omega_m = \omega_n \bmod p$ and $\Phi_m(x) =\Phi_n(x)^{\phi(p^k)}\bmod p$.
Then $p \Bbb{Z}[\omega_n] = \prod_{j=1}^{\phi(n)/a} (p,\omega_n-\zeta_j)$ where $a$ is the order of $p$ modulo $n$ and the $\zeta_j= \omega_n^{d_j},gcd(n,d_j)=1$ are the roots of $\Phi_n(x)$ in $ \Bbb{Z}[\omega_n]/(p)$ whose $p$-th powers are distinct, ie.
$\Phi_n(x) = \prod_{j=1}^{\phi(n)/a}\prod_{l=1}^a (x-\zeta_j^{p^l}) \in \Bbb{Z}[\omega_n]/(p)[x]$ and $$p \Bbb{Z}[\omega_m] = \prod_{j=1}^{\phi(n)/a} (p,\omega_m-\zeta_j)^{\phi(p^k)} = \prod_{ r \in \Bbb{Z}/(n)^\times / \langle p \rangle} (p,\omega_m-\omega_n^r)^{\phi(p^k)}$$
Since we are in abelian extension the decomposition/inertia group are the same for each of those prime ideal.
The inertia group are the $\sigma\in Gal(\Bbb{Z}[\omega_m]/\Bbb{Z}), \sigma(\omega_m) = \omega_m^r$ such that $\omega_m^{p^kr}=\sigma(\omega_n) = \omega_n=\omega_m^{p^k}$ ie. $r \equiv 1 \bmod n$, and the decomposition group are the $\sigma\in Gal(\Bbb{Z}[\omega_m]/\Bbb{Z})$ such that $\sigma(\omega_n) = \omega_n^{p^l}$ ie. $r \equiv p^l \bmod n$, that is $D_p$ is the preimage in $\Bbb{Z}/(m)^\times $ of $ \langle p \rangle \subset \Bbb{Z}/(n)^\times$.
Then the game is to show $$\zeta_{\Bbb{Z}[\omega_m]}(s) = \prod_\mathfrak{q} \frac{1}{1-N(\mathfrak{q})^{-s}} = \prod_{\chi \bmod m} L(s,\tilde{\chi})$$ where $L(s,\tilde{\chi})$ is the Dirichlet L-function of the primitive character underlying each character $\chi$ modulo $m$.


Finding $\lim_{x\rightarrow 0}\frac{x^{6000}-(\sin x)^{6000}}{x^{6002}}$

Question:
Finding $\displaystyle \lim_{x\rightarrow 0}\frac{x^{6000}-(\sin x)^{6000}}{x^{6002}}$

Try: I have tried using series expansion of $\displaystyle \sin x = x-\frac{x^3}{3!}+\frac{x^5}{5!}+\cdots $
So $$\lim_{x\rightarrow 0}\frac{x^{6000}-\bigg(x-\frac{x^3}{3!}+\cdots \bigg)^{6000}}{x^{6002}}$$
$$\lim_{x\rightarrow 0}\frac{x^{6000}-x^{6000}\bigg[1-\frac{x^2}{6}+\cdots \bigg]^{6000}}{x^{6002}} = 1000$$
Could some help me how to solve it without Series expansion or D L Hopital Rule
Thanks
Answer: For integer $n\ge0,$
$$\dfrac{x^{n+1}-\sin^{n+1}x}{x^{n+3}}=\dfrac{x-\sin x}{x^3}\cdot\sum_{r=0}^n\dfrac{x^r\sin^{n-r}x}{x^n}$$
Now $$\dfrac{x^r\sin^{n-r}x}{x^n}=\left(\dfrac{\sin x}x\right)^{n-r}$$
Now use Are all limits solvable without L'Hôpital Rule or Series Expansion
Answer: Hint: $\;\dfrac{x^{6000}-(\sin x)^{6000}}{x^{6002}} = \dfrac{1-\left(\dfrac{\sin x}{x}\right)^{6000}}{x^2}=\dfrac{1 - \dfrac{\sin x}{x}}{x^2}\cdot \left(1 + \dfrac{\sin x}{x} + \ldots\right)\,$


How to calculate the single value decomposition. when eigenvalue=0?

Question: After calculating the eigenvalues. I got 0 as one of my 2 eigenvalues. Hence, one of the singular values of my matrix is 0.
So
$\sigma_1 = 0$
But then when Im trying to find U:
 $$u_1 = \frac{1}{\sigma_1} Av_1$$
What do I do? $\frac{1}{\lambda_1} $ is undefined.
Answer: The correct form of this equation (since we can't divide by $\sigma_1$) is
$$
\sigma_1 u_1 = Av_1
$$
Since $\sigma_1=0$, we are free to select $u_1$. Keep in mind, however, that $u_1$ must be chosen to be length $1$, and orthogonal to all other $u_k$.
So, what you do is as follows: first, find all the other vectors $u_k$. Then, select any vectors for $\sigma=0$ such that, all together, the $u_k$ form an orthonormal basis.
You may want to use Gram Schmidt in order to do this.


Using Find, Grep, Awk, or Sed To Rename Server After Cloning

Question: My client tells me they have cloned a VM in VMWare of an Ubuntu Linux server. Now it's my job to get into all the files and find out what still has the old server name of "bishop" and change it to something else. Also, the IP address is changed and I need to search for that too.
How would you typically use find, grep, awk, or sed to find these files and then change them rapidly? In the end, I want to make a Bash script.
Of course, I'm not expecting you to tell me every file, but just want to know the technique for finding files with "x" in it and then switching that rapidly with "y".
Comment: The fact that this is a VM is not an issue in respect of the question so I have removed the [tag:vmware] tag
Comment: Do you want to find files that contain the string 'bishop' and/or some ip address ?
Answer: I'd actually strongly suggest that you make this a TWO step operation:

Find the files with the old hostname and old IP:
find / -print | xargs egrep -l 'oldhost|10.10.10.10' > filelist
Then edit the resulting list and remove any that you know one should not change.
Then use the resulting list, put it in a shell script and doing something like the following command sequence using the filelist as input:

.
cp filename filename.20110624     # lets be safe!
if test $? -ne 0
then
    echo 'filename copy bad'
    exit 1
fi

cat filename.20110624 | sed 's/oldhost/newhost/g
s/10.10.10.10/10.20.20.20/g' > filename # the newline between commands is needed

if test $? -ne 0
then
    echo edit failed
    cp filename.20110624 filename
    if test $? -ne 0
    then
        echo unable to restore filename
        exit 1
    fi
    exit 1
 fi

 exit 0

Comment: Maybe someday, stackoverflow will support alternating code and bulletpoint blocks, until then I added a "." to have some plain text so the formatting would come out right.  I also added a comment to point out the newline between sed `s///` commands is significant
Comment: thanks DerflK for your edits! I was trying to do what you did for me.


3rd order differential equation with variable coefficients

Question: How to do I solve this differential equation?
$$ x^3 u′′′ + x^2 u′′ + x u′= 0. $$
The series solution method is not working in this case.
Comment: one solution is $$u(x)=C$$ $C$ const.
Comment: But there should be 2 more solutions as well
Comment: yes we have additionally $$x\sin(\ln(x))$$ and $$x\cos(\ln(x))$$
Answer: Thos is an example of a Cauchy-Euler equation.  See this link to Wikipedia for a solution method in the second-order case.  The same thing works in third order if you can solve the cubic equation for the exponent, which should be possible here.  Be ready to deal with the "complex" case.
Cauchy-Euler equations have a regular singular point at zero, so a Maclaurin series as you apparently tried does not generally work.
Answer: Start by putting $v=u'$ and dividing through by $x^3$ to get $v''+\frac{v'}{x}+\frac{v}{x^2}=0$.
The $\frac{1}{x}$ suggests putting $v=w(\ln x)$. That kind of substitution is less common, but it can still be useful. The point is that we have $v'=\frac{w'}{x}$ or $w'=xv'$, that is what we need here so that higher derivatives are multiplied by higher powers of $x$.
We have $v'=\frac{w'}{x},v''=\frac{w''}{x^2}-\frac{w'}{x^2}$ so we get $$w''+w=0$$
That is an easy equation for $w$. We solve it to get $$w=A\cos t+B\sin t$$ Remember that $t=\ln x$. So we have $$v=A\cos(\ln x)+B\sin(\ln x)$$
Now we have to integrate that. The obvious thing to try is to differentiate $x\cos(\ln x)$. That gives $\cos(\ln x)-\sin(\ln x)$. Similarly, differentiating $x\sin(\ln x)$ gives $\cos(\ln x)+\sin(\ln(x)$. So we have finally $$u=Cx\cos(\ln x)+Dx\sin(\ln x)+E$$.
Comment: @OscarLanzi Not quite sure what the problem is. Is that clearer?
Comment: @OscarLanzi ? $x\cos(\ln x)$ gives $\cos(\ln x)+x(-\sin(\ln x)\frac{1}{x})=\cos(\ln x)-\sin(\ln x)$
Comment: Ok, now it works.
Comment: Now? I didn't change it!
Comment: Got my functions mixed up when I differentiated the first time.
Answer: Considering $$x^3 u′′′ + x^2 u′′ + x u′= 0$$ let us start with $v=u'$ and now $$x^3 v′′ + x^2 v′ + x v= 0$$ Change variable using $x=e^y$ $$\frac {dv}{dx}=\frac {dv}{dy}\frac {dy}{dx}=\frac {1}{x}\frac {dv}{dy}$$  $$\frac {d^2v}{dx^2}=-\frac {1}{x^2}\frac {dv}{dy}+\frac {1}{x ^2} \frac {d^2v}{dy^2} $$ All of this makes $$\frac {d^2v}{dy^2}+v=0$$ which is simple to integrate (at least for $v(y)$).


Matplotlib & Data Visualization: Compare overlapping histograms

Question: I'm comparing several histogram together, to evaluate the variance over the years:

The problem is that, while it shows the variance, it is hard to discern one type from another. I'm wondering what would be a better way to make this plot?
Answer: Those bars are not really stacked but overlapped. You can stack them by doing:
import matplotlib.pyplot as plt
import numpy as np

mu, sigma = 0,100
x1 = mu + sigma*np.random.randn(1000)
x2 = mu + sigma*np.random.randn(1000)
x3 = mu + sigma*np.random.randn(1000)

#Stack the data
plt.figure()
plt.hist([x1,x2,x3], bins=30, stacked=True, normed = True)
plt.show()

, which will result in this:

However to perceive a quantile variability in different distributions the boxplot is usually more adequate:
plt.boxplot([x1,x2,x3])
plt.show()

, results in:

You can also take a look into plots such as Probability Plot, or StackPlot. If you really need to compare the variance value (and not just the distribution globally) you can plot the lines for variance (or standard deviation if the order of magnitude is big) over your plots.
Comment: You have an argument that allows you to remove the fill from the bars in matplotlib histogram. Try adding histtype='step' as argument. It should only show you the contours of the histogram (https://micropore.wordpress.com/2011/11/15/python-histogram-histtypestep/). Also take a look at this: http://matplotlib.org/examples/statistics/histogram_demo_multihist.html
Comment: My mistake. I wrongly used `stack`. It should be `overlap`. The `boxplot` would lose the histogram shape, which I think is really important here.
Comment: Than use the Violin Plot (http://matplotlib.org/examples/statistics/violinplot_demo.html). What features are you trying to compare (variance is just a number)?
Comment: No, no,no. I don't need to compare this specfic here, e.g. variance etc. . I just want to compare the histogram. The problem is that overlapping made them hard to look at. The half-transparent setting (`alpha`) makes it hard to discern one from another.


How Do I Compare 2 Git Branches By Commit Message And Not By Hash?

Question: If I do git log develop..feature it is going to show me the difference between the develop and feature branches using the hash id.
However, I have rebased one of my branches and the commit hashes are now different even when the commits are the same (because the parent is different).
How can I compare the two branches by commit message?
Note: I am aware that comparing this way is not reliable. It isn't the only diff I will be doing to determine the difference between the branches but it will help me identify any extra commits.
Answer: It depends on what you mean by "compare".
The three-dot notation can tell you all the commits they don't have in common.
git log develop...feature

If you want to find what commits they have in common by commit message, you could try diffing a sorted list of all the commit title lines.
diff -u <(git log --format='%s' feature..master | sort) <(git log --format='%s' master..feature | sort)

Or you can rebase them both on top of a common parent where they got the same commits from in the first place. The redundant commits will go away.
Comment: @opticyclic Rather than editing the answer, could you clarify the question? It's not clear to me how you want this compare to happen. Perhaps you could provide a concrete example?
Comment: The first line of the question states that using dot notation compare using hash id. The next line explains why I can't use that. The question then asks how to compare by commit message. Your reference to three-dot notation is is not a valid answer because it compares by hash id which I said I can't use. The part about rebasing on a common parent also doesn't answer the question on how to compare by commit message. The middle part is the only relevant part of the answer.
Comment: @opticyclic I answered with more than the literal question because this smells like an [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem); I expect it's easier to let Git fix the branches with rebase than to try and compare and fix them manually. But, again, the situation is not clear to me. Could you please clarify? An example of your branches, how you got them into this situation, and the sort of comparison you'd like to see would help. I realize this was years ago, but it's an interesting problem.


CSS display:table-layout issue

Question: Here's my problem:
In the jsbin code here. I am getting gray border of #wrapper hidden to the right.
However this happens only in firefox. Chrome is working fine.
Can you please let me know the fix for firefox?
Answer: See this example: http://jsbin.com/avupak/1/edit
Relevant CSS
width:100%;
-webkit-box-sizing: border-box;
-moz-box-sizing: border-box;
box-sizing: border-box;

box-sizing: border-box; computes the width including borders
Answer: Only setting the width to 100% would also do.
width:100%;

CSS display:table-layout issue


Output Cache ASP .Net MVC

Question: Estou estudando sobre o output cache no Asp .NET MVC para melhorar o desempenho da minha aplicação. Pelo que vi ele pode ser armazenado em vários lugares como:
<code>· Any
· Client
· Downstream
· Server
· None
· ServerAndClient
</code>
Dúvida
Caso eu faça o cache de uma página, quando usar cada uma das situações? Não encontrei nenhum lugar onde explique qual a diferença do armazenamento entre elas.
Fonte: https://docs.microsoft.com/pt-br/aspnet/mvc/overview/older-versions-1/controllers-and-routing/improving-performance-with-output-caching-cs
Comment: Acho que [neste site](https://docs.microsoft.com/pt-br/dotnet/api/system.web.ui.outputcachelocation?redirectedfrom=MSDN&view=netframework-4.7.2) voce consegue encontrar maiores referencias.
Comment: Neste site apenas explica onde é armazenado cada um, eu gostaria de saber quando usar cada uma delas e por que.
Answer: Na maioria das vezes o cache Any vai atender a expectativa, pois ele realiza o cache Local/Proxy/Server.
Se você tem um ecommerce e ele possui uma página de categorias, onde o usuario selecionará qual a categoria ele deseja acessar.
Este tipo de informação pode ser "cacheada" em qualquer lugar, quando o usuario tentar acessar e ele tem os arquivos na propria maquina, vai apenas fazer uma requisição HEAD no server e se o cache estiver valido ele responde com os dados que está na maquina local, se um usuario que nunca acessou o seu sistema tentar acessar, ele vai utilizar o cache da ultima requisição.

Você não deve realizar o cache de informações sensiveis no server/proxy, veja a seguinte situação:
<code>  [OutputCache(Duration = 3600, VaryByParam = "none")]
    public string GetName()
    {
        return "Olá " + User.Identity.Name;
    }
</code>
Se o usuario Cesar realizar a requisição, vai retornar normalmente "Olá Cesar", porém, quando o José realizar a requisição após o Cesar, vai retornar a informação "Olá Cesar".

Relacionado ao cache local, se voce precisar de uma atualização completa na página, e inviabilizar o acesso à pagina antiga, você precisaria alterar o endereço do seu site, para que o usuario realize a requisição e não utilize o cache local.
Ref: link 1 link 2 link 3

Output Cache ASP .Net MVC


ntpdate -t does not timeout with the timeout value

Question: I am using ntpdate to update the time. If there is no internet connection or the NTP server is down, ntpdate fails, but ntpdate takes a long time to exit (approximately 20 to 23 seconds). I read the manual of ntpdate and I see -t timeout option.
-t timeout Specify the maximum time waiting for a server response as the value timeout, in seconds and fraction. The value is rounded to a multiple of 0.2 seconds. The default is 1 second, a value suitable for polling across a LAN.
I thought to try this option but it is taking the same amount of time to exit. I thought if we specify timeout then ntpdate will exit within specified seconds. I used the command like this
ntpdate -u -t 10 <ip>
Did anyone try the timeout option in ntpdate?
Comment: If your Internet connection is intermittent or unreliable, consider using an NTP client/server designed for this scenario such as chrony.
Answer:
ntpdate will exchange information with ntp server at least three times, before setting time on client to trust this ntp server.
Therefore, if you add -t 10 option on ntpdate, it may run about 30s.
You can add -d option for more information on checking with ntp server.
# ntpdate -dv -4 time.stdtime.gov.tw
16 Jul 14:57:36 ntpdate[652]: ntpdate email@example.com Wed Jun 22 03:15:52 UTC 2022 (1)
Looking for host time.stdtime.gov.tw and service ntp
126.96.36.199 reversed to 118-163-81-61.hinet-ip.hinet.net
host found : 118-163-81-61.hinet-ip.hinet.net
transmit(126.96.36.199) // first time
receive(18.104.22.168)
transmit(184.108.40.206) // second time
receive(220.127.116.11)
transmit(22.214.171.124) // third time
receive(188.8.131.52)
transmit(18.104.22.168) // fourth time
receive(18.104.22.168)

server 22.214.171.124, port 123
stratum 2, precision -25, leap 00, trust 000
refid [192.168.0.3], root delay 0.000397, root dispersion 0.019592
reference time:      e67cdd88.16fefc9c  Sat, Jul 16 2022 14:47:36.089
originate timestamp: e67cdfe7.80de2e29  Sat, Jul 16 2022 14:57:43.503
transmit timestamp:  e67cdfe7.5e43017f  Sat, Jul 16 2022 14:57:43.368
filter delay:  0.29337    0.31583    0.28447    0.31200
            ----       ----       ----       ----
filter offset: -0.003258  -0.024422  -0.004762  -0.008031
            ----       ----       ----       ----
delay 0.28447, dispersion 0.00401, offset -0.004762

16 Jul 14:57:43 ntpdate[652]: adjust time server 184.108.40.206 offset -0.004762 sec


Divisibility properties of point counts for elliptic curves over finite fields

Question: Let $E$ be an elliptic curve over the finite field $\mathbb{F}_q$ (of characteristic $p$). My questions are about the following ideal $I_E \subset \mathbb{Z}$:
$$I_E=\langle \#E(\mathbb{F}_q), \#E(\mathbb{F}_{q^2}), \#E(\mathbb{F}_{q^3}), \#E(\mathbb{F}_{q^4}), \#E(\mathbb{F}_{q^5}), \dots \rangle$$
For convenience let $n_E$ be the positive generator of $I_E$.
At LMFDB one can view a lot of examples of the numbers $\#E(\mathbb{F}_{q^d})$, which suggest "rules" obeyed by the $n_E$. After inspecting many examples I noticed the following two "rules" in particular:

If $E$ is ordinary then $n_E>1$.

Here is an example with $q=5$ and (apparently) $n_E=3$. Here is a supersingular $E$ with $n_E=1$.

If $E$ is supersingular then $p \nmid n_E$.

On the other hand, if $E$ is ordinary then frequently $p \mid n_E$, e.g. in this example.
My questions: Are the above "rules" actually correct? If so, why are they true? Is there any conceptual interpretation of the number $n_E$?
Answer: As $E(\Bbb F_q)$ is a subgroup of $E(\Bbb F_{q^r})$ for all $r$, then
$|E(\Bbb F_q)|\mid|E(\Bbb F_{q^r})|$ for all $r$, and so
$n_E=|E(\Bbb F_q)|$,
If we write $|E(\Bbb F_q)|=q+1-a$, where $a$ is the "trace of Frobenius",
one characterisation of supersingularity is that $p\mid a$. Therefore
$E$ is supersingular iff $n_E\equiv1\pmod p$.
By Hasse, if an elliptic curve over $\Bbb F_q$ has only one point, then $q=2$.
There is such a curve; it is supersingular.
Comment: Minor correction: $\#E(\mathbb{F}_q)=1$ also happens for $q=3$ and $q=4$.


Can't use numbers in certain instances for perl hash keys?

Question: Quick question...
Why does the first key work but not the rest? That is, the second key throws a syntax error. I've used numbers as keys before, but as soon as I write 'to' the script turns black (that is, not comment colour which is used for keys normally). If I take away the 'to' it works and throws an error on the next key.
Can I not have a number and letter combination starting with a number?
my %ranges = (
    under10 => "x < 10000",
    10to20  => "10000 <= x < 20000",
    20to30  => "20000 <= x < 30000",
    30to40  => "30000 <= x < 40000",
    40to50  => "40000 <= x < 50000",
    50to60  => "50000 <= x < 60000",
    60to70  => "60000 <= x < 70000",
    70to80  => "70000 <= x < 80000",
    80to90  => "80000 <= x < 90000",
    90to100 => "90000 <= x < 100000",
    100plus => "100000 <= x",
);

Comment: did you tried it with qoutestrings?
Comment: no, good point - I've gotten so used to not using them, as I rarely use numbers as keys. Why can't I do it without them?
Answer: Put them in quotes. The documentation says:

The => operator is mostly just a more visually distinctive synonym for a comma, but it also arranges for its left-hand operand to be interpreted as a string if it's a bareword that would be a legal simple identifier.

Identifiers have to begin with a letter or underscore, so 10to30 is not a legal identifier. As a result, it doesn't get converted to a string.
Answer: you need to qoute them :)
my %ranges = (
    'under10' => "x < 10000",
    '10to20'  => "10000 <= x < 20000",
    '20to30'  => "20000 <= x < 30000",
    '30to40'  => "30000 <= x < 40000",
    '40to50'  => "40000 <= x < 50000",
    '50to60'  => "50000 <= x < 60000",
    '60to70'  => "60000 <= x < 70000",
    '70to80'  => "70000 <= x < 80000",
    '80to90'  => "80000 <= x < 90000",
    '90to100' => "90000 <= x < 100000",
    '100plus' => "100000 <= x",
);

Comment: Why though? If it is smart enough to use numbers or text or text and numbers together in that order, why not numbers and text?! weird!
Comment: See the documentation I quoted in my answer.


How To Get The Current Year Using Vba

Question: I am trying to get the current year using VBA. Is there any pre-build function available to find out the current year?
Comment: Even though it's in Excel, if you are asking how to do something in VBA, it relates more to programming. I'm migrating this to Stack Overflow.
Comment: See the answer here: http://stackoverflow.com/a/23479761/630284
Answer: Year(Date)

Year(): Returns the year portion of the date argument.
Date: Current date only.
Explanation of both of these functions from here.
Comment: For those who might get this error that I received above: I found it was because I had a label whose `Name` value was also "Year". A little troubling that VBA let me name something the same as what a reserved keyword is... Changing my label to "YearLabel" allowed me to use `Year(Date)` successfully.
Comment: Thanks for adding that, @TylerH.
Comment: This gives a Compile error: "`Wrong number of arguments or invalid property assignment`"
Answer: Try =Year(Now()) and format the cell as General.

How To Get The Current Year Using Vba


PLSr: Generating predicted value using regression coefficient

Question: I perform PLS with pls package in R using plsr function.
Why am I unable to get the same predicted Y value as when I use the predict function as when I dot product the regression coefficients with the test data. I have read from other answers that I should be able to do this, especially if the method is set to simpls.
EDIT:
I did some further testing and it seems like there is a bias/intercept constant that I have not added in. However, from elsewhere I read that with mean centering, the intercept should be 0 and I believe plsr has already mean centered all my data by default. I am wondering why is there still an intercept and also how can I find this intercept constant from the plsr model?
EDIT again:
I have found the coef(,intercept=TRUE) function! However, my question still remains, why is there an intercept when my data has been mean centered (Both X and Y)?
Many thanks for your help!! Let me know if I can clarify anything at all!
Answer: pls::plsr centers both $\mathbf X$ and $\mathbf Y$, and the corresponding intercepts are in $Xmeans and $Ymeans.
So in order to predict using the coefficients that map $\mathbf{Y_c} = \mathbf { X_c} \mathbf B$, you need to

center $\mathbf X$: $\mathbf X_c = \mathbf X - \bar x$ ($\bar x$ is $Xmeans)
matrix-multiply by $\mathbf B$: $\mathbf Y_c = \mathbf X_c \mathbf B$
"uncenter" $\mathbf Y$: $\mathbf Y = \mathbf Y_c + \bar y$ (= add $Ymeans)

library (pls)
yarn.pls <- plsr(density ~ NIR, 6, data = yarn)

X_c <- yarn$NIR [1,, drop = FALSE] - yarn.pls$Xmeans
Y_c <-  (X_c %*% yarn.pls$coefficients [,1,])
Y_c + yarn.pls$Ymeans

gives:
      1 comps  2 comps  3 comps  4 comps  5 comps  6 comps
[1,] 90.53581 90.49171 99.13326 98.94135 99.39108 99.54403

which is the same as (up to some small numerical error)
predict (yarn.pls, yarn [1,]) [,1,]

After putting the 3 steps above together, we can separate the calculations involving the centering:
$$\mathbf Y = (\mathbf X - \bar x) \mathbf B + \bar y\\
= \mathbf X  \mathbf B - \bar x \mathbf B + \bar y \\
= \mathbf X  \mathbf B + \beta_0$$
with
$\beta_0 = - \bar x \mathbf B + \bar y$
leaving us with a single intercept, which is the intercept from coef (intercept = TRUE)
This way:
coef1 <- coef (yarn.pls, intercept = TRUE)
X1 <- cbind (1, yarn$NIR [1,, drop = FALSE])
X1 %*% c

we get the same prediction as above using all 6 components:
         [,1]
[1,] 99.54403

Comment: thank you for such clarity!!! but just to solidify what I have in mind, adding the intercept is what you mean by 'uncenter' Y right?
Comment: @sunnydk: should be clear now. One important point is that we have 3 different intercepts: *two* for X and Y in the "low-level" formulation, and one "high-level" intercept that combines the two (with the coefficients): for PLS it is always important to spell out which intercept is meant.
Comment: I was just going to edit my previous comment; but yes! combining your answer with theGD's, I was able to work out that the intercept is made up of both ymeans and coef times xmeans. Thank you very much!!
Answer: For PLS, mean-centering is equivalent to adding a constant "1" as a variable to every observation. In other words, it is matter of algebra to account for mean-centering with an intercept term. One can also account for scaling too. See below:

Sorry for bad notation. So, $n$ is number of variables, $B$ is regression coefficients obtained by mean-centered and scaled data, $s$ stands for stddev and $\bar{x}$ and $\bar{y}$ are the means. $B_0$ here is the intercept term. Ignoring $s$ terms gives the version for mean-centering only case.
Comment: thank you for reminding me about the constant '1'. However, do you mind elaborating a little more about what you mean by 'equivalent' (perhaps intuitively if possible, I can follow the above algebra but I am just trying to make the link in my mind between the bias constant and mean-centering)? Also, should your Bnewi be equals to (sy*Bi)/sxi instead, since xi is from the X matrix?
Comment: In PLS, the aim is to find $B$ that maximizes covariance. If you add 1's as a variable, the best (according to aim of PLS) $B$ for that variable ($B_0$ or intercept) is the one that the rest of the model will vary around, basically the average. In other words, since the unbiased estimate (very vaguely means an estimate where no error is "expected") for intercept is the average, you can exploit this directly (mean-centering). Alternatively, you can let the algorithm find the best estimate for the intercept and this corresponds to adding 1s.
Comment: thank you for your answer!! there is quite a bit to unpack here mentally, but combining your answer and cbeleities' have helped me make much more sense already :)


What is the "u=" and "i=" class in CSS?

Question: I am trying to modify the CSS in a Drupal 7 theme from being statically coded to one which is generated by views.  The person before me, wrote the slideshow directly into the page--front.tpl.php file and this makes it next to impossible for users to change the slideshow without getting a Drupal developer involved.
In the "static" page, there are some tags which, due to my ignorance, I don't know how to deal with.  For exampe:

I know what the "id" is and I know what the "class" is, but I have no idea what the "u=" references.  I have also seen:

Can someone tell me the terminology associated with this tag so I can research it?  Typing in "<div u=" in Google is bringing up tons of unrelated results
Comment: You can run across a picture of a cat in Drupal, does that mean you should be able to ask a question on identifying it here too? You're asking for someone to identify the meaning of HTML attributes - whether they are standard are non-standard, that doesn't have anything to do with Drupal. Someone's already given you a response, so you have an answer, the question is fundamentally off-topic, and now it's closed, per the way every Q+A site in this network operates. What "point in the right direction" do you feel I could give you here? The closure has been explained, and your question answered
Comment: You shouldn't need to work with the 'u' and 'i' attributes when writing CSS
Comment: Excellent @Cilive  Something I run across in DRUPAL, while trying to THEME in CSS for which I say I don't have the correct terminology is for you to be ever so helpful and say "Off Topic, go somewhere else."  If this is off topic, why have those tags?  If you are so knowledgeable, why not a point in the right direction?
Comment: Even better.  Not only are you not helpful, but insulting to boot.  Did I ask about content or something I din't know was/wasn't specific to Drupal theming conventions?  Please don't try to defend your obstructionist answer as following policy - if you KNEW what an answer was, why not say it AND THEN point me to the right forum?  Too "end user helpful centric" for you?
Comment: @Clive is correct: `` is just an HTML tag with non standard attributes. There is nothing in it that is Drupal specific, which means a Drupal standard installation doesn't output anything like that. You are asking about the meaning of those attributes, and for Drupal they don't have any meaning.
Comment: If you were asking which module outputs those attributes, then the question could be more on-topic, but unfortunately we cannot look for every existing module to tell you which module outputs that. Since you have access to the site you are developing, you would have more chances than us to find which module would output that; we don't have access to any sites you are developing, so we should work hard to find that module, or be very lucky to guess it.
Comment: That's not the point.  My point is Cilive could have stated your first sentence exactly which would have explained why it's off topic as opposed to the the curt approach he decided to take.  I am not saying the answer is wrong, I am saying his approach to the answer is wrong.  It is not helpful and obstructionist followed by a condescending, sarcastic response.  It was in page--front.tpl.php and not a module (see OP)
Comment: @Allan Chill out, the only sarcasm in this comment thread has been from you (read it back if you don't believe me, it's all still there). There hasn't even been a hint of an insult from anyone, not sure what you're talking about there. It isn't my fault you didn't know this was a plain-HTML, and not Drupal, question. No one's blaming you for that, we all learn as we go, but given that you'd already received and acknowledged an answer as correct, and the question is fundamentally off topic, I don't understand what your issue with it being closed is, or what message you expected from me
Answer: Those attributes ("u" and "i") are not standard attributes. They seems from this JOSS slider or another slider.
Examine your Drupal modules and the View your want to modify to know which slider are you using. Then check the documentation for that slider.
Comment: Thank you. This is what I was looking for "they are not standard attributes." Now I know how to approach and what I need to do in views. And yes...it appears it is JOSS slider"

What is the "u=" and "i=" class in CSS?