Should there be no foot sound when jogging?

Question: I heard that when jogging/running that there should be no foot thud sounds when landing. Is it true?
Answer: You ask:

Is it true?

Well, to an extent, yes.  But don't worry too much about achieving silence.
A thud sound whilst running generally indicates that the heel is striking the ground on the downstep more than it should.  However, rather than focus on reducing the sound or feel of the thud, you should improves the mechanics of your running and the thud will reduce as a result.
The thud is caused because the impact shock is not absorbed by the flesh of the foot arch or the muscles in the calf.  Instead, the force travels up through the bone at the front and can lead to shin splints in the muscles there.
To counter this, you need to spend as much of your time running on the balls of your feet as you can.
Try rolling your foot landing rather than coming down at an angle.  You can reduce your stride to achieve this.  Watch a video of a long-distance runner to see how they run without much impact against the ground.
Comment: Here's what I like about this answer: "...rather than focus on reducing the sound or feel of the thud, you should improves the mechanics of your running and the thud will reduce as a result." Too much running advice out there focuses on symptoms without discussing underlying causes.
Comment: "You need to spend as much of your time running on the balls of your feet as you can." Mid-foot is better as staying on the balls of your feet will engage your calf muscle which will tire out over longer distances. Mid-foot strike engages the glute and quad while using the hamstring to propel forward. Staying on the ball of your foot will use the calf muscle for landing and propelling which is great for sprinting/quick steps but bad for distance.
Answer: My answer is 'Yes'.
Moreover, I think that making no footstep sound with intention is a good way to naturally get technical skill, or to easily improve your running technic.
Because doing that, indirectly, you will absorbe shocks with you calfs and your thighs (muscles), and not with your bones. Your bone legs will not be hurt because of the non-max outstretched stride.
Answer: No "thud" sound?  Yes.  No sound at all?  No.
In general, the reason you don't want to hear much noise is because it means your body is going up and down and not forward. The "thud" sound means you are using excess force (energy) down against the ground when what you want is to maximize the amount of energy you are using to go forward. Since energy is a zero-sum game when running, that means you won't go as fast or as far as your potential energy allows.


SQL agrupando os dados iguais, mas mostrando as fotos dos registros

Question: Tenho um SLQ que retorna a quantidade de registros repetidos agrupados,
Porem minha necessidade e que list todas as fotos dos registros duplicados
SELECT count(Longitude) AS registros, id_local, Longitude,Latitude,nome_local,endereco,pic FROM `locais` GROUP BY Longitude,Latitude ORDER BY `nome_local` ASC

A consulta me retorna que tenho 2 registros em uma mesma latitude, eu preciso que me de o nome das 3 fotos que se encontram em cada registro, porem agrupado apenas com o nome do local
Answer: Para esta situação, eu utilizaria o GROUP_CONCAT() do MySQL, desta maneira, para cada nome diferente encontrado durante o agrupamento, eu teria o valor separado por vírgula. Ficaria algo como abaixo (imaginando que as fotos estejam no campo pic):
SELECT count(Longitude) AS registros, id_local, Longitude,Latitude,nome_local,endereco, GROUP_CONCAT(pic) as pics FROM `locais` GROUP BY Longitude,Latitude ORDER BY `nome_local` ASC

Desta maneira, no código faria as tratativas necessárias. O resultado para essa coluna seria algo como "foto1, foto2, foto3...".


Restrict ssh user to connection only

Question: Setting up an SSH jump box to allow some outside people to access an internal server.
I have the authentication stuff all working where they will use a private/public key pair to authenticate. I'll then be using IP tables to allow access to RDP to the server they need to work on. I will then have them use the tunnel feature on Putty so they will only have to do a localhost:port on RDP window once connected instead of running commands in the box. The above is all working fine.
Since these are outside people, I DO NOT want them having access to anything on the jump box at all. I simply want them to authenticate ONLY and then remote using the Windows RDP.
Right now when they authenticate, it opens the command line in putty and it needs to stay open so they can remote. My question is this. Is it possible to restrict the users to only be able to authenticate and nothing else? I do not want them to have any shell access at all. Maybe something similar to what /bin/false does EXCEPT the connection stays open to allow the RDP without them being able to even type commands.
Ideally, I would like their putty session to not work anymore except to keep the connection alive until closed. Not sure if that is possible though. I can't seem to find a straight answer anywhere either.
I am a bit new to Linux so please bear with me if anything I ask is stupid.
Comment: https://askubuntu.com/questions/48129/how-to-create-a-restricted-ssh-user-for-port-forwarding
Answer: You can take a look at bash's restricted mode, and use it as the users' login shell.
From GNU's website:

If Bash is started with the name rbash, or the --restricted or -r option is supplied at invocation, the shell becomes restricted. A restricted shell is used to set up an environment more controlled than the standard shell. A restricted shell behaves identically to bash with the exception that the following are disallowed or not performed:

Changing directories with the cd builtin.
Setting or unsetting the values of the SHELL, PATH, ENV, or BASH_ENV variables.
Specifying command names containing slashes.
Specifying a filename containing a slash as an argument to the . builtin command.
Specifying a filename containing a slash as an argument to the -p option to the hash builtin command.
Importing function definitions from the shell environment at startup.
Parsing the value of SHELLOPTS from the shell environment at startup.
Redirecting output using the >, >|, <>, >&, &>, and >> redirection operators.
Using the exec builtin to replace the shell with another command.
Adding or deleting builtin commands with the -f and -d options to the enable builtin.
Using the enable builtin command to enable disabled shell builtins.
Specifying the -p option to the command builtin.
Turning off restricted mode with set +r or set +o restricted.

By the way, I'm not sure that a jump box with SSH tunnels is ideal for your use case. Maybe you should consider hosting a private VPN server to allow people from external networks to access your internal servers.
I would recommend SoftEther VPN server because it is lightweight, powerful, and really easy to configure. The configuration process of the server is also consistent across different platforms and operating systems that you might choose to use.

Restrict ssh user to connection only


Get username in custom user twig template

Question: I have a twig template for the compact display of a user where I am able to output the defined content fields. Is there a way to output the username of that user?
Answer: The display name is not a field you can configure in {{ content }}. You can get it directly from the user entity:
{{ user.displayname }}

Reference for the php method: AccountInterface::getDisplayName
Comment: Should have more votes IMO. This is the equivalent of the preprocess function in the accepted answer but called from the Twig template.
Comment: Definitely the best solution.
Comment: @KrishnaMohan, the question is about the user template. If you have a different template this is probably not cached per user. By bubbling up a cache context
you can cache per user: `{{ {'#cache': {'contexts': ['user']}} }}`. Caveat: Drupal is optimized to cache per user permissions. Adding a per user context to random templates could cause a severe impact on cache performance, depending on the amount of users registered on the site.
Comment: it works. but how can I prevent it's being cached. sometimes it's displaying prevoiusly logged in username. Or If I login with other user in other browser and come back and refresh here, the user name is getting changed.
Answer: {{ user.name.0.value }}
should give you what you want.
The preferred way is inject the username as a variable in a preprocess function. To do so, tweak that in your theme's .theme file
<?php

/**
 * Implements hook_preprocess_user().
 */
function yourtheme_preprocess_user(&$variables) {
  /** @var User $account */
  $account = $variables['elements']['#user'];

  $variables['username'] = $account->getDisplayName();
}

then in your user template you can use that as {{ username }}.
Edit:
To get the full URL:

In your template {{ path('entity.user.canonical', {'user': user.id}, {}) }}
In yourtheme_preprocess_user(): $variables['user_url'] = Url::fromRoute('entity.user.canonical', ['user' => $account->id()])->setAbsolute()->toString();
Comment: The later is preferred so that `hook_user_format_name()` runs to do anything that a site may want for usernames.
Comment: Can the full URL of the user page be retreived in a similar way using the user preprocess?
Comment: Updated response to include 2 methods to get the user page url
Comment: Thanks so much for your help. The template method works fine, but the preprocess gives me the following error:
Recoverable fatal error: Object of class Drupal\Core\Field\FieldItemList could not be converted to string in Drupal\Core\Routing\UrlGenerator->doGenerate() (line 192 of core/lib/Drupal/Core/Routing/UrlGenerator.php).
Comment: if you use `yourtheme_preprocess_user()` solution, you need to add `use Drupal\Core\Url;` in your theme's .theme file.
Answer: This answer is specifically for getting the username in menu.html.twig
The solution of passing the username through macro worked for me.
Setting a variable and passing it into the macro is needed as these are out of scope of the macro in menu.html.twig
{% set username = user.account.name %}
{{ menus.menu_links(items, attributes, 0,logged_in,username) }}
{% macro menu_links(items, attributes, menu_level,logged_in,username) %}
{{ username }}
{% endmacro %}

Comment: This answer would be more helpful if it also showed how you setup the `logged_in` and `username` variables in a `hook_preprocess_menu()`.


Why is my script not writing to file?

Question: I have written a script to remove excess spaces from a foreign language text. When I execute the script in Windows command prompt, I receive no errors. Everything looks perfect. However, the output file which I specified in my script is not created nor the input file modified. I tried creating a blank document 'corpus_1' for the script to write to. Then I tried just writing back to the input file. Either way, the specified file remains unmodified. How do I get my script to write to a file? What am I missing in my code?
def lettersWhitespace():

    replacements = {'  ':' ', 'c ':'c'}

    with open('C:\\Users\\Charles\\corpus.odt','w+') as infile, open('C:\\Users\\Charles\\corpus_1.odt', 'w') as outfile:
        for line in infile:
            for src, target in replacements.iteritems():
                line = line.replace(src, target)
            outfile.write(line)

EDIT: I believe that I have found the problem. It appears that my first line, 'def lettersWhitespace():' is redundant. As written, the script is defining a function, but not calling that function. Does this sound correct?
Comment: `w+` wipes the file. I hope that wasn't your only copy.
Comment: Thanks for the input. I do have several backups, but I'm not even coming up with a wiped file after running the script. Nothing is being modified.
However, when I finally get this to work, should I just have 'w' rather than 'w+'?
Answer: Both w and w+ truncate the file. Suppose you have a file containing a, b, c (each in a newline):
with open('testfile.txt', 'w') as f:
    f.write('a\nb\nc')

and you open it in r you can read the file:
with open('testfile.txt', 'r') as f:
    print(f.read())
# a
# b
# c

If you open it in w+ mode it's truncated (empty):
with open('testfile.txt', 'w+') as f:
    print(f.read())
#

You probably wanted a "non-truncating" read/write mode starting at the file beginning: r+ (or if you want the file handle to be at the end of the file: a+)
with open('testfile.txt', 'r+') as outp, open('testfile.txt', 'r') as inp:
    for line in inp:
        line = line.replace('a', 'b')
        outp.write(line)

which modifies the file as you write:
with open('testfile.txt', 'r') as f:
    print(f.read())
# b
# b
# c

A very handy summary of the file modes can be found in this StackOverflow answer of @And.
Comment: Thank you. I am a newbie and had understood 'w' as 'write', 'r' as 'read', 'a' as 'append', and '+' as 'read and write'. It's starting to look like there's a bit more to the picture, so I'm going to dig into some more tutorials on file open commands.
For the script in question, does it appear that that is indeed the reason that my source text is not being altered? I was also wondering if python has difficulty working with text which is not in .txt files? (I'm currently using .odt files as my input and output.)
Comment: @CharlesR In windows there is a difference between binaries and raw data files, it might be necessary to open files in `b`-mode (just add the `b` somewhere to the mode like `rb+`. Not sure if `.odt`-files are binaries. Also I'm a bit confused. Did it work? Note that if this anser fully answers your question please don't forget to [accept it](http://stackoverflow.com/help/accepted-answer).
Comment: I still have not found a way to get it to work. Just now I tried changing the mode to rb+ . I also have switched to using a .txt file for testing purposes. In addition, I tried commenting out the nested 'for' statement as well as changing the output method to print(). Still, nothing happens except that my shell accepts the query and gives me a new command line. Here is my most recent test:
def lettersWhitespace():
    with open('C:\\Users\\Charles\\Test.txt','rb+') as infile:

        for line in infile:
                print(line)
Comment: if nothing happens there is nothing to print. That seems to indicate your file is empty. Are you trying to verify my answer (if yes you need to re-create the file after the `w+` opening) or is this about your original script?
Comment: The new input file ('Test') contains four lines of text. The question is simply, what is the error in my original code which would cause the script to execute without returning either output or error message. I wasn't sure if your answer was saying that the error is in the file mode, so I tried to verify by using various modes. No such changes have yet had an effect on the outcome. I am still running different variations on my original code in order to try to find something that will give me an output.
Comment: Can you share your file or a similar file?
Comment: Sure. My test text file reads:
Ny ob, t x h a i s \n
e e j N E E g \n
t   x ia l is \n
li nts ia l l u b \n
However, is it possible that my problem is being caused by starting the script with a 'def' statement? Should I just be starting the script from 'with open(etc.)'?
Comment: you do call the function, right? So there is a `lettersWhitespace()` (without indentation) at the end of your script - or it's inside a `__name__ == '__main__':` block?
Comment: No. I shortened the 'replacements' set dramatically for sake of posting; but otherwise, what I posted was the entire script.
Comment: In that case just append (without indentation) a call to the function: `lettersWhitespace()`

Why is my script not writing to file?


How to access a component function from main.js in Vue 3 with Composition API

Question: Using Vue 3 and composition API I have a component that have this function:
const retrieveSignedJWT = async (callback) => {
  if (jwtUrl.value && !callback) {
    //console.log("There's no callback use the by default URL")
    await fetch(jwtUrl.value)
      .then(async (response) => {
        const data = await response.text();
        // check for error response
        if (!response.ok) {
          // get error message from body or default to response statusText
          const error = (data && data.message) || response.statusText;
          return Promise.reject(error);
        }
        let jwt = data;
        token.value = data;
        decodeToken(jwt);
        retrieveCategories();
      })
      .catch((error) => {
        errorMessage.value = error;
        console.error("There was an error!", error);
      });
  } else {
    //Function has a callback
    token.value = callback;
  }
};

What I need to do is to find a way to expose the previous component function so I can call it from the main.js. The scenario is that I'm creating an IIFFE with Vue 3 and Vite (a widget that the end user will load from a script) and hooking up a public function to it so the user can use it at any point in their code. That function can have or not have a callback that will expose a token implemented.
import { createApp } from "vue";
import "@/assets/styles/index.scss";
import App from "./App.vue";
import store from "./store";
let div = document.createElement("div");
document.body.appendChild(div);
div.setAttribute("id", "my-widget");
window.myWidget = {
  load: function (endUserRetrievedJWT) {
    if (endUserRetrievedJWT) {
      const endUserJWT = endUserRetrievedJWT();
      //Calling my component function w a call back
      retrieveSignedJWT(endUserJWT);
    } else {
      //Calling my component function without a call back
      retrieveSignedJWT();
    }
  },
};
createApp(App).use(store).mount("#my-widget");

So basically I'm trying to find a way to invoke the parent component function from the main.js file in order to manage how to save a token to the state of my application. That token can come from a default URL or in the shape of a callback function that the end-user will pass as an argument to the globally expose function coming from main.js.
Answer: main.js is meant for setting up your Vue application. Don't use this for any other code! It won't work.
Simply create a separate .js file (e.g. utils.js) and export the function from there.
I'm not completely sure what you're trying to achieve exactly but it looks like an authentication process for logging in. What you're probably trying to do is call the login/logout functions from within a Vue component? This could be as simple as
// Separate file, e.g. `authentication.js`
export const login = (cb) => {
  // do not name the callback function (cb) 'callback' as that may get you unexpected behavior
  // Your code
  somePromise().then((res) => {
    cb(res);
  });
}

// Your component
import { login } from 'path/to/authentication';

login(myCallback);

const myCallback = (res) => {
 ...
};

Comment: Thanks, @paddotk. Can you develop a little bit? What I understand is that, instead of creating a function inside a component, I have to create some sort of "service" that export that functionality globally and then call it from the main.js file. Is that right? The main problem with that is that the function will grab some data from the component.
Comment: A 'service' is basically just a separate javascript class. You can use that too, but it's probably not necessary. A simple javascript file with your function exported will do. As for the data, you can pass any arguments into the function.
Comment: Is this some sort of plugin? Just trying to find the best practice for this approach inside a Vue Implementation. Also, I know that I can pass parameters to the exported function but that parameter is in the state of a component. So I will have to have access to that state in the main.js when I run the globally exposed function.
Comment: I've updated my answer. I don't know if this is what you mean but perhaps it'll help
Comment: Thanks @paddotk, your answer helped me to get to the final approach
Answer: I finally ended up exposing the function in the mounted hook like this:
onMounted(() => {
  window.myWidget = {
    load: retrieveEndUserJWT,
  };
  retrieveDataAttributes();
  callWebsocket(websocket.value);
});

Then inside the same component, I create a method that will process the callback:
const retrieveEndUserJWT = async (endUserRetrievingTokenCallback) => {
  const jwt = await endUserRetrievingTokenCallback();
  processingToken(jwt);
};

And in the processingToken method, I deal with that token coming from the end-user callback function. I still have to navigate the pros and cons of exposing the function in the mounted hook.


Gradient clipping when training deep neural networks

Question: When would one want to perform gradient clipping when training a RNN or CNN? I'm especially interested in the latter. What would be a good starting value for clipping? (it can of course be tuned)
Comment: Choosing a good value of gradient clipping depends on the network and the data, so there's no way to know ahead of time what a good choice is. A choice that's too small will make training take a long time, and a choice that is too large won't effectively curb exploding gradient when it happens. https://stats.stackexchange.com/questions/530288/choosing-a-clip-gradient-for-lstm-deepar/530301#530301
Answer: You would want to perform gradient clipping when you are getting the problem of vanishing gradients or exploding gradients.
However, for both scenarios, there are better solutions:

Exploding gradient happens when the gradient becomes too big and you
get numerical overflow. This can be easily fixed by initializing the
network's weights to smaller values. If this does not work it is
likely that there is a bug in the code.
Vanishing gradient happens when the optimization gets stuck in a
saddle point, the gradient becomes too small for the optimization to
progress. This can be fixed by using gradient descent with momentum
or RMS prop or both (also known as the Adam optimizer).

Starting values for the upper bound of gradient clipping would be something smaller than the largest number the variable can take.
For the lower bound, I would say it is problem specific but perhaps start with something like 1e-10.
Comment: I'm not sure if the context of this answer is supposed to exclude RNNs, but if it doesn't, then both of the proposed solutions are *not* better than gradient clipping especially in the case of RNNs.
Comment: Sorry I was thinking more in the context of CNNs, feel free to edit


Setting up a linux switch

Question: I have a C++ Program to sniff each and every packet that crosses my linux box. However i need to now get my linux box to listen to every traffic in my network. I could buy a managed switch and set up port spanning, but i aint paying 200+ £'s for a switch and plus gives me a chance to learn.
My Network
                                            +---------Computer A
                                            |
Internet-----Router------Switch-------------+---------Linux Box
                                            |
                                            +---------Computer B

Proposed Network
                                            +---------Computer A
                                            |
Internet-----Router------Linux Box--------Switch
                    eth0^         ^eth1     |
                                            +---------Computer B

How would i setup this in linux. Do i just configure both the ethernet on different IP Address on the same network. Or am i completelly on the wrong track
My System

Fedora 13.

Thanks
Comment: Bridge the interfaces...
Comment: Y'know.. You could buy a cheap hub (yes, hub) and then use that.  Hubs just retransmit all traffic to all other interfaces.
Comment: I want to manually set this up. So bridging would mean bonding the two interfaces together.
Comment: I have tcpdump to sniff all traffic.  Did you actually write your own? :O
Comment: @Tom, yea using libcapp, it was a freaking nightmare. Na i dont wana buy. I wana do :D
Comment: Cool, impressive work.
Comment: @Shahmir Javaid: bridge and bonding are two different things.  you want to use bridge (hint: `brctl`)
Comment: You could simply enable ipforward on the linux box and point de df gw of the other machines to the linux box. Or you could try multicast =)
Comment: Thanks Javier, I realized that after some further research
Comment: @voodooo im guessing the de=the and the gw=gateway df=fd && !=xz | zz i can go on but im lost. What is it with Net-workers creating abbreviations for everything, and Doctors with thier latin-izing every freaking word. You gotta be more elaborate cause I suck at networking. :D
Comment: Sory for the late response. Enable IPForward: sysctl -w net.ipv4.ip_forward=1 | and point de default gateway at computer A to the ip address of the linux box.
Answer: I'm assuming your current network is on a private subnet and that your router performs NAT for external communication (i.e. to the outside world all your machines look to have the same IP address). You have two options here:

set the machine up as a bridge
set the machine up as a NAT router

To set it up as a router:

have eth0 setup as if it is a client to the current router
have eth1 on a different subnet
have the server configured to hand out addresses on this subnet to the other machines via DHCP
add the approptiate iptables rules so the box performs NAT for outgoing connections from the eth1 side
if you have any manual port forwarding setup on the router at the moment this will need to be set to point to the new box and the new box set to forward these connections further into the network

Exactly how you do some of the above perminantly (i.e. so it survives a reboot) depends on your Linux distribution and any other local network setup you need to respect. http://www.linuxjournal.com/article/8172 shows an example of a bridge setup using a RedHat/Fedora/CentOS style system, some minuor alterations may be needed for Debian/Ubuntu or Slackware or others.
Answer: brctl addbr br0
ifconfig eth0 up
brctl addif br0 eth0
ifconfig eth1 up
brctl addif br0 eht1
ifconfig br0 <your.ip.address>

note that neither ethernet interface has an IP address; the single address is assigned to the bridge (br0).
Answer: I think this is what you are looking for. :D
Don't forget Promiscuous mode.
ifconfig eth0 promisc
ifconfig eth1 promisc

Comment: i want to manually set this up. Can i define promiscous mode in the ifcfg-eth0 files or the ifcfg-bond file
Comment: Easy way would be rc.local. More correct way would be to write a bit of code for ifup-post, something like this: http://linux.derkeiler.com/Mailing-Lists/RedHat/2007-10/msg00178.html
Answer: You could purchase a hub to replace the switch. It is cheap, the difficulty is merely to find one, now everyone prefers to use a switch. On the Internet you should be able to find one for a small cost.
The hub broadcasts all traffic to each port, by design.
In this case the current network topology would be enough (and set the Linux eth0 to promiscuous mode - otherwise the NIC will drop all packets that are not targeted to the Linux box. If you use wireshark to listen to the network, the soft can set the NIC in promiscuous mode automatically).
If you really want to use the "planned network", you can either

if you want the Linux to play an active network role, your best choice is to implement NAT-masquerading ; because I assume you don't have a class C network or, anyway, enough IPs to have the luxury of making your box a router. In this case, the WAN side should be your public (Internet) address, and the LAN side is a private network (like 10.1.1.1/24).
or, as suggested in a comment, make your box a bridge.

I would definitely go for the hub.
Comment: @ring0: That doesn't look cheap though compared to a cheap switch, even a small Gbit one. And remember that if anything connects at 10Mbit it will be the other side of the switch to the 100Mbit devices. A 10/100 switching hub is essentially two hubs (1x 100Mbit, 1x 10Mbit) with a switch between so even in promiscuous mode you might not see all packets on one port. @Jake: yes, that can be a performance issue, which is why everyone uses switches where affordable (which is everywhere these days) unless they need hub-like behaviour.
Comment: Oh yea didnt think of a hub. But my future plan will be to make my linux box plug and play for that i might as well go through the headache of this bridging malarchy. +1 for the HUB. i cant believe i forgot that. Ive been programming to long. Basic Network Leacture back in UNI 101. :D
Comment: You might be hard pressed to find a true dumb hub these days, as even the cheap and nasty devices you can get for less than a tenner are switches rather than hubs.
Comment: Amazon has some seemingly http://www.amazon.com/Netgear-DS104-4-Port-Uplink-Button/dp/B00000J4LQ/ref=pd_sim_e_1
Comment: You can also force a switch into hub mode on a temporary basis. But its a bit macguyver.
Comment: Wouldn't all the devices you want to sniff have to be on the hub? Sounds like a performance nightmare...
Comment: @David A manageable switch would be more expensive. The Amazon hub above is 10/100Mb, but as I said, the "current plan" (all devices connected to the hub => no switch) would be used in the *hub* case. Thus all packets would be seen (at the expense of performance and privacy).
Comment: @Jake yes to begin with all the devices will be. But for future im thinking to implement linux as a primary firewall. And this firewall may then be set-up on different networks.
Comment: @ring and @dave, ive got managed switches already setup to test the sniffing. However i want to learn and develop a firewall like structure, therefore need to worry about first setting up my linux as a switch based device and then worry about outside and inside interface malarchy. So i really ment it from that perspective, so i can understand a bit more about networking and bridging. :D

Setting up a linux switch


Does it make sense to cut a continuous variable to intervals?

Question: I'm trying to fit a two-class logistic model, using many many features. When inspecting one of the features, I binned it so I could inspect its behavior. In each bin I count the number of 'good class' occurrences, and divide by total number occurrences. I see that in the upper bins, there's a higher probability for a 'good class'.
.
For this kind of functional form to enter the model, I would have to add higher-order terms [for instance, by using natural splines]; and this, I'm afraid, would cause my model to over fit.
So, I thought I could 'help' the regression by explicitly dividing the variable into a few different intervals; the knots would be based on so-called 'eye-balling'. Thus, in each interval the variable could have a different linear coefficient [or I could even make it a fixed number by making the variable categorical].
I hope I've made myself clear, and sorry for the non-mathematical explanation - I'm just trying to make my point clear as possible. Thanks for the help!
Answer: If you use $n$ bins, you will use $n-1$ degrees of freedom. For the same cost in degrees of freedom, you can fit $n-1$ restricted cubic splines. Thus, there should definitely not be more overfitting if you use splines.
Conversely, discretizing continuous variables is almost always a bad idea, see, e.g., here or here or here. In your case, defining the bins by "eye-balling" will tempt you to change them "just a little bit" to get a better fit... I am not saying you will give in to this temptation, but you will need to account for this leeway in model creation in some way.
Bottom line: don't discretize, use restricted splines.
Comment: Yes, in fact they do. Median splits are the most common way of discretizing a continuous variable into two equal bins. Everything these sites warn about in median splits is also true in splitting data into more bins. And when you start to have leeway in determining cut points, you run into "optimal cut point selection", which messes up your inference; see points 10 and 11 in the third link above.
Comment: thanks for the quick reply; and especially for the links; they all talk about "median splits" - a term I wasn't familiar with before.
Comment: +1 to @StephanKolassa Splines are good. Another possibility here, mentioned in the question, is polynomial terms. The curve in the question looks like it would be well fit by a cubic. One advantage of polynomial terms is that they can be easier to interpret.


Free object is a free group in the category of groups

Question: I have a question and would appreciate a clear answer.
Firstly, I will provide an introduction regarding my understanding, and then I will ask my question.
Let's begin with the definition of a universal morphism from an object $ X $ to a functor $F$:
Let $\mathcal{C}$ and $\mathcal{D}$ be two categories. A universal morphism from object $X\in ob(\mathcal{D}) $to functor $F: \mathcal{C} \longrightarrow \mathcal{D} $ is a pair $ (A, f) $, where $A$ is an object of $\mathcal{C}$ and $ f: X \longrightarrow F(A) $ is a morphism in $\mathcal{D}$. This pair satisfies the universal property, i.e., for every $ B \in \text{ob}(\mathcal{C}) $ and any morphism $ g: X \longrightarrow F(B) $, there exists a unique map $h: A \longrightarrow B$ such that the diagram commutes: $ g = F(h) \circ f $.
Now, if $(\mathcal{C}, F) $ is a concrete category and $f $ is the canonical injection, then the universal morphism $ (A, f) $ is called a free object on a set $ X $.
On the other hand, if $\mathcal{C} $ is the category of groups $\textbf{Grp} $, then the free object reduces to the free group in the category $\textbf{Grp} $.
However, the definition of a free group is:

Free group on generators $x_1,...,x_m,x_1^{-1},...,x_m^{-1}$ is a group whose elements are words in the symbols $x_1,...,x_m,x_1^{-1},...,x_m^{-1}$ subject to the group axioms. The group operation is concatenation.

Now, my question is how to reduce a free object into a free group in the category of groups.
I know that the faithful functor $F$ of the concrete category $(\textbf{Grp}, F)$ is the forgetful functor.
Comment: I'm glad you specified you want a "clear answer", because here I was ready to try to answer as unclearly as I could manage...
Comment: "At best, passive-aggressive" -- sheesh. I think a more charitable explanation could be that English is not OP's native language?
Comment: The free group on $X$ is the image of the set $X$ under the functor $\mathsf{F}\colon \mathsf{Set}\to\mathsf{Group}$ which is the left adjoint of the forgetful underlying set functor $\mathsf{U}\colon\mathsf{Group}\to\mathsf{Set}$. I have to think how to relate it to the formalism you have set up, and don't really have the time this instant. Possibly later today. And my previous point is that it is a given that you want clear/good answers. Making a point of explicitly requesting one at the top is at best passive-aggressive, and at worst condescending and insulting.
Comment: @user43208 English is my fourth language.
Comment: Congratulations, but that doesn't matter!  My point is that I think you're being a bit harsh to this newish user. I think you could have left it at the quip in your first comment.
Comment: Okay! Iam thank you so much if you provided to me a clear answer.
Comment: Maybe OP's first sentence reads "I have a question and I would like someone to clarify my confusion with a clear answer, that I am frustrated because I can't find"?
Comment: Note that what you describe is not the "free group on $x_1,\ldots,x_m,x_1^{-1},\ldots,x_m^{-1}$". It is the free group on $x_1,\ldots,x_m$. See my answer.
Comment: See my answer for how to resurrect, i.e., rewrite, what you wrote into something that works.
Answer: Two things before we start:

You are misquoting the definition of free group. What you describe is the "free group on $x_1,\ldots,x_m$", not on "$x_1,\ldots,x_m,x_1^{-1},\ldots,x_m^{-1}$". The symbols $x_1^{-1},\ldots,x_m^{-1}$ are part of the construction, but they are not part of the definition. So here, the set $X$ is going to be $X=\{x_1,\ldots,x_n\}$, and not the set $\{x_1,\ldots,x_m,x_1^{-1},\ldots,x_m^{-1}\}$.

I am going to change your notation, because the use of $F$ is of course natural for "functor", but in this case it conflicts with the usual notation for "free group" and "underlying set." So instead, I am going to take the "data" functor denoted as $\mathsf{U}$, for "underlying set". Your category $\mathcal{C}$ is $\mathsf{Group}$, and $\mathcal{D}$ is $\mathsf{Set}$. So the particular instance of the "free object on $X$" here would be:

A universal morphism from the set $X$ to the "underlying set functor" $\mathsf{U}\colon\mathsf{Group}\to\mathsf{Set}$ (the forgetful functor that sends a group to its undderlying set, and a group morphism to the corresponding function between underlying sets), is a pair $(A,f)$ where $A$ is a group, and $f\colon X\to \mathsf{U}(A)$, the underlying set of $A$. This pair satisfies the universal property; i.e., for every group $B$ and every set map from $X$ to the underlying set of $B$, $g\colon X\to\mathsf{U}(B)$, there exists a unique group homomorphism $h\colon A\to B$ such that $g=\mathsf{U}(h)\circ f$.

You want to understand the "usual" explicit description of "free group on $x_1,\ldots,x_m$" in terms of this definition. (See point 1 above for what the group is free "on").
So, $X$ is going to be the set $x_1,\ldots,x_m$. The group $A$ is going to be the group you describe: the set of "reduced group words" in the alphabet $x_1,\ldots,x_m$; that is, terms of the form
$$w_1^{\epsilon_1}\cdots w_k^{\epsilon_k}$$
where $w_i\in \{x_1,\ldots,x_m\}$, $\epsilon_i\in\{1,-1\}$, $k\geq0$, and where "reduced" means that there is no $i$ such that $w_i=w_{i+1}$ and $\epsilon_i=-\epsilon_{i+1}$. The group operation on $A$ is "concatenate and reduce", which means that you take the words $w_1^{\epsilon_1}\cdots w_k^{\epsilon_k}$ and $v_1^{\delta_1}\cdots v_{\ell}^{\delta_{\ell}}$, and you write the word
$$w_1^{\epsilon_1}\cdots w_k^{\epsilon_k}v_{1}^{\delta_1}\cdots v_{\ell}^{\delta_{\ell}},$$
and then if there are any instances of $x_k$ followed by $x_k^{-1}$, or $x_k^{-1}$ followed by $x_k$, you remove those two letters and you check the result. This is a group, and the easiest way to verify the hard part (associativity) is to use van der Waerden's trick.
So this is the group $A$. The set map $f\colon X\to \mathsf{U}(A)$ is just the inclusion map, since each element of $X$ is itself a reduced word in $A$.
To verify that this is a universal morphism (equivalently, that $(A,f)$ is the free group on $X$) we must show that given any group $B$ and any set map $g\colon X\to B$, there is a unique group morphism $h\colon A\to B$ such that $g=\mathsf{U}(h)\circ f$. This morphism $h$ is defined as follows: given a reduced group word $w_1^{\epsilon_1}\cdots w_k^{\epsilon_k}\in A$, with $w_i\in \{x_1,\ldots,x_m\}$, $\epsilon_i\in\{1,-1\}$, we define
$$h(w_1^{\epsilon_1}\cdots w_k^{\epsilon_k}) = g(w_1)^{\epsilon_1}\cdots g(w_k)^{\epsilon_k}.$$
It is then a bit of work, but not too hard, to verify that this is indeed a group homomorphism and that it satisfies $g=\mathsf{U}(h)\circ f$. Moreover, it is the unique map that will do this, because $A$ is generated by $f(X)$, and the values of any morphism $\mathfrak{h}\colon A\to B$ is completely determined by its value on the generating set $f(X)$; so if $\mathfrak{h}$ satisfies that $\mathfrak{h}(x_i) = g(x_i)$, then $\mathfrak{h}=h$.

The more common way to think about the free group on $X$ within category theory is in terms of adjoint functors. Specifically, we have the concretizing functor $\mathsf{U}\colon\mathsf{Group}\to\mathsf{Set}$, the underlying set functor, which is faithful. We define a functor $\mathsf{F}\colon \mathsf{Set}\to\mathsf{Group}$ with the following property: given a set $X$ and a group $G$, there is a natural bijection
$$\mathsf{Set}(X,\mathsf{U}(G)) \cong \mathsf{Group}(\mathsf{F}(X),G).$$
The functor $\mathsf{F}$ is called the left adjoint of $\mathsf{U}$, and $\mathsf{U}$ is the right adjoint of $\mathsf{F}$; $\mathsf{F}$ is the "free group functor", that associates to each set $X$ the free group on $X$. The standard way to express the universal property is the following, which elides the functor $\mathsf{U}$:

The free group on $X$, $(\mathsf{F}(X),\iota)$, is a group $F(X)$ and a set function $\iota\colon X\to \mathsf{F}(X)$ such that for every group $G$ and every set function $g\colon X\to G$, there exists a unique group homomorphism $h\colon F(X)\to G$ such that $g=h\circ\iota$.

Throughout we are abusing notation by considering a function between groups to be the same as the corresponding function between underlying sets.
So we usually think of "free object on $X$" as the image of $X$ under a functor that is the left adjoint of a concretizing functor, rather than as an object whose image is a universal arrow in the category $\mathsf{Set}$. This gives the "free commutative ring with unity on $X$" (the polynomial ring $\mathbb{Z}[X]$); the "free vector space over $F$ on $X$" (the vector space $F^{(X)}$), the "free topological space on $X$" (the set $X$ together with the discrete topology on $X$), etc.

Another way to think about the free group on $X$, entirely within the confines of group theory (without reference to functors), is to think of "the free group on $X$" as "the most general group that contains $X$ among its elements". In that sense, the set of reduced group words can be thought of as "all group elements we could construct starting from the elements of $X$, subject only to the rules that are inherent in any group: associativity, existence of inverses, and that inverses cancel each other out." You can think of it as taking products of elements, but "leaving everything that is not obvious indicated". We know that $x$ and $x^{-1}$ multiply to the identity, but in general we do not know what $x_1$ and $x_2$ will yield when multiplied... so we just "leave it indicated" by writing $x_1x_2$. When we identify the elements of $X$ with specific elements of a group $G$ via the map $f\colon X\to G$, we can now "evaluate" these products, because now we know which elements of the known group $G$ they are, and the morphism $h$ is just the "go ahead and evaluate all the expressions left indicated" morphism.

Note that like any universal object, there is in general no such thing as "the" free group on $X$. Rather, the free group on $X$ is only determined up to (unique) isomorphism, and there are many constructions that may reify it. "The set of reduced group words on $X$" is one such construction. If you want to see two other constructions, I recommend George Bergman's An Invitation to General Algebra and Universal Constructions, which is published by Springer. There is an older version available as a PDF from his website. In particular, take a look at the extensive discussion of three ways to construct "the free group" in Chapter 3.
Comment: I did enjoy reading in that work by Bergman.
Comment: Thank you very much for this interesting explanation.
Comment: Alright, I have a question. Do you mean to say that when you select a particular group from within a category of groups, not every group can be extended to a universal morphism?Since we pick a specific group and not arbitrary. [Arturo Magidin] (https://math.stackexchange.com/users/742/arturo-magidin)
Comment: On other word, is the universal morphism from a set $X$ to a functor $F$ is unique?
Comment: So, if it's not unique, then the definition of the free group we provided might not align with the general definition of a free group within the category of groups? The free group we've defined could be just one of several possible instances,and we have just prove that the one we defined is one of these possibilities. Please correct to me if Iam missing something.@Arturo Magidin
Comment: @Santa-claus It's not unique because you can always construct another copy of it on a disjoint set. But it is unique up to unique isomorphism, which is all that matters. Category theory cares about what it *does*, not what it is. The "reduced group words" construction is one way to construct an object that does what we want. Inpointed you to a source that discusses two other constructions. * Of course** "it aligns". Because it satisfies the definition. We verify it does. We prove it. Did you think we just say "voila!"?
Comment: @Santa-claus the question"when you select a particular group from within the category of groups, not every group can be extended to a universal morphism?" makes no sense to me. First because there is no apparent connection between the first and second clauses of the question. Second, because groups don't get "extended", which in this context means "enlarged". If you mean, "not every group $A$ is part of a pair $(A,f)$ that is a universal morphism", then, no, not every group is a free group. The universal morphism is unique up to unique isomorphism (in the appropriate category).
Comment: @Arturo Magidin Indeed, that aligns perfectly with my sentiments.
Answer: Many definitions in mathematics (like the one you gave of "free group") are constructions. They tell you how to build a certain object.
Other definitions (like the one you gave of "free object") are properties -- for any given mathematical object, you can check whether or not it satisfies the desired property.
The central idea of category theory is to identify properties which uniquely determine important constructions up to canonical isomorphism. The definition of "free object" applies in the setting of any concrete category, but the constructions of free objects in these categories will be different.
So, you need to show two things:
(i) The construction of free groups that you provided satisfies the desired universal property;
(ii) Any two free objects on the same set are canonically isomorphic.
Part (i) is an elementary abstract algebra exercise. Part (ii) is easy if you have already learned the Yoneda lemma -- have you?
Comment: You don't need the Yoneda lemma for (ii). I don't think it is even helpful.
Comment: I disagree; the fact that any two objects satisfying the same universal property are isomorphic is essentially the same as the statement of the Yoneda lemma.
Comment: But the fact that any two objects satisfying the same universal property are isomorphic is (at least in this case) immediate from two applications of the universal property without any thoughts about natural transformations or conditions on the size of hom-sets.
Answer: Maybe I dont completely understand your question but I dont think "universal morphism" is a thing.
This is the definition of a free object that Im familiar with: let $C$ be a concrete category, namely exists a faithful $U:C\to \textbf{Set}$ (Underlying set).
Given a set $X$, a free object with the basis $X$ is a pair $(A,i)$ where $A\in ob(C)$ and $i:X\to U(A)$ is an injective function, such that: for any $B\in ob(C)$ and a function $g:X\to U(B)$, exists a unique morphism $f:A\to B$ such that $g=U(f)\circ i$.
It follows from this definition that free objects whose bases are equinumerous are isomorphic. Specifically, a free object on a given set is - if exists - unique up to isomorphism, so simplistically we can call it the free object on a set $X$. In the category of groups we can explicitly build a free group on any set, so this is the free group on that set.
Comment: "Universal morphism" is a thing; for example, the canonical embedding $u_X: X \to UFX$ of a set $X$ (into the underlying set of the free group it generates) is universal among all functions $X \to UG$ into groups. This language is used for example in Mac Lane's Categories for the Working Mathematician (he uses e.g. the terminology "universal element" $u_X \in \hom(X, UFX)$ for the functor $\hom(X, U-)$ that is represented by $FX$).
Comment: i havent ever seen this so i mean... i guess you could understand free objects without it (if the OP uses this book then maybe i wasnt much of a help tho)
Comment: Not really meaning to pile on, but I'll add that for the purposes of discussing universal properties in a suitably general manner, the faithfulness of $U$ is unimportant, and injectivity of $i$ is similarly a red herring. The discussion centers on whether every functor of type $\hom(X, U-)$ is representable.
Answer: Perhaps what the OP meant to say is that one way to construct a free group on a set $X = \{x_1, \ldots, x_n\}$ is to

Start by taking a free monoid $M$ on a set $\{x_1, \ldots, x_n, x_1^{-1}, \ldots, x_n^{-1}\}$ (where the $x_i^{-1}$ are new symbols that we adjoin to $X$), whose elements are formal "words" in letters drawn from the new set, multiplied by concatenation, and then

Pass to the monoid whose elements are equivalence classes of words, taking the equivalence relation to be the the least equivalence relation containing pairs $(x_i x_i^{-1}, e)$ and $(x_i^{-1} x_i, e)$ where $e$ is the empty word, and such that whenever $(a, b)$ and $(c, d)$ belong to the equivalence relation, then so does $(ac, bd)$. Equivalence classes $[a]$ are multiplied by the rule $[a] [b] = [ab]$.

The resulting monoid is a group (define $[w]^{-1}$ to be the equivalence class of the word obtained by writing the word $w$ in reverse and replacing each of its letters by its formal inverse). It takes a few lines to verify that all this works, but it's straightforward. Let's denote this group by $F(X)$. Letting $UG$ denote the underlying set of a group $G$, there is a function $X \to UF(X)$ that sends $x_i$ to $[x_i]$.
Now let $G$ be a group, and suppose we are given a function $f: X \to UG$ to the underlying set of $G$. This extends to a function
$$\{x_1, \ldots, x_n, x_1^{-1}, \ldots, x_n^{-1}\} \to G$$
by sending $x_i^{-1}$ to $f(x_i)^{-1} \in G$. This function further extends uniquely to a monoid homomorphism $\hat{f}: M \to G$, in a pretty obvious way. Finally, by passing to equivalence classes as described above, this uniquely induces a homomorphism $F(X) \to G$ between groups, defined by the rule $[w] \mapsto \hat{f}(w)$ (check this is well-defined!). Since everything here was forced, the uniqueness clause for the universal property is assured.
I've left a certain number of details to you to check, but this should give the idea.


Need help figuring out a regular expression for these strings

Question: I'm just learning regex and I've been trying to figure out a pattern which only matches the following strings:
-a
-A
--add
--Add
a
A
add
Add

I'm using this in Perl if it matters. If you answer, could you explain your regex so I can try and learn what I've been doing wrong?
Comment: Check out RegExr, it's a great tool for testing regexes: http://gskinner.com/RegExr/
Comment: Another resource to check out is the perl module Regexp::Assemble
Answer: ^(-?[aA]|(--)?[aA]dd)$

^ matches the beginning of the string
-?[aA]:

-? matches - or nothing
[aA] matches a or A

(--)?[aA]dd:

(--)? matches -- or nothing
[aA] matches a or A
dd matches dd

(x|y) matches x or y
$ matches the end of the string

You should invest some time carefully reading http://perldoc.perl.org/perlrequick.html. I still need a reference handy when I write regular expressions.
Comment: Works perfectly, and thanks to your explanation I fully understand regex and I'm making some more for my application. Many thanks!


Como pegar o value do segundo botão em JavaScript

Question: Estou ajustando um sistema de venda de fotos, para adicionar as fotos no carrinho por JS.
O script funciona 100%, somente com PHP, mas quero fazer sem sair da página, usando JS.
Tenho a seguinte situação:
Cada foto, tem 3 opções de tamanho: P, M, G.
Com JS, ele sempre adiciona a primeira opção, não pega o value="< ?php echo $preco_digital['id']; ? >" do tamanho/form/botão seguinte.
Como faço para o JS ir para o registro seguinte, quando clico no botão específico?
Formulário (1 para cada tamanho):
<form style="padding: 0" method="POST" id="add-carrinho" action="carrinho-add.php">
    <input type="hidden" name="id_foto" id="id_foto" value="<?php echo $r['id']; ?>" />
    <input type="hidden" name="id_evento" id="id_evento" value="<?php echo $r['id_evento']; ?>" />
    <input type="hidden" name="id_preco" id="id_preco" value="<?php echo $preco_digital['id']; ?>" />
    <input type="hidden" name="id_promo" id="id_promo" value="<?php echo $promocao['id']; ?>" />
    <button id="add-carrinho" class="compra-foto" type="submit" role="button">
        <?php echo $preco_credito; ?> Créditos

    </button>
</form>

Javascript:
$(function(){
    // Executa assim que o botão de salvar for clicado
    $('.compra-foto').click(function(e){

        // Cancela o envio do formulário
        e.preventDefault();

        // Pega os valores dos inputs e coloca nas variáveis
        var id_foto = $('#id_foto').val();
        var id_evento = $('#id_evento').val();
        var id_preco = $('#id_preco').val();
        var id_promo = $('#id_promo').val();

        // Método post do Jquery
        $.post('carrinho-add.php', {
            id_foto:id_foto,
            id_evento:id_evento,
            id_preco:id_preco,
            id_promo:id_promo
        }, function(resposta){
            // Valida a resposta
            if(resposta == 1){
                // Limpa os inputs
                $('input, textarea').val('');
            }
        });

    });
});

Answer: Vinicius, quando você clica em qualquer um dos botões de enviar, o jquery pega os dados do primeiro formulário, você não está dizendo pra ele os inputs de qual formulário você quer que ele pegue. você pode resolver isso fazendo isso.
Faça as seguintes alterações:

Mude o type de "Submit" para "Button" em seus botões de todos
  os formulários

ficando assim
<button id="add-carrinho" class="compra-foto" type="button" role="button">

E seu javascript atualize para ficar assim.
$(function(){
    // Executa assim que o botão de salvar for clicado
    $('.compra-foto').click(function(){

        // Pega os valores dos inputs e coloca nas variáveis
    var id_foto = $(this).closest("form").find("input[name='id_foto']").val();
    var id_evento = $(this).closest("form").find("input[name='id_evento']").val();
    var id_preco = $(this).closest("form").find("input[name='id_preco']").val();
    var id_promo = $(this).closest("form").find("input[name='id_promo']").val();

        // Método post do Jquery
        $.post('carrinho-add.php', {
            id_foto:id_foto,
            id_evento:id_evento,
            id_preco:id_preco,
            id_promo:id_promo
        }, function(resposta){
            // Valida a resposta
            if(resposta == 1){
                // Limpa os inputs
                $('input, textarea').val('');
            }
        });

    });
});

Espero ter ajudado.
Se funcionar por favor, marque ao lado a resposta como correta, ficaria grato :)


How to set Exim envelope domain to From domain

Question: I've set up DKIM on Exim with the domain set like:
DKIM_DOMAIN = ${sender_address_domain}

However, the domain is always set to the same domain (my primary domain), which causes DMARC validation to fail, because of alignment, when sending emails for other domains (I host several websites).
From reading the documentation, I think the sender_address_domain is the envelope address and not the From field. How can I change the envelope address so that it matches the From field of a given email (I assume this will also allow SPF alignment to be correct)?
Also, for security, is it possible to have a whitelist of allowable domains, so Exim refuses to send emails that have another domain in the From field?
Comment: It looks like you are using the Debian exim config, as you use `DKIM_DOMAIN` in capitals? As far as I know this is initialized at startup so using a per-message variable to set it will not work. As I read it you need to specify the list of possible domains here, the correct one should be selected.
Comment: @wurtel Correct, I'm using Debian's. But, I'm not sure that makes any difference, they just do `dkim_domain = DKIM_DOMAIN`. So, how might I configure the variable per-message as you suggest?
Comment: Same thing, `dkim_domain` is set at startup as far as I know. Note that it's a list, perhaps include all domains this exim might send email for.
Comment: @wurtel Tried setting `DKIM_DOMAIN = foo.com : bar.com`, but this just results in DKIM not getting set at all. Even if that worked, I'm still unsure how I would change the sender domain for the SPF as well.
Answer: Add the rewrite rule to the Exim4 configuration:
*@+local_domains "${local_part}@${domain}" F

The rule rewrites the Envelope-from header to match the From header, allowing DMARC alignment to work correctly.
Recommendation is to append it to the end of the section to avoid conflicts with current rules.
You can find this configuration in the file /etc/exim4/conf.d/rewrite/10_from_rewrite or in the section called rewrite/31_exim4-config_rewriting of the file /etc/exim4/exim4.conf.template (for Debian). It depends on the type of your configuration – called single monolithic or split Exim4 config file with possible need to run the command update-exim4.conf.
Restart Exim after reconfiguration using systemctl restart Exim4.

The rule explanation:

* of the *@+local_domains = for all "local_parts" e.g. users.
+local_domains = for all domains served by Exim4 server (defined in dc_other_hostnames and dc_readhost variables) and not by other domains (redirection attempts etc.).
${local_part}@${domain} composes RFC2822 compliant e-mail address from From field. Variables are described in exim4 documentation – string expansions. You can use just "$header_from:", but addresses in a form as "John Doe <email@example.com>" will fail and get recorded to paniclog because of the "John Doe" part.
F = target field of rewrite operation is the Envelope-from. see exim4 documentation – address rewriting.
Comment: This has stopped working. Not sure if something has changed, but `$local_part` and `$domain` seem to refer to the address that is currently being rewritten. So, this rewrite rule just outputs the same domain. The `+local_domains` has also never worked at restricting the domains that are rewritten. I've reverted to my previous answer which still seems to work.
Comment: I added `${address:}` to my answer which should resolve the previous concern with using the header. I think the reason that `+local_domains` doesn't work is because it is matching against the envelope address before rewriting, and will therefore always be the same. We want to match against the From header or the result of the transformation instead.
Answer: Add the rewrite rule:
*       "${address:$header_from:}" F

Where ${address:$header_from:} extracts the address from the From header and F means to rewrite the Envelope-from field.
In debian this can be added by creating a file such as /etc/exim4/conf.d/rewrite/10_from_rewrite. This rule rewrites the sender field to match the From header, allowing DMARC alignment to work correctly.

For the whitelist, we can add a system filter. First create the filter file (e.g. /etc/exim4/exim.filter):
# Exim filter

if not foranyaddress "domain1.com, domain2.com" ($thisaddress is "${domain:$header_from:}")
then fail "${domain:$header_from:} not permitted to send email" endif

Then in the main part of the config somewhere add system_filter = /etc/exim4/exim.filter.
I've not figured out any way to use the domainlist/local_domains within this filter.


Strange guitar warble sound

Question: I have a Jackson Dinky DK2M with two Seymour Duncan Humbucker pickups and a floyd rose tremolo on it.
It is producing a strange `warble' noise on G string, especially on higher frets like 12th, 15th, 17th. It only appears when I use a high gain (distortion) and more noticeable when using neck pickup. I cannot hear it when using a clean channel.
It is like you are tuning your strings according to another string, e.g., high E open vs. B 5th, and they are slightly out of tune. The tone sounds like vibrating, not smooth, and seems killing the sustain.
I know it is hard to describe and thus I recorded a bit, you can checked it out here. It is recorded through a Peavy Vypyr 30W using a `6505' model, the pre-gain is set to highest. I used a 1mm pick. I played 15th, 17th and 12th on G string. The quality may not be so good since I used a iPhone to record it. If anyone will like to hear a better one I can surely do it.
I also searched and found some suggestions to this, but none of them solve my problem, here's a list and my result:

I have tuned the action and bridge high enough such that there is no bret buzz.
I changed the string several times but it does not work. From D'addario to Ernie Ball to Fender, from .09 to .11.
It is not an amp problem, I took it to a local store and tried several amps.
I took it to a local store, and this confused all the techs there.
Lower the pickup. I make both pickups the lowest but no use.
Someone said it is the magnet not synchronized, see here. I followed what he suggested, but not help at all.
Some said it is called 'stratitis'. I am not so sure what it means, but looks like it is related to the pole piece on a Fender Stratocaster, and they suggest to lower the pickup and change the strings, which I already did.
Some articles said it is related to a plain or a wounded G string. I am not sure if I have tried this, since I did used a set of D'addario wounded string, but cannot recall if I used it on this guitar.
I found some one has the same problem as me here.

This is really freaking me out. I really hope some one can help me to solve it.
Thanks in advance.
Comment: Probably you have tried these but let me ask anyway: Did you try playing pretty far from your amp? Can you disconnect the bridge pickup and try? And finally does it also happen when you bend and sustain?
Answer: I came across your post today and I think you have identified the same problem I have with my Peavey Wolfgang.  I hear the warble on my guitar at frets 16-19 both plugged and unplugged - suggesting that it's nothing to do with amplification or electrics. And there is no string buzz.
My research on Google suggests that it is magnetic pull on the strings from the pickup:
http://acapella.harmony-central.com/showthread.php?2116481-Strat-G-string-WARBLE-!

From Fender.com
PICKUPS
Set too high, pickups can cause myriad inexplicable phenomena. Depress
  all the strings at the last fret. Using a 6" (150 mm) ruler, measure
  the distance from the bottom of the first and sixth strings to the top
  of the pole piece. A good rule of thumb is that the distance should be
  greatest at the sixth-string neck pickup position, and closest at the
  first-string bridge pickup position. Follow the measurement guidelines
  in the chart below as starting points. The distance will vary
  according to the amount of magnetic pull from the pickup.
Bass Side Treble Side

Texas Specials 8/64" (3.6 mm) 6/64" (2.4 mm)
Vintage style 6/64" (2.4 mm) 5/64" (2 mm)
Noiseless™ Series 8/64" (3.6 mm) 6/64" (2.4mm)
Standard Single-Coil 5/64" (2 mm) 4/64" (1.6 mm)
Humbuckers 4/64" (1.6 mm) 4/64" (1.6 mm)
Lace Sensors As close as desired (allowing for string vibration)

I will do some tests on my guitar regarding this and hope to report back.
Comment: Thanks. As I said in my original questions, I have tried to lower my pickup, even a lot lower than the recommended settings, but the strange warble sound still exists. I am not sure whether there is warble when unplugged, it may have but I simply cannot hear it. The overdrive may be emphasizing this effect.
Comment: Magnet pull is common with single coils like a strat, not with humbuckers like the Wolfgang has.
Answer: There is no one single cause for this type of problem. But if you have eliminated mechanical factors such as floating bridge instability, loose truss rod, or microphonics from metal parts coupling into the pickups magnetically, you should take a look at your electronics. Actually that should be the first thing, perhaps.
I have cured this problem in the past simply by replacing old, worn volume pots, switches and jacks.
The problem is that when your guitar vibrates, these vibrations transfer to the contacts between the potentiometer's wiper and the resistive element, and to other electrical contacts.  Not only is that form of microphonics, but actually a form of distortion: as the note vibrates the guitar, the vibration causes a disturbance in that note's own waveform going through the electronics.
As I write this I'm plucking a guitar which had an atrocious case of this some years ago; the only thing different is electronics. It was awful. In certain positions on the neck on the G string, oh boy. There was a nasty flutter intermodulating with the note. It drove me crazy!
One thing I did differently was installing the new potentiometer in a loose way. The guitar has a wood top (not a plastic pick guard). I kept the little metal "nub" on the potentiometer, and drilled a matching hole in the wood that then nub fits into when the pot is installed. This nub prevents the pot's body from rotating when the knob is turned, even though the pot is not fully tightened. The knob has enough free play that you can wiggle it in place. The arrangement prevents vibrations from transferring from the body to the pot.
Answer: Just wanted to add I've been trying to eliminate what appears to be the same issue on a custom Warmoth Strat using YJM Fury pickups. I've cranked the action up (and the pickup height down) to absolutely absurd settings and it doesn't help. I don't hear it unplugged, nor do I hear it if the amp is "clean", but hear it with any kind of overdriven sound (both through an amp as well as a VST amp emulation - so that rules out amp issues in my mind). It's super frustrating. I'm going to check for mechanical issues next (as the previous poster described).
Comment: I will be very glad if you can have any clue of this issue. Please keep me updated.
Answer: I am having the same problem with my Parker P-38. I could not find anything about this problem since I got the guitar, and just now found all of you poor souls with the same issue. I'm glad to not be alone in this mess, and I'm also sorry for you all. While I have ignored it for about a year, I need to sell it now and I can't get any interest with this problem. The G and B string both have this harmonic ghost note sort of sound, mainly above the 12th fret. The G is by far worse and is intolerable (and embarrassing) to play by itself. I would love to record it (I believe it's much worse than the sound clip already added) and it might make it easier to diagnose. If this would help, let me know and tell me how to do it.
I had the Parker professionally set up (new strings, intonation, tuning, action, and pickup height) thinking that it would fix this "warbling." It did not. I took it back a few times before he told me that it was a problem with the bridge (Wilkinson floating) and he could not fix it. I really don't see how that could be the problem. If this could be the problem, please verify so I can quit calling the tech a liar.
I've noticed that the G and B magnet poles on both piezo pups are notably taller than all of the others. When I lowered the pups as far down as they can go, the warbling was not as severe, but still highly noticeable. I really don't want to put more money into this guitar because I won't make my money back.
This is what I'm thinking... very controversial-- I'm thinking, that the pup poles on the G and B string should be ground down carefully to about half the height. I'm also thinking that if there is a way to partially demagnetize them, it would probably help.. Thoughts and suggestions anybody?
Answer: The problem may lie with the guitar itself. If you haven't already, have the guitar checked for any type of warpage. It may not look warped but that means your warp just isn't visible. Also have it check for fret bore, which is uneven or 'bored' frets and check for any cracks, pits, blemishes, or damaged spot on your fretboard. I sympathize with these problems your having and I hope these suggestions help. I've had similar problems with a rattling sound on my Raven guitar which has EMG H4A passive humbucker. It was being caused by the position of my Floyd Rose.
Answer: To me that doesn't sound like anything wrong with the guitar at all - admittedly it is tricky from just that single clip, but it sounds to me much more like an artifact of the high gain and distortion, especially as this doesn't happen when you use a clean tone from your amp. Does your sustain get killed when you are on clean tone?
For some amps I wouldn't even think this sound would imply a fault - has it just started doing this?
Comment: This sound clip in fact does not show the problem very clearly,  I want to redo it again.
Comment: I did not notice this after I changed my original set of strings. Thus, I am not sure whether it was there from the very beginning. But definitely a long time ago (I bought it last year)
Answer: This intrigued me so I did some Googling and found some things which might help:

Maybe using lighter strings might help.
As for the "Strattis" link you posted, the user 'Martian' mentions a method of pressing the strings until they are barely touching all of the magnets on the humbuckers and then slowly releasing the strings until they are back to normal.

Lay the guitar flat on its back and with both hands, gently press down on all 6 strings until the respective strings barely touch all 3 of their respective magnets simultaneously. Wait a second and then, slowly start relieving the hand pressure until the strings have been restored to their original position.
It is possible that there is a dead spot on the guitar.  This might be able to be fixed with a fat finger, or by replacing the neck.  Hopefully you might know someone with a fat finger sitting around and can borrow it to see if it works before buying one.
Aside from the above, the other person who had the same issue as this with his Dinky traded it back in to Sam Ash and got a replacement.  If your guitar is new-ish and still under warranty, this could be a possibility also.

If it were me though, I would try lighter strings (you mentioned you went to heavier strings ), the fix for possible Strattis, and the Fat Finger before any thoughts of taking the guitar apart or replacing it.

If possible, can you record some more sounds for us to listen to.  You can directly say in the recording what note you are playing and play it afterward.  You can also play other strings, different tone/gain/clean settings (like you mentioned in your post), and chord strumming so that we can hear the differences.
All the best!
Comment: Thanks! I have tried point 2, but no use. I bought the guitar from GuitarCenter online one year ago, even with extended warranty. I don't think they will honor replacement request. But sending to them costs too much time, I want to try to fix it first.
Comment: BTW let me record it again and see if you have any idea what is going on.
Answer: Avoid buying good guitars online - they also have defects or not being stored in the recommended environment parameters for extended period of time.  I once got a Jackson DK2 online and it took me weeks to fixed and adjust.
Problems with the guitar is always either mechanical or electrical, and usually not difficult to deduct the reason to troubleshoot.
First, press the string G very hard down to the fret, play it to produce the "warble" tone. Simultaneously, feel with your another hand where the string vibrates when it shouldn't be (e.g. string behind nuts, string behind bridge. Also do the same for other good strings (E, A, D) and feel the comparative difference. If you spot a difference, then you will need to do some filing (with a file or sand paper) or filling (with wood chips or plastic).
Then, check the amplitude of the vibration (with your eyes) for any abnormality - you can discern it by comparing to other strings that does not produce the warble sound. This check against any surface contact.
If there is a mechanical issue, at least by now you should be able to locate it - and hence solve it.
If you find absolute nothing mechanically (which I doubt - usually it's a mechanical issue) then it's an electrical issue.
First check the tone for pickup setups (by switching the pickup switch) see if all positions produce the same warble sound. If you find one humbucker setup produce bigger warble sound then the other then either it's a humbucker defect (hopefully not) or the wire connection. Assuming the latter (which you can fix), remove the humbucker and re-solder the connection - if you have a multi-meter check across all wire connection for consistent resistance.
If not, i'm sorry, get a new humbucker.
Answer: The answer is: usually cheap or poor quality woods.
I once built a guitar from an old chipboard benchtop and it sounded gross -- all grindy and grungy sounding, especially the higher notes. Also, loose or broken truss rods can cause similar issues.
Answer: All the other answers are wrong.
It has to do with the unwound G string. It's the physics of the string. An unwound G string will always have this problem.
Buy a WOUND g string, and the problem will go away entirely. Try D'Addario EJ120 (or any other set that has a wound g).
Most guitarists are not aware of this.
Comment: I noticed that this also happens to my Strat... however it is worse at my Dinky. I did read articles recommend replacing to a wound G string, but that will make it really hard to bend. I have not tried this, maybe I should
Answer: To interject some engineering. The warble is an ossiliation. Take a simple example. Suspend a spring from the ceiling and attach a weight. Pull down on the weight and let go. The weight rises an falls in a periodic motion at a given frequency just like the guitars string. Now supposed you attach a second spring to the original weight the add a second weight. Pull down on the second weight and let go. Now you have a very complex system. The weights bob at periodic intervals but not in a consistent way. One weight bobs for a while then stops. The second bobs and stops as energy transfers from spring to spring. Since no object is truly rigid your guitar is a multitude of springs. To minimize the crazy ossociliations. Make everything as massivse and stiff as possible. Enter the fixed tail stop and heavy a$$ les paul. Stiff as a brick unwanted ossociliations are minimized. Trem systems are the worst. Try a peice of light foam under the trem springs. This is the best solution I know. Get a massive trem block. By the way similar ossociliations occur in electrical systems especially when high gain is used. Push pull amps driven hard work gear with 2 strineteg played. Try six an you get mud.
Answer: I currently have a similar issue with my fender jaguar. It began right after I tightened the truss rod 1/4 a turn to straighten a slight archer's bow. The issue occurs only on my low E string, most prominently on and around the 12th fret. I tried lowering the pickups, changing the string, adjusting the bridge saddles for height. Raising the bridge saddle helped a little bit, but didn't kill the problem. None of the aforementioned prevailed. So I untightened the truss rod 1/4 of 1/4 of a turn and it did not help. I am reluctant to untighten it back to its original position because I fear the archer's bow may return. I swapped the saddles, but no good. This didn't happen before I adjusted the truss rod, so I will try loosening it again by another 1/4 of 1/4 of a turn. This sucks because it is one of those truss rods you have to take the neck off to get to, so it is extremely laborious. I will let you know if I prevail. I wish you the best!
Answer: Try replacing the G with a wound string.
I know this is an old thread, but it has helped me, so here's my experience (hope could be useful to anyone):
3 years ago I bought my first strat-like, a cheap second hand guitar that was a school for me: disassembled completely, adjusted truss rod, saddles, changed pickups, installed a phase switch... so well, I'm not an expert, but I've learned a bit.
2 weeks ago I bought a used Yamaha Pac612v, near mint condition! It was wonderful, but I noticed something odd on 3rd string.... maybe old strings, and guitar not adjusted as I also saw strings pulling too hard the bridge... that was what I thought.... but I was completely wrong.
I adjusted the truss rod and changed bridge springs for heavier ones. No luck, problem still there. Maybe saddle/machine head... some verifications, even changed 3rd machine head with 4th: no luck. Tried 'magnetic flux' indications, same as before. Checked how neck was screwed to the body, pickups and buttons cavities, pups height, all the screws I found: first string broken. Well, I was about to change them anyway. Change string, now D'Addario 9-42 and original bridge springs: well, bridge on perfect position, but problem persist.....
In the meantime, push-push splitting switch broken trying a cosmetic change. Damn!! Just one thing remaining that I can do before taking the guitar to a luthier and thinking inside there's an structural problem with this guitar: replace G for a wound G.
YES! That's it!!! I can not say warble disappeared 100%, but near! I can not believe it was just that! But makes sense, right? Single coils are vintage style, staggered, central poles are taller than the others and alnico v. When this kind of SC were created, most of G string were wound. So now we have in a very short piece of string, single wire string, a powerful magnet influencing oscillation. So wound G for now is helping a lot. And maybe in a near future I'll take the guitar to be reviewed by someone who really knows about it, to fix what I've broken and to change the SC pups.

Strange guitar warble sound


Pointwise convergence when $f_n$ is defined as sum of other functions

Question: I have this problem:
Let  $f: [0,1] \longrightarrow \mathbb{R}$ be a continuous function. Now, define:
$$ f_n (x) = \sum_{i=0}^{n} {n \choose i}x^{i}(1-x)^{n-i}  f( \frac{i}{n})$$
Show that $f_n$ converges pointwise to f.
I worked (succesfully) the case in which $f$ is the identity map. However, I cant't handle the general case. Could someone give me pointers in how to deal with this kind of problem ? (when the functions in the sequence not only depends on $n$ and $x$, but also in some other index. Thanks in advance!
PD: This problem was left by my teacher to prepare the third exam in my class of probability theory.(the last thing we saw was the Law of large numbers). So maybe there is a natural way of attacking this problem using only probability theory but (honestly) I don't see how.
Comment: https://en.wikipedia.org/wiki/Bernstein_polynomial#Approximating_continuous_functions
Comment: I see, so it's a famous problem. Thanks a lot!
Answer: We could also take i.i.d. random variables $(X_i)_{i \in \mathbb{N}}$ with probability $P(X_i=0) = (1-x)$ and  $P(X_i=1) = x$. Let now $S_n = \sum_{k=1}^n$ X_i. By the strong law of large numbers we have $$\frac{1}{n} S_n \rightarrow x$$
$\mathbb{P}$-almost sure. This also implies convergence in probablity. Since $f$ is bounded on $[0,1]$, we see that
$$\sum_{k=0}^n \binom{n}{k} x^k (1-x)^{n-k} f(k/n) = \mathbb{E}[f(S_n/n)] \rightarrow \mathbb{E}[f(x)] = f(x).$$
This gives pointwise convergence. In the proof of the Wikipedia article - see here - is shown that this convergence is uniformly in x. (This proves the famous Weierstraß theorem on the approximation of functions $C[0,1]$ by polynomials.)


XSLT - Get nodes by checking content

Question: I've xml like follows,
<doc>

  first sentence

second sentence

     third sentence

</doc>

I need to get
 nodes which have space at the first. in this case I need to get first and third
 nodes.
can you suggest a method how can I do this in xslt?
Comment: have you tried starts-with() function of xslt??
Answer: try out the following
  <xsl:template match="@* | node()">
    <xsl:copy>
      <xsl:apply-templates select="@* | node()"/>
    </xsl:copy>
  </xsl:template>

  <xsl:template match="p">
    <xsl:if test="starts-with(.,' ')">
      <xsl:copy-of select="."/>
    </xsl:if>
  </xsl:template>

Do note I haven't mentioned stylesheet element in my answer.

XSLT - Get nodes by checking content


What time signatures are used in flamenco and related styles?

Question: Recently I've been trying to get some flamenco spirit into my playing. I've already experimented with some 'gypsy' scales and it feels good, but I've also realized that part of the secret is in using less common time-signatures.
What time-signatures could be considered as typical for these styles?
Comment: do you have a song or piece as an example?
Answer: You need to understand that flamenco rhythm is very different from classical or popular music (it has more in common with Indian Classical music and the concept of "Tala").
All flamencos talk about rhythm using the notion of "compas" - these are rhythmic "styles", that include a time structure, as well as a feel and a usual tempo. For example, Bulerias compas is a cycle of 12 beats, but sometimes just 6, played fast (and faster!), and has a feeling of "fiesta", both dark and light, about it. So when one of your answers refers to a Paco de Lucia song as "Bulerias", he is mistaking the name of the compas for a song title. And Paco is certainly not playing in 3/4 - the "uncommon" time signature you are hearing is coming from taking 12 fast beats and grouping them by threes and twos (Google "hemiola"), and with Bulerias in particular, often throwing in an accent on beats 7 and 8 (with 12 being the first strong beat of the cycle).
"12 being the first strong beat"?!? Another amazingly confusing thing about flamenco rhythm is that traditionally flamencos count the 12 beat cycles (Bulerias, Solea, Alegrias, etc.) starting with 12, instead of 1. This seems just perverse, until someone reminds you that they refer to the "clock" - and the clock starts ticking at 12...
OK, ok, there are simpler compas - "Tango" is four 4/4 measures (but the cycle usually ends on beat 3... And it doesn't really have anything to do with Argentine tango...). Obviously, people can go on and on about this topic - it's a great stew of old dance forms (think Sarabande, from Bach's time?), ethnic music (Zambra, from the Jews of Cordoba), modern "whacked" rhythms (they rap to flamenco in Jerez) and ancient mystery (compas really are like Indian Talas).
But since you are just looking for flamenco "spirit", and aren't necessarily committed to Total Immersion Flamenco Obsession, I would first just focus on trying to hear those squirrely groups of twos and threes, and how they shift throughout the 12-beat cycle. But if you get curious, there's lots of material on the web (a good start is the Wikipedia article on flamenco - and then just start typing the names of various compas into YouTube). Have fun!
Answer: Based on a quick survey of my flamenco music books, here is a quick survey:
Gypsy Kings: 18 songs in 2/4, 7 songs in 4/4.
Django Rheinhart (gypsy, not flamenco): 13 songs, all 4/4.
Paco de Lucia: 8 in 3/4, 1 in 4/4, 4 in 3/8, 1 in 6/8.
Sample YouTubes by signature:

2/4. Bamboleo, by Gypsy Kings.
3/4. Bulerias by Paco de Lucia.
4/4. Garrotin.
3/8. Cepa Andaluza by Paco de Lucia, a buleria.
6/8. Percusion Flamenca by Paco de Lucia.

Generally, bulerias and sevillanas are 3/4 time; rumbas are 4/4.
Disclaimer: my ex was a flamenco dancer and I never throw books out.
Comment: Anyway Im surprised as I expected that majority of flamenco songs to be something else then 3/4 and 4/4. It has to be something in the rhythm and style how its played that gave me that impression.
Comment: In Seguiriyas the measures are 3/4, 6/8, 3/4, 6/8 etc...
Answer: The most typical 12/8, with different 3 + 2 and so on patterns, maybe the most remebered/accessible/stereotyped being 3+3+2+2+2 (Bulerías)
Wikimedia One of the Typical Flamenco Signature
At least the one I remember most exportable and fusionable...

(Even

, which are not spanish)
But they can use as 6/8, 3/4 and even 4
I'm not expert but I am spanish :)
Comment: The last link is dead.
Comment: It's Yes - Heart of the Sunrise
Answer: I'm not affiliated to it but in my own opinion the following web page:
flamencopolis.com/archives/472
is the most extensive online document I've found as of yet on the issue of compas.
It breaks down all of the main time signatures of flamenco palos. Unfortunately it is only available in Spanish, as is the case of many of the best ressources on flamenco. I submit it just in case...
However, as was said above in another answer, if you aren't looking to fully immerse yourself in flamenco and only add some flamenco spirit to other forms of playing, I wouldn't focus on the time structure (which could take years to discover all of the ins-and-outs of) and would focus on the scales and chord voicings that are specific to flamenco, as well as some of the right hand techniques.
Comment: You may want to include that you are not personally affiliated with that site as to spare this post a flag for spam.
Comment: The original link is dead, but here's the most recent archived version: https://web.archive.org/web/20161010134521/http://www.flamencopolis.com:80/archives/472


Choice of a loss function

Question: Im running an xgboost model to try and find important predictors for a disease from a list of almost 1000 covariates. The prevalence of the disease in my cohort is about 10%.
Given the imbalance data, would the  precision-recall AUC or the logloss be a more appropriate matrix to assess the model fit? Is it appropriate to use logloss when classes are not balanced?
Also, playing with hyperparameters tuning, it seems like adding scale_pos_weight is benificial, but should i avoid doing this if i use logloss?
Thank you!
Answer: Yes, the log loss is appropriate. It's a proper scoring rule (see the tag wiki for more info). It can indeed be used with "unbalanced" data. Precision and recall are improper, so don't use them.
More information here (admittedly boilerplate): Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? Do not use accuracy to evaluate a classifier:
Why is accuracy not the best measure for assessing classification models?
Is accuracy an improper scoring rule in a binary classification setting?
Classification probability threshold
The same problems apply to sensitivity and specificity, and indeed to all evaluation metrics that rely on hard classifications. Instead, use probabilistic classifications, and evaluate these using proper scoring rules.
Comment: AUC is somewhat less bad than just plain accuracy, [take a look at this thread](https://stats.stackexchange.com/q/339919/1352). And yes, log loss can be used in any kind of situation. Predicting rare classes is hard, and it's even more important to use proper scoring rules in this setup.
Comment: Thank you Stephan for the answer! So just as a quick follow-up: my understanding that with unbalanced data you should indeed not use accuracy, partly because with severely unbalanced data the model can be very accurate even if it always predicts the negative class. But I thought that the same should not hold when using the precision-recall AUC, which specifically focuses on the positive class only, and is also insensitive to the specific probability threshold for classification. Is that not accurate?
Comment: And just to confirm, despite the class imbalance, you suggested that the logloss is an appropriate scoring rule (so, in other words, logloss is OK to use when you have unbalanced data?)
Comment: Thanks for all of this! So final question: I read that over/under sampling is actually not needed in most situations (https://stats.stackexchange.com/questions/357466/are-unbalanced-datasets-problematic-and-how-does-oversampling-purport-to-he). Yet, when tuning the hyperparameters through CV, it seems like adding weights to make the gradient of a positive sample more influential improves the prediction (https://stats.stackexchange.com/questions/243207/what-is-the-proper-usage-of-scale-pos-weight-in-xgboost-for-imbalanced-datasets). Should this not be done however?
Comment: "Improves the prediction" in what sense? I only skimmed the thread, and I don't see what KPI is used. I have a hard time imagining this improves the probabilistic predictions as measured by proper scoring rules. Conversely, I could easily imagine it "improves" accuracy, by biasing predictions towards the majority class. But that is not an argument, since [accuracy is not a good evaluation measure](https://stats.stackexchange.com/q/312780/1352). There is a lot of misinformation floating around about how to evaluate classifiers.

Choice of a loss function


Form returned by drupal_get_form() won't submit

Question: I am currently implementing an additional registration page on my drupal site, using drupal_get_form(), but I seem to be missing something cucial.
While the registration form is printed at the path I created, I can not submit the form - drupal will clear both the user and the email field and return an error claiming the form could not be submitted, because the fields were empty. They were not.
Do I need to hook up a custom validation handler? Something else?
My code so far:
function my_module_menu() {
  $items['customer/register'] = array(
    'page callback' => 'drupal_get_form',
    'page arguments' => array('my_module_customer_registration'),
    'access callback' => TRUE,
  );

 return $items;
}

function my_module_customer_registration() {
  $form = drupal_get_form('user_register_form');
  return $form;
}

Comment: I'm not sure if it is the problem or not but it looks like you have superfluous code. You should be able to call the form directly from your hook_menu. `'page arguments' => array('user_registration_form')`. Could be an issue with your form name and submit handlers calling it the way you are.
Comment: I tried your suggestion, but now the form doesn't render at all.
Comment: If you copied my comment it is because I had a typo. `'user_register_form'` not registration form. If that doesn't work what kind of errors are you getting? You might have to mess with the access callback or maybe `'type' => MENU_NORMAL_ITEM,
Answer: It's not working because the form builder function user_register_form() is expecting a couple of arguments, $form_id & $form_date, that you're not passing (see the drupal_build_form() docs to know more about what they're for).
If you want to provide a wrapper function around the user reg form, you can do something like this:
function my_module_customer_registration($form, &$form_state) {
  $form = user_register_form($form, $form_state);
  return $form;
}

Comment: It was indeed $form + $form_state I missed :) Thanks. I thought with drupal_get_form they were superfluous, but obviously I was wrong.
Answer: Why don't you  add user_register_form in to hook menu
function my_module_menu() {
  $items['customer/register'] = array(
    'page callback' => 'drupal_get_form',
    'page arguments' => array('user_register_form'),
    'access callback' => TRUE,
  );
 return $items;
}

Comment: This does work and is the solution I tried based on Sareed's comments. If I could approve two solutions I would, as this is works. The reason for using the attempted workaround with drupal_get_form was to do some additional form manipulation (showing/hiding fields) however and as Shawn pointed out, what I really missed was pasing the references to $form + $form_state


Binomial GLM in R: Is there any overdispersion test, like AER package?

Question: I have (mydata) as example:
mydata <- read.csv("https://stats.idre.ucla.edu/stat/data/binary.csv")
## view the first few rows of the data
head(mydata)

mydata$rank <- factor(mydata$rank)
mylogit <- glm(admit ~ gre + gpa + rank, data = mydata, family = "binomial")

summary(mylogit)

I try to see if I have model overdispersion:
458.52/394 #[1] 1.163756 Almost 1!!!

But I not sure ... there is the package AER that makes the overdispersion test for Poisson and Binomial Negative and for Binomial there is any option?
Thanks in advance!
Comment: Try this: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#overdispersion
Answer: By Ben Bolker and others (https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#overdispersion)
overdisp_fun <- function(model) {
    rdf <- df.residual(model)
    rp <- residuals(model,type="pearson")
    Pearson.chisq <- sum(rp^2)
    prat <- Pearson.chisq/rdf
    pval <- pchisq(Pearson.chisq, df=rdf, lower.tail=FALSE)
    c(chisq=Pearson.chisq,ratio=prat,rdf=rdf,p=pval)
}

overdisp_fun (mylogit)

#      chisq       ratio         rdf           p
#397.4901989   1.0088584 394.0000000   0.4412888

Not overdispersion model!!!


Find the real eigenvalues if any

Question: Consider the 2nd order boundary value problem
$ x^2 y''- \lambda xy'+ \lambda y=0 \ ,$
subject to
$ y(1)=0 \ \ and \ \ y(3)=y'(3) \ .$
Find the real eigenvalues if any.
Answer :
My approach is as follows:
Let $ \ \ x=e^z \ ,$  then $ \ z=\log (x) \ .$
Now $ \ \frac{dy}{dx}= \frac{1}{x} \frac{dy}{dz} \ \ and \ \ \frac{d^2y}{dx^2}=\frac{1}{x^2} \frac{d^2y}{dz^2}-\frac{1}{x^2} \frac{dy}{dz}  $.
Thus, the equation becomes
$ \frac{d^2y}{dz^2}-(1+\lambda ) \frac{dy}{dz} +\lambda y=0 \ .$
But now I can't proceed to find the eigenvalues.
Please, help me out.
Comment: What is the characteristic polynomial of your transformed equation? What do the roots look like?
Answer: The characteristic equation can be factored as
$$ r^2 - (1+ \lambda)r + \lambda = (r-1)(r-\lambda) $$
If $\lambda = 1$, there's a double root and the solution takes the form
$$ y = Ae^z + Bze^z = Ax + Bx\ln x  $$
Otherwise, there are two distinct roots
$$ y = Ae^z + Be^{\lambda z} = Ax + Bx^{\lambda} $$
It's up to you to determine $\lambda$ values that satisfy the boundary conditions

Find the real eigenvalues if any


Como posso fazer um botão de carregar mais usando o jquery?

Question: Bom dia pessoal, estou fazendo um site que tem os posts, porém não quero usar o sistema de paginação, quero fazer um botão que carregue mais post, daí queria saber como faço..

projeto

MAIS

Esse é o código que fiz até agora, será quem alguém pode me ajudar? Fazendo com que quando o botão "MAIS" seja clicado, ele carregue uma nova linha com mais 3 posts..
Desde já agradeço..
Comment: Vê este link: https://code.tutsplus.com/tutorials/how-to-create-infinite-scroll-pagination--wp-24873
Comment: https://www.sitepoint.com/jquery-infinite-scrolling-demos/ tai ^^
Comment: Esse daqui é bem simples, explica como implementar com o banco de dados tbm http://www.w3bees.com/2013/09/jquery-infinite-scroll-with-php-mysql.html
Answer: @TássioGonçalves, você pode fazer com os seguintes passos:

Atribui o event handler click ao botão "Carregar mais".
Conta o número de itens existentes na página.
Faz pedido de dados a uma página PHP que vai buscar no BD
Recebe esses dados e insere-os na página.

index.html

A

B

C

<button class="mais">Carregar mais</button>

PASSO 1 E 2
jquery-script.js
// Atribui um event handler ao botao  - evento click
$(".mais").on('click', function () {
        var nItens = $itens.find(".item").length; // Conta o numero de itens
        var dadosEnviar = { offset : nItens };    // Dados que serao enviados ao BD. Será o número de ítens na página.
       pedidoAjax(dadosEnviar);
});

Pedido AJAX que espera a resposta do servidor no formato JSON. Nesse caso, um vetor de strings no formato JSON.
var pedidoAjax = function(dadosEnviar) {
    $.ajax({
        data: dadosEnviar,        // dados a serem enviados
        dataType: "json",         // Formato do dado **recebido como resposta do servidor**.  html, json, text ...
        method: "post",           // post ou get
        url: "myPHPscript.php",   // url invocada
        success: function (dados, textStatus, jqXHR) {

            // Executado em caso de sucesso.
            console.log(dados + " " + textStatus + " " + jqXHR + " ");

            inserirItens(dados);
        },
        error: function (jqXHR, textStatus, errorThrown) {

            // Executado em caso de erro.
            alert(jqXHR + " " + textStatus + " " + errorThrown);
            error();
        }
    });
}

Você pode ver mais nesse post.
PASSO 3
/**
 * Funcao que insere os dados na lista
 * @param {string[]} dados. Vetor de strings. FORMATO: ["nome 1", "nome 2", "nome 3"]
 */
var inserirItens = function(dados) {
    function addItem(indice, dado) {
        // Cria item da lista
        var $li = $("

");

        // Configura-o
        $li.addClass("item");
        //...
        $li.html(dado);

        // Insere o elemento no DOM
        $li.appendTo($itens);
    }

    $.each(dados, addItem); // Para cada elemento no vetor `dados`, invoca a funcao `addItem`
}

O código PHP pode ser algo assim
myPHPscript.PHP
//Altera o cabeçalho para que o PHP saiba que o retorno será uma string JSON
header('Content-Type: application/json; charset=utf-8');

// Recebe os dados (Número de ítens existentes na página do cliente)
$contentType = $_SERVER["CONTENT_TYPE"];
$response_array['contentType'] = $contentType;

$aDados = json_decode(file_get_contents("php://input"), true); // Recebe dados como array associativo.

// Busca no BD
// Se o BD for MySQL ou PostgreSQL, use LIMIT e OFFSET para fazer a busca, ex:
// "SELECT nome FROM tabel LIMIT 3 OFFSET ".$aDados["nItens"].";";

// Cria array com os dados do BD
$dados = array();
while ($r...)               // $r guarda o resultado do BD
{
   $dados[] = $r["nome"];
}
// agora $dados é um array que contém todos os nomes buscados no BD.

// Converte o array para JSON e "retorna"
echo json_encode($response_array);

PASSO 4
/**
 * Funcao que insere os dados na lista
 * @param {string[]} dados. Vetor de strings. FORMATO: ["nome 1", "nome 2", "nome 3"]
 */
var inserirItens = function(dados) {
    function addItem(indice, dado) {
        // Cria item da lista
        var $li = $("

");

        // Configura-o
        $li.addClass("item");
        //...
        $li.html(dado);

        // Insere o elemento no DOM
        $li.appendTo($itens);
    }

    $.each(dados, addItem); // Para cada elemento no vetor `dados`, invoca a funcao `addItem`
}

Answer: Gabriel o que pretendes implementar é algo tipo infinitscroll, tens muitos exemplos, fiz um para ti.
http://codepen.io/H3ld3rM3/pen/ozPBOW
Comment: Muito obrigado, mas o que seria o $("div:hidden") que tem no codigo?
Comment: Essa solução não carrega mais 3, e sim exibe mais 3, o ideal seria carregar realmente, ou seja, desse jeito eu teria que trazer todos os meus registros de uma vez.
Comment: @TássioGonçalves, você pode buscar os dados por AJAX. Olha esse post https://pt.stackoverflow.com/a/297125/91181 . A diferença desse outro post é que não precisará coletar dados do form.
Comment: Carregar e ocultar não será a melhor solução para nenhuma necessidade, acabaria por pesar o carregamento da página e até mesmo atrapalhando um possível trabalho de SEO.