How do I allow partial updates via POST request in Django?

Question: Working on a small Django app and I've been asked to set up partial updates via POST request. I'm aware that PATCH works for partial updates out of the box but I don't have access to the front end and that's not what I've been asked to do.
I was told it's a quick one line change, so I'm assuming I have to update the serializer as explained in the DRF docs (https://www.django-rest-framework.org/api-guide/serializers/#partial-updates) but I'm not sure where to do that exactly.
Serializers:
from rest_framework import serializers
from cat.models import Cat
class CatSerializer(serializers.ModelSerializer):

    class Meta:
        model = Cat
        fields = (
            'id', 'name', 'breed', 'birth_date', 'added_at', 'description')

Viewset:
from rest_framework import viewsets
from cat.serializers import CatSerializer
from cat.models import Cat

class CatViewSet(viewsets.ModelViewSet):
    queryset = Cat.objects.all().order_by('-birth_date')
    serializer_class = CatSerializer

Answer: The custom router is what you need. Change the router's mapping as you want. Do something like this:
class CustomRouter(SimpleRouter):
    routes = [
        Route(
            url=r'^{prefix}{trailing_slash}$',
            mapping={
                'get': 'list',
                'post': 'create'
            },
            name='{basename}-list',
            detail=False,
            initkwargs={'suffix': 'List'}
        ),
        DynamicRoute(
            url=r'^{prefix}/{url_path}{trailing_slash}$',
            name='{basename}-{url_name}',
            detail=False,
            initkwargs={}
        ),
        Route(
            url=r'^{prefix}/{lookup}{trailing_slash}$',
            mapping={
                'get': 'retrieve',
                'post': 'partial_update',  # <-- this is the needed part
                'put': 'update',
                'patch': 'partial_update',
                'delete': 'destroy'
            },
            name='{basename}-detail',
            detail=True,
            initkwargs={'suffix': 'Instance'}
        ),
        DynamicRoute(
            url=r'^{prefix}/{lookup}/{url_path}{trailing_slash}$',
            name='{basename}-{url_name}',
            detail=True,
            initkwargs={}
        ),
    ]

Comment: Appreciate the answer! I can't seem to get this method working. All POST requests to the resource endpoint on Postman return "Method \"POST\" not allowed." Regular POST requests work fine though. I tried your code with PUT instead but received a 400 saying the other fields are required.
Comment: @Wayne, my mistake, I forget the `url_path` attribute. I updated my answer.
Comment: Ok this is strange. So that now works for partial updates, and I can still post new objects. I can also get all objects from the endpoint, but when I give a specific ID to return I get back "Method \"GET\" not allowed."

Does the url_path argument route all urls that match to this action?
Comment: It seems I found the better way. I dive deeper and understood, this is the DRF's router task. You should change the mapping part. Just add `'post': 'partial_update'` and it will work as you need.


Adding a whole IPv6 /64 block to an network interface on debian

Question: I tried adding a whole IPv6 (/64) block to an interface using
ip route add local 2001:41d0:2:ad64::/64 dev lo

like described here on my Debian server, but I seem to be missing something.
If I ping for example 2001:41d0:2:ad64::fe locally everything works just fine, but if I try from a remote machine it does not work.
I then tried adding the route on eth0:
ip route add local 2001::41d0:2:ad64::/64 dev eth0

Now I couldn't even ping any example address locally!
I'm a bit lost since I seem to be missing something but I can't find the answer here.
For short:
I want 2001:41d0:2:ad64::/64 to be bound to eth0 so that every IP this block is containing will be reachable from the internet on my machine.
I hope someone out there can point me the right way.
Thanks in advance.
The guide provided by the ISP does require me to add each IPv6 to the interface explictly. I want it to be implict.
Working configuration with explict ip address binding
/etc/network/interfaces:
auto eth0
iface eth0 inet static
        address my.ip.v4
        netmask 255.255.255.0
        network my.network.address.ip
        broadcast my.broadcast.address.ip
        gateway my.gateway.ip

iface eth0 inet6 static
        address 2001:41d0:2:ad64::fe
        netmask 64
        gateway 2001:41d0:2:adff:ff:ff:ff:ff
        up ip addr add 2001:41d0:2:ad64::1/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::1/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::2/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::2/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::3/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::3/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::4/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::4/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::5/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::5/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::6/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::6/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::7/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::7/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::8/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::8/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::9/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::9/64 dev eth0
        up ip addr add 2001:41d0:2:ad64::a/64 dev eth0
        down ip addr del 2001:41d0:2:ad64::a/64 dev eth0

Solution try #1
I tried reenableing the local route as @kasperd suggested.
Content of my /etc/network/interfaces
auto lo
iface lo inet loopback
    post-up ip route add local 2001:41d0:2:ad64::/64 dev lo
    pre-down ip route del local 2001:41d0:2:ad64::/64 dev lo

auto eth0
iface eth0 inet static
        # <snip of ipv4 config>

iface eth0 inet6 static
        address 2001:41d0:2:ad64::fe
        netmask 64
        gateway 2001:41d0:2:adff:ff:ff:ff:ff

Local routing table:
# ip -6 route show table local
local ::1 dev lo  proto none  metric 0
local 2001:41d0:2:ad64::fe dev lo  proto none  metric 0
local 2001:41d0:2:ad64::/64 dev lo  metric 1024
local fe80::225:90ff:fe06:6bbe dev lo  proto none  metric 0
ff00::/8 dev eth0  metric 256

Output of traceroute (my local home PC):
  1    <1 ms    <1 ms    <1 ms  fritz.box [xxx]

  2    20 ms    21 ms    24 ms  2002:c058:6301::1
  3    21 ms    22 ms    24 ms  10gigabitethernet6.switch2.fra1.he.net [2001:470
:0:150::1]
  4    44 ms    31 ms    40 ms  100ge3-1.core1.ams1.he.net [2001:470:0:2d4::1]
  5     *        *        *     Zeitüberschreitung der Anforderung.
  6     *        *       35 ms  ams-5-6k.nl.eu [2001:41d0::8d1]
  7    37 ms    39 ms    36 ms  rbx-g2-a9.fr.eu [2001:41d0::ab1]
  8    37 ms    70 ms    36 ms  chi-3-4m.il.us [2001:41d0::176]
  9  Zielhost nicht erreichbar.

Ablaufverfolgung beendet.

traceroute6 on server:
traceroute to 2001:41d0:2:ad64::23 (2001:41d0:2:ad64::23), 30 hops max, 80 byte packets
 1  2001:41d0:2:ad64::a (2001:41d0:2:ad64::a)  0.028 ms  0.009 ms  0.008 ms

ping6 on server:
PING 2001:41d0:2:ad64::23(2001:41d0:2:ad64::23) 56 data bytes
64 bytes from 2001:41d0:2:ad64::23: icmp_seq=1 ttl=64 time=0.029 ms
64 bytes from 2001:41d0:2:ad64::23: icmp_seq=2 ttl=64 time=0.057 ms
^C
--- 2001:41d0:2:ad64:23 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.029/0.043/0.057/0.014 ms

tcpdump output (while pinging and tracerouting on remote server):
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

tracert to the gateway:
Routenverfolgung zu vss-3-6k.fr.eu [2001:41d0:2:adff:ff:ff:ff:ff] über maximal 3
0 Abschnitte:

  1    <1 ms    <1 ms    <1 ms  fritz.box [2002:5476:1b4c:0:c225:6ff:fe40:b2b0]

  2    23 ms    22 ms    26 ms  2002:c058:6301::1
  3    24 ms    40 ms    23 ms  10gigabitethernet6.switch2.fra1.he.net [2001:470
:0:150::1]
  4    28 ms    37 ms    39 ms  100ge3-1.core1.ams1.he.net [2001:470:0:2d4::1]
  5     *        *        *     Zeitüberschreitung der Anforderung.
  6    38 ms    33 ms     *     ams-5-6k.nl.eu [2001:41d0::8d1]
  7    36 ms    39 ms    38 ms  rbx-g2-a9.fr.eu [2001:41d0::ab1]
  8    36 ms    35 ms    35 ms  vss-3-6k.fr.eu [2001:41d0:2:adff:ff:ff:ff:ff]

Ablaufverfolgung beendet.

ping to the gateway:
Ping wird ausgeführt für 2001:41d0:2:adff:ff:ff:ff:ff mit 32 Bytes Daten:
Antwort von 2001:41d0:2:adff:ff:ff:ff:ff: Zeit=36ms
Antwort von 2001:41d0:2:adff:ff:ff:ff:ff: Zeit=34ms
Antwort von 2001:41d0:2:adff:ff:ff:ff:ff: Zeit=38ms
Antwort von 2001:41d0:2:adff:ff:ff:ff:ff: Zeit=57ms

Ping-Statistik für 2001:41d0:2:adff:ff:ff:ff:ff:
    Pakete: Gesendet = 4, Empfangen = 4, Verloren = 0
    (0% Verlust),
Ca. Zeitangaben in Millisek.:
    Minimum = 34ms, Maximum = 57ms, Mittelwert = 41ms

So it's still just working locally (server) but not from remote (my pc).
Comment: So have you tried a traceroute from a remote machine?  Where does the trace fail?
Comment: If all the hosts are in the same network, you shouldn't had any route.
Comment: Your second `ip route` command has a typo in the IPv6 address.
Comment: @Spack I want eth0 to listen to a whole ipv6 /64 prefix (incoming traffic), @michael-hampton fixed, it was just while typing this question, @Zoredache It fails at the ISPs Gateway `2001:41d0:2:adff:ff:ff:ff:ff` after that it will just timeout.
Comment: From research I've done in the past, this isn't possible. The reason it works locally is because your routing table *knows* where to route packets. If you added this prefix as a static route in your edge router then you'd see LAN clients able to connect.
Comment: i'm having the same question/problem.
What is the working solution here? i noticed the mentioned ip address : 2001:41d0:2:ad64::/64
isn't that short 1 digit/hexgroup?
shouldn't that be : 2001:41d0:2:ad64:xxxx::/64
Comment: @TD_Nijboer When `::` is used in an IPv6 address, it means enough 0 groups are inserted to make the total length 8 groups. `/64` means the first 64 bits of the address, which in this case will ignore the last 64 bits inserted by `::`.
Comment: @TD_Nijboer For short 2001:41d0:2:ad64:: is the same as 2001:41d0:0002:ad64:0000:0000:0000:0000
Answer: I have needed something similar in the past. I found that there are three steps needed to get this working:

You need to route a prefix to the host
You need a local route on the host
Applications need to set the IP_FREEBIND or IP_TRANSPARENT option on sockets

The proper way to get a prefix routed to the host involves contacting your provider, if they do not already provide you one. They may have a DHCPv6 server, which can delegate a prefix to you, if you just send the right DHCPv6 request to it.
If a real routed prefix is for some reason impossible for you to get, but you have access to use as many addresses as you want from a link prefix available on one of your network interfaces, you can turn part of that into a routed prefix by having a daemon respond to neighbor discovery requests for every IPv6 address in that range.
Using such a daemon is not recommended other than as a last resort, since it will needlessly consume memory from all your neighbors. There are a few implementations of such a daemon, one that looks promissing is ndppd. (I have no specific experience with it, since I only learned about it after I had written my own with my link prefix hardcoded.)
It looks like you already got the local route working. As you noticed, it has to be assigned to the lo interface in order to work.
Finally the applications using addresses from this range need an IP option in order to be able to bind to addresses, which are not explicitly assigned to a specific network interface on the host. Here is a code fragment, that can be used:
const int one = 1;
setsockopt(fd, SOL_IP, IP_FREEBIND, &one, sizeof(one));

Comment: @Arya When I needed such a thing I put the command in `/etc/rc.local`
Comment: I've got a static prefix routed to my server by the ISP (they do not provide DHCP. Neither fo IPv4 nor IPv6). I want all IPv6 in this /64 block to be reachable from the outside (most of my applications bind to :: which should be all available addresses if I'm not mistaken). Now I want all these IPs to be available on eth0 so if I try to connect to any IPv6 in that block any application listening to the specified port will be able to answer (for example each IP should respond to ping correctly).
Comment: If you bind to :: the IP_TRANSPARENT option is not needed. With a prefix routed to my server and the local route in place, I can bind to :: and receive connections made to arbitrary IPv6 addresses in that range. ping6 works as well. I am testing this on Ubuntu 12.04, but I would expect it to work on any recent kernel on other distributions as well. If it is not working for you, I suggest you take a look on the network traffic using `tcpdump -pni eth0 'host 2001:41d0:2:ad65::fe'`
Comment: It's still not working. I provided a few more information on the configuration in my question, maybe this helps.
Comment: You say that you have a /64 routed to the server. But the examples in the hosting provider's guide only has a link prefix and no routed prefix. And the tcpdump and traceroute6 output looks like the addresses are not routed to the server. Have you managed to get a single IPv6 address working using the documentation from the provider?
Comment: Yes, this works perfectly fine.
Comment: Then can you show us the configuration, which does work, as well as tell us what your routed IPv6 prefix is?
Comment: In your latest update, you mention two different /64 prefixes. Is 2001:41d0:2:ad64::/64 the link prefix and 2001:41d0:2:ad65::/64 the routed prefix? I cannot reach any of the IPv6 addresses you mentioned from those prefixes. Could you show the output of a traceroute from your server towards 2a01:4f8:d16:701:7a18:3d30:d395:28bb?
Comment: `2001:41d0:2:ad64::/64` is the only prefix. `2001:41d0:2:ad65::/64` was just writtern because I wanted to avoid attacks and for some privacy reasons and may or may not even exist. Where's the difference between a routed prefix and a link prefix anyway? (I'm not really good with network architecture ;) )
Comment: The question about the difference between a link prefix and a routed prefix would kind of deserve to be asked as a separate question and not just in a comment. What you are trying to do is best done with a routed prefix. But you should still keep the link prefix. A configuration with a routed prefix but no link prefix would be somewhat non-standard. I think you probably only have a link prefix and no routed prefix. You should ask your provider for a routed prefix. If they won't give you one you can use a daemon like previously mentioned to use part of the link prefix as if it was a routed pref
Comment: @kasperd awesome solution. Millions of thanks to you. Because its very important for my project
Comment: I'm having a hard time getting this to work on Vultr.
Comment: @ChandraNakka which VPS provider are you using?
Comment: @Arya I know nothing about that provider. I have used the setup described here on Hetzner, where it works just fine.
Comment: @kasperd How exactly would you route a prefix? That's the part I'm confused about.
Comment: @Arya That's something the ISP is supposed to do for you. With many of the providers who don't do IPv6 properly you can work around the lack of routed prefix by setting up ndppd.
Comment: @kasperd how can I make the "ip route add local" command persistent that I don't have to execute it every time after a system reboot?
Comment: Is there any proxy server that uses IP_FREEBIND or IP_TRANSPARENT? I have tried Tinyproxy and Squid so far and they don't work with local route method. I'm trying to create a proxy server for all the IPs in the subnet.
Comment: @Arya No idea. That question might be suitable for [softwarerecs.se].
Comment: unfortunately it looks like with this method of assigning IPs, the IPs can't be used for outgoing connections. They only work for incoming connections. For example, I can connect to the machine with any IP with the /64 block, but on the same machine if I do "wget --bind-address=2001:19f0:9002:073e:0000:0000:0000:3000 http://google.com" I get the message "failed: Cannot assign requested address." but I can connect to the same machine with "2001:19f0:9002:073e:0000:0000:0000:3000". If I add the same address in the interfaces file then wget will also work. Is there any workaround for this?
Comment: @Arya It works for outgoing connections. All you need to do is to use `IP_FREEBIND`. If the application you want to use doesn't support `IP_FREEBIND` you will have to change a few lines of source code to make it work.
Comment: I'm writing a simple C program to fetch a page with IP_FREEBIND, I will share my results
Comment: @Arya sorry for long delay. I'm using Ubuntu 16.04 x64 server on DigitalOcean VPS services.
Comment: @Hikaru-Shindo i'm facing the same issue, my app says 'failed: Cannot assign requested address.', did you found a way around?
Answer: It's 2019 year now. One word: ip_nonlocal_bind (since 4.3 kernel as i may know).
Use ndppd + sysctl  net.ipv6.ip_nonlocal_bind = 1, last one allows you to bind to any IPv6 address(no IP_FREEBIND needed in this case).
Guess you did so:
ip add add local 2001::41d0:2:ad64::/64 dev lo
ip route add local 2001::41d0:2:ad64::/64 dev eth0
sysctl  net.ipv6.ip_nonlocal_bind = 1

ndppd.conf will look like:
route-ttl 30000

proxy eth0 {

   router no

   timeout 500
   ttl 30000
   rule 2001::41d0:2:ad64::/64{
       static
   }
}

run ndppd and now you can bind to any address(of added block) and use it as it added itself.
Comment: when trying this ndppd responds with "(error) Unable to create socket." Anyone have any clue why?
Comment: what if application listens on [::]? i tried ndppd but it's not working...
Comment: regardless of that i can't ping any random ipv6 either. But I see the echo request in tcpdump
Answer: You need to add the subnet to your loop back interface and then add a route between lo and eth0 and also enable non-local binding for ipv6.
Just modify your networking configuration (/etc/networking/interfaces) as is in the following and restart your networking daemon:
auto eth0
iface eth0 inet static
        address my.ip.v4
        netmask 255.255.255.0
        network my.network.address.ip
        broadcast my.broadcast.address.ip
        gateway my.gateway.ip

iface eth0 inet6 static
        address 2001:41d0:2:ad64::fe
        netmask 64
        gateway 2001:41d0:2:adff:ff:ff:ff:ff
        up ip addr add 2001:41d0:2:ad64::/64 dev lo
        up ip route add local 2001:41d0:2:ad64::/64 dev eth0
        up sysctl net.ipv6.ip_nonlocal_bind=1
        down ip -6 addr del 2001:41d0:2:ad64::/64 dev lo
        down ip route del local 2001:41d0:2:ad64::/64 dev eth0
        down sysctl net.ipv6.ip_nonlocal_bind=0

I use this approach for setting up ipv6 tunnels on my Ubuntu machines and it works fine. The whole IP range is ping-able from remote machines.


How to put header in url using volley in Kotlin?

Question: My code-
val accessTokenRequest: JsonObjectRequest = JsonObjectRequest(Request.Method.GET, url,
            Response.Listener { response ->
            },
            Response.ErrorListener { error ->
                Toast.makeText(activity,error.toString(), Toast.LENGTH_LONG).show()
            }
    )

    AppController.instance!!.addToRequestQueue(accessTokenRequest)

Header I wat to put -
"Search"&
"Authorization"
Comment: Possible duplicate of [Add custom headers in volley request](https://stackoverflow.com/questions/33054019/add-custom-headers-in-volley-request)
Answer: Try using the following code to add headers
val accessTokenRequest: JsonObjectRequest = object : JsonObjectRequest(
    Request.Method.GET, "", JSONObject(),
    Response.Listener<JSONObject?> {
        TODO("not implemented") //To change body of created functions use File | Settings | File Templates.
    }, Response.ErrorListener {
    TODO("not implemented") //To change body of created functions use File | Settings | File Templates.
}) {
    @Throws(AuthFailureError::class)
    override fun getHeaders(): Map<String, String> {
        var params: MutableMap<String, String>? = super.getHeaders()
        if (params == null) params = HashMap()
        params["Authorization"] = "Your authorization"
        //..add other headers
        return params
    }
}

Note: To generate kotlin code from java try using Ctrl + Shift + Alt + K or simply copy paste it on a kotlin file.
Comment: That code results in an error UnsupportedOperationException at java.util.AbstractMap.put. remove the first 2 lines in getHeaders() and just instsantiate a new map right away val params: MutableMap = HashMap()
Comment: Thanks a lot, you made my day.
Answer: As @chitgoks said @Abilash code occurs UnsupportedOperationException at java.util.AbstractMap.put error. So I rewrote his code to fix it.
val accessTokenRequest: JsonObjectRequest = object :
            JsonObjectRequest(Request.Method.GET, "", JSONObject(), Response.Listener<JSONObject?> {
                // Add something you want to run on success
            }, Response.ErrorListener {
                // Add something you wanna run on error
            }) {
            @Throws(AuthFailureError::class)
            override fun getHeaders(): Map<String, String> {
                val params = HashMap<String, String>()
                params.putAll(super.getHeaders())
                params["Accept"] = "application/json"
                //add your header like above line
                return params
            }
        }


Let $A\sim A'$ and $B\sim B'$ then $A\times B\sim A'\times B'$

Question: Let $A\sim A'$ and  $B\sim B'$ then  $A\times B\sim A'\times B'.$
Well I know that  $A\sim A'$ means that exists a bijective function $f$ between the set  $A$ and $A'$. Also $B\sim B'$ means that exists a bijective function $g$ between $B$ and $B'$.
I want a bijective function $h:A\times B \rightarrow A'\times B'$. I'm kind of stuck here.
Answer: Let $f\colon A\to A', g\colon B\to B'$ be bijections. Then $h\colon A\times B\to A'\times B'$ defined by $h(a,b):=(f(a),g(b))$ is also bijective.
Answer: Let $h:A\times B\to A'\times B'$ be defined by $h(a,b)=(f(a),g(b)),$ where $a\in A,b\in B,$ and $f$ and $g$ are the given bijections. Then, it is easy to check $h$ is a bijection (check injectivity and surjectivity.)


After updating the microsoft sdk to 6.5 professional, an "Windows Mobile 6.5 Device" on platform list

Question: After I installed the latest SDKs I found related to Windows Mobile 6.5, does not appear an "Windows Mobile 6.5 Device" in the plataform list.
This is normal? If not what should be the needed installs. The SDKs were installed on a clean Windows XP virtual machine since the application must be also compiled to PPC2003 (which is the setting on the host machine).
Any ideas?
Edit: It's on deploy list. It never brings an WinMo 6.5 Device, just emulatores. It only brings an WinMo Pro 6 device.
Comment: With what version of Visual Studio?  And which platform list, exactly (the output for your app, the "deploy to" list, the "target platform" list when creating a new app, something else?)
Comment: A reboot of your PC may be required after installing the SDK. Like Chris hinted to, I think WM6.5 may require VS2008 (not higher).
Comment: The list of devices to Deploy, @ctacke.
Comment: I see the VS2008 tag now. Thanks Fabricio. Did the reboot do any good on your Virtual Machine?
Comment: I retagged to show that I'm using VS2008, @jp2code
Comment: No. I rebooted a fez times since install, but it never brings an WinMobile 6.5 Device option, only emulators.
Answer: To install and use WM6.5 target you need to have VS2008 Pro or better (remeain in the VS2008 chain, no VS2010 or 2012); VS2008 SP1 installed; WM6 SDK installed and finally WM6.5 DTK (developer toolkit).
See also wm65 requirements at http://www.microsoft.com/en-us/download/details.aspx?id=5389
excerpt:
 - Microsoft .NET Compact Framework v2 SP2.
 - Important: "Visual C++ Smart Device Programmability", an option of
   Microsoft Visual Studio Setup, must be selected and installed during
   Visual Studio installation.


Função MAP em Lista de Dicionários

Question: Aplique a função map na lista de emprestimos para extrair os valores da chave valor_emprestimos na lista valor_emprestimos_lista. Faça também a conversão de str para float.
emprestimos: [{'id_vendedor': '104271', 'valor_emprestimos': '448.0', 'quantidade_emprestimos': '1', 'data': '20161208'}, {'id_vendedor': '21476', 'valor_emprestimos': '826.7', 'quantidade_emprestimos': '3', 'data': '20161208'}, {'id_vendedor': '87440', 'valor_emprestimos': '313.6', 'quantidade_emprestimos': '3', 'data': '20161208'}, {'id_vendedor': '15980', 'valor_emprestimos': '-8008.0', 'quantidade_emprestimos': '6', 'data': '20161208'}, {'id_vendedor': '215906', 'valor_emprestimos': '2212.0', 'quantidade_emprestimos': '5', 'data': '20161208'}, {'id_vendedor': '33696', 'valor_emprestimos': '2771.3', 'quantidade_emprestimos': '2', 'data': '20161208'}, {'id_vendedor': '33893', 'valor_emprestimos': '2240.0', 'quantidade_emprestimos': '3', 'data': '20161208'}, {'id_vendedor': '214946', 'valor_emprestimos': '-4151.0', 'quantidade_emprestimos': '18', 'data': '20161208'}, {'id_vendedor': '123974', 'valor_emprestimos': '2021.95', 'quantidade_emprestimos': '2', 'data': '20161208'}, {'id_vendedor': '225870', 'valor_emprestimos': '4039.0', 'quantidade_emprestimos': '2', 'data': '20161208'}]
Minha Resposta
valor_emprestimos_lista = map(lambda valor_emprestimo : float(emprestimos[:]['valor_emprestimos']), emprestimos)
print(list(valor_emprestimos_lista))
Erro
TypeError                                 Traceback (most recent call last)
 in ()
----> 1 print(list(valor_emprestimos_lista)) # [448.0, 826.7, ..., 4039.0]
 in (valor_emprestimo)
----> 1 valor_emprestimos_lista = map(lambda valor_emprestimo : float(emprestimos[:]['valor_emprestimos']), emprestimos)
TypeError: list indices must be integers or slices, not str
Comment: Bem vindo ao Stack Overflow. Sua publicação parece ter alguns problemas, nós somos um site de perguntas e respostas, e sua experiência aqui no Stack Overflow em Português poderá não ser a melhor por causa disso. Queremos que você se dê bem aqui e consiga o que quer, mas para isso precisamos que você faça a sua parte. Veja algumas orientações que vão te ajudar: [Guia de sobrevivência do Stack Overflow em Português](https://pt.meta.stackoverflow.com/q/8045)
Comment: Por favor, esclareça seu problema ou forneça detalhes adicionais a fim de destacar exatamente o que você precisa. Da forma como está escrito atualmente é difícil dizer com exatidão o que você está perguntando.
Answer: No contexto que tu tá usando, emprestimos[:] é a "mesma coisa" que emprestimos ainda que sejam tecnicamente objetos distintos porque retorna uma cópia da lista inteira, então emprestimos[:]['valor_emprestimos'] está tentando acessar a chave 'valor_emprestimos' de emprestimos, da mesma forma que se tu tivesse acessando emprestimos['valor_emprestimos'], onde emprestimos é uma lista de dicionários, e não um dicionário, e portanto não tem a chave 'valor_emprestimos'.
O que o erro diz é que listas deveriam ser acessadas por posição, como em emprestimos[0], emprestimos[1], emprestimos[-1], etc. Sendo uma lista de dicionários, emprestimos[0]['valor_emprestimos'] é o que tu realmente quer, daí a necessidade de usar uma solução como map() pra iterar sobre a lista inteira aplicando uma função.
Além disto, o map() aplica uma função a uma lista, então a tua função lambda usa a variável valor_emprestimo como um item da lista emprestimo, e portanto na primeira iteração, valor_emprestimo é igual a emprestimos[0]. Ou seja, valor_emprestimo['valor_emprestimos'] vai retornar o resultado esperado porque é equivalente a emprestimos[0]['valor_emprestimos'].
valor_emprestimos_mapa = map(lambda valor_emprestimo : float(valor_emprestimo['valor_emprestimos']), emprestimos)
valor_emprestimos_lista = list(valor_emprestimos_mapa)
print(valor_emprestimos_lista)

Neste caso é possível replicar a funcionalidade deste mapa, já que a função pretendida não é complexa, com compreensão de listas desta forma:
print([float(emprestimo['valor_emprestimos']) for emprestimo in emprestimos])

Comment: Se quer positivos tem muita coisa que melhorar, começa pela pergunta. Ela realmente tem que ser uma pergunta pois o conjunto perguntas e respostas não é inicialmente destinado para eu, você ou o autor da pergunta. Tudo que nós produzimos é para ser consumido por ferramentas de automação, seja Motores de Busca, AIs de auto completar código, AIs de auto programação, NPL,......... o foco não é o helpdesk pessoal o nosso foco é o futuro e o coletivo.
Comment: Obrigado amigo. Me ajudou bastante
Comment: Foi por isso que eu usei as aspas, o que eu quis dizer é que não tem razão pra usar essa expressão naquele contexto, e é por isso que ele está recebendo aquele erro. Dentro do contexto dele, vai ser a mesma coisa,mesmo que sejam objetos diferentes.
Comment: mudei a resposta pra esclarecer porque é a mesma coisa neste contexto
Answer: Eu fiz assim:
valor_emprestimos_lista = list(map(lambda emprestimo:
float(emprestimo['valor_emprestimos']), emprestimos))
print(valor_emprestimos_lista)

Função MAP em Lista de Dicionários


How to convert $\lim_{n\to\infty}\frac{\pi}{n}\sum_{k=1}^{n}\sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right)$ into an integral?

Question: The question asked is:

$\displaystyle \lim_{n\to \infty} \frac{\pi}{n} \sum_{k=1}^{n} \sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right) = ? $
a) $\dfrac{2\pi}{5} \hspace{3em}$
b) $\dfrac{5}{2} \hspace{3em}$
c) $\dfrac{2}{5} \hspace{3em}$
d) $\dfrac{5\pi}{2}$

The solution is given as:src)
\begin{align*}
&\lim_{n\to \infty} \frac{\pi}{n} \sum_{k=1}^{n} \sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right)  \\
&= \frac{2}{5} \int_{0}^{5\pi/2} \sin\left(\frac{\pi}{2}+x\right)\,\mathrm{d}x
= \frac{2}{5} \int_{0}^{5\pi/2} \cos x \,\mathrm{d}x \\
&= \frac{2}{5} \left[ \sin x \right]_{0}^{5\pi/2}
= \frac{2}{5} \left[ \sin\frac{5\pi}{2} - \sin 0 \right] \\
&= \frac{2}{5}\sin\left(2\pi + \frac{\pi}{2}\right)
= \frac{2}{5}\cdot\sin\frac{\pi}{2}
= \frac{2}{5}
\end{align*}
My question is how they got
$$ \lim_{n\to \infty} \frac{\pi}{n} \sum_{k=1}^{n} \sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right) = \frac{2}{5} \int_{0}^{5\pi/2} \sin\left(\frac{\pi}{2}+x\right)\,\mathrm{d}x. $$
Thanks in advance!
Comment: https://en.wikipedia.org/wiki/Riemann_sum This is what you are looking for.
Answer:
$$ \lim_{n \to \infty} \frac{b-a}{n} \sum_{k=1}^n f( a+ k( \frac{b-a}{n})) = \int_a^b  f(x) dx$$

$$\displaystyle \lim_{n\to \infty} \frac{\pi}{n} \sum_{k=1}^{n}
\sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right) = ? $$

Taking $f(x) = \sin(x)$,
By direct comparison,
$$ a= \frac{\pi}{2}$$
And,
$$ \frac{ 5 \pi}{2} \frac{k}{n} = \frac{k}{n} (b-a)  $$
Or,
$$  \frac{5 \pi}{2} = b -a$$
Or,
$$  b = 3\pi$$
Hence, this is equal to:
$$\frac{2}{5} \int_0^{3\pi} \sin x = \frac{2}{5}$$
Note: Note I multiplied by $ \frac{2}{5}$ to make it $ \frac{b-a}{n}$ form

The solution
If instead of taking,
$$ f(x) = \sin(x)$$
You took,
$$ f(x) = \sin( \frac{\pi}{2} + x)$$
Then,
$$ f(a + k \frac{b-a}{n}) = \sin( \frac{\pi}{2} + a +k \frac{b-a}{n} )$$
Compare this with the question,

$$\displaystyle \lim_{n\to \infty} \frac{\pi}{n} \sum_{k=1}^{n}
\sin\left(\frac{\pi}{2}+\frac{5\pi}{2}\cdot\frac{k}{n}\right) = ? $$

hence,
$$ a= 0 $$
$$ \frac{5\pi}{2}  = b$$
Comment: What was the downvote for ?


Python - read 1000 lines from a file at a time

Question: I've checked this, this and this.
The 3rd link seemed to have the answer yet it didn't do the job.
I can't have a solution where the whole file is brought to main memory, as the files I'll be working with will be very large. So I decided to use islice as shown in the 3rd link. First 2 links were irrelevant as they used it for only 2 lines or read 1000 characters. Whereas I need 1000 lines. for now N is 1000
My file contains 1 million lines:
Sample:
1 1 1
1 2 1
1 3 1
1 4 1
1 5 1
1 6 1
1 7 1
1 8 1
1 9 1
1 10 1

So if I read 1000 lines at a time, I should go through the while 1000 times, yet when I print p to check how many times I've been in through, it doesn't stop at a 1000. It reached 19038838 after running my program for 1400 seconds!!
CODE:
def _parse(pathToFile, N, alg):
    p = 1
    with open(pathToFile) as f:
        while True:
            myList = []
            next_N_lines = islice(f, N)
            if not next_N_lines:
                break
            for line in next_N_lines:
                s = line.split()
                x, y, w = [int(v) for v in s]
                obj = CoresetPoint(x, y)
                Wobj = CoresetWeightedPoint(obj, w)
                myList.append(Wobj)
            a = CoresetPoints(myList)
            client.compressPoints(a) // This line is not the problem
            print(p)
            p = p+1
    c = client.getTotalCoreset()
    return c

What am I doing wrong ?
Comment: the `f` is probably not consumed so you end up reading the same and same 1000 lines every time. This will never terminate. You have to use the alternative formulation for `islice` (`itertools.islice(iterable, start, stop[, step])` this one and not `itertools.islice(iterable, stop)` this)
Answer: As @Ev.kounis said your while loop doesn't seem to work properly.
I would recommend to go for the yield function for chunk of data at a time like this:
def get_line():
    with open('your file') as file:
        for i in file:
            yield i

lines_required = 1000
gen = get_line()
chunk = [next(gen) for i in range(lines_required)]

Comment: No, it will only repeat the steps in the for loop. Yield can be interpreted as "return this input and come back exactly here when asked to". Have a look at doc for Generators: https://docs.python.org/3/howto/functional.html#generators
Comment: But won't it try to open the same file `1M` times for each line ? it will slow down the program, is it not ?
Comment: @MKesper
And how do if the file is over so I can stop iterating and reading ?
`if not chunk: break` didn't work. Any ideas ?
Comment: Managed to fix this by wrapping it by `try` and if an exception is thrown then I break. Thanks!


Why doesn't CatBoost Encoding cause target leakage?

Question: I'm currently working on a fraud detection problem with a dataset of 300,000 rows and 500 columns, 70 of which are categorical with over 10 categories each. I'm facing memory constraints and exploring target encoding as a solution to deal with the categorical columns.
I've recently come across the CatBoost Encoder, which is often praised for its ability to prevent target leakage. However, I'm struggling to understand why this method prevents target leakage.
What's the intuitive explanation of this method?
Edit:
My question is similar to this one: How do Ordered Target Statistics work for CatBoost?. It's different because the author in that question is asking about the meaning of the history. I understand that, but I don't understand why the method stops target leakage.
Answer: If you read the CatBoost paper, you can see that they answer this intuitively. Paraphrasing:

The assumption of a time ordering allows you to encode each row with data from the "past", which prevents information from the "future" from leaking into the encoding.

That's the intuitive explanation.
Mathematically:
Iteratively splitting your data into "past" and "future" and conditioning your encoding on the past introduces a significant amount of variance in the encoding of your categories. The variance is like random noise added over the target encoding and partially breaks the relationship between the target and your categorical column.
Partially breaking the relationship prevents target leakage and improves your model's ability to generalise.
This method can also be better than random noise because it doesn't require introducing a prior assumption about the distribution of your category's values.
This combination is powerful and is responsible for Ordered Encoding's success.


pandas grouping data frame - python

Question: I have this dataset,
PRODUCT_ID   SALE_DATE   SALE_PRICE PROVIDER
1            01/02/16    25         1
1            02/10/16    60         1
1            01/11/16    63         2
1            09/10/16    65         3
2            11/11/15    54         1
2            13/01/16    34         2
3            19/05/14    45         1
3            15/10/15    38         1
3            16/06/14    53         2
3            18/10/15    58         2

This is a combined dataset, data provided by different providers, there is no common identifier for each sale. The issue here is each data provider will have slightly different date and price for each sale. So I am trying to group them together as single sale with a group id. So the business logic here is data provider 1 is the first to get sale data, so for a product id, if the sale date from provider 2 or 3 is within 1 month time and the price is within 10$ difference (more or less), we consider them as same sale, else it will be considered as different sale. So, the output should look like,
PRODUCT_ID   SALE_DATE   SALE_PRICE PROVIDER  SALE_GROUP_ID
1            01/02/16    25         1          1
1            02/10/16    60         1          2
1            01/11/16    63         2          2
1            09/10/16    65         3          2
2            11/11/15    54         1          3
2            13/01/16    34         2          4
3            19/05/14    45         1          5
3            15/10/15    38         1          6
3            16/06/14    53         2          5
3            18/10/15    58         2          7

How do I achieve this in pandas, can someone help please? Thanks.
Comment: `18/10/15` and `11/11/15` are within one month and have $10 in price difference. Are they same sale?
Comment: Nope. Because, they have different product_id. PRODUCT_ID should be the same to be considered as same sale..
Comment: `19/05/14` and `16/06/14` have same PRODUCT_ID, are within one month, have $10 in price difference, and are same sale. Do their sale IDs need to be in sequence?
Comment: Yes, that is why they are considered as same sale. Product_ID is same, within one month and have 10$ difference. So considered as same sale and sale_group_id is same. Nope, they don't need to be in sequence.
Answer: We can using groupby after sort_values
df.SALE_DATE=pd.to_datetime(df.SALE_DATE,format='%d/%m/%y')
df=df.sort_values(['PRODUCT_ID','SALE_DATE','SALE_PRICE'])
df1=df.copy()
df1=df.copy().sort_index()

df1['SALE_GROUP_ID']=df.groupby([df['PRODUCT_ID'],(df['SALE_DATE'].diff().abs()/np.timedelta64(1, 'M')).lt(1).ne(1).cumsum(),df['SALE_PRICE'].diff().lt(10).ne(1).cumsum()]).ngroup().add(1).sort_index().values
df1
Out[294]:
   PRODUCT_ID  SALE_DATE  SALE_PRICE  PROVIDER  SALE_GROUP_ID
0           1 2016-02-01          25         1              1
1           1 2016-10-02          60         1              2
2           1 2016-11-01          63         2              2
3           1 2016-10-09          65         3              2
4           2 2015-11-11          54         1              3
5           2 2016-01-13          34         2              4
6           3 2014-05-19          45         1              5
7           3 2015-10-15          38         1              6
8           3 2014-06-16          53         2              5
9           3 2015-10-18          58         2              7

More info: you require 3 condition when get the group ,
 1. PRODUCT_ID
 2. Date different within in 1 month
 3. SALE_PRICE different less than 10

(df['SALE_DATE'].diff().abs()/np.timedelta64(1, 'M')).lt(1).ne(1).cumsum(), is to getthing the group of date within one month
(df['SALE_DATE'].diff().abs()/np.timedelta64(1, 'M')).lt(1).ne(1).cumsum()
Out[274]:
0    1
1    2
3    2
2    2
4    3
5    4
6    5
8    5
7    6
9    6
Name: SALE_DATE, dtype: int32

df['SALE_PRICE'].diff().lt(10).ne(1).cumsum() is to get the value within 10
df['SALE_PRICE'].diff().lt(10).ne(1).cumsum()
Out[275]:
0    1
1    2
3    2
2    2
4    2
5    2
6    3
8    3
7    3
9    4
Name: SALE_PRICE, dtype: int32

After the groupby we just combine (getting the intersection of each group condition), then we using ngroup to get the group Id
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/161557/discussion-between-ds-user-and-wen).
Comment: @ds_user sorry I can not using the chat ,, you can change to `df['SALE_DATE'].diff()/np.timedelta64(1, 'D')`  this part lt(50) is to handle the bin size
Comment: Sorry, I still don't understand this.  So to mention the day difference, I have to modify 'lt' alone or this also np.timedelta64(1, 'D') to np.timedelta64(50, 'D') . Is this correct ? `df['sale_date'].diff().abs()/np.timedelta64(1, 'D')).lt(50).ne(1).cumsum(),df['net_sale_price'].pct_change().lt(0.1).ne(1).cumsum()` ?
Comment: @ds_user you just need to np.timedelta64(1, 'D'), this is to convert the days object to int

pandas grouping data frame - python


Segmentation Fault When Using MySQL PDO

Question: I've just started working on a new site in Laravel, and I'm having trouble with the connection to a MYSQL (version 5.5.25a) database. The mysql server is on the same local network, but on a different machine to the web server (OSX server). This set up has been working fine for me in Codeigniter, YII, straight up PHP, etc, for a long time.
I've set up Laravel's /config/database.php with my database credentials, however the site times out whenever its on a page that makes a database connection using Laravel's PDO database classes. It gives me the dreaded Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data. error in chrome.
If i enter the credentials incorrectly (wrong password/username/dbname), Laravel gives me a real MYSQL error message SQLSTATE[28000] [1045] Access denied for user..., so it seems like it can communicate the server fine.
If I do a quick and dirty database call using mysqli and log the result with chromePHP, everything appears to work fine, and it returns the correct number of rows from the database, no timing out, no problems.
$mysqli = new mysqli('192.168.1.178', 'username', 'password', 'dbname');

if ($mysqli->connect_error){
    die ("Connect error: " . $mysqli->connect_error );
}

if ($result = $mysqli->query("SELECT username FROM user LIMIT 10")) {
    ChromePhp::log($result->num_rows);
    /* free result set */
    $result->close();
}

UPDATE
I checked the logs, and I get nothing from mysql, but the apache logs show the following error when it times out:
child pid 60418 exit signal Segmentation fault (11)

I've also noticed that phpinfo() shows that the PDO Client API version is set to mysqlnd 5.0.7-dev - 091210 - $Revision: 300533 , I wonder if this mismatch between the Client version and the MYSQL server version could be causing a problem?

Why could php/mysql be timing out when I try to use PDO with mysql?
Comment: Seems I'm not the only one with this problem, and it could be specific to the apple setup I am using. https://discussions.apple.com/thread/2141886?threadID=2141886&tstart=0
Comment: `phpinfo()` on the web server shows that the PDO Client API version is set to `mysqlnd 5.0.7-dev - 091210 - $Revision: 300533`, I wonder if this could be causing a problem?
Answer: You have 2 environments CLI & browser. Both should work with your ip or "localhost" but in my experience one environment works with your IP in the DB configuration and the other works with "localhost" in the DB configuration.
Comment: Might be related http://stackoverflow.com/questions/6538484/php-or-mysql-crash-when-retrieving-a-big-database-record-5mb (Laravel is using PDO MySQL).
Comment: My site is on a virtual domain, so I connect to it via `http://some-made-up-address.dev` through my browser. As you can see above, I connect to the database via IP address, as it is not within the scope of localhost. This set up hasn't caused me any problems in other frameworks. If I enter the same credentials into a codeigniter  or yii set up, it connects to the database fine.
Answer: I had issues with this error while trying out the Laravel 4 beta. I'd see the Segmentation fault: 11 when running migrations from the command line.
Turns out, my issue was a side-effect of having MAMP installed. The version of php MAMP provides only runs in browsers. Out of the box, OS X comes with it's own version of php. This is the version OS X will run when you use php in the command line.
A quick fix is to add the MAMP php directory to your .profile or .bash_profile files. There's a better guide on how to do this here.

Segmentation Fault When Using MySQL PDO


Lilypond "\set associatedVoice" trouble with stanzas

Question: I'm transcribing a song in which two verses have slightly differing notes. I decided to use \set associatedVoice to achieve this. It partly works but I'm having trouble getting the final words of the second stanza to appear:

The conclusion of the lyric of verse 2 should be "... and it slows and ends." I can't figure how why "and ends" isn't appearing. I'd appreciate any help. Here's the lilypond code of this MWE:
\version "2.19.12"
\language english

global = {
  \clef treble
  \key c \major
  \time 4/4
  \tempo 4=120
}

vocalNotes = \relative c' {
  \global
  c4 d e f | %1
  g a
  << {\voiceOne b16 c d e }
     \new Voice = "alternative" {\autoBeamOff \voiceTwo b4 } >>
  d4 | %2
  c1 | %3
}

textVerseOne = \lyricmode {
  \set stanza = #"1. "
  This is a text. | %1
  Here it goes ver -- y fast and | %2
  ends. | %3
}

textVerseTwo = \lyricmode {
  \set stanza = #"2. "
  A se -- cond text, | %1
  and it \set associatedVoice = "alternative" slows
  \set associatedVoice = "vocalMelody" and | %2
  ends. | %3
}

\score {
  <<
  \new Voice = "vocalMelody" {\autoBeamOff \vocalNotes }
  \new Lyrics \lyricsto "vocalMelody" \textVerseOne
  \new Lyrics \lyricsto "vocalMelody" \textVerseTwo
  >>
  }

Answer: Well, the \set associatedVoice = "vocalMelody" is scheduled to happen when the next associated note is played and there is no next associated note in the Voice = "alternative" any more.  Maybe you are better off doing this as a melisma on "vocalMelody" here.
Comment: The case I'm trying to solve is more complicated than a melisma could address, unfortunately! I'm struggling to understand the situation here--why doesn't `\set associatedVoice =  "vocalMelody"` at the end of m2 just switch the text back over to the `vocalMelody` variable beginning at m3? -s
Answer: I don't understand why your idea doesn't work, but you can use \skip instead of \set associatedVoice. They syntax is a bit illogical - \skip must be followed by a note duration, but when you use \lyricsto the duration is ignored and it skips one note.
\version "2.18"
\language english

global = {
  \clef treble
  \key c \major
  \time 4/4
  \tempo 4=120
}

vocalNotes = \relative c' {
  \global
  c4 d e f | %1
  g a
  << {\voiceOne b16 c d e }
     \new Voice = "alternative" {\autoBeamOff \voiceTwo b4 } >>
  d4 | %2
  c1 | %3
}

textVerseOne = \lyricmode {
  \set stanza = #"1. "
  This is a text. | %1
  Here it goes ver -- y fast and | %2
  ends. | %3
}

textVerseTwo = \lyricmode {
  \set stanza = #"2. "
  A se -- cond text, | %1
  and it slows \skip16 \skip16 \skip16 and | %2
  ends. | %3
}

\score {
  <<
  \new Voice = "vocalMelody" {\autoBeamOff \vocalNotes }
  \new Lyrics \lyricsto "vocalMelody" \textVerseOne
  \new Lyrics \lyricsto "vocalMelody" \textVerseTwo
  >>
  }

If all else fails, you can use \lyricmode instead of \lyricsto, and append a duration to the syllables in the same way as for notes. You can then use \associated voice to get the syllables centred under the notes.
\version "2.18"
\language english

global = {
  \clef treble
  \key c \major
  \time 4/4
  \tempo 4=120
}

vocalNotes = \relative c' {
  \global
  c4 d e f | %1
  g a
  << {\voiceOne b16 c d e }
     \new Voice = "alternative" {\autoBeamOff \voiceTwo b4 } >>
  d4 | %2
  c1 | %3
}

textVerseOne = \lyricmode {
  \set stanza = #"1. "
  \set associatedVoice = "vocalMelody"
  This4 is a text. | %1
  Here it goes16 ver -- y fast and4 | %2
  ends.1 | %3
}

textVerseTwo = \lyricmode {
  \set stanza = #"2. "
  \set associatedVoice = "vocalMelody"
  A4 se -- cond text, | %1
  and it
  \set associatedVoice = "alternative" slows4
  \set associatedVoice = "vocalMelody" and | %2
  ends.1 | %3
}

\score {
  <<
  \new Voice = "vocalMelody" {\autoBeamOff \vocalNotes }
  \new Lyrics \textVerseOne
  \new Lyrics \textVerseTwo
  >>
  }

Comment: Personally, I usually use `\lyricmode` with explicit syllable lengths. it's a bit more typing, but it always does what it says on the tin.
Comment: Yes, \skip can be useful, and I do use it at times. The situation I'm trying to solve here, though, is complex in ways not captured by my MWE, so I'm hoping to get to a solution that uses instantiation of a new voice to help make the resulting output clear to a reader.
Comment: It took a little doing, but this solved my issue! Switching over to `\lyricmode` and adding all the syllable lengths did it -- now my `associatedVoice` construct works as I'd hoped. Thank you! If you want to write it up as an answer, I'll happily accept it.


How can use one script for many id's?

Question: I have this code and i need use many id's and not only one like map1 but map1, map2, map3 and more.. For example ( $usamap = $('#map_id'); ) How can do this?
$usamap = $('#map1');
    function bindlinks() {
        $('*').unbind();
        $("#unbind_link").bind("click", function (e) {
            e.preventDefault();
            $usamap.mapster("unbind");
            $usamap.width(720);
            bindlinks();
        });
        $("#rebind_link").bind("click", function (e) {
            e.preventDefault();
            $usamap.mapster(default_options);
        });

        $("#unbind_link_preserve").bind("click", function (e) {
            e.preventDefault();
            $usamap.mapster("unbind", true);
            bindlinks();
        });
        $("#tooltip").bind("click", function (e) {
            e.preventDefault();
            var state = !$usamap.mapster('get_options').showToolTip;
            $('#tooltip_state').text(state ? "enabled" : "disabled");
            $usamap.mapster("set_options", { showToolTip: state });
        });
        $("#single_select").bind("click", function (e) {
            e.preventDefault();
            var state = !$usamap.mapster('get_options').singleSelect;
            $('#single_select_state').text(state ? "enabled" : "disabled");
            $usamap.mapster("set_options", { singleSelect: state });
        });
        $("#is_deselectable").bind("click", function (e) {
            e.preventDefault();
            var state = !$usamap.mapster('get_options').isDeselectable;
            $('#is_deselectable_state').text(state ? "enabled" : "disabled");
            $usamap.mapster("set_options", { isDeselectable: state });
        });

Comment: Pass in the selector to a method.
Comment: I dont now how, you can show me an example
Comment: Use class names instead.
Comment: Diodeus sorry but i dont now how. Can you show me an example?
Answer: You can pass the map object into your bindlinks. The assumption I'm making is that you have multiple maps but a single control, and you want to make the controls apply to a different map every time.
function bindlinks(map) {
    // This is really bad..., affecting all links on the page
    // You should only unbind handlers you bound
    $('*').unbind();

    $("#unbind_link").bind("click", function (e) {
        e.preventDefault();
        map.mapster("unbind");
        map.width(720);
        bindlinks();
    });
    $("#rebind_link").bind("click", function (e) {
        e.preventDefault();
        map.mapster(default_options);
    });

    $("#unbind_link_preserve").bind("click", function (e) {
        e.preventDefault();
        map.mapster("unbind", true);
        bindlinks();
    });
    $("#tooltip").bind("click", function (e) {
        e.preventDefault();
        var state = !map.mapster('get_options').showToolTip;
        $('#tooltip_state').text(state ? "enabled" : "disabled");
        map.mapster("set_options", { showToolTip: state });
    });
    $("#single_select").bind("click", function (e) {
        e.preventDefault();
        var state = !map.mapster('get_options').singleSelect;
        $('#single_select_state').text(state ? "enabled" : "disabled");
        map.mapster("set_options", { singleSelect: state });
    });
    $("#is_deselectable").bind("click", function (e) {
        e.preventDefault();
        var state = !map.mapster('get_options').isDeselectable;
        $('#is_deselectable_state').text(state ? "enabled" : "disabled");
        map.mapster("set_options", { isDeselectable: state });
    });
}

// Call it like
bindlinks( $('#map_id') );

// And later
bindlinks( $('#map_other_id') )

How can use one script for many id's?


Is there a poisson variant for cohen's D/ Hedges' G?

Question: I've got a problem with some count data of mine.
I am using a cohen's D/ Hedges G to compare the effect size between two different treatment groups in my experiment, and then repeating that process for a different strain of fly with the same treatment, and comparing effect sizes for the two different groups to see if effect of treatment varies by strain.
However the actual data I get out at the end is count data in a poisson distribution, not a normal distribution.
Is there an effect size coefficient that is designed for poisson distribution, or should I just stick to these ones and take it with a pinch of salt?
Answer: Effect sizes only compare means (or whatever, like $\eta^2$). They make no assumptions about distributions. So they are applicable regardless of the conditional distribution.
Of course, their interpretation hangs on whether the means are the only thing we are interested in. Some events may have a tiny probability but a huge impact if they do occur, so the mean is very small, but in such a case, we should also be concerned about the variance.
Comment: A bit of counterpoint: because an effect size is traditionally defined using a standard deviation as the reference unit, the concept implicitly *does* make a (mild, nonparametric) distributional assumption: namely, that the SD is suited to this purpose.  As the distribution grows further from Normal--particularly as it becomes more skewed--that assumption is worth questioning, as the OP indicates.
Comment: @whuber: yes, I agree. I tried to convey this in my second paragraph.
Comment: I didn't get that message from your paragraph because "concerned about the variance" could mean many things.


maven plugin jacoco: Coverage checks have not been met. See log for details

Question: I understand that if we set
<jacoco.skip.exec>true</jacoco.skip.exec>

in pom.xml, it might skip the skip the jacoco checking. However, in my case the tests are conducted on TeamCity, on which we have no control of the environment configuration, therefore it might be that the above flag is not set. As a result, it shows:
Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.8:check (verify) on project xxxx: Coverage checks have not been met. See log for details.

on the failure of two tests which I decorated with annotation @Ignore
I thought those two tests were skipped but the integration test failed because jacoco still picked them up.
How can I do from the code level to disable jacoco checking?
Answer: If you remove the goal from pom.xml, it will no longer require the check:
Remove this line inside of your Jacoco plugin in the pom.xml file.
<goal>check</goal>


Im having trouble figuring this integral out can someone help? Not allowed to use polar coordinates.

Question: $$c\int_{-1}^{1} \int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}\sqrt{1-x^2-y^2}dy \:dx=1 .$$ Find $c.$ I went with the substitution say $b=1-x^2$ in the first integral. Then I went with : $\cos t= {y \over \sqrt{b}}$ Could anyone finish the integral with this, i'm very inteerested as to find out how to solve this?
Comment: Are you allowed to use geometry? The integral is the volume of a hemisphere of radius $1$.
Comment: I'm not you can type that out as an alternative , id love to see that, but I was curious with this idea how it is done?
Answer: In the inner integral over $y$, use the substitution
$$y=\sqrt{1-x^2}\cdot \sin\theta$$
for $-\frac{\pi}2\le \theta\le\frac{\pi}2$.
Then in (part of) your remaining integral over $x$, use the substitution
$$x=\sin\phi$$
for $-\frac{\pi}2\le \phi\le\frac{\pi}2$.
You should be able to finish from here.


NSString to NSDate

Question: I got a string that contains the current date by using this :
NSString *date = [[NSDate date] description];

At a different point I want to retrieve the date from this string and I used the following code:
[NSDateFormatter setDefaultFormatterBehavior:NSDateFormatterBehavior10_4];
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
//[NSDateFormatter setDefaultFormatterBehavior:NSDateFormatterBehaviorDefault];
[dateFormatter setDateFormat:@"YYYY-MM-DD HH:MM:SS ±HHMM"];

NSDate *dateFromString = [[NSDate alloc] init];
dateFromString = [dateFormatter dateFromString:<NSString containing date>];

I am getting dateFromString as nil 0x0. What am I doing wrong?
Answer: You can't invent format string syntax and expect it to work; you need to actually use a documented format.  (Seriously, "MM" meaning "month", "minute" and "GMT offset minutes" all at the same time?)
As the documentation points out, the 10.4 formatters use Unicode format strings.
Try "yyyy-MM-dd HH:mm:ss ZZZ" instead.
Also, Objective-C source is ASCII. Don't put characters like ± in there and expect it to work in any context; instead, use strings files.
Comment: That's changed with the move to newer compilers since I wrote my original answer (https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/Strings/Articles/CreatingStrings.html#//apple_ref/doc/uid/20000148-SW1).
Answer: Also, dateFromString: returns an autoreleased NSDate, so:
NSDate *dateFromString = [[NSDate alloc] init]; // <- non freed instance
dateFromString = [dateFormatter dateFromString:<NSString containing date>]; // <- new autoreleased instance
....
[dateFromString release]; // <- wrong

You may want to:
//NSDate *dateFromString = [[NSDate alloc] init];
NSDate *dateFromString = [dateFormatter dateFromString:<NSString containing date>];
....
//[dateFromString release]; // dateFromString is autoreleased

Hope this could save someone :)
Comment: what is dateFormatter?
Comment: From the question itself: NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
Answer: Made an NSString extension for that.
// Simple as this.
date = dateString.dateValue;

Thanks to NSDataDetector, it recognizes a whole lot of format.
// Tested in GMT+1 (Hungary).
@"2014-01-16" dateValue is <2014-01-16 11:00:00 +0000>
@"2014.01.16" dateValue is <2014-01-16 11:00:00 +0000>
@"2014/01/16" dateValue is <2014-01-16 11:00:00 +0000>
@"2014 Jan 16" dateValue is <2014-01-16 11:00:00 +0000>
@"2014 Jan 16th" dateValue is <2014-01-16 11:00:00 +0000>
@"20140116" dateValue is <2014-01-16 11:00:00 +0000>
@"01-16-2014" dateValue is <2014-01-16 11:00:00 +0000>
@"01.16.2014" dateValue is <2014-01-16 11:00:00 +0000>
@"01/16/2014" dateValue is <2014-01-16 11:00:00 +0000>
@"16 January 2014" dateValue is <2014-01-16 11:00:00 +0000>
@"01-16-2014 17:05:05" dateValue is <2014-01-16 16:05:05 +0000>
@"01-16-2014 T 17:05:05 UTC" dateValue is <2014-01-16 17:05:05 +0000>
@"17:05, 1 January 2014 (UTC)" dateValue is <2014-01-01 16:05:00 +0000>

Part of eppz!kit, grab the category NSString+EPPZKit.h from GitHub.

ORIGINAL ANSWER: Whether you're not sure (or just don't care) about the date format contained in the string, use NSDataDetector for parsing date.
//Role players.
NSString *dateString = @"Wed, 03 Jul 2013 02:16:02 -0700";
__block NSDate *detectedDate;

//Detect.
NSDataDetector *detector = [NSDataDetector dataDetectorWithTypes:NSTextCheckingAllTypes error:nil];
[detector enumerateMatchesInString:dateString
                           options:kNilOptions
                             range:NSMakeRange(0, [dateString length])
                        usingBlock:^(NSTextCheckingResult *result, NSMatchingFlags flags, BOOL *stop)
{ detectedDate = result.date; }];

Answer: You might check out TouchTime.
https://github.com/jheising/TouchTime.
It's a direct port of the awesome strtotime function in PHP in 5.4 for Cocoa and iOS. It will take in pretty much any arbitrary format of date or time string and convert it to an NSDate.
Hope it works, and enjoy!
Answer: As previously answered, that is what happens when you invent a new sytax.
Based on Apple NSDateFormatter Class Reference documentation

There are many attributes you can get and set on a style date
  formatter,  ...
  You are encouraged, however, not to change individual settings.  Instead you should accept the default settings established on initialization and specify the format using setDateStyle:, setTimeStyle:

This is specially important for the output, which is different for every locale. By default NSDateFormatter observes the current user's locale settings. So the same NSDate could be 22.11.2011 18:33:19, or Nov 22, 2011 6:33:19 PM, or 2011-11-22 下午6:33:19 or even २२-११-२०११ ६:३३:१९ अपराह्, all for the same input and with the same code.
Consider for example this code
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];

[dateFormatter setDateStyle:NSDateFormatterShortStyle];
[dateFormatter setTimeStyle:NSDateFormatterShortStyle];

// ShortStyle used: 11/23/37 3:30pm
NSString *dateString = [dateFormatter stringFromDate:date];

Check Apple docs for the different styles of NSDateFormatterStyle or this excellent tutorial

NSString to NSDate


Choosing among proper scoring rules

Question: Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences between them. (Exhibit A: Wikipedia.)
Picking the model that maximizes the logarithmic score corresponds to picking the maximum-likelihood model, which seems like a good argument for using logarithmic scoring. Are there similar justifications for Brier or spherical scoring, or other scoring rules? Why would someone use one of these rather than logarithmic scoring?
Comment: Some hints are in the nomenclature.  "Cost functional" is from optimization or optimal control system engineering.  There is no "best".  To have a "good" means you must have a measure of goodness.  There are infinite numbers of families of measures of goodness.  A trivial example is: what is the best path?  If you are marching to your execution - make it a long pleasant one.  If you are going to your Fields metal, make it shortest.  System expertise helps you select the measure of goodness.  When you have the measure of goodness, then you can find "best".
Comment: I took the liberty of editing the title to make it more precise/informative. If I misinterpreted it, sorry and feel free to revert back the change.
Comment: You may be interested in [Merkle & Steyvers, "Choosing a strictly proper scoring rule" (2013, *Decision Analysis*)](https://doi.org/10.1287/deca.2013.0280).
Answer:
Why would someone use one of these rather than logarithmic scoring?

So ideally, we always distinguish fitting a model from making a decision. In Bayesian methodology, model scoring & selection should always be done using the marginal likelihood. You then use the model to make probabilistic predictions, and your loss function tells you how to act on those predictions.
Unfortunately in the real world, computational performance often dictates that we conflate the model-selection and the decision-making and so use a loss function to fit our models. This is where subjectivity in model selection creeps in, because you've got to guess  just how much different kinds of mistake will cost you. The classic example is a diagnostic for cancer: overestimating someone's probability of cancer is not good, but underestimating it is much worse.
As an aside, if you're looking for guidance on how to pick a scoring rule, you might also want to look for guidance on picking a loss function or designing a utility function, as I think the literature on those two topics is a lot more voluminous.
Comment: 1) Yep. 2) Not personally, no. Scoring rules aren't "fashionable" in the bit of ML I work in. Having a quick poke around on [Scholar](http://scholar.google.co.uk/scholar?q=spherical+scoring+rule&btnG=&hl=en&as_sdt=0%2C5), it seems they're a bit dated in general. [This paper](http://faculty.engr.utexas.edu/bickel/Papers/QSL_Comparison.pdf) looks like it'd be interesting to you though. 3) By performance I meant "computational performance", not "predictive performance".
Comment: 1) Are you saying that Brier scoring is essentially a "loss function in disguise"--that is, even though it masquerades as a utility-function-agnostic scoring/comparison rule, it's actually used because people have specific preferences over the types of errors the model makes?
Comment: 2) Do you have any specific examples of settings in which someone might choose Brier or spherical scoring over log scoring (= marginal likelihood, as I understand it) for those reasons?
Comment: 3) Why would it perform better to bake your loss/utility function assumptions into the model than to fit to marginal likelihood and use your loss/utility function when actually making the decision? It seems like for ideal learning algorithms there should be no gap between these.
Answer: (as to my intuition)
the less the difference between predicted & actual - the more precise the forecast is. Just for the purpose of maximization of differencies - different scores are used - either log (log-loss) or difference between squares (Brier score). Everything is obvious in the mathematical formulation of the score & your needs: e.g. if you consider the bigger difference between forcasted & actual value to be the bigger errorness you'd like to take into account - of course, you'll choose log-loss, if any error (independently of size) is error for your purposes - of course, you can see Brier score (in any case, Brier score is for binary categorical outcomes -- and even in multiclass classification you will use one-vs-rest binary comparison logics). Here can see figures for log-loss & Brier. And here three properties a scoring rule should have.
Sometimes for your purposes you can even choose improper scoring rules - it's up to the purpose of your classification
You can always pick any loss_function (or cost function) or even design your own appropriate for the purposes of your estimator in training step && evaluating scores in testing step is just a mathematical formulation of how good the points are forecasted. This goodness you evaluate according this mathematical formulation - chosen according the nature of outcomes (numerical or categorical, but discrete anyway for classification)
P.S. In statistics: statistical estimation should be unbiased, effective, careful. And Variance Components used in estimation include:
Continuous dependent - Dependent,
Categorical dependent - Random effects,
Categorical predictors - Fixed effects,
Continuous predictors - Covariates,
.. and their interactions.
So, In probability theory applied to statistically meaningful data I prefer to follow the same rules for scoring: unbiased, effective, careful -- though it is really not so easy to develop such model [for any distribution of features given, much depends on the size of sample and its inner variation for comprehensive modelling] -- applied mathematics, I think, should help in designing appropriate loss-functions & scores based on them (just for your task's   objectivity & convenience).
ALSO the choice of score can depend on the nature of process you explore: either stochastic or deterministic or dynamic, I think. You should be aware of either you need probabilistic or deterministic model based on your task. With first you can input in loss (or/and score) the size of errorness, with second you need just any score for binary outcomes, with third even improper naive linear score would be enough for me sometimes
Anyway, with statistically meaningful data (feature engineered & selected correctly) modelling becomes easier & scores simplier are affordable

Choosing among proper scoring rules


Expected value of inverse?

Question: If I have a random variable $V$ that is normally distributed with some $\mu$ and $\sigma$, then what is the expected value of $1/V$? I tried doing by delta method, and I get expected value $1/\mu$, but I also read that this is not the correct result.
To clarify:
My random variable $V$ is a variable that can be measured. The data given back to us are mean and standard deviation of the measurements. (I understand that I should be using the t-distribution to best model it, as we only have an average of 4 reads, but I am trying to work it out for the Normal case first.)
The value $V$ is indicative of protein binding to ligand. I wish to understand fractional occupancy ($\alpha = \frac{current signal}{saturation signal}$), in which case I need to know the value of $V$ at saturation, which also can be estimated from the data (take the max value of all values present, this will be a consistent under-estimation). Therefore, if $V \sim N(\mu, \sigma^2)$, I expect $\alpha \sim N(\frac{\mu}{max}, \frac{\sigma^2}{max^2})$. I hope I did the math right.
From the Hill equation,
$$
\alpha = \frac{\mbox{ligand}^n}{\mbox{ligand}^n + K_d} ,
$$
therefore
$$
K_d = \frac{\mbox{ligand}^n}{\alpha} - \mbox{ligand}^n.
$$
Therefore, if $V$ were a random variable, then alpha is a random variable, therefore Kd should be a random variable. I'm trying to find how to model $K_d$ here. My thought was to use the delta method to get an approximation, and that is how I led myself to this question I've posted here.
Comment: This question at math.SE may help, although it also only references Wikipedia and does not give a proof (if you have access to the book by Johnson, Kotz & Balakrishnan cited on the Wikipedia page, this may help): http://math.stackexchange.com/questions/646428/mean-and-variance-of-reciprocal-normal-distribution
Comment: ericmjl - I just noticed I typed the inequality the wrong way around in my previous comment. I decided to delete it so as not to mislead anyone with it. The correct inequality is [here](http://en.wikipedia.org/wiki/Jensen%27s_inequality#General_inequality_in_a_probabilistic_setting) -- i.e. when $E(1/X)$ exists and variance is not zero $E(X)E(1/X)>1$.
Comment: Oh, I meant to say, since your values are *measurements*, you should be careful of specifying normality; normal distributions have density at 0, while measurements don't.
Comment: @Glen_b: Ah yes, I understand that the delta method always gives us an approximation. I think this is sufficient for us, since we are dealing with real-world data. We are modelling the binding coefficient of a ligand as a random variable, which is proportional to the inverse of the fluorescence signal produced.
Comment: Allow me to include additional information into the question.
Comment: Many thanks to @Quantibex for helping me with edits!
Comment: Great, this is really great, I'm learning a lot as I go along here. Thanks a lot!
Comment: Note that the expectation of the inverse of a normal variable can easily be infinite!
Answer: Since your values are measurements, I assume the actual density close to 0 is not an issue and we can safely assume convergence for the Taylor expansion.

(I understand that I should be using the t-distribution to best model it, as we only have an average of 4 reads, but I am trying to work it out for the Normal case first.)

This appears to be a common misconception. Having small samples doesn't cause approximately normal data (or its mean) to have a t-distribution; if your reasons for assuming approximate normality are good, small samples doesn't change that.
Your discussion doesn't make it clear how $V$ relates to $\alpha$.
It's also not clear how finding the expectation of $1/V$ tells you about $K_d$. I feel like there are important details missing.
In any case, I'll do the expansion:
\begin{align}
g(X) &= g(\mu_X + X - \mu_X)  \\
     &= g(\mu_X) + (X - \mu_X)\cdot g'(\mu_X) + \frac{(X - \mu_X)^2}{2!}\cdot g''(\mu_X) + \frac{(X - \mu_X)^3}{3!}\cdot g'''(\mu_X) + ...
\end{align}
Taking expectations term by term:
\begin{align}
E(g(X)) &= g(\mu_X) + g'(\mu_X)\cdot E(X - \mu_X) + \frac{g''(\mu_X)}{2}\cdot E[(X - \mu_X)^2] +  ...  \\
        &=g(\mu_X) + 0 + g''(\mu_X)/2\cdot \text{Var}(X) + ...   \\
        &\approx g(\mu_X) + g''(\mu_X)/2 \cdot\text{Var}(X)
\end{align}
When $g(X) = 1/X$,
$\quad\quad\quad\quad\ \ \approx \frac{1}{\mu_X} - \frac{1}{2\mu_X^2} \sigma^2_X $
An approximation for the variance can also be obtained.
Comment: I don't see how those equations suggest $K_d$ is proportional to $1/\alpha$. Similarly if $\alpha=V/V_\text{max}$ and both are random variables, you can't really treat $V_\text{max}$ as a constant in computing expectations: for example $E(X/Y)\neq E(X)/E(Y)$
Comment: alpha is proportional to V, but Kd is proportional to 1/alpha, that is how the variables are related to one another. V is the binding signal, alpha is the fractional binding, which is V/Vmax (where Vmax is also estimated from data), and Kd is proportional to 1/alpha by the Hill equation.

In any case, your answer was really helpful. Thank you!
Comment: Ah, I see. There was an error made when my question was edited to introduce LaTeX formatting. I have updated the equation properly.
Comment: If $\text{ligand}^n$ is a constant, say $c$, then $K_d=c(1/\alpha -1)$ which isn't $\propto 1/\alpha$
Comment: I also see your point on Vmax... So how would you treat it in this case?
Comment: But for the purposes of computing the expected value, I could ignore the c, and ignore the -1 term, or am I wrong here? Specifically, the c would provide a proportionality term, but the expected value of -1 would be -1, which is shifting the expected value.
Comment: You can't really *ignore* it. But yes, you can easily account for it.

Expected value of inverse?


light confirm script

Question: http://lalalafactory.blogspot.com/2008/02/light-alet-lightalertjs.html
Can anyone tell me what to change that JS confirm button will also work. I add
<code>function confirm(obj){
    new LightAlert(obj);
}
</code>
but still works like alert not confirm.
Answer: Not sure, you would have to get in and modify their code. You can use the jQuery dialog and acheive this.

light confirm script


Scripted install of Debian backdoor/rootkit

Question: We have a number of servers (100+) that we need to increase a certain type of security on. (sortof internal, sorry NDA).
We have thought about using a rootkit of some sort that would be able to keep us access if the main root password has been compromised. It is a time sensitive problem (think high SLA).
What we are after is a script that can take a server and password list and add this backdoor to. We have other software to automate other things, but not this unfortunately.
This is NOT a fishing or hacking goal, we own and have full rights to said servers, we just don't know much about scripting or backdoor type stuff :/
Comment: "This is NOT a fishing or hacking goal, we own and have full rights to said servers, we just don't know much about scripting or backdoor type stuff :/" - Doesn't that just sum up the problem and the solution ?
Comment: Your idea is not necessarily non-sense but I advise you to read up on the subject big time ! Make sure you are not creating a vulnerability by introducing this "protection".
Comment: I'm stuck in a bureaucratic quagmire but yes that is the case. Unfortunately the solution is not as easy as that, because I'm neither the technical team nor management. I'm in a very shit position :(
Comment: You want to install rootkit software to create a backdoor to increase security? OMG...
Answer: So you're defending yourself from a potential vulnerability by introducing another one ?
Adding attack surface won't solve your problem, just use SSH keys, disable password use,  store the keys encrypted (and backed-up offline, think usb/cd in a safe), keep you servers updated and you will thwart most attacks.
Every compromised machine MUST be disconnected as soon as possible (think seconds or minutes, not hours !) from your infrastructure and taken to the lab for analysis and maybe recovering files but take care doig that.
Comment: You don't understand, sorry. It isn't from external attacks, it is from .... 'internal'. I can't say anything more specific due to NDA. I know it seems absurd that this situation exists, but I'm sure you all know of the epic waste of government money that can happen.


trocar comandos no shell

Question: O comando
python foo.py

Quando chamado no terminal, roda o arquivo python enviado. Imagine que eu tenho um arquivo python que sempre precisa ser chamado da seguinte forma:
xvfb-run python foo.py

para funcionar.
Existe alguma maneira de eu escrever algo do tipo no Shell:
python = xvfb-run python

Assim, sempre que eu chamar python ele vai chamar xvfb-run python?
Answer: Não sei se você pode atribuir algo a palavra python, em específico. Ela deve estar atrelada aos binários do python e deve ser meio complicado mudar. O que eu sei é que você pode criar um alias pro seu comando, algo como
$ alias meu_comando = 'xvfb-run python'

O uso seria $ meu_comando foo.py . Leia sobre alias aqui
EDIT:
De acordo com este link:

An alias can be created with the same name as the core name of a command (i.e., a command without any options or arguments). In such case, it is the alias that is called (i.e., activated) first when the name is used, rather than the command with the same name.

Portanto, você pode fazer o que você quer. Só não sei se é uma boa idéia...
Answer: Um alias resolveria seu problema, mas caso não resolva desenvolva outro arquivo python que chame a seguencia que você precisa.
bar.py

import subprocess
subprocess.call("xvfb-run python foo.py")
Depois chame seu arquivo com o comando criado

$python bar.py

Mas reforço que o Alias seria uma alternativa melhor.
Comment: Eu gostava bastante de usar o commands porém foi descontinuado.
https://docs.python.org/2/library/commands.html para o python 3

`
import commands
result = commands.getstatusoutput("xvfb-run python foo.py")

`

trocar comandos no shell


How do I calculate the run rate in cricket?

Question: I know the run rate is calculated by dividing the number of runs scored by number of overs.
How do I calculate the run rate midway through an over? If say a team scores 223 runs in 43.5 overs, what would the run rate be?
Answer: Run rate is calculated for how many runs are scored in an over i.e. how many runs per six balls. So you just need to convert completed overs in balls. Then calculate the run rate with respect to six balls.
So in the above example completed balls is 43 X 6 = 258 + 5 balls. So total completed balls is 263. So now calculate with respect to six balls. So if 223 runs in 263 balls then how many runs in 6 balls? So 223 X 6 / 263 = 5.087 runs per over.


matching height of empty div (adjacent)

Question: I am using Bootstrap framework and using the 2 column grid.
html

   <?php the_content(); ?>

The content has titles, links, subtitles, and text. Which increases the height of that column. I want the column beside it to match its height (so that the image shows) Without a set height the image doesn't show
Answer: Have your own css as below and add the same along with the row class as below,
HTML

      <?php the_content(); ?>

/*
 * Row with equal height columns
 * --------------------------------------------------
 */
.row-eq-height {
  display: -webkit-box;
  display: -webkit-flex;
  display: -ms-flexbox;
  display:         flex;
}

Reference: http://getbootstrap.com.vn/examples/equal-height-columns/

matching height of empty div (adjacent)


How can I display a node create form in a Panel pane?

Question: I'm trying to build a Panel page with two columns that are stacked side-by-side. I need the left hand column to display node/add/story, and the right hand column to display node/add/page. How can I do this?
Comment: will you be saving the at the same time ?
Comment: I hadn't thought that far ahead, but it seems that formblock.module will display both forms side by side, but as soon as the "save" button is clicked on one form the content of the other form are lost ... I suppose it is, technically, still a solution to my original question, but I don't see how it could be of much use on a production site.
Comment: i realized there was going to be a problem once i submitted the answer :) I think there will be a solution to your problem, too.
Answer: One simple solution is installing formblock. But that won't allow to save the nodes at the same time.
If you dont mind programming, this should be of help.
Comment: Thanks, that's useful. I'm trying to fix http://drupal.org/node/937600 so I guess I'll wait until the user who posted the issue replies so I know what the actual issue is I'm trying to fix :)


How do I add custom validations to Webform?

Question: I have created the country and state fields with the Webform module, and I want to add a field validation for the state field when user select US as country.
How can I achieve this?
Comment: Not necessarily.  The linked question is more generic, this is specific to Webform which has different options.
Comment: I think this is a duplicate question: http://drupal.stackexchange.com/questions/22952/custom-validation-for-a-form
Answer: Make use of the validate function in hook_form_alter:
<code> if ($form_id == 'webform_client_form_14') {
    $form['#validate'][] = 'cork_creation_validate_custom';
 }

function cork_creation_validate_custom($form, &$form_state) {
  if (!preg_match('/^[A-Za-z]+$/', $form_state['values']['submitted']['name'])) {
            // $form['submitted'][$key]['#attributes']['class'] = 'error';
            form_set_error('name', t('Please enter alphabets only for Name.'));
 }

  }
</code>
Comment: This is described on the doc page [Custom coding: Adding advanced validation or submit code](https://www.drupal.org/node/1291574), which has more details.
Comment: @colan the webform module do not adhere 100% tends to implement  its own solutions, so it's good to know it works here.
Answer: The Webform Validation module should add the features you need.

This module adds an extra tab to each webform node, allowing you to
  specify validation rules for your webform components. You can create
  one or more of the predefined validation rules, and select which
  webform component(s) should be validated against those. By using the
  hooks provided by this module, you can also define your own validation
  rules in your own modules.

If you need something more in depth or with possibly more control then check out the answer linked to in the question comments.


Ler dados de entrada até EOF no URI Online Judge

Question: Olá, estou tentando resolver o seguinte problema abaixo:

Porém, estou tendo problemas em colocar essa condição do EOF em java. Aqui no stack encontrei tentativas de soluções, porém elas são relacionadas ao uso de arquivos, e pro meu problema ai não se utiliza arquivo.
Por exemplo:
while (fileReader.nextLine() != null) {
    String line = fileReader.nextLine();
    System.out.println(line);
}

Mas isso é com o uso de arquivo, que não é o meu caso.
Meu problema é representar esse EOF em java. Segue a parte do meu código que estou com problema:
public class Main {
public static void main(String[] args) {

    while(?){   //Condição de parada EOF
      ... código ...
    }

Se me recordo bem, em C eu poderia fazer algo como...
while (number != EOF);

Mas como eu posso fazer isso em java?

Resolvi o problema em C, fica assim:
int main(){
int N = 0;
while(scanf("%d",&N) != EOF){
... código ...
}

Mas gostaria de resolver em java, até o momento estou sem solução, alguma dica?
Comment: a condição de parada não seria o número de entrada que você indicou? 4 ou 7 por exemplo? nesse caso poderia colocar um contador dentro do while que vá incrementando até chegar no número, e quando o número for maior ou igual a condição de parada, ele para o while. int contador = 0; while(numeroEntrada > contador){ contador ++; }
Comment: @Geferson, não, 4 e 7 representa a dimensão da matriz. Pelo que entendi do problema meu programa deve receber várias entradas, tipo:
4
7
8
10
50
...
E o critério de parada é o EOF. Então depois de passarem todas as entradas, devo imprimir todas as respectivas matrizes. Porém não entendi como devo fazer esse critério de parada, ou seja, como representar esse EOF.
Answer: Simples, utilize um objeto BufferedReader e chame o método readLine() até que ele retorne null. Lembrando que o método readLine() da classe BufferedReader() retorna uma String. E como no seu exercício você está lidando com um número, basta converter a String para int. Exemplo:
import java.io.*;

public class Main {

    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        String linha;

        while ((linha = br.readLine()) != null) {
            int n = Integer.parseInt(linha);
            // TODO: lógica
        }
    }

}

Funcionando com alguns valores de exemplo no Ideone.
Eu já tinha resolvido esse exercício no URI há alguns anos (sou o 7º colocado ali), e fiz isso utilizando exatamente o mesmo código de leitura que postei acima (exceto a lógica do problema em si, obviamente).
Comment: Não deu certo. Eu informo uma linha com entradas: 10 5 9 8 25, dou enter e ele não faz nada. Se em seguida eu informo mais uma entrada (após o enter) ele imprime o resultado. Estou um pouco confuso quanto a condição de entrada do problema, não sei se devo representar o EOF como -1 ou outra coisa. Meu palpite é de que devo fornecer uma linha com várias entradas, ou seja, 10 20 30 5 9 45 7... e em seguida imprimir as respectivas matrizes.
Comment: @FilipiMaciel vi que você atualizou sua pergunta **após** eu ter dado um exemplo concreto aqui, então pelo jeito não tinha contribuído. Ajustei um pouco mais minha resposta. Espero que dessa vez sim dê certo :).
Comment: Devo ter me esquecido de voltar aqui ontem por estar envolvido com o problema. Caraca Tayllan, que bacana! sua solução deve ter uma lógica muito boa. Eu resolvi o problema com base na sua resposta ai, ficou um pouco diferente, mas basicamente é a mesma coisa. A parte do EOF ta resolvido, agora é o run time hehe mas isso eu resolvo la no URI, muito obrigado!
Comment: Como poderia ser solucionada em Python? Não consigo encontrar em lugar nenhum algo a respeito, e por conhecidencia, é uma questão do URI também: https://www.urionlinejudge.com.br/judge/pt/problems/view/1215
Answer: Também da para fazer da seguinte maneira
 BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
 while(br.ready()){...

assim quando acabar o arquivo sairá no laço while


Multiple instances of an application, how to close the first and keep the last?

Question: I want to update an application without any service interruption, the idea is to inform the first instance of the application (aka the old version) when the last (aka the new version) is ready to start.
Example:

old version is running
new version starts
new version detects that old version is already running
new version prepares to serve
when ready, new version instructs old version to close
when closed, old version informs new version
new version starts serving

When the time comes, the procedure is repeated with a newer version.
This way the service is interrupted for the briefest possible time.

My idea was to use a system-wide mutex, unfortunately they are not available in mono, and it's probably not possible to do step 5 with this technique anyway.
Another approach whould be to use a pipe, this is not available on mono too.
My last idea was to use a socket bound to loopback address on a random (constant) port, do you think this is fine or there is a better way to handle situations like mine?
All the code must run on both linux Mono and .NET.
Thanks for reading, Fabio Iotti.
Comment: It may be little bit overkilling. Which kind of application you're talking about? A service? Something with user interface?
Comment: If there is not user interface (where a close request may wait indefinitely) then I'd simply ask process to terminate (`Process.Close()`) and start waiting (`Process.WaitForExit`). Old version will always serve pending requests before quitting and new version will be pending until done. No need for IPC.
Comment: It's a web server, so a service, a console application whould be fine anyway.
Answer: Kind of a hack, but could you just write to a known file

Application Starts and checks file for other apps
Writes "my id - running" if no app found otherwise write "my id - starting" if another running app is found
running app checks file for any new apps marked as starting"
If a starting app is found stop service and write "my id - shutdown", then terminate app
Starting app will poll file until it finds there are no apps running, then it will start service and write "my id - running"

This looks like a link to a similar issue:
Simple cross-platform process to process communication in Mono?
Comment: Interesting, I'd avoid polling if possible, anyway.
Thanks for the link.


How to spawn a child_process with node.js and save the output to a file

Question: I am trying to spawn a child process /bin/bash from my Node.js app and use use my app stdin, stdout, and stderr for it, and at the same time I want to capture everything entered (stdin) and outputted (stdout) into a file called inAndOut.log.
var spawn = require('child_process').spawn;
var shell = spawn('/bin/bash', ['-l'], {stdio: 'inherit'});

Comment: https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options Here there are examples. Is this what you are looking for?
Comment: you've gotten any luck?
Answer: Create a writable stream, and pipe the stdout to it:
const { spawn } = require('child_process');
const fs = require('fs');

const writeStream = fs.createWriteStream("output-file");
const shell = spawn('/bin/bash', ['-l'], {});

shell.stdout.pipe(writeStream);


Replicate Oracle to MySQL

Question: I am developing a web apps, this web application will be using MySQL.
Now I need to replicate my client's Oracle database into MySQL, only a few tables will be involved.. a table can be up to 2-3 million rows.
I only have SELECT privilege on this Oracle, so don't ask me to install any kind of service on the Oracle machine. I have complete control on the MySQL side however.
The replication is only one way (Oracle to MySQL). I can write a simple script to truncate MySQL table and repopulate it every night but I think this is very inefficient, there must be a better way.
Is there any free tools I can use? Expensive database replication system is definitely out of the question.
Answer: If you are developing a webapp anyways, I would also create a script php/asp/whatever that queries the oracle tables and imports the data that you want into mysql - every X minutes/hours (cron or windows service).
You can keep track of the data that was imported on the last run by saving the last item imported (oracle table unique key ('id')) in a "special" mysql table that will indicate where you left off and the data that now needs to be imported.
Comment: You do realize I still need to truncate MySQL table every time I start the job and start over from 0? The data could have changed, even though the oracle key id is still the same.
Comment: Why are you truncating the MySQL DB? Do you need only that day's data? and after you truncate you reload all the data or just some of it?
Answer: Sounds like you may be limited to truncate and fill by only having SELECT.
By any chance do the records have a timestamp to let you know when they were last changed?
If you don't have any other access then I am not thinking they would be willing to configure something like Oracle's Change Data Capture feature. If they would configure this for you then you could take an initial dump of the table, then incrementally read changes from the associated Change Table, then purge the Change Table when you were done. Very much roll your own. An intro to CDC is here: link .. You would just need select access to the Change Table associated with your Oracle table and ability to purge the changes once you applied them to your database.
A product I used to use ($$$ so I won't go into details but it is an Oracle Golden Gate related product) does a checksum on source and target rows first, then copies across the rows that are different. This takes some processing power to generate the checksums but can greatly reduce the amount of data moved across the wire.

Replicate Oracle to MySQL


Prefix html before radiobutton (input field)

Question: I'm looking to insert some extra html before an input radio button. This is my html now:

        <input type="radio" id="edit-field-geslacht-und-female" name="field_geslacht[und]" value="female" class="form-radio">
        Female

        <input type="radio" id="edit-field-geslacht-und-male" name="field_geslacht[und]" value="male" checked="checked" class="form-radio">
        Male

        <input type="radio" id="edit-field-geslacht-und-other" name="field_geslacht[und]" value="other" class="form-radio">
        Other (transgender...)

Now I would like to add the following html before every radio input:

But how can I do this?
UPDATE:
I've tried to do it with theme_form_element function and now have this:
function theme_form_element($variables)
{
    if($variables['element']['#type'] == 'radio')
    {
        dump($variables);
    }
}

My dump shows me this array:
array (size=1)
  'element' =>
    array (size=27)
      '#type' => string 'radio' (length=5)
      '#title' => string 'Female' (length=6)
      '#return_value' => string 'female' (length=6)
      '#default_value' => string 'male' (length=4)
      '#attributes' =>
        array (size=0)
          empty
      '#parents' =>
        array (size=2)
          0 => string 'field_geslacht' (length=14)
          1 => string 'und' (length=3)
      '#id' => string 'edit-field-geslacht-und-female' (length=30)
      '#ajax' => null
      '#weight' => float 0.001
      '#input' => boolean true
      '#process' =>
        array (size=1)
          0 => string 'ajax_process_form' (length=17)
      '#theme' => string 'radio' (length=5)
      '#theme_wrappers' =>
        array (size=1)
          0 => string 'form_element' (length=12)
      '#title_display' => string 'after' (length=5)
      '#pre_render' =>
        array (size=1)
          0 => string 'ctools_dependent_pre_render' (length=27)
      '#after_build' =>
        array (size=1)
          0 => string 'conditional_fields_element_after_build' (length=38)
      '#defaults_loaded' => boolean true
      '#tree' => boolean true
      '#array_parents' =>
        array (size=3)
          0 => string 'field_geslacht' (length=14)
          1 => string 'und' (length=3)
          2 => string 'female' (length=6)
      '#processed' => boolean true
      '#required' => boolean false
      '#name' => string 'field_geslacht[und]' (length=19)
      '#value' => string 'male' (length=4)
      '#ajax_processed' => boolean false
      '#sorted' => boolean true
      '#after_build_done' => boolean true
      '#children' => string '<input type="radio" id="edit-field-geslacht-und-female" name="field_geslacht[und]" value="female" class="form-radio" />' (length=119)

I've tried to change the #children variable and add my html before it. But when I do that no element on my page is showing ... .
Answer: You can override the theme_radio function in your own theme :
function mytheme_radio($variables) {
  $element = $variables['element'];
  $element['#attributes']['type'] = 'radio';
  element_set_attributes($element, array('id', 'name','#return_value' => 'value'));

  if (isset($element['#return_value']) && $element['#value'] !== FALSE && $element['#value'] == $element['#return_value']) {
    $element['#attributes']['checked'] = 'checked';
  }
  _form_set_class($element, array('form-radio'));

  if ($its_my_field) {
     return '<input' . drupal_attributes($element['#attributes']) . ' />';
  }
  else {
     return '<input' . drupal_attributes($element['#attributes']) . ' />';
  }

Maybe try with the more generic function theme_form_element.
Make sure to add conditions to only affect the form or the element you need like in the example written above.
Comment: I've tried to do it with theme_form_element but didn't work, updated my topic. Could you take a look,
Comment: Can you give me the entire code of your theme_form_element function please ?
Comment: You should have something like this : `$output .= ' ' . $prefix . '' . $element['#children'] . $suffix . "\n";` and `return $output` at the end of your function.
Answer: You can use jQuery selector -

        <input type="radio" id="edit-field-geslacht-und-female" name="field_geslacht[und]" value="female" class="form-radio">
        Female

        <input type="radio" id="edit-field-geslacht-und-male" name="field_geslacht[und]" value="male" checked="checked" class="form-radio">
        Male

        <input type="radio" id="edit-field-geslacht-und-other" name="field_geslacht[und]" value="other" class="form-radio">
        Other (transgender...)