java clearing internal memory cache

Question: I am trying to test a Java program that I wrote, in order to prove its efficiency. I have many different tests to run, and I am trying to be precise in the runtime analysis. The problem is that different tests may access information on the disk that is common. So, in order to be "fair" in my experimental results I would like to somehow programmatically clear the internal memory being used by my Java program, in between experiments. In other words, I want each experiment to have the same "empty memory/cache".
I tried reading in a large file in between experiments. I also tried restarting my machine. Interestingly, the times are much much worse when I read in a large file, then when I simply restart the machine (say 40 sec to 5 sec). What is the correct way to clear internal memory (i.e., avoid the artificial speedup for experiments from common disk accesses) beyond restarting my machine between each experiment, which is not feasible?
Comment: But why do you think an empty disk cache is a realistic scenario? You'd just be testing the very rare worst case.
Comment: Thanks. I agree that I am testing the worst case, and that in practice, things will be better. But, I would still like to show that even in that worst case scenario, my program works well. (I dont want the referees of the paper I am writing about my system to have an excuse to complain about the cache issue.)
Comment: There's no Java API for this, but there might possibly be a OS specific way to do it. So which operating system are you using ?
Comment: See http://stackoverflow.com/questions/478340/clear-file-cache-to-repeat-performance-testing for a few options.
Comment: This is hard.  Modern operating systems cache files and directory pages that have been recently used, and from what I've read controlling this in Windows is difficult at best.  And even if the OS doesn't cache a file, the disk drive may.
Comment: It should be noted that some Java implementations *do* cache files in the sense that if a class in loaded in one Java process the file image may be "shared" with other Java processes that come along later.  But I don't think any standard Windows JVM does this -- it's mainly for big server setups.
Answer:
What is the correct way to clear internal memory

Restart the computer. As you say this might not be the slowest option.
Computers are design to be efficient as possible, making them deliberately inefficient means making some assumption about how inefficient you want to make it, so there is no standard way of doing this.
One way of clearing the cache for files you have written is to delete them. If you want to clear the cache for file in disk you want to keep, you can write a large set of files, larger than your main memory and delete them.
If you are using Windows it doesn't use all the memory to cache files in any case.  What you can do is to write large files of a few hundred MB at a time and time how long this takes.  When the cache is exhausted you see a sudden jump in the time it takes from Java and you know at this point you have probably clearer the cache. After this happens, delete them and your cache is likely to be empty as a result.
This can help you get reproducible results but this may not be exactly the same as rebooting.  Note: unless you expect to reboot your computer every time you run your application, the time it takes after a reboot may not be meaningful anyway.
BTW: The disk cache you should be worrying about the in the OS.  Java doesn't cache files except in buffers you create and in the code you write so these should be under your control.

java clearing internal memory cache


OpenCL to OpenGL texture problems

Question: I'm trying to use OpenCL to draw to a cl_image that I got from a OpenGL texture and then render that texture.  The problem is when I run my code on CL_DEVICE_TYPE_CPU it works fine however when I run on CL_DEVICE_TYPE_GPU the texture appears to be some random pixels.  I'm new to OpenCL and not sure what's goin on so I'll post code below, also using OpenCL on OSX.
Host Code:
<code>#import "GLView.h"
#import <GLKit/GLKit.h>
#import <OpenCL/OpenCL.h>
#import "kernel.cl.h"

#define WIDTH 500
#define HEIGHT 500

static GLfloat squareVertexData[] =
{
    -0.5f, -0.5f, 0.0f,     0.0f, 0.0f,
    0.5f, -0.5f, 0.0f,      1.0f, 0.0f,
    0.5f, 0.5f, 0.0f,       1.0f, 1.0f,
    -0.5f, -0.5f, 0.0f,     0.0f, 0.0f,
    0.5f, 0.5f, 0.0f,       1.0f, 1.0f,
    -0.5f, 0.5f, 0.0f,      0.0f, 1.0f
};

@interface GLView ()

@property (strong, nonatomic) GLKBaseEffect *effect;
@property (nonatomic) GLuint vertexArray;
@property (nonatomic) GLuint vertexBuffer;
@property (nonatomic) GLuint tex;
@property (nonatomic) cl_image clImage;
@property (nonatomic) dispatch_queue_t queue;

@end

@implementation GLView

#pragma mark - Lifecycle methods

- (void)awakeFromNib
{
    NSOpenGLPixelFormatAttribute attrs[] =
    {
        NSOpenGLPFADoubleBuffer,
        NSOpenGLPFAOpenGLProfile,
        NSOpenGLProfileVersion3_2Core,
        NSOpenGLPFAColorSize, 8,
        NSOpenGLPFAAlphaSize, 8,
        0
    };

    NSOpenGLPixelFormat *pixelFormat = [[NSOpenGLPixelFormat alloc] initWithAttributes:attrs];
    [self setPixelFormat:pixelFormat];

    NSOpenGLContext *context = [[NSOpenGLContext alloc] initWithFormat:pixelFormat shareContext:nil];
    CGLEnable([context CGLContextObj], kCGLCECrashOnRemovedFunctions);
    [self setOpenGLContext:context];
}

- (void)dealloc
{
    [self.openGLContext makeCurrentContext];

    glDeleteBuffers(1, &_vertexBuffer);
    glDeleteVertexArrays(1, &_vertexArray);

    glDeleteTextures(1, &_tex);
    gcl_release_image(self.clImage);
}

#pragma mark - NSOpenGLView methods

- (void)prepareOpenGL
{
    GLint swapInterval = 1;
    [self.openGLContext setValues:&swapInterval forParameter:NSOpenGLCPSwapInterval];

    glGenVertexArrays(1, &_vertexArray);
    glBindVertexArray(self.vertexArray);

    glGenBuffers(1, &_vertexBuffer);
    glBindBuffer(GL_ARRAY_BUFFER, _vertexBuffer);
    glBufferData(GL_ARRAY_BUFFER, sizeof(squareVertexData), squareVertexData, GL_STATIC_DRAW);

    glEnableVertexAttribArray(GLKVertexAttribPosition);
    glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 20, (char *)NULL + (0));

    glEnableVertexAttribArray(GLKVertexAttribTexCoord0);
    glVertexAttribPointer(GLKVertexAttribTexCoord0, 2, GL_FLOAT, GL_FALSE, 20, (char *)NULL + (12));

    glBindVertexArray(0);

    self.effect = [[GLKBaseEffect alloc] init];
    self.effect.transform.projectionMatrix = GLKMatrix4MakeOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f);

    glGenTextures(1, &_tex);
    glBindTexture(GL_TEXTURE_2D, self.tex);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, WIDTH, HEIGHT, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);

    self.effect.texture2d0.name = self.tex;

    CGLShareGroupObj sharegroup = CGLGetShareGroup(self.openGLContext.CGLContextObj);
    gcl_gl_set_sharegroup(sharegroup);

    self.clImage = gcl_gl_create_image_from_texture(GL_TEXTURE_2D, 0, self.tex);

    self.queue = gcl_create_dispatch_queue(CL_DEVICE_TYPE_GPU, NULL);

    dispatch_sync(self.queue, ^{

        cl_ndrange range = {
            2,
            { 0 },
            { WIDTH, HEIGHT },
            { 0 }
        };

        mykernel_kernel(&range, self.clImage);
    });

    glClearColor(0.65f, 0.65f, 0.65f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    glBindVertexArray(self.vertexArray);

    [self.effect prepareToDraw];

    glDrawArrays(GL_TRIANGLES, 0, 6);

    glBindVertexArray(0);

    [self.openGLContext flushBuffer];
}

@end
</code>
Kernel Code:
<code>kernel void mykernel(write_only image2d_t image)
{
    size_t x = get_global_id(0);
    size_t y = get_global_id(1);

    uint4 color;
    color.x = 255;
    color.w = 255;

    write_imageui(image, (int2) (x,y), color);
}
</code>
Comment: Where you assign the values to 'color' in the kernel, is there a reason why you only set the 'x' and 'w' components?
Comment: no reason, just testing to see if it works, i see a red texture when on cpu, but weird random looking noise on gpu
Comment: Do you also get the random noise if you do set all components? Perhaps not setting two of them can be the source of some randomness? The kernel itself looks fine otherwise, very similar to one that I've used for testing.
Comment: I set them all now to 255, and when I run on CPU I see white texture, however on GPU it's always black, really weird what's happening
Comment: The only other thing I can suggest is trying to use GL_NEAREST instead of GL_LINEAR
Comment: Nah it's still happening, thanks anyways, I'll sleep on it for now and try again tomorrow, I think I'll try with PBO's instead of textures
Comment: I would suggest finding one of the many CL/GL interop examples and verify it runs correctly on your hardware, and then adjust your code to be closer to the example.
Answer: I'm not clear why you're asking for:
<code>    NSOpenGLPFAColorSize, 8,
</code>
This seems like not a lot of color. Possibly it'd work better if you ask for 32-bits (8 per pixel, plus alpha at end?):
<code>    NSOpenGLPFAColorSize, 32,
</code>
That's the default created by Scene Kit, anyhow.

OpenCL to OpenGL texture problems


Is there a way to quickly copy the same ordering and layout from manage fields to manage displays?

Question: I have a content type I made in Drupal that has over 50+ fields and all sorts of fieldsets, divs, and horizontal tabs.  I got the Manage Fields ordering and look to be what I want, but the manage display is all out of order and missing some field groups like the divs, fieldsets, and horizontal tabs.  I would need to add them all back in one-by-one!  Is there a quick way I can just click a button that would automatically take the existing layout from "Manage Fields" and use that for "Manage Display" for a content type?
Answer: Since this has been posted for more than a week and nobody has responded, then I am going to assume that there is no quick and easy way.  I just ended up recreating the display by hand.  Oh well.


how to repeat a vxworks 6.9 shell command in the shell interpreter

Question: Is there a single key to step through past vxworks v6.9 shell interpreter commands, like  for Windows and Linux?
'h' shows a list of command history.
How can I step through this and select?
Answer: It uses the VI key bindings by default.
<code>esc-K  - go back in history
esc-J  - go forward in history
</code>


Passing item data to PayPal with ActiveMerchant (payments pro)

Question: I'm hoping somebody can help me with this issue that's got me stuck.
I'm using ActiveMerchant in my rails app to process cart transactions with the Paypal gateway.
The transactions are successful, but when I look at the history in my PayPal account, there is no information about the items associated with the transaction. Occasionally, one will come through with the item data, but most of the time, no items show up in the details.
Here's what's happening in my checkout controller:
def process_order
    @items = Array.new

    @donations.each do |d|
      item = Hash.new
      item[:name] = d.project.title
      item[:quantity] = 1
      item[:description] = "Donation from website"
      item[:amount] = (d.pledge.to_i*100).round
      @items << item
    end

    @response = GATEWAY.purchase(price_in_cents, credit_card, purchase_options)
    if @response.success?
      @validTransaction = true
    end
end

private

 def purchase_options
  {
  :items => @items,
  :ip => request.remote_ip,
  :billing_address => {
    :name     => params[:first_name],
    :address1 => params[:last_name],
    :city     => params[:city],
    :state    => params[:state],
    :country  => params[:country],
    :zip      => params[:zip]
    }
  }
end

def validate_card
  unless credit_card.valid?
    credit_card.errors.full_messages.each do |message|
      @errors += message + "
"
    end
  end
end

def credit_card
  @credit_card ||= ActiveMerchant::Billing::CreditCard.new(
  :brand               => params[:card_type],
  :number             => params[:card_number],
  :verification_value => params[:card_verification],
  :month              => params[:date][:month],
  :year               => params[:date][:year],
  :first_name         => params[:first_name],
  :last_name          => params[:last_name]
  )
end

I think that's all the pertinent stuff there, so if anyone can see why this wouldn't work, it'd be much appreciated.
Thanks!
Answer: I've had a problem similar to this, and it was because I was passing HTML code into the "description" value (my descriptions store HTML code). I got around it by replacing the HTML-tagged description with my own static one and it worked fine.
You might want to test with more static data to see what is causing the problem specifically, if you haven't fixed the problem already


How to create all possible XMLs?

Question: How can I create all possible XMLs of these XSD:
<code><?xml version="1.0" encoding="utf-8"?>
<xsd:schema xmlns="http://beep2000/client/beep.de" xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://beep2000/client/beep.de">
    <xsd:complexType name="TextType">
        <xsd:sequence>
            <xsd:element name="Text" minOccurs=1">
                <xsd:simpleType>
                    <xsd:restriction base="xsd:string">
                        <xsd:maxLength value="255"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="StreetType">
        <xsd:sequence>
            <xsd:element name="Street" type="xsd:string" minOccurs="1"/>
            <xsd:element name="HouseNumber" type="xsd:int" minOccurs="1"/>
            <xsd:element name="Suffix" type="xsd:string" minOccurs="1">
                <xsd:simpleType>
                    <xsd:restriction base="xsd:string">
                        <xsd:maxLength value="1"/>
                        <xsd:enumeration value="A"/>
                        <xsd:enumeration value="B"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="BodyType">
        <xsd:sequence>
            <xsd:element name="FirstPart" type="TextType" minOccurs="1"/>
            <xsd:element name="SecondPart" type="TextType" minOccurs="0"/>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="AdressType">
        <xsd:sequence>
            <xsd:element name="Street" type="StreetType" minOccurs="0">
            <xsd:element name="PostalCode" minOccurs="0">
            <xsd:simpleType>
                <xsd:restriction base="xsd:string">
                    <xsd:maxLength value="5"/>
                    <xsd:pattern value="[0-9]{5}"/>
                </xsd:restriction>
            </xsd:simpleType>
            <xsd:element name="City" type="xsd:string" minOccurs="1"/>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="RequestEnvelope">
        <xsd:sequence>
            <xsd:element name="Title" type="xsd:string" minOccurs="0"/>
            <xsd:element name="Forename" type="xsd:string" minOccurs="1"/>
            <xsd:element name="Surname" type="xsd:string" minOccurs="0"/>
            <xsd:element name="PersonalID" minOccurs="0">
                <xsd:simpleType>
                    <xsd:restriction base="xsd:string">
                        <xsd:maxLength value="20"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:element name="Adress" type="AdressType" minOccurs="0"/>
            <xsd:element name="Body" type="BodyType" minOccurs="0"/>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="ResponseEnvelope">
        <xsd:sequence>
            <xsd:element name="Result" minOccurs="1">
                <xsd:simpleType>
                    <xsd:restriction base="xsd:string">
                        <xsd:maxLength value="3"/>
                        <xsd:enumeration value="OK"/>
                        <xsd:enumeration value="NOK"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:element name="Errorcode" type="xsd:integer" minOccurs="0"/>
            <xsd:element name="Errortext" minOccurs="0">
                <xsd:simpleType>
                    <xsd:restriction base="xsd:string">
                        <xsd:maxLength value="255"/>
                    </xsd:restriction>
                </xsd:simpleType>
            </xsd:element>
            <xsd:element name="Comment" type="TextType" minOccurs="0"/>
        </xsd:sequence>
    </xsd:complexType>
    <xsd:element name="MsgResponse" type="ResponseEnvelope"/>
    <xsd:element name="MsgRequest" type="RequestEnvelope"/>
</xsd:schema>
</code>
There are some fields which are optional and some field which are mandatory. So there is a variety of XMLs. How can I find all possible structures of XMLs?
I expect something like:
XML No.1:

Field 1 --> mandatory
Field 2 --> mandatory
Field 3 --> optional
......

XML No.2.:

Field 1 --> mandatory
Field 2 --> mandatory
Field 4 --> mandatory
.....
Comment: You are aware, that this are infinitely many?
Comment: A xsd (schema) is xml format so any xml parsing method will work.  The optional fields are the ones with minOccurs=0.
Comment: I know that  minOccurs = 0 signalises that this field is optional but I what I need is to find a way to find out how to create all possible XML structures ?!
Answer: Generating all possible XML files that conform to this schema would take an infinite amount of time and they would occupy an infinite amount of space.
However, you can generate a selection of sample documents using tools such as those described here:
How to generate sample XML documents from their DTD or XSD?
Comment: Sorry but this does not help me. Why should it take an infinite amount of time? I do not understand it...
Comment: There's an infinite number of documents that conform to any non-trivial schema, and you said you wanted them all.

How to create all possible XMLs?


How to transform this probability equation?

Question: This formula is taken from the book Mathematics for Machine Learning (p. 177, Formula 6.8). I don't know how to transform from the left to the right side. I searched for probabilities rules, I looked through the symbols and I totally don't understand.
$$
P_X(S) = P(X \epsilon S) = P(X^{-1}(S)) = P(\{\omega \space \epsilon \space \Omega : X(\omega) \varepsilon S \})
$$
Comment: These are all notational manipulations originating in the basic language of mathematics (set theory, mostly).  The first can be taken as a definition of the "$P_X$" notation, while the others explain what "$X\in S$" might mean and then define$X^{-1}(S).$  None of them has anything to do with rules of probability. If your book doesn't define this notation, then a good reference would be an elementary book on set theory, such as that by Paul Halmos.  You don't need to learn set theory: the notation will be defined within the first few pages.
Comment: A quick skim of that book indicates nobody is going to learn mathematics from it--too much is assumed to be a "review" and a great deal goes either undefined or badly defined.  If you must use it, then consider supplementing it with introductory textbooks on relevant subjects.
Comment: @whuber: For your information, I notice that the book has an [issues page](https://github.com/mml-book/mml-book.github.io/issues) where you can post your comments, especially if you believe the book is failing in its goal to "motivat[e] people to learn mathematical concepts" for machine learning.
Comment: Thank you PeterO. and whuber for your guidance!
Answer: In order to understand this equation you will need to go back to the definition of a probability measure in measure theory.  The book you are working from is not a good introductory book for that purpose, so I recommend you start by reading an introductory text on probability and measure.  In any case, I will give you an explanation of how this equation comes about.$^\dagger$  In this explanation I will refer to "measureable" sets, for purposes of rigor, but you can ignore that part until you know more about measure theory.

Explaining the equation: The symbol $P$ in this equation is referring to a probability measure on the abstract sample space $\Omega$.  We call the elements $\omega \in \Omega$ "outcomes" in the sample space and we refer to (measureable) sets of these outcomes as "events".  The probability measure takes an event and tells us its probability.
Now, in most applications of probability theory, we want to work with random numbers, and to do this, we label each of the outcomes in the sample space by a number.  In the present case, $X(\omega)$ is the number ascribed to the outcome $\omega$ in the sample abstract space.  Suppose you want to find the probability that the number $X$ falls within some particular set of numbers.  To do this, we define a new probability operator $P_X$ which takes in a set of numbers instead of a set of outcomes in the abstract sample space.  This is really a definition of the new probability measure:
$$P_X(\mathcal{S}) \equiv P(X \in \mathcal{S})
\quad \quad \quad \text{for all measureable } \mathcal{S}.$$
Since $X: \Omega \rightarrow \mathbb{R}$ is a function mapping the outcomes in the abstract sample space to the real numbers, the pre-image $X^{-1}$ for this function is:
$$X^{-1}(\mathcal{S}) \equiv \{ \omega \in \Omega | X(\omega) \in \mathcal{S} \}.$$
Using standard properties of the pre-image, we have the event equivalence:
$$X(\omega) \in \mathcal{S}
\quad \quad \iff \quad \quad
\omega \in X^{-1}(\mathcal{S}).$$
Putting all these things together means that for any measureable set $\mathcal{S}$ we have:
$$P_X(\mathcal{S})
= P(X \in \mathcal{S})
= P(X^{-1}(\mathcal{S}))
= P(\{ \omega \in \Omega | X(\omega) \in \mathcal{S} \}).$$
This final equation essentially just says that the probability that $X$ is in the set $\mathcal{S}$ (using the probability measure $P_X$) is the same as the probability that we have an $\omega$ with $X(\omega) \in \mathcal{S}$ (using the probability measure $P$).  The purpose of the equation is to show how the measure $P_X$ relates back to the foundational probability measure $P$.

$^\dagger$ I don't really like the kind of notation used in the book, so I'm going to use some slightly different symbols/fonts for this equation; it is the same thing in slightly different notation.


What is the probability of rolling an even number and a number greater than 8 for two rolled die?

Question: This is one of the questions asked in the swirl package of R. One of my students however, has pointed out that the correct option of this question is the wrong answer.
His assertion is that the correct probability is:
18/36 + 10/36 - 6/36 = 22/36
Instead of,
18/36 + 10/36 - 4/36 = 24/36
His point seems correct, since the last term for intersection should be 6/36 buts its showing as 4/36 as the correct option.
Comment: Are you sure your title correctly reflects the question?  The answers would lead one to suspect the question actually was about the event "the sum of the dice is even **or** exceeds 8".
Comment: Reference is to the event for getting an even number and at the same time a number exceeding 8 in the first throw itself.
Comment: Then neither of the answers presented in this question comes even close to giving the correct probability.
Answer: Why should it be 6/36? There are only four events in the interaction set, $(6, 6), (5, 5), (6, 4), (4,6)$.
You can check quickly by
<code>d1=sample(1:6, 10000, replace=T); d2=sample(1:6, 10000, replace=T)
sum((d1+d2) %% 2 == 0 & d1+d2 > 8) / 10000

> [1] 0.106

6/36
> [1] 0.1666667
4/36
> [1] 0.1111111
</code>
Answer: The student has incorrectly calculated the answer to the question

What is the probability that the sum is even or greater than $8$?

using the formula  $P(A\cup B) = P(A)+P(B)-P(A\cap B)$ where $A$ is the event that the sum is even ($18$ outcomes) and $B$ the event that the sum is greater than $8$ ($10$ outcomes). As juod's answer points out, the event $A\cap B$ has only $4$ outcomes in it, not $6$ as the student contends and the OP concurs.  Are they possibly thinking of the set
$A\cap B$ as being
$$\left\{(6,6), (6,5), (5,6), (5,5), (6,4), (4,6)\right \} ??$$
What the swirl manual has is the perfectly correct answer to the question highlighted above. It would be interesting to find out what question the swirl manual is answering. Is it possible that the OP mistyped the question actually asked in the swirl manual, replacing the or by an and in the title?
Comment: This answer would fit our site's model much better if it were to confine its remarks to the question rather than impugning the capabilities of the OP.  Please reconsider whether you really want this post to remain up.
Comment: It is easy to enumerate the number of possible answers which are the 4 possibilities listed in both answers.  Neither answer suggested in the question is correct.
Comment: @whuber I have edited the post, and would like to have it remain up in its present version. If you feel that it is still inappropriate, please delete it as you have the power to do.
Comment: Thanks, but I needed a clarification to clear out the confusion. I would want to delete the post.
Comment: @MichaelChernick The intersection set suggested by the student was wrong, hence the correction.
Comment: Shiv, I believe @Michael's point is not that the student's suggestion was wrong, but that *both* the student's and the other option are grossly incorrect.  That is what led me to suggest in a separate comment that you might not have correctly described the original question.


Auto add shipping line item

Question: I want to auto add flat rate shipping cost to any order and display it in the shopping cart summary.
I am using Commerce Shipping and Commerce Flat Rate modules. I can't know how to implement it using rules.
I found this post but it didn't help me.
what I need is to automatically add $10 on every order for shipping, I will disable the Shipping Service Panel.
Answer: You can do this with Rules. Here is an example:
<code>{
    "rules_auto_add_shipping_service" : {
      "LABEL" : "Auto-add shipping service",
      "PLUGIN" : "reaction rule",
      "OWNER" : "rules",
      "REQUIRES" : [ "rules", "commerce_order", "commerce_shipping", "entity" ],
      "ON" : { "commerce_order_update" : [] },
      "IF" : [
        { "data_is" : { "data" : [ "commerce-order:type" ], "value" : "commerce_order" } },
        { "commerce_order_contains_product" : {
            "commerce_order" : [ "commerce_order" ],
            "product_id" : "YOUR-SKU-HERE",
            "operator" : "\u003E=",
            "value" : "1"
          }
        }
      ],
      "DO" : [
        { "commerce_shipping_delete_shipping_line_items" : { "commerce_order" : [ "commerce_order" ] } },
        { "commerce_shipping_method_collect_rates" : {
            "shipping_method_name" : "flat_rate",
            "commerce_order" : [ "commerce_order" ]
          }
        },
        { "commerce_shipping_rate_apply" : {
            "commerce_order" : [ "commerce_order" ],
            "shipping_service_name" : [ "" ]
          }
        }
      ]
    }
}
</code>
Just change the SKU or the other conditions to suit your needs.
Answer: You should be able to do this out of the box with FLAT RATE.
Just add your new FLAT RATE for $10 (yoursite.com/admin/commerce/config/shipping then click ADD NEW FLAT RATE) and it will show up in the shipping pane already selected when checking out.
It will also create a new line item on the cart summary for SHIPPING.
This will add a $10 flat rate shipping method to every order
 - if this is not what you are looking for please clarify, hope this helps.

Auto add shipping line item


Is a sample i.i.d or is a collection of random variables i.i.d.?

Question: Basic terminology question.
I hear "let the sample be i.i.d." and "let these random variables be i.i.d." being used interchangeably.
Even Wikipedia uses both:

A collection of random variables is independent and identically distributed if...

vs.

It is commonly assumed that observations in a sample are effectively i.i.d.

Are there different nuances or are they equivalent?
Also, are the points in a sample regarded as repeated measurements of one random variable, or one measurement each of multiple i.i.d. random variables?
Answer: From Wikipedia, two Random Variables (RVs) (remark: you can generalize this to any number of RVs) are independent and identically distributed (i.i.d.) if their Cumulative Distribution Function (CDF) is the same for any element of the domain $I$ and if their joint CDF factorizes in the product of the marginal CDFs. This means that:
$${\begin{aligned}&F_{X}(x)=F_{Y}(x)\,&\forall x\in I\\&F_{X,Y}(x,y)=F_{X}(x)\cdot F_{Y}(y)\,&\forall x,y\in I\end{aligned}}$$
(Note that this also imply that their pdfs are the same (almost everywhere, i.e. on the whole domain except for sets of measure zero, but this is a technical condition so don't worry about it)
Realizations of an RV are usually referred to as samples, i.e. roughly speaking their outcome. The assumption that samples generated by an RV are i.i.d. simply refers to the fact that underlying RVs, whose realizations you observe in the samples, are i.i.d.
So replying to your questions:

they are essentially the same thing.

You can regard it as repeated measurements of one random variable since the CDFs of two i.i.d. RVs are the same.
Comment: You absolutely *can* regard it as repeated measurements of the same random variable, but in the mathematical formulations I think it's more common to regard them as a different random variable each time.


Intercept QMail SMTP emails

Question: Here is the situation, we have a linux web server that runs Plesk and I need a way to intercept emails sent to and from the server. Plesk uses qmail to send and receive emails.
We have sorted the receiving emails problem by adding a mailgroup to each email address that includes an "archive@" address. Then we use a PHP script to read the "archive@" mailbox and puts the emails into a MySQL database.
My problem at the moment is how to capture the messages sent from the server. Has anyone had any experience in doing this?
Thanks
Peter
Answer: this can be done via QUEUE_EXTRA (http://www.google.com/search?q=queue_extra), for which you need to recompile qmail. some hints on doing this on plesk here (http://translate.google.com/translate?hl=en&sl=pt&u=http://www.maglione.com.br/artigos/artigostecnicos/pleskqextrahowto).

Intercept QMail SMTP emails


Paginação utilizando PHP

Question: Fiz uma paginação utilizando PHP que está funcionando perfeitamente, porém, ao chegar na penúltima página(10), a última página(11) ela vem como resultado "solto" e não vem estilizada como se fosse um "bug" Segue a imagem com o erro:

Segue o código:

                <?php
            $pagina_atual = (!isset($_GET['p']))? 1 : $_GET['p'];
            $max_results = 1;
            $prev = ($pagina_atual - 1);
            $next = ($pagina_atual + 1);
            $total_pages = ceil($paginas / $max_results);
            $pagination = '';
            if($paginas > 1) {
                $pagination .= '
«
 ';
            }

            for($i = max(1, $pagina_atual - 4); $i <= min($pagina_atual + 4, $total_pages); $i++) {
                if(($paginas) == $i)
                {
                    $pagination .= $i;

                }
                else
                {
                if($pagina_atual == $i) {
                        $pagination .='
'.$i.'
';
                } else {
                        $pagination .= '
'.$i.'
';
                    }
                }
            }
            if($pagina_atual < $total_pages)
            {
                $pagination .= '
»
';
            }
            if($next == $i) {
                $pagination .= '
'.$i.'
';
            }

            echo $pagination;

            ?>

Comment: Tente enumerar o For a partir do zero, já que 0 é um index.
Answer: Você deve começar a enumerar as suas paginas a partir do zero(Somando +1 para a exibição).
O if para exibir o botão da pagina anterior precisa de mais uma regra para não exibir quando estiver na primeira pagina.
O primeiro if de dentro do for não é necessário, sendo que você só possui dois casos, pagina ativa e não ativa.
O ultimo if não é necessário pois nunca será chamado.
$pagina_atual = (!isset($_GET['p'])) ? 1 : $_GET['p'];
$max_results = 1;
$prev = ($pagina_atual - 1);
$next = ($pagina_atual + 1);
$total_pages = ceil($paginas / $max_results);
$pagination = '';

if ($paginas > 1 && $pagina_atual > 1) {
    $pagination .= '
«
 ';
}

for ($i = max(0, $pagina_atual - 4); $i < min($pagina_atual + 4, $total_pages); $i++) {
    $page = $i + 1;

    if ($pagina_atual == $page) {
        $pagination .= '
' . $page . ' ativo
';
    } else {
        $pagination .= '
' . $page . '
';
    }
}
if ($pagina_atual < $total_pages) {
    $pagination .= '
»
';
}

Comment: Continua com mesmo erro ;/ Aparece o ultimo número.
Comment: @matheus teste novamente, veja se não esqueceu de algo, verifiquei o código linha a linha e está ok

Paginação utilizando PHP


Google BigQuery Return Users Count

Question: I am trying to get number of return users from Google Big Query to match with what I have in GA.
The query I wrote is:
SELECT
  SUM(CASE WHEN totals.newVisits IS NULL THEN 1 ELSE 0 END) AS Return_Visitors
FROM `table`
WHERE date='20161109'

But I got more return users than what I got from GA.
I found the the definition of totals.newVisits is "Total number of new users in session (for convenience). If this is the first visit, this value is 1, otherwise it is null."  That's why I used when totals.newVisits is null.
Is my logic right?
Thank you
Answer: Does it work for you?
select
SUM(CASE WHEN totals.newVisits IS NULL THEN 1 ELSE 0 END) AS Return_Visitors
FROM `table`
where totals.visits = 1
and date = '20161109'

Comment: Hi Will, unfortunately, it does not work for me. I guess when the totals.visitor equals to 1, it means it's the first time visit. it's not a return_visitor. Do you have other thoughts? Thank you
Comment: Just now I realized that I wrote a different summation from yours. I just fixed that. Could you please try the query again? I used the flag `totals.visits = 1` because that's the only difference between G.A data and BigQuery. In G.A, there's a filter to consider only `total.visits = 1` so that's why we usually see less data there.
Comment: Thank you so much Will! The solution solved my problem!

Google BigQuery Return Users Count


Match a string with comma-separated sub strings, ensuring no leading/trailing commas, with regex in Kotlin

Question: There are many variations on this type of question, but none that I could see match exactly what I need - apologies if I've missed a different answer.
As part of a unit test, I want to test whether a given string will match a regex pattern. The string will usually take a comma-separated form using the same set of substrings in the same order:
accounts,leads,contacts,opportunities
However, sometimes some of the substrings might not be present. For example, the string might be:
accounts,contacts
Or:
leads,opportunities
The string could also be empty.
What I want is a regex pattern that will always match variations on this string and confirm that there are no trailing or leading commas around it. So far I have:
"^[^,]?(accounts,?)?(leads,?)?(contacts,?)?(opportunities)?[^,]?$".toRegex()

But this doesn't work, as the regex will still accept one trailing space or comma, or letters other than commas around it. I'm not great at Regex, so any help is appreciated.
Examples of what needs to match:
accounts,leads,contacts,opportunities // match
<emptystring> // match
accounts,opportunities // match, substrings are in correct order
leads,opportunities // match, substrings are in correct order
leads,accounts,contacts // no match, wrong order of substrings
accounts,leads,contacts, // no match, trailing comma
,accounts,leads,contacts // no match, leading comma
apples,bananas,contacts // no match, invalid substrings

Comment: You want `^(?:accounts)?(?:,leads)?(?:,contacts)?(?:,opportunities)?$`, don't you? See [this demo](https://regex101.com/r/IaOFrq/1).
Answer: You need to use
^(?:accounts)?(?:(?:,|^)leads)?(?:(?:,|^)contacts)?(?:(?:,|^)opportunities)?$

See the regex demo. This is basically a chain of optional groups in a specified order:

^ - start of a string
(?:accounts)? - optional accounts string
(?:(?:,|^)leads)? - an optional , or start of string and then leads string
(?:(?:,|^)contacts)?  - an optional  , or start of string and then a contacts string
(?:(?:,|^)opportunities)? - an optional  , or start of string and then an opportunities string
$ - end of string.
Comment: This doesn't seem to work if the string doesn't start with 'accounts'. I want the regex to also match 'leads,opportunities', for example (I'll add this to my list of examples). Any ideas on how that would look?
Comment: @DrChoppy Yes, see an update.


Python script that reads a Json and converts it to a JavaScript

Question: I have a JSON that I can catch her from the Web or download on my Linux... And I need to assemble a standardized code from this JSON.
I need to assemble a JavaScript always standard with as modifications that come from my JSON.
I need to change two values in my JavaScript.
Today currently works right, plus I that manual mount, wanted to assemble this automatica using Python or JavaScript.
My JSON:
{
    "home": [
        {
            "base_url": "home",
            "redirect_url": "/",
            "type": "www.testemestradoolpt.com.br"
        }
    ],
    "portal": [
        {
            "base_url": "portal",
            "redirect_url": "/",
            "type": "www.testemestradoolpt.com.br"
        }
    ],
    "digital": [
        {
            "base_url": "digital",
            "redirect_url": "/",
            "type": "www.testemestradoolpt.com.br"
        }

    ]
}

My Java Script:

'use strict';

const querystring = require('querystring');

 exports.handler = (event, context, callback) => {
     const request = event.Records[0].cf.request;
     const headers = request.headers;
     const uri = request.uri;

     var origin = request.origin[0].custom.domainName;
     console.log(`Request #PATH# is "${uri}"`);
     console.log(`Request #INCOMING REQUEST ORIGIN# set to "${origin}"`);

     const regex = /^\/(digital|home|portal|)[^\/]*/i;  // **<<<< HERE**
     if (regex.test(request.uri))
     {
             /* Set custom origin fields*/
             request.origin = {
                 custom: {
                     domainName: 'www.testemestradoolpt.com.br' // **<< HERE**
                 }
             };
             request.headers['host'] = [{ key: 'host', value: '*'}];
             /**origin = request.origin.custom.domainName;  enable it for testing*/
    }
    /** for testing purposes don't use unless you need to, big performance hit
    console.log(`###### - > Request HOST "${request.headers.host[0].value}"`);
    console.log(`###### - > Final Request Origin Server "${origin}"`);
    */
    callback(null, request);
};

If you notice I just need to add my base_url that comes from my JSON in the regular JavaScript expression field and only once the type that and the URL in the other field, the URL will always be the same.
I made a python script that brings me the base URL, more I need a python script that mounts to min the Javascript with the variables of the JSON as the Javascript above I sent, and I'm having a lot of trouble doing that , follow what I could do in Python:
import json

with open('dataload.json') as json_file:
    data = json.load(json_file)
    for p in data ['home']:
         print(p['base_url'])

Comment: have you tried using [string.Template](https://docs.python.org/3.7/library/string.html#template-strings)?
Comment: no,I am with the doubt PQ I have never programmed in Python.
Answer: You can use string.Template to easily substitute values in a string,
i am not entirely sure what parts of the json you want to use to fill the template. below is one approach.
import json
from string import Template

# load the template
with open("template.js", "rt") as template_file:
    js_template = Template(template_file.read())

with open('dataload.json') as json_file:
    data = json.load(json_file)

# select data
base = data["home"][0]["base_url"]
url = data["home"][0]["type"]

# fill the template with the selected data
print(js_template.safe_substitute(base=base, url=url))

template.js
'use strict';

const querystring = require('querystring');

 exports.handler = (event, context, callback) => {
     const request = event.Records[0].cf.request;
     const headers = request.headers;
     const uri = request.uri;

     var origin = request.origin[0].custom.domainName;
     console.log(`Request #PATH# is "${uri}"`);
     console.log(`Request #INCOMING REQUEST ORIGIN# set to "${origin}"`);

     const regex = /^\/($base)[^\/]*/i;  // **<<<< HERE**
     if (regex.test(request.uri))
     {
             /* Set custom origin fields*/
             request.origin = {
                 custom: {
                     domainName: '$url' // **<< HERE**
                 }
             };
             request.headers['host'] = [{ key: 'host', value: '*'}];
             /**origin = request.origin.custom.domainName;  enable it for testing*/
    }
    /** for testing purposes don't use unless you need to, big performance hit
    console.log(`###### - > Request HOST "${request.headers.host[0].value}"`);
    console.log(`###### - > Final Request Origin Server "${origin}"`);
    */
    callback(null, request);
};

output
'use strict';

const querystring = require('querystring');

 exports.handler = (event, context, callback) => {
     const request = event.Records[0].cf.request;
     const headers = request.headers;
     const uri = request.uri;

     var origin = request.origin[0].custom.domainName;
     console.log(`Request #PATH# is "${uri}"`);
     console.log(`Request #INCOMING REQUEST ORIGIN# set to "${origin}"`);

     const regex = /^\/(home)[^\/]*/i;  // **<<<< HERE**
     if (regex.test(request.uri))
     {
             /* Set custom origin fields*/
             request.origin = {
                 custom: {
                     domainName: 'www.testemestradoolpt.com.br' // **<< HERE**
                 }
             };
             request.headers['host'] = [{ key: 'host', value: '*'}];
             /**origin = request.origin.custom.domainName;  enable it for testing*/
    }
    /** for testing purposes don't use unless you need to, big performance hit
    console.log(`###### - > Request HOST "${request.headers.host[0].value}"`);
    console.log(`###### - > Final Request Origin Server "${origin}"`);
    */
    callback(null, request);
};

Comment: Hello thanks for your huge help so that's exactly what I wanted

I tested it worked correctly.

more missing a detail that I can not do, maybe you can help me.

on the line

    const regex = /^\/(home)[^\/]*/i;  // **<<<< HERE**

I need to return all positions of the example json:

    const regex = /^\/(digital|home|portal|)[^\/]*/i; // **<<<< HERE**

And from the test I did it always returns home only, even though json has the values of home, portal and digital.

I tried to do everything for him to read the other keys, but as I am new to python I'm having a hard time


Sanity check question about correlation between log-normal random variables

Question: Recently I came across the following result in a paper, which I can't seem to find right.
Anyway the authors had cited Distributions in Statistics: Continuous Multivariate Distributions by Johnson & Kotz, page 20 as the source. Since I do not have access to this book I am unable to check on the derivation of this result. I was hoping someone could shed some light on how to prove this.
Let $X$ and $Y$ denote a pair of bivariate log normal random variables with their means and variances given by the ordered pairs $(\zeta_{X}, \eta_{X})$ and $(\zeta_{Y}, \eta_{Y})$ and with correlation coefficient $\rho_{(X, Y)}$, derived from the bivariate normal with marginal distributions having parameters $(\xi_{X}, \nu_{X}^2)$ and $(\xi_{Y}, \nu_{Y}^2)$ and correlation coefficient $\rho_{N}$.
The expression for $\rho_{(X, Y)}$ is given as
\begin{equation}
\rho_{(X, Y)} = \dfrac{\mathrm e^{(\rho_{N} \nu_{X} \nu_{Y})} - 1}{\sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)}}
\end{equation}
The below is a manipulation I came up with, any error in that would be completely mine.
Upon rearranging we get
\begin{equation}\begin{aligned}
\rho_N &= \dfrac{\ln (1 + \rho_{(X, Y)} \sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)})}{\nu_{X}\nu_{Y}}\\
&= \dfrac{\ln (1 + \rho_{(X, Y)} \frac{\eta_{X}\eta_{Y}}{\zeta{X}\zeta{Y}})}{\sqrt {\ln ((\frac{\eta_{X}}{\zeta_{X}})^2 + 1)\ln ((\frac{\eta_{Y}}{\zeta_{Y}})^2 + 1)}}.
\end{aligned}
\end{equation}
I am very interested in learning if this result would hold true between $X$ and $Y$'s correlations if we had a quad-variate situation as in there were 2 more random variables which are also correlated with $X$ and $Y$ would this result still be true ? I suspect it should be, but I am newbie and due to lack of a credible source I was wondering if some one would be kind to confirm it.
Comment: I am not following. How do additional dimensions matter? Are you interested in a partial correlation between $X$ and $Y$ given the values of another two variables?
Comment: @StasK I think i am not interested in partial correlations. I could be wrong about that  though. The situation i am interested in is that i want to derive a formula for distribution of 4 correlated Log normal random variables, to do this i convert to Gaussian by taking logs, I am aware of a linear relationship between the Gaussian variables, but this relationship brings in a few $\rho_N$ type terms. Upon converting back this multivariate Gaussian variance to it's lognormal variance. I would like to use the last formula in the post to replace them with correlations of the log-normal.
Answer: The moments of the lognormal distribution are usually derived from the moment generating function of the normal distribution. If $X\sim N(\mu,\sigma^2)$, then it has the mgf
$$
M_X[t] = \mathbb{E} \exp(tX) = \exp( \mu t + \sigma^2 t^2/2 )
$$
Then if $Y = \exp(X)$ is the lognormal variable of interest, we can find, for instance,
$$
\mathbb{E}[Y] = \mathbb{E} [\exp(X)] = M_X(1) = \exp( \mu + \sigma^2/2 )
$$
and
$$
\mathbb{E}[Y^2] = \mathbb{E} [\exp(2X)] = M_X(2) = \exp( 2\mu + 2\sigma^2 )
$$
from which
$$
\mathbb{V}[Y] = \mathbb{E}[Y^2] - \mathbb{E}^2[Y] = \exp( 2\mu + 2\sigma^2 ) - \exp( 2\mu + \sigma^2 )
$$
$$ = \exp( 2\mu + \sigma^2 ) [ \exp(\sigma^2) - 1 ] = [ \exp(\sigma^2) - 1 ]\mathbb{E}^2[Y]
$$
You may have to do something like that with a four-variate normal distribution and its multivariate mgf.
In my opinion experience working with lognormal distributions, it is only practical when the log-variance $\sigma$ is less than 1. Beyond that, the sensitivity of pretty much any reasonable summary of the distribution hinges critically on whether you got the tail behavior correctly. In most situations (as is true with the original normal distribution, as well, but is not exacerbated by exponentiation), the shape of the right will be different, and you can easily see a twofold difference due to 0.1 change in $\sigma$... and you don't want that to happen.
Comment: See a little update on being careful with making strong assumptions.


validation for input type number with minimum and maximum range

Question: I want to validate if the input type number crosses the range.
<input type="number" min="-10" max="10"  required="true" message="you can give score -10 to +10 only">

I tried required = true but not working i want to show if the value crosses the range show message like you can give score -10 to +10 only
I want to allow while entering itself, not when submitting.
How can i do that..?
Answer: You can look at this jsfiddle i have created.
https://jsfiddle.net/6bkws7zd/
HTML
<input id="txtNumber" type="number" message="you can give score -10 to +10 only" />
you can give score -10 to +10 only

Jquery
$("#txtNumber" ).keyup(function() {
  if($('#txtNumber').val()<-10 || $('#txtNumber').val()>10 ){
      $('#errorMsg').show();
  }
  else{
    $('#errorMsg').hide();
  }
});

You can add required field to input control if you want to make it required.
Answer: Instead of hard coding "10" and "-10" in your function, you can get the min and max values from the element and check if it's within range.
$(function () {
   $( "input" ).change(function() {
   var max = parseInt($(this).attr('max'));
   var min = parseInt($(this).attr('min'));
   if ($(this).val() > max)
   {
      $(this).val(max);
   }
   else if ($(this).val() < min)
   {
      $(this).val(min);
   }
 });
});

@Caspian also provided a working fiddle: http://jsfiddle.net/Ddk67/75/
Comment: I think that additionally to the `change` event you should also wire up the `keyup` event. Otherwise it's possible to manually enter a value which exceeds the max or is below the min and there is no correction or adjustment.
Answer: This is the vanilla js way.

document.querySelector('input').addEventListener('input', e=>{
  const el = e.target || e

  if(el.type == "number" && el.max && el.min ){
    let value = parseInt(el.value)
    el.value = value // for 000 like input cleanup to 0
    let max = parseInt(el.max)
    let min = parseInt(el.min)
    if ( value > max ) el.value = el.max
    if ( value < min ) el.value = el.min
  }
});
<input type="number" min="-10" max="10"  required="true" title="you can give score -10 to +10 only">
Comment: I like your approach. showing the reverse value as validation.
Comment: Cannot input negative sign with keyboard first.
Answer: $(document).on('change', 'input', function() {
    var max = parseFloat($(this).attr('max'));
    var min = parseFloat($(this).attr('min'));
    if (parseFloat($(this).val()) > max) {
        $(this).val(max);
    } else if (parseFloat($(this).val()) < min) {
        $(this).val(min);
    }
});

Comment: It would be helpful to explain what your code does with comments or additional commentary.


3-4 outside linebacker vs. a 4-3 outside linebacker

Question: It looks like a 3-4 outside linebacker is more of a standup DE with some additional duties that include playing in space. This position appears to be more of a 'point of attack' style position. A 4-3 linebacker seems to play more off the line and is more of a 'play in space' kind of player that is more likely to pick up TE, backs for coverage and have more outside contain responsibility. I rarely see 4-3 linebackers get many sacks.
Is it common for a team that uses a 3-4 to have one of their outside linebackers be more of a blitzing linebacker and the other be a play in space person? Then the inside linebacker to the side of the blitzer, shift out wide. So in a base defense a 3-4 would be very similiar to a 4-3, but with 1 DE standing up. This is how I think the cowboys used Demarcus Ware. So now that he is switching to a DE in a 4-3, its very similiar to his previous position.
Comment: Welcome to Sports SE! Don't consider it being "dinged," but rather learning the ropes. If you have several questions, feel free to ask, but I suggest to decomposing your questions unless they rely on one another to which they can be combined. See [this](http://meta.sports.stackexchange.com/a/335/527) for more info.
Comment: I think these rely on one another. They seem like they are all related.
Comment: I only see one question however. The rest is supplemental commentary. Please elaborate.
Comment: OLB's in a 3-4 are pseudo-defensive ends. While they do occasionally have coverage assignments, (like TEs routes and RB screens), their general goal is to blitz. See players like Joey Porter, Clay Matthews, DeMarcus Ware as you mentioned, Terrell Suggs. *From my experience watching the NFL* generally the OLB in a 3-4 that blitzes is the Right OLB. As to why this is, I'm not entirely sure to be honest. Perhaps they want to blitz the blindside of right handed quarterbacks. [3-4 Defense - Linebackers](http://en.wikipedia.org/wiki/3%E2%80%934_defense#Linebackers)
Comment: so does the ILB to the side of the blitzer generally cover the role of the OLB in a 4-3? Or do teams do some other kind of shift. If the OLB is at the point of attack, teams still need someone to cover in space. This tells me at a 3-4 and 4-3 are just semantically different. It just means one of your DEs is standing up.
Answer: The main difference between a person suited for a 3-4 outside linebacker vs a 4-3 outside linebacker is their ability to get off the ball and play at the line of scrimmage.  Starting with a head of steam from 4-5 yards away is very very different from lining up across from a man and beating him off the ball.  This takes more initial burst and leverage.  While blitzing from the second level takes timing and agility and speed.
At almost all levels your DEs are the core of your defense.  Coaching pee-wee through college I can say my first pick would be a left defensive end (left because most QBs are right handed).  You don't know how many stud players I have tried to "turn" into a DE but they eventually went back to linebacker.  I call these players my "thinkers".  At 4-3 outside linebacker you want your "thinkers" there.  These are players who read the ball well, analyze plays, reacts to blocking patterns, reacts to route patterns and so on.
Now your 3-4 outside linebacker is a "doer".  This guy just wants to go.  You can line him up on the line and he will play like a D-lineman.  You put him in coverage though and you usually have two options.  He can cover the short flat or he can man up on a TE or back.  Put him in space he will think too much.  You don't want your attack dog thinking when a crime is taking place - and a team trying to get a yard on my defenses is a crime!
There are very very few players I have coached that are both "thinker"s and "doer"s.  Well maybe there were a lot but there were very very very few that were really good players that could do both.  If I labeled you a "thinker" or a "doer" those are big skills and you were a good player, and if you were a hybrid you were probably 2nd-3rd string.
Some other notes:

some 3-4 outside LBs can move to DE.  Most of the time they were picked to OLB position because of their size.  If their penetration outweighs their lack of size then they may merit a move up to a 4-3 DE if schemes change.  You will hardly ever see a OLB move to a 3-4 DE position which is generally a two gap technique (and "doer"s can't do 2-gap).
some 4-3 DE have moved to OLB.  If a DEs size is being taken advantage of they may move to an OLB position in a 3-4.  Every once in a while they may transition to a 4-3 OLB but that is basically saying they aren't good at getting after the QB.
a 3-4 OLB is not a glorified DE.  He may line up 90% of the plays during a specific game like a DE but that is just that game.  What happens in the NFL if an OLB does this (I have never coached a 3-4 and have very very rarely seen it used in college) and what I would do, is simply pass every down.  You keep your 3 big bodies with 2-gap technique with no exotic blitzes (because the left OLB is just their left DE) and I will throw laddered slants to your left side all day.  If you do this you are leaving potentially your corner, safety, and ILB on my TE, RB, and WR.  This doesn't compute right?  If your corner doesn't play shallow zone my TE and RB catches outs all day.  If your corner does play a shallow zone my WR runs streaks all day (and when your safety starts cheating posts).
a 3-4 OLB is a hybrid that can play on the line or at the second level.  With a good schemed 3-4 an OLB may "blitz" (play as a DE) maybe 30% of the time.  Now if they have a very favorable matchup maybe 80-90%.  How do I get to 30%?  Well you have 2 OLBs.  That would cover 60% of the snaps.  You will rush 3 on maybe 15-20% of the snaps.  The others should be filled with ILB or safety blitzes.
this question is really around pro football.  The 3-4 isn't really relative at any other level.  Some FCS schools run the 3-4 sometimes.  To run it right not only do you have to have 3 stud DL to run 2-gap but you also have to have intelligent players that understand all the scheming involved to run a 3-4 right.  It isn't that players can't or aren't smart enough it is just that only at the pro level is there enough time involved to learn how to do it without opening up glaring holes because of assignment mistakes.
at most an OLB in a 4-3 will blitz 10-15% of the time.  Generally your 4-3 OLB has to be faster because they need to cover a 3rd of the field instead of a 4th - this is a HUGE difference.  Also generally speaking size isn't much of a concern for a 4-3 OLB with some major colleges playing guys around 200 pounds (and lighter).
if you have a 4-3 OLB that gets after the QB well then chances are most of their pass rushes have them coming in as basically a DE in nickel and dime packages.  Note that this is more of a high school or college thing.  If an OLB rushes the passer well in the pros he moves to a 3-4 team or to DE so he makes more money.

Special Note:  For those with ****ass high school coaches that ran a 3-4.  Chances are you had one of your OLB line up on the line (who acts like a DE), probably on the weak side.  Then you have the other 3 LBs plus 2 corners on the second level.  Then you had a free safety, then a "rover" or some other sort of roaming strong safety.
Guess what?  That isn't a 3-4.  That is a 4-3 in Cover 1 (or simply a 4-4 with the SS assuming role of OLB).  High school coaches like to say they are running a 3-4 to make themselves feel current or smart.  In 25+ years of coaching I have not seen a true 3-4 run at the high school level.  Even with the fake 3-4 it puts undo pressure on your corners and even if you have an athlete advantage over me I can probably scheme my guys open (yes we would pass most downs).
Answer: The main difference in the 4-3 and 3-4 defenses are the gap responsibilities.  There are also many variations of each defense.  The Eagles were known to run a variation of the 4-3 under Andy Reid called the "wide 9" referring to the 9 gap the players would line up on.
With the 3-4 the defense ends have to be able to stop the run as well as rush the passer.  The reason is that they control most of the middle of the field along with the two inside linebackers. Usually one outside linebacker will rush the passer and the other will be responsible in coverage (zone or man).
In the 4-3 defense, the defense ends have less run stopping responsibility and more pass rushing responsibility.  The 2 defensive tackles and usually 1 or two of the linebackers will be responsible for the run depending on the situation.
Either way, players will be lined up and responsible for certain gaps.  In the 4-3 defensive ends have the wide gaps, where in a 3-4 linebackers have the wide gaps (with wide 9 being more like a 3-4).
There is a good article that covers the gaps and responsibilities more, the link is at the bottom.
I think the main idea is that offense and defense will continue to evolve and players will fit differently in different schemes.  The 3-4 linebacker may fit in as a 4-3 end or a 4-3 linebacker and vice versa.  Depends on the player and the scheme.
4-3 vs 3-4 pass rush
Answer: Most defenses match their lineups to the offensive team. Instead of having a left outside linebacker and a right outside linebacker in a 3-4 defense, most teams have a strong-side linebacker and a weak-side linebacker. The strong side linebacker lines up opposite the side of the offensive line with the extra lineman or tight-end. What each defender does on a given play is also determined reactively from what the offense does but I think it's safe to say that the weak side linebacker blitzes more often than the strong side linebacker.
Answer: Bum Phillips was one of the first I recall using a  3-4, with Robert Brazile as the pass rush WS OLB.
The famous Broncos "Orange Crush" Defense was 3-4 as well.
But it was Lawrence Taylor who revolutionized the 34 OLB position.
Compared to 4-3 OLB, the 3-4 requires a more agile, faster, quicker, and athletic player. Most 4-3 OLB would be shifted inside to play the SAM backer in a 34 defense.


Possible way to write below code in java 8

Question: I have a code, which is working as required, but I want to re-write it in Java 8.

This code will produce a map.Each list item will have all the servers allocated to it.
public static Map<String, List<String>> agg(){
        List<String> list = Arrays.asList("Item A", "Item B", "Item C");
        List<String> servers = Arrays.asList("Server A", "Server B", "Server C", "Server D");
        Map<String, List<String>> map = new HashMap<>();
        for (int i = 0; i < list.size(); i++) {
            ArrayList<String> temp = new ArrayList<>();
            for (int j = 0; j < servers.size(); j++) {
                temp.add(servers.get(j));
            }
            map.put(list.get(i), temp);
        }
        return map;
    }

Output
Item C ::[Server A, Server B, Server C, Server D]
Item B ::[Server A, Server B, Server C, Server D]
Item A ::[Server A, Server B, Server C, Server D]

What would be the lambda equivalent?
Answer: Create a stream of your "items", and collect them into a map where the keys are the objects from that stream, and the values are copies of the server list.
import java.util.*;
import java.util.function.*;
import java.util.stream.*;
import static java.util.stream.Collectors.toMap;

...

List<String> servers = Arrays.asList("Server A", "Server B", "Server C", "Server D");
Map<String, List<String>> map = Stream.of("Item A", "Item B", "Item C")
    .collect(toMap(Function.identity(), (__) -> new ArrayList<>(servers)));

Answer: Let's simplify code before converting it to lambda expressions
 public static Map<String, List<String>> agg(){
    List<String> list = Arrays.asList("Item A", "Item B", "Item C");
    List<String> servers = Arrays.asList("Server A", "Server B", "Server C", "Server D");
    Map<String, List<String>> map = new HashMap<>();
    for (int i = 0; i < list.size(); i++) {
        map.put(list.get(i), new ArrayList<>(servers));
    }
    return map;
}

I just simplified creating an array copy.
The pipeline to convert from a form of data to another in Java 8 contains the following steps:

stream
map (convert from a form to another)
reduce (filter out unwanted values)
collect (collect results)

Because your data does not need to be converted, just collected in a different structure, map and reduce are not needed
public static Map<String, List<String>> agg(){
    List<String> list = Arrays.asList("Item A", "Item B", "Item C");
    List<String> servers = Arrays.asList("Server A", "Server B", "Server C", "Server D");

    Function<String, String> keyFunction = key -> key;
    Function<String, List<String>> valueFunction = key -> new ArrayList<>(servers);

    return list.stream()
            .collect(Collectors.toMap(keyFunction, valueFunction));
}

Those functions can be inlined and the result would be:
    public static Map<String, List<String>> agg(){
    List<String> list = Arrays.asList("Item A", "Item B", "Item C");
    List<String> servers = Arrays.asList("Server A", "Server B", "Server C", "Server D");

    return list.stream()
            .collect(Collectors.toMap(key -> key, key -> new ArrayList<>(servers)));
}

Comment: Your answer is correct but sorry I am unable to understand it. What happened to keyFunction and valueFunctions?
Comment: Understood your answer. But I have one question. Like you said we can have Stream, Map, Reduce & Collect operation. Let's say I want to append list name with each sever name, e.g: Item A_Server A, Item A_Server B ..., in that case how i will write map method?
Comment: In that case, you need to change the value function to something different. In this case you would need a map function to convert those values: Function> valueFunction = key -> servers.stream().map(server -> key + "_" + server).collect(Collectors.toList());
Comment: Thanks, I did the same thing.
Comment: Glad I could help. The streams are a powerful tool to be used in case you understand how they work.
Comment: correct, I am still learning Java 8. Do you have any references to learn from? If yes, please share.
Comment: I don't have a specific reference you can learn from, just some ideas to make the learning process easier: 1. Install an IDE that suggests changing existing code to Lambda expressions (like Intellij IDEA). Make sure you understand what happens in the new code. 2. Split your problems in stream -> map -> reduce -> collect flows and replace them with Lambda equivalent structures. 3. Search / ask questions on StackOverflow on how to solve specific scenarios.
Answer: A simple forEach approach would be
Map<String, List<String>> output = new HashMap<>();
list.forEach(s -> output.put(s, servers));

Answer: Try this one:
list.stream()
            .map(item->new AbstractMap.SimpleEntry<>(item,servers))
            .collect(Collectors.toMap(Map.Entry::getKey,Map.Entry::getValue));

Comment: Nice approach!.There is one small possible issue: The list of servers is shared across multiple endpoints and can lead to unexpected issues further the line
Comment: @nucandrei what *exactly* do you mean?
Comment: If you alter the list in one place, those changes will be reflected for everybody using a reference to that list. There are two options to avoid this, unless it's the desired behavior: 1-Create a copy. 2-Use an immutable structure.


Keycloak user data encoding

Question: By default users' data fields (e.g. first name and last name) are encoded using ISO-8859-1. People from many countries can't properly create accounts as their personal data is silently messed up. How can I fix it?

The DB receives already damaged names. By default all columns are ISO-8859-1-encoded, but converting them to UTF-8 doesn't help.
Manual account modification from admin console has same effect.
Change of default server (Wildfly) encoding to UTF-8 doesn't do anything.
Answer: The KEYCLOAK-3259 bug states, that the encoding of the forms depends on the default encoding of the JVM. This will be fixed in Keycloak
2.1.0.CR1.
Did you try to change the JVM default encoding by passing <code>-Dfile.encoding=UTF8</code> to the Java commandline or Java options?
Comment: Thank you for your response. I tried it before by modifying standalone.xml using CLI and now with the JVM flags, but failed twice.
Answer: It turned out, that it's a bug involving connection with MySQL, you can follow it on Keycloak Jira.

Keycloak user data encoding


Is this the right way to evaluate MSE, RMSE, MAE, MAPE after creating a model on data passed through a standard scaler?

Question: Create the Scaler object
scaler = preprocessing.StandardScaler()

Fit your training data on the scaler object
scaled_X_train = scaler.fit_transform(training_data)

scaled_X_train = pd.DataFrame(scaled_X_train, columns = training_data.columns)

scaled_X_train.head()

Train your model here
Fit your testing data on the scaler object
scaled_X_test = scaler.fit_transform(testing_data)

scaled_X_test = pd.DataFrame(scaled_X_test, columns = testing_data.columns)

scaled_X_test.head()

Predict on test data using the trained model and the scaled test data
If y_test_actual is the scaled actual y &

If y_test_predicted is the scaled predicted y

Then would this be the right way to evaluate MSE, RMSE, MAE, MAPE
testing_mse = mean_squared_error(y_test_actual, y_test_predicted, squared=True)

testing_rmse = mean_squared_error(y_test_actual, y_test_predicted, squared=False)

testing_mae = mean_absolute_error(y_test_actual, y_test_predicted)

testing_mape = mean_absolute_percentage_error(y_test_actual, y_test_predicted)

Would these metrics be the same if the model was trained without performing any scaling?
The metrics would most likely be different. My question is how to get the right metrics when performing scaling of all features including the target variable ?
Answer: First of all, data scaling or normalization is useful for many models but there are cases where it's not needed. For example trees in general does not depend on scaling. The same should apply e.g. to gradient boosting methods, but there might be some special cases, see this link.
Regarding your question, metrics calculated on scaled data will be usually different. But when you use model which is not scale-dependent like the trees, the results (the performance) of the model should be in the same ballpark.
Note also the fact that some metrics are scale-dependent and some not. Typically squared and absolute based metrics (like MSE or MAE) will return different results. On the other percentage based errors (like MAPE) will not be influenced by scaling much (but depends on type of scaling, especially when you move "0").
I would recommend to try it by yourself. My recommendation would be following - use scaling for models that are scale-sensitive and report metrics in scaled/transformed space as in the original space. This might be easily achieved using inverse transform (see sklearn docs - you can do something like scaler.inverse_transform(y_test_predicted) and then calculate the metrics in the original space as well).
Advantage is that you can use scaled data for models which will work better in most cases and keep the original metrics as well (so you will see the real error for you data, e.g. what is true absolute error for temperature forecasting since its value is much useful than taking the one from scaled space).


ServiceStack.Text Serializer as default serializer/deserializer

Question: We are just starting to develop our new API client using ASP.net Web Api. Having found ServiceStack.Text i do not want to use JSON.NET serialization feature but replace them with ServiceStack.
How to set this serializer as default in Web Api?
Thanks a lot!
Answer: You can create a custom MediaTypeFormatter which uses ServiceStack's serializer/deserializer. Looks like some folks have already created formatter for this here: https://github.com/WebApiContrib/WebAPIContrib/wiki/Media-Type-Formatters
To make it as a default formatter, just insert this new custom formatter in the beginning of the global configuration's formatters list.


combining two taxonomy fields into one exposed filter dropdown

Question: I have created two taxonomy fields (home team and away team) with a content type.
I have created a views to display the list content of content type. In view, I have added taxonomy filter for home team and away team which is showing two taxonomy filter in the views page.
I would like to combine both taxonomy filter into single exposed filter.
Comment: Hello and welcome. Please put a bit of effort into your question. For example phrase title as a question, and use sentences in it's body. It'll make it easier to read for us non-native speakers. And showing a bit of effort you have put into solving it before you asked will greatly help, too. People aren't too eager to look for an answer if person who asked didn't.
Answer: You can follow this using below step:

Go to Administration -> Structure -> Views
Edit your existing view or create new new.
Add Filter for taxonomy for 'home' and 'away'
Click on "Rearrange" filter icon in Filter section
Click on "Create new filter group" and move filter "Home taxonomy" and "Away taxonomy" in new filter group.
Select Operator as "Or" in new filter group and save the form.
Comment: but there should be only one dropdown a combination of home and away team
Comment: i tried global:combine filter but it coming in search form i need a dropdown
Comment: please help me i am i deep trouble
Comment: @mayur, I have worked on this but with other scenario. I am working on this and will let you know.
Comment: @mayur please help yourself and 1) read my comment under your question, 2) review suggested edit by Jayendra Kainthola and either accept it, or not, but if not, then do some cleanup yourself so we could be sure we know what you need.
Comment: please help me??
Comment: @mayur, I am looking into this. If I found anything, I will let you know.
Answer: Create only one taxonomy vocabulary like Team Type. Then add

home team and away team

as terms in that vocabulary. Then add the new vocabulary as filter in view.
OR
Otherwise use form alter, then alter the dropdown field in the exposed for and set the options in the select field manullay.
Refer this: How to alter filter in exposed filters form trough form_alter hook
Comment: 1st option wont work because home and  away team both are coming from teams only
Comment: you mean **home** and **away** are coming from a single vocabulary called **team**?
Comment: yes and filter having teams names dos'nt matter its a home team or away team
Comment: Please help me out i am i deep trouble


find out the volume of solid removed?

Question: i have sphere that has an equation $$x^2+y^2+z^2=1$$ a cylindrical hole $x^2+(y-1/2)^2$=$1/4$ is cut through it . find the volume of the portion cut. i don't know what to do, i was thinking of using cavalieri's principle , and integrate with respect to $dz$ but i could not imagine the equation of sheets that would be formed . i guess we can proceed with polar , but i don't know how? please help
Comment: A tricky calculus I problem for sure.
Comment: You could integrate a triple integral in cartesian coordinates.
Comment: i dont know how to go about that
Answer: To integrate in yucky cartesian coordinates you would do for the sphere:
$$V_\textrm{sphere} =\int_{-1}^1dx\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}dy\int_{-\sqrt{1-y^2-x^2}}^{\sqrt{1-y^2-x^2}}dz = 8*\int_0^1dx\int_0^{\sqrt{1-x^2}}dy\int_0^{\sqrt{1-y^2-x^2}}dz$$
(to see this worked out in some detail see: https://www.youtube.com/watch?v=pIf78H50iNc)
To get the missing cylinder, you need to integrate over the region of the cylinder, Same $z$ bounds, but just integrate $x$ and $y$ over the cylindrical region given by your equation.
viz. $$\int_{-1/2}^{1/2}dx\int_{(1/2) -\sqrt{1/4-x^2}}^{(1/2) +\sqrt{1/4-x^2}}dy\int_{-\sqrt{1-y^2-x^2}}^{\sqrt{1-y^2-x^2}}dz $$
Comment: but the problem is that though cylinder is drilled , the part that is drilled is not a cylinder right?
Comment: No it is, at least, the sides are. The top and bottom are bounded by the sphere so the $z$ integral is just like the one above. The $x$ and $y$ integrals go over the region of the cylinder.
Comment: I added the cylinder part. Basically you just solve for $y$ in terms of $x$ from your equation that defines the cylinder. Also $x$ goes from $-r$ to $r$ where $r$ is the radius of the cylinder.
Comment: oh i get it, that was nice , although if i had changed cylinder into a sphere of radius of 1/2 centred at (0,1/2,0) both the integrals would still remain the same right ?
Comment: No because the $z$ limits would change to be $\pm \sqrt{(1/4)-y^2-z^2}$ as it is you integrate $z$ up to the unit sphere.
Comment: k sorry to disturb you , but in the second equation i am still not sure why did you set the limits as $\sqrt{1-x^2-y^2}$ and  $-\sqrt{1-x^2-y^2}$
Comment: i understood the change of limits in the sphere as  mentioned in comments , but not the part in the original equation
Comment: The $z$ integral limits come from your first equation (solve for $z$) of the sphere. Your removed cylinder is bounded by the sphere in the $z$ direction so you use that equation. For $x$ and $y$ you use the other equation since that bounds those directions. If you have further comments we can chat about it later.
Comment: no thats it , thank you :)

find out the volume of solid removed?


Are only linear transformations associative

Question: I am teaching myself linear algebra and calculus from Apostol Calculus Volume 2 and I find it very satisfying. Today I came came across theorem 2.5 in chapter linear transformations and matrices which states that:

If $T: U \to V, S: V \to W ,R: W \to X$ are three functions, then we have:
$$R(ST)= (RS)T$$

Proof:

Both functions $R(ST)$ and $(RS)T$ have domain $U$ and values in $X$. For each $x$ in U, we have:
$[R(ST)](x)= R[(ST)(x)]= R[S[T(x)]]$ and $[(RS)(T)](x)=(RS)[T(x)]=R[S[T(x)]]$

This is a very natural proof and I understand it. What I fail to grasp is the use of linear transformation and the structure of vector spaces in the proof. I mean it is not at all obvious where does the proof makes use of the axioms of linear transformation and vector spaces.
So is the theorem valid for non linear transformations and/or sets which are not vector spaces? If no, then please point out the assumptions of the proof and its scope.
Comment: The theorem says "functions," not "linear transformations." Indeed, function composition is associative, no matter whether the function is linear.
Comment: @symplectomorphic  Can I view linear transformations as functions?
Comment: yes, of course... a linear transformation is a specific type of function.
Comment: What about non linear transformations? I think they can also be viewed as functions which makes them associative as well. Am I correct?
Comment: not only are you correct, that's what I said in my first comment, and *that's what the theorem in Apostol's book says*. The only assumption of the theorem is that $R$, $S$, and $T$ are functions. It does not assume anything more about them (except that their domains and ranges are specified so that function composition makes sense). In particular, it does not assume they are linear transformations.
Comment: @symplectomorphic Okay this answers my question, if you make it an answer, I shall accept it.
Answer: The theorem says "functions," not "linear transformations," so in fact the theorem Apostol proves answers your question: yes, function composition is associative, no matter whether the functions involved are linear. You are right to see that the proof doesn't depend in any way on linearity -- because, indeed, linearity is not assumed, nor is it needed.
Comment: Just a quick question, can I use the theorem to prove the associativity of matrix multiplication?  If it is possible, then it would save me a lot of computations which seem artificial.
Answer: Perhaps the core reason is that:

functions between sets are associative in the same sense as above.
"a linear transformation is what a linear transformation does", which is to say that a linear transformation is completely determined by where it sends each element of its domain.

Category theory is a subject that is good at formalizing answers to questions of this sort, I encourage you to look it up.
Comment: I am studying introductory linear algebra so perhaps category theory is out of question.
Comment: I think the definition of linear transformation that you provided also holds for functions.


Use AWS API Gateway RestApi to get information about key

Question: I am looking in to start using AWS API Gateway for my rest api.
I have been looking into their reference guide for the gateways internal rest api, link here
I want to find information about a user from their key but my issue is that the rest api only seems to handle the key id, not the key value.
How can i get the key id from this rest api? Keep reading and testing queries in postman but i can't seem to  find a way to identify a user by their api key..
Does anyone else know how to do it?
Thought i was close with this query apikey:by-key but this is by-keyId
Answer: You should not use API Gateway Keys to identify users. Their purpose is for usage plans and request limits. I suspect you want some sort of authentication system with users and if that's the case you need to implement AWS Cognito as the authorizer of the API Gateway (check here). From a Cognito pool you can extract information about your users.


What does "dominant" mean in music?

Question: C7 is a dominant 7th chord.
G or G7 is the dominant chord of the C-major scale.
Is the word dominant related between these two?
The word "dominant" relates to several items in music. In Common Practice Harmony, the 5th scale degree is called the Dominant. In a Major scale, the 4-note chord build on this note (stacking thirds) is a Major-Minor Seventh commonly called a "dominant" chord. (Major-Minor here describes the quality of the third and seventh of the chord, respectively.) The C7 chord has the same set of intervals as other Dominant-Seventh chords and is thus called a "Dominant Chord." In one case, there is a reference to the 5th scale degree and in the other, there a reference to the interval pattern of a chord.
Why is a C7 chord named a dominant seventh chord?
Comment: It's worth noting that the naming of the dominant seventh chord happened before dominant seventh chords were used much outside of **literally** being a dominant chord (resolving down a fifth). In the modern day, we use seventh chords all the time, even when they don't have this function (the D7 at the beginning of taxman by the Beatles has no dominant funciton), but we still call them dominant seventh chords because that's what they've always been called.  It's kind of like how we still say "dial" a number even though phones haven't had "dials" for years!
Comment: It means both. Like many words both in music and English in general, "dominant" has different meanings in different contexts.
Comment: it actually has both as @Todd Wilcox put it. To expand on that, there is "dominant" wherever there is a "7nth", and there's a tonal dominant which could simply be a triad. The whole jazz genre, if you will, depends on the first defintion, that is, maintaining harmonical structure on the brink of tension for prolonged periods of time.
Answer: A dominant is a very specific idea in functional harmony. The dominant's job is to take you back to the tonic. Why a dominant has so much pull in functional harmony is that a dominant chord utilizes the leading tone along with having the common tone with the tonic chord which is the dominant itself.
In terms of scale degrees the dominant is the scale degree where you build chords that tonicizes the tonic of that scale. Naturally, when you build a 7th chord off of G in C Major you get a G7 which by no accident is known as a dominant 7th. A C7 is a dominant chord and its job is to take you to the tonic of F or tonicize F.
Comment: @NeilMeyer - what? A dominant chord DOES NOT have a major seventh in it. A dominant chord, by definition, has 1, maj 3, P5 and min7 notes . There will never be a major 7 note in there. Unless you're confusing the major 7th note of the tonic, which will manifest itself as the major 3rd of the actual dominant chord. A minor 7th chord will comprise 1, m3, P5 m7, which will be called just that. Minor seventh.
Comment: "A dominant is a very specific idea in functional harmony. The dominant's job is to take you back to the tonic" I disagree somewhat with this. This to me is more a function of the major seventh, which the Dominant chord usually has, but there does exist unusual keys with minor sevenths in the dominant that sound different.
Comment: @Tim he means the leading tone i.e. the major seventh degree with respect to the tonic
Answer: I'm not 100% sure what you're asking, but:

Yes, C7 is a dominant seventh chord. This doesn't necessarily mean that it is the dominant, just that it is the same "quality" (ie, the same make-up) as a dominant seventh chord, which is a major triad with a minor seventh.
G is the dominant chord of C major

This means that G7 is the dominant seventh chord of C major.
The dominant is always a perfect fifth above its tonic. This means that a dominant seventh chord will always have some urge to resolve down a perfect fifth. Although G7 is the dominant of C, a C7 can still appear in C major, but it will want to resolve down a perfect fifth to F. (It doesn't have to, but it wants to!)
Answer: The description  "Dominant 7th" is used in two ways.  It can mean any major triad with added minor seventh.  Or it can mean the ACTUAL dominant 7th chord in a given key.
If we're in C major, G7 is the dominant 7th chord of the key.  But D7, A7, Bb7, F#7... are all "dominant 7th" type chords.  (And yes, all those chords can exist in a piece of music in C major).
Answer: "I Am Confused About Dominant."
At first, this is not an easy concept when you have a seventh in the fifth of a scale and a 7th is the seventh chord as well. After a formal class in jazz theory, I got it.  I will try to answer your question.
We'll use the C scale.  The seventh note in  C is B.  It is the seventh note, seventh chord, but we are talking about the fifth of the scale, specifically the 7th note of the fifth chord of a scale.  The seventh note of the fifth can make the chord either a Major 7 chord or a dominant 7 chord.
The seventh in a dominant 7 CHORD refers to the notes in the chord.  Stack the chord in thirds.  like this, C E G B, or 1, 3, 5, 7.  The seventh note of the C Chord is B.  In C Major 7 the scale is exactly that, C E G B.  If you flatten the 7th note, the chord becomes C E G  and B flat.  It is a dominant seventh chord when you flatten the 7th note in the chord.   Dominant 7th chords are the fifth in a Major scale with the formula C E G Bb.  In C that is G7.  In F, it is C7.  It is sometimes referred to as the V7 Chord to distinguish it as the fifth and not the seventh of the scale.
Answer: Some of the answers seem to add to the confusion!
The dominant chord in a key is so called as it is a pushy sounding chord. It tends aurally to want the tonic chord to follow it. No, it isn't always the case that the tonic comes next, but that's part of the tension that music can bring. You expect one thing, and something different happens.
The dominant seventh chord has an extra note to the triad of 1,3 and 5. The sequence is followed, so it's a 7. That's where confusion sets in. It's not the 7th of the scale that the dominant chord would be the key of. In other words, G7, as a dominant of C, would have F natural as the 7th part of it. Not F#, which is the 7th of the G scale.
That's because G7 - a.k.a. G dominant 7th - is a chord from the C key, not its own G key.I know that sounds adaft, as we find G7 in lots of songs that are in G. However, most times, the following chord is C (or Cm), so the G7 needs the 7th part of it to be from key C rather than its own key,G.
Musically, what's happening is the 3 is a semitone below the tonic root, and the 7 is a semitone above the tonic 3. To resolve, they both move a semitone each, the smallest change possible, and everything sounds fine again. The tritone - very unstable sounding - resolves to a nice major (or minor) 3rd interval.
So, in summary, the dominant 7th chord has all its notes from the tonic it's pushing to, as in E7 has a D as its 7th, even though the key of E has D# in it. It has become part of the tonic it's aiming for, so needs a D natural.This 7th can also be called a m7 or b7 of the dominant's own key.
I was hoping to uncomplicate the concept, but it's not a particularly simple answer after all!
Answer: In counterpoint studies, which study the musical practice before the notion of "everything comes from harmony" posed by J.P.Rameau, a dominant was simply a note that was most often used throughout the melody.
As I recall.
Comment: I'm not sure  which bonehead downvoted you, but you are indeed correct, although I thought the source was J.J. Fux. So, historically, the 5th of most modes was the most used note and hence termed the 'dominant' note in many melodies. And chords built off of the 5th are therefore named dominants. And to the point of the OP, the only chord built off of a major scale, having a major 3rd, perfect 5th and flat 7th is a seventh chord built off of the 5th degree or dominant.
Answer: In a more literary definition, the adjective dominant, as applied in music, details that the function of whatever structure it describes sets up a resolution to a structure with a tonic function. Generally, for chords and scales, dominant will mean one of two things:

The chord or scale described contains both a major 3rd interval and a minor 7th interval. For example, a C dominant seventh chord (C7) has both E and B♭ in it, hence the name "dominant". It resolves easily to an F major chord, which has tonic function. Also, scales based off of a dominant chord will usually be called dominant scales.
The chord or scale described is constructed from the (perfect) fifth degree of a scale. This is based off of the names for scale degrees where 1-2-3-4-5-6-7 as scale degrees become Tonic-Supertonic-Mediant-Subdominant-Dominant-Submediant-Subtonic(or leading tone if a semitone below the tonic). Notice that the fifth degree is named "Dominant". Thus in any mode, the chord built off of the fifth is called the dominant, regardless of whether it actually contains the ♮3 and ♭7.

So: The word dominant has two meanings in music: It can describe the quality of a structure, or the types of intervals found within it; or it can describe the function of the structure it describes.
(Confusingly, The two meanings are not always the same, as certain chords can have dominant function without being of the dominant seventh quality, and vice versa. The most common instance is where they are in conjunction: In B major, an F♯7 chord earns the moniker "dominant" for both reasons: Its interval qualities of the ♮3 and ♭7, and its construction upon the 5th, or dominant, of the key B major.)

What does "dominant" mean in music?


Bulk import images & create nodes

Question: Is there a way to bulk import images from server and auto create nodes for each image? this is for Drupal 7
Answer: The module Bulk Media Upload is the one that I have used in the past for that matter. From the module page:

With Bulk Media Upload you can upload a bunch of files and a new node, taxonomy term, or other entity will be created for every file. The files are selected via a browser file upload dialog and can be stored in a file or image field.
  Also default values can be chosen for the other fields assigned to the new node.

I believe you will not have any difficulties, it is pretty straightforward. Also, if you come across some minor bugs, I have a couple of patches commited in its issue queue awaiting community review.
Alternatively, one can use the Feeds module to import nodes. That would be more helpful with existing files. It might require to list the files into a CSV file (for *nix systems copy/paste from terminal the output of ls /path/to/files).
Comment: For existing files a possibility is to use the [Feeds](https://drupal.org/project/feeds) module to import nodes, maybe along with some columns in a CSV file.
Comment: The only problem with that module is it can't import existing files.I'd have to upload on the go. Is there any tricks to select existing files?

Bulk import images & create nodes


Why half coversed or coversed trigonometric functions are being deprecated?

Question: As you can see here there are some names for some trigonometric functions that I can't find in any text or math related papers today.
In my opinion this kind of approach will also make it easier to reason about practical problems that can be solved with a trigonometric approach, in particular this set of functions makes it clearer on why the definition cyclometric functions exist.
So why this good old approach is not being considered anymore ?
Comment: Mainly because with modern calculators we dont need tables anymore ...  . Many of the old deprecated functions existed so that one could table them!
Comment: @kjetilbhalvorsen Sometimes it's more user-friendly to focus on one thing than another, even when the two might be otherwise mechanically equivalent with necessarily identical answers (e.g.: finding the sagitta [i.e: versin] of a ring, rather than the radius minus the cosine).
Answer: They're not deprecated; they're largely just forgotten.
In the present day, trigonometry is taught in courses for non-mathematically inclined people who have no desire to take a math course except in order to get rewarded with a good grade for obeying and working hard.  They do not suspect that mathematics is a subject in which one derives things logically from other things and that one knows that something is true by understanding its derivation, rather than by being handed a dogma by authorities.  They bring in tuition money.  It's a racket.  Racketeers get paid to pretend that that is education.  If you tell the truth to such (using the term loosely) students, that mathematics is an intellectual endeavor, they complain that other instructors don't require them to know that.
Textbooks written for that kind of audience is where people learn trigonometry nowadays, including many people who actually want to understand mathematics.
It is largely forgotten that there is actually such a thing as an advanced trigonometry book for an audience consisting of mathematicians.  Some of these are of that sort.
Comment: (I didnt downvote) In what sense do you think this answers the question?  Would it have any value to bring back versed function to be used today? I thought they mainly existed to make tables, simplifying practical use. With calculators we dont need so many tables!
Comment: @kjetilbhalvorsen yes but 1) if you are a programmer you still have to write trigonometric functions in such a way that your hardware will be able to compute them, which means reducing this obscure functions in terms of additions and multiplications basically 2) to use math the right way you have to identify the functions or the set of functions that can solve your problem, plus if you manage to solve more problems with the same functions, it's good starting point for possible optimizations. It's about both your mindset and how to implement things.
Comment: I'm actually more of a programmer and I think that there is still a lot of value in this even today, in fact I spent a lot of time trying to figure out a good way to compute trigonometric functions ( without tables or more common and unoptimized approaches ) but I didn't had any idea about this, I'm quite sure that with this notions I would have been able to find Bhaskara or Padé approx much quicker, or just reasoning about what I was trying to do in a simpler way. Probably there are even better approx. I didn't downvote this btw.
Comment: user2485710:  Yes, but the versed functions are such as $\cos \theta +1$, I dont seee how they help you with what you talk about!
Comment: @kjetilbhalvorsen : You could post part of your comment as an answer if you expanded it somewhat.  If the process of table-making is served by those functions, might they also contribute to understanding the software used in calculators, etc.? $\qquad$


Get a user's kerberos ticket updated over VPN

Question: We run a Windows 2008 network (2008 DC level).  There are many users who work only from home and connect to the network via VPN.
Recently group membership was changed for many of these users and the membership isn't making it to their machine.  I need a way I can update their Kerberos ticket with their security groups.  Normally this happens at login, though these users are not connected to the network at login and don't get the updates.
gpupdate /force doesn't update the local Kerberos ticket. Using gpresult /r is being used to review current security groups for the user.
How can I get these updates to these users?
Comment: Have them connect to VPN before AD auth. Every VPN client I've used is able to be configured in this way.
Comment: Sounds like a good solution which I can ask the VPN team about.  Are there any other solutions since I have no control over the VPN configuration?
Answer: Try using a klist purge as login script, group policy scheduled task, etc. If run in the user context this should provide the response you want without changing the vpn setup.
I use a small script of a similar nature to refresh computer group memberships for deploying software without rebooting.
Comment: Everyone at my workplace was like, "Nope, gotta log out at least." Simply running the `klist purge` worked like a charm, and is way more convenient than logging off in the middle of a workday!