Append the count of the occurrence of the word in python Dataframe

Question: My original data

I want to convert the text data into a dataframe which will contain the 500 words like the below picture in which each sentence will contain the occurrence of that word in the particular sentence (Row of a dataframe.)
Final Output_data

I have performed text preprocessing and all with NLTK.
Comment: _Kindly help._ With what, what specifically is the issue? Please provide a [mcve], and see [ask], [help/on-topic].
Comment: I need to create term docoment matrix of the dataframe like the above shown.
Answer: <code>from sklearn.feature_extraction.text import CountVectorizer
count_vect = CountVectorizer()
X_train_counts = count_vect.fit_transform(twenty_train.data)
</code>
https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html
Comment: This will only form a vector martrix. But I donot want that. I want to get the occurrence of  each word in a sentence in the dataframe.


Can the Kivy language access inherited layouts and widgets?

Question: Can the kivy language access inherited layouts and widgets? I want to create one basic BoxLayout that contains the styling and title Label for my widget. I want to be able to inherit from this widget and add additional widgets in different positions.
from kivy.app import App
from kivy.lang import Builder
from kivy.uix.boxlayout import BoxLayout

Builder.load_string('''
<SimpleBar>:
    canvas.before:
        Color:
            rgba: 0, 0.5, 0.5, 1
        Rectangle:
            pos: self.pos
            size: self.size
    BoxLayout:
        id: my_layout
        Label:
            text: "hi"

<NewBar>:
    Label:
        text: "2"
''')

class SimpleBar(BoxLayout):
    def log(self, value):
        print(value)

class NewBar(SimpleBar):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        print(dir(self))

class GeneralApp(App):
    def build(self):
        return NewBar()

if __name__ == '__main__':
    GeneralApp().run()

Above is my basic running widget.
I want NewBar's "2" Label to be located before SimpleBar's 'hi' Label like below.
<NewBar>:
     BoxLayout:
         id: my_layout
         Label:
             text: "2"
         Label:
             text: "hi"

I know that - can negate items. However, <-NewBar> removes all of my styling.
Is there any way to do this in the kivy language?
Comment: made a little edit, it supports even indicies now ^^
Answer: Here's a fun thing: you don't need to specify all classes used in kv lang in the lang itself - you can also add them using Factory.register method later in code. Here's an example:
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.lang import Builder
from kivy.factory import Factory

from functools import partial

Builder.load_string('''

<MyWidget>:
    Foo
    Bar
''')

class MyWidget(BoxLayout):
    pass

class MyApp(App):
    def build(self):
        Factory.register('Foo', cls=partial(Label, text='foo'))
        Factory.register('Bar', cls=partial(Label, text='bar'))
        return MyWidget()

if __name__ == '__main__':
    MyApp().run()

Let's use it to create a template base widget we later fill with various content. We use a placeholder that we later replace with another widget:
<BaseWidget>:
    orientation: 'vertical'
    Label:
        size_hint: None, 0.1
        text: 'title'
    Placeholder

In the Python code we register a placeholder class in the __init__ method of this base template class.
class BaseWidget(BoxLayout):
    def __init__(self, **args):
        # unregister if already registered...
        Factory.unregister('Placeholder')
        Factory.register('Placeholder', cls=self.placeholder)
        super(BaseWidget, self).__init__(**args)

Now let's define a content class.
<TwoButtonWidget>:
    Button:
        text: 'button 1'
    Button:
        text: 'button 2'

And finally create a customized class that use our base class as a template and replaces its placeholder with a content class. This class don't have its own kivy rules (these are moved into content class) so when we inherite from our base template, no extra widgets are inserted.
# content class
class TwoButtonWidget(BoxLayout):
    pass

# Base class subclass
class CustomizedWidget(BaseWidget):
    placeholder = TwoButtonWidget # set contetnt class

A full example:
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.lang import Builder
from kivy.factory import Factory

Builder.load_string('''
<BaseWidget>:
    orientation: 'vertical'
    widget_title: widget_title
    placeholder: placeholder
    Label:
        size_hint: None, 0.1
        id: widget_title
    Placeholder
        id: placeholder

<TwoButtonWidget>:
    button1: button1
    Button:
        text: 'button 1'
        id: button1
    Button:
        text: 'button 2'

<ThreeButtonWidget>:
    orientation: 'vertical'
    Button:
        text: 'button a'
    Button:
        text: 'button b'
    Button:
        text: 'button c'
''')

class BaseWidget(BoxLayout):
    def __init__(self, **args):
        # unregister if already registered...
        Factory.unregister('Placeholder')
        Factory.register('Placeholder', cls=self.placeholder)
        super(BaseWidget, self).__init__(**args)

class TwoButtonWidget(BoxLayout):
    pass

class ThreeButtonWidget(BoxLayout):
    pass

class CustomizedWidget1(BaseWidget):
    placeholder = TwoButtonWidget

class CustomizedWidget2(BaseWidget):
    placeholder = ThreeButtonWidget

class MyApp(App):
    def build(self):
        layout = BoxLayout()
        c1 = CustomizedWidget1()
        # we can access base widget...
        c1.widget_title.text = 'First'
        # we can access placeholder
        c1.placeholder.button1.text = 'This was 1 before'

        c2 = CustomizedWidget2()
        c2.widget_title.text = 'Second'

        layout.add_widget(c1)
        layout.add_widget(c2)
        return layout

if __name__ == '__main__':
    MyApp().run()

You can easily extend it and for example, have multiple placeholders.
Applying this to your case:
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.uix.label import Label
from kivy.lang import Builder
from kivy.factory import Factory

from functools import partial

Builder.load_string('''

<SimpleBar>:
    canvas.before:
        Color:
            rgba: 0, 0.5, 0.5, 1
        Rectangle:
            pos: self.pos
            size: self.size
    BoxLayout:
        Placeholder
        Label:
            text: "hi"

<NewBarContent>:
    Label:
        text: "2"
''')

class SimpleBar(BoxLayout):
    def __init__(self, **args):
        # unregister if already registered...
        Factory.unregister('Placeholder')
        Factory.register('Placeholder', cls=self.placeholder)
        super(SimpleBar, self).__init__(**args)

class NewBarContent(BoxLayout):
    pass

class NewBar(SimpleBar):
    placeholder = NewBarContent

class MyApp(App):
    def build(self):
        return NewBar()

if __name__ == '__main__':
    MyApp().run()

Comment: **Outrageous** – both that this is even feasible *and* that this isn't actually documented anywhere. Seriously. This technique catapults the Kivy language into a full-blown widget-oriented object-oriented language complete with user-defined widget inheritance rules. That said, your last example is more than a little busted: the `SimpleBar` widget can no longer be instantiated, because it fails to define the `placeholder` property. Ideally, you'd default `placeholder` to an invisible null widget class. Does Kivy even provide an invisible null widget class, though?
Comment: @CecilCurry `kivy.uix.widget.Widget` is exactly this - a default empty base widget. If you add `placeholder = Widget` into `SimpleBar` you can use it directly.
Answer: If you want to do a composite widget, that accept new children and add them in one specific "container" widget, you have to do some python.
Basically the idea is to override add_widget so that once the basic structure of the widget is there, the new widgets are added using the new method.
Let's say you have this NestingWidget
class NestingWidget(BoxLayout):
     title = StringProperty()

     def activate(self):
         # do something
         pass

with this rule
<NestingWidget>:
    Label:
        text: root.title

    BoxLayout:

    Button:
        on_press: root.activate()

and you want to use it like this:
FloatLayout:
    NestingWidget:
        title: 'first item'
        Image:
            source: '../examples/demo/pictures/images/Ill1.jpg'

this won't work immediatly, because the Image will be added as a direct child of NestingWidget, so it'll be under the Button.
Howether, you probably remarked that some widgets in kivy can accept new nested widgets, while being already complex on their own.
The trick to do that is — as said before — to override add_widget.
First, let's add an id to our container.
<NestingWidget>:
    Label:
        text: root.title

    BoxLayout:
        id: container

    Button:
        on_press: root.activate()

then, let's use it in add_widget.
class NestingWidget(BoxLayout):
    …
    def add_widget(self, *args, **kwargs):
        if 'container' in self.ids:
            return self.ids.container.add_widget(*args, **kwargs)
        else:
            return super(NestingWidget, self).add_widget(*args, **kwargs)

Answer: With simple kv no, because if you put something in the widget (e.g. Label: ...), it'll call <widget>.add_widget() method and when such a method is called without additional parameters, it will by default place the widget after whatever was already placed before it. Therefore you can either search through the kivy/lang/parser.py file and add such a functionality(PR welcome), or do it within python in hm... __init__ or wherever you'll like to add the widget (maybe after some event).
To do that within python you can call <widget (or self)>.add_widget(<child>, index=<where>) according to the docs. For example:
from kivy.app import App
from kivy.lang import Builder
from kivy.uix.boxlayout import BoxLayout
from kivy.properties import ListProperty

Builder.load_string('''
#:import Factory kivy.factory.Factory
<Ninja@Label>:

<SimpleBar>:
    BoxLayout:
        id: my_layout
        Label:
            text: "hi"

<ChildWithBenefits>:
    placebefore:
        [(Factory.Label(text='I am the first!'), 0),
        (Factory.Ninja(text='No, I am!'), 2)]
''')

class SimpleBar(BoxLayout):
    def log(self, value):
        print(value)

class ChildWithBenefits(SimpleBar):
    placebefore = ListProperty([])
    def __init__(self, *args, **kwargs):
        super(ChildWithBenefits, self).__init__(*args, **kwargs)
        for child, index in self.placebefore:
            print child.text
            print type(child)
            self.add_widget(child, index=index)
        self.log('Hello!')

class GeneralApp(App):
    def build(self):
        return ChildWithBenefits()

if __name__ == '__main__':
    GeneralApp().run()


How does Windows 2008 remember drive letter / mount point associations?

Question: We use storage array features to clone LUNs and then map these "new" LUNs out again to the same or different hosts.  A typical case is when we refresh between environments, so imagine cloning a LUN holding a bunch of VMs from source host TEST1 to destination host TEST2.  Sometimes the destination host will assign the drive automatically, and sometimes it won't.  The storage array changes the cloned LUN's serial number (but we could modify it back) but for the rest the cloned LUN contents are identical to the source.  In the case of a refresh (so in this example remove the old LUN from TEST2 and then map the cloned LUN in it's place) the LUN ID also stays the same.  The LUNs are accessed over a Fibre Channel SAN and are thus not seen as removable devices.  Needless to say, in the situation here we are not cloning the boot LUN itself either.
I found Microsoft KB234048 describing the behavour with Windows 2000 but I suspect a lot has changed in the last 10 years.
If anyone has details, including special cases like if they are MSCS shared disks, or multiple disks with matching signatures (i.e.cloned disk is maped to same server), please share!
Answer: Well, actually not much has changed - the associations are still stored in the registry within the HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices key. Some insight about what to put in there / how to interpret the data comes from this 3rd party docs.
Comment: I didn't see mount points in the MountedDevices registry key.  How are these associations known?  Also, many MountedDevices entries are more than 12 bytes, how do I interpret these?
Comment: The mount points are filesystem references to volume objects - they are saved as JUNCTION objects in NTFS which you can see when using "dir" for example: `18.05.2011  22:36       MountTest [\??\Volume{}\]`. The reference is taken to the value within the MountedDevices key. Values in MountedDevices which are longer than 12 bytes are typically removable storage devices referenced by their device string and a volume GUID in Unicode.
Answer: For cluster storage, Windows looks at the MBR Signature or GPT Guid (depending on disk format) to identify the disk so that it can be used on other nodes in the cluster.  If two disks show up with duplicate signatures/guids I believe the first one is used and the others are left offline.  If you want a cloned disk to show up as a new disk, you'll need to change the signature/guid.


PhoneGap - Cordova 2.1 Geolocation on Android is inconsistent

Question: I am having a major issue, Geolocation is very inconsistant between Android devices.
The Geolocation on my Android Tablet is working (Samsung Galaxy), where as on my newest device Samsung Galaxy S3 - it doesn't even return a response (unless I enable enableHighAccuracy, but then it stops the other devices from working). I've tested the Geolocation in the browser and on the Google Maps app which return correct responses (and immediate responses). On the HTC it also works fine
My manifest settings are as follows
<code><uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.wifi" />
<uses-feature android:name="android.hardware.location" />
<uses-feature android:name="android.hardware.location.gps" />
<uses-feature android:name="android.hardware.location.network" />
</code>
Are there any code-changes that I am supposed to do to PhoneGap to better support GeoLocation? Even testing it out on the DEFAULT example yields the exact same results.
Testing devices

Samsung Galaxy Tablet (Android 4.0.3) . Geolocation works fine
Samsung Galaxy S3 Mobile (Android 4.1.1). Geolocation only works in enableHighAccuracy on PhoneGap. Works without GPS enabled with GoogleMaps, and works on the Internet Browser
HTC (Android 2.3.3) . Geolocation works fine
Apple iPad, iPhone and iPod (I wish Android could somehow have control over hardware like apple does :()
Comment: You mean that you want geolocation response even if geolocation is disabled ?
Comment: sorry left out alot of words from frustration haah. No Geolocation IS enabled, I ment I have to put in "enableHighAccuracy" for the new device, but it ruins the other ones... I'm editing my question now
Comment: it's better to post your testing device's Android OS versions.
Answer: Try restarting your device. Worked for me.....
Comment: Thanks^_^ ,,, This fixed my issue ,Am using Android 4.1.2 S2 .No need to set enableHighAccuracy  to True
Answer: Based on the information given, it is OS 4.1.x with PhoneGap 2.1 specific issue. After briefly searched from Google, it is a known issue, and no solution yet. Recommend to report bug directly to PhoneGap team, though they are busy migrating to new repo.
Reference: https://groups.google.com/forum/?fromgroups=#!topic/phonegap/ESrHAjFHgFU
Comment: I will try to muck around with the settings and hopefully find a solution
Comment: 4.1.1 is too new for PhoneGap. Let's wait for next PhoneGap release ( or you can try development builds )


How do I show the Auth0 login page when using FeathersJS

Question: I'm using FeathersJS as a REST service but I need the user to be able to login with Auth0. How do I show the Auth0 login page to the user?
Answer: After checking FeathersJS website, I saw you can run it paired with express. So it should be alright to use Auth0's express quickstart to build that connection. I have referenced said doc below should help you on your path. Thanks!
https://auth0.com/docs/quickstart/backend/nodejs/01-authorization


F# linq - How to filter for only unique values?

Question: my seq linq is
<@ db.Dictionaries
|> Seq.filter(fun x -> x.ID_Line = l1 || x.ID_Line = l2) @>
|> fun pquery ->
    <@ seq { for cd in db.DeltaCompares do
                for cl1 in %pquery do
                    for cl2 in %pquery do
                        yield
                            if cd.IID1 = cl1.IID && cd.IID2 = cl2.IID then
                                Some(cl1, cl2)
                            else None } @>
    |> query |> Seq.choose id
    |> Seq.collect(fun a -> [fst a; snd a])
    |> List.ofSeq

but here I need only unique values here...
Answer: Seq.distinct, or Seq.distinctBy will do what you want.
Presumably the query function can handle them, if you use it inside the quotation - but don't quote me on that.


How to use apply to subtract tuple's elements in DataFrame

Question: I have the following dataset
data = {'A': [[(1,5), (7,10), (20,22)],[(50, 52), (100, 110)], [], [(15,17)]]}
dt = pd.DataFrame(data)
print(dt)

                             A
0  [(1, 5), (7, 10), (20, 22)]
1       [(50, 52), (100, 110)]
2                           []
3                   [(15, 17)]

and I want to get a new column which is a list that each element in that list is difference of elements of tuples. For instance for row1 there are three tuples as (1,5), (7,10), (20,22) and I want to get a list that shows [4,3,2] these elements are coming from 5-1= 4, 10-7=3, and 22-20=2 .
The desired outcome could be such as:
                             A          B
0  [(1, 5), (7, 10), (20, 22)]  [4, 3, 2]
1       [(50, 52), (100, 110)]    [2, 10]
2                           []         []
3                   [(15, 17)]        [2]

I can use loop but it is not an efficient way so I tried a better solution as following:
dt['A'].apply([(lambda x: i[1]- i[0])(x) for i in x.tolist()])

Note: that x.tolist() is my attempt to make x iterateble.
My solution doesn't work, any help would be appreciated
Answer: dt['B'] = dt['A'].apply(lambda x: [b-a for a, b in x])
print(dt)

Prints:
                             A          B
0  [(1, 5), (7, 10), (20, 22)]  [4, 3, 2]
1       [(50, 52), (100, 110)]    [2, 10]
2                           []         []
3                   [(15, 17)]        [2]

Comment: @JoetheSecond each `x` is one _cell_ from column `A`. So first `x` is `[(1, 5), (7, 10), (20, 22)]`, second `x` is `[(50, 52), (100, 110)]` and so on. So when you put this `x` to for loop `for a, b in x`, then you will iterate over every element `[(a1, b1), (a2, b2), ...]` of this list.
Comment: Thanks for responding and it is working but I have a question to get a better understanding of coding.

You are taking each `x` as tuple so you can get `a` and `b` from each tuple, but I thought  each `x` should be the whole `list`
`example: [(1, 5), (7, 10), (20, 22)]`
in that cell. Why `x` is a tuple and not the whole `list`?


Python parse string containing functions, lists and dicts

Question: I'm trying to find way to parse string that can contain variable, function, list, or dict written in python syntax separated with ",". Whitespace should be usable anywhere, so split with "," when its not inside (), [] or {}.
Example string: "variable, function1(1,3), function2([1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': "dict_item_1"}"
Another example string: "variable,function1(1, 3) , function2( [1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': "dict_item_1"}"
Example output ["variable", "function1(1,3)", "function2([1,3],2)", "['list_item_1','list_item_2']", "{'dict_key_1': "dict_item_1"}"]
edit:
Reason for the code is to parse string an then run it with exec("var = &s" % list[x]). (yes i know this might not be recommended way to do stuff)
Answer: I guess the main problem here is that the arrays and dicts also have commas in them, so just using str.split(",") wouldn't work. One way of doing it is to parse the string one character at a time, and keep track of whether all brackets are closed. If they are, we can append the current result to an array when we come across a comma. Here's my attempt:
s = "variable, function1(1,3),function2([1,3],2),['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"

tokens = []
current = ""
open_brackets = 0

for char in s:
    current += char

    if char in "({[":
        open_brackets += 1
    elif char in ")}]":
        open_brackets -= 1
    elif (char == ",") and (open_brackets == 0):
        tokens.append(current[:-1].strip())
        current = ""

tokens.append(current)

for t in tokens:
    print(t)

"""
    variable
    function1(1,3)
    function2([1,3],2)
    ['list_item_1','list_item_2']
    {'dict_key_1': 'dict_item_1'}
"""

Comment: I was thinking of the same idea using a `list` as a stack of brackets, but the `open_brackets` counter works the same and is simpler.
Comment: Yeah i just thought that regex/python would have had way of doing it instead of writing algorithm my self. I will have to do that then i suppose.
Answer: Have you tried using split?
>>> teststring = "variable, function1(1,3), function2([1,3],2), ['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"
>>> teststring.split(", ")
['variable', 'function1(1,3)', 'function2([1,3],2)', "['list_item_1','list_item_2'],{'dict_key_1': 'dict_item_1'}"]

Comment: Oh yeah i forgot to mention it might or might not have whitespace after ",". For example "['variable', function1(1, 3)'"  would not work.
Answer: Regular expressions aren't very good for parsing the complexity of arbitrary code. What exactly are you trying to accomplish? You can (unsafely) use eval to just evaluate the string as code. Or if you're trying to understand it without evaling it, you can use the ast or dis modules for various forms of inspection.
Comment: Reason for the code is to parse string into list and then run it with exec(var = list[x]).
Comment: @SacredCoconut: Is there a reason not to bulk parse it all at once?
Comment: Sorry i don't know what you mean by bulk parse.
Comment: @SacredCoconut: As in, why not just do `allvals = eval(thestring)`, then index the resulting `tuple` for the results?
Comment: Oh that might work, i have most likely misunderstood how eval works
Comment: Only downside is that might want to know what the function, variable name is. But other than that that does seem most simple way. I think that or building the algorithm my self will be enough for now.


Interstitial Ads In Xcode 8

Question: I made an iOS app with xcode, I am using webview, I have already implement google admob banner ads but now i also want to show Interstitial ads in my app without click any button, I mean when my url loads Interstitial ads shoud show. I am using this code, but not works
interstitial = GADInterstitial(adUnitID: "")
    let request = GADRequest()
    // Request test ads on devices you specify. Your test device ID is printed to the console when
    // an ad request is made.
    request.testDevices = [ kGADSimulatorID, "2077ef9a63d2b398840261c8221a0c9b" ]
    interstitial.load(request)

    if interstitial.isReady {
        interstitial.present(fromRootViewController: self)
    } else {
        print("Ad wasn't ready")
    }

and in AppDelegate.m
FirebaseApp.configure()

But ads doesn't show, I can't see any error in xcode log, please help me, thanks
Answer: Do not skip to import your AppDelegate to "GoogleMobileAds" and connect the GADInterstitialDelegate. And try this in the AppDelegate if your ID's are true:
   var interstitial: GADInterstitial!
   var window: UIWindow?

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {

    GADMobileAds.configure(withApplicationID: "yourApplicationID")
    return true
}

func showInterstitial(_ notification: NSNotification) {

    interstitial = GADInterstitial(adUnitID: "youradUnitID")
    let request = GADRequest()
    interstitial.delegate = self
    // Request test ads on devices you specify. Your test device ID is printed to the console when
    // an ad request is made.
    //request.testDevices = [ kGADSimulatorID, "2077ef9a63d2b398840261c8221a0c9b" ]
    interstitial.load(request)
}

func interstitialDidReceiveAd(_ ad: GADInterstitial) {

    if(self.interstitial.isReady){
        interstitial.present(fromRootViewController: (self.window?.rootViewController)!)

    }
}

func interstitialDidFail(toPresentScreen ad: GADInterstitial) {
    NSLog("")
}

If there is a error you can show the failure in the console

Interstitial Ads In Xcode 8


Returning a memory mapped InputStream from a content provider?

Question: The the client side of a content provider consumer I can do something like this, to get a proper InputStream for reading the picture:
InputStream is = getContentResolver().openInputStream(pictureUri);

It is a nice API, and will on the server side, the actual content provider result in a call to:
public ParcelFileDescriptor openFile(Uri uri, String mode) throws FileNotFoundException {
  // Open a proper ParcelFileDescriptor, most likely using openFileHelper(uri, mode)
}

But what if the picture mapped to the URI is not to be found on the filesystem, but as a memory resource, or generated on the fly.
Can I create a memory mapped File or InputStream, or anything else, so that I am not required to save a temporary file to disk, just to be able to return it to my content provider consumer?
Answer: This is tough. You might be able to get away with using anything that can use a Socket interface.  I haven't done it, but this is what makes me think so:
http://developer.android.com/reference/android/os/ParcelFileDescriptor.html#fromSocket(java.net.Socket)
And a Socket could, in theory, be an Internet resource, or most anything...if you're willing to work at the Socket level.  I would probably just give up and create the temporary file.  Perhaps that makes me a coward.
Comment: That is a solution that might work, but not as I want to do it. I really want to be able to consistently use the `ContentResolver#openInputStream()` method for reading all kinds of remote data. Regardless of how the server creates/gets the data. It is a question of not exposing the server implementation to the client.
Comment: @PeyloW did you find a solution to this? I am looking for a solution too.
Comment: @Sunny None that is not a hack. So I ended up saving temp files and sending file descriptors.
Comment: creating a temporary file does work and is faster, better yet I'm creating a provider for caching another provider, so I have a solution that's simple and faster. One contentprovider deals with caching and pass the request to another one for generation. The only downside is wasting SD space.
Answer: As of android-9, you can make a pipe using ParcelFileDescriptor.  You can stuff up to 64k in it and be done with it, or you can set up another thread to fill it after its been read.  See the answer here for more detail:
Custom ContentProvider - openInputStream(), openOutputStream()


Pegar ultimo key do auto increment PDO

Question: Estou com um problema quanto ao meu sistema que esta o seguinte,
try {
$sql = "SELECT Nid+1 AS Nid  FROM noticias ORDER BY Nid DESC LIMIT 1";
$stmt = $DB->prepare($sql);
$stmt->bindValue(":Nid", intval($_GET["Nid"]));

$stmt->execute();
$results = $stmt->fetchAll();
$Npasta = $results[0]["Nid"];

} catch (Exception $ex) {
echo $ex->getMessage();
}

Estou pegando o ultimo ID registrado e criando uma pasta com o mesmo numero porem quando eu excluo o ultimo id a contagem passa a ser errada pois EX: inseri o id 998, apos inserir apaguei, no proximo que irei inserir ele ira pegar 998 novamente , onde o certo era pegar o id 999; Alguma luz sobre isso?
Answer: As duas saidas menos ruins para esse problema, que eu vejo são:
1) Crie uma coluna chamada excluido ou ativo pode ser bit ou boolean, no lugar de deletar o registro, faça um update nessa coluna, isso resolve parcialmente o problema de buracos na sequência.
2) Crie uma nova tabela com duas colunas uma o id e a outra o valor do último insert, para cada registro grave na sua tabela principal e nessa nova e sempre faça a leitura do último valor nessa tabela.
Comment: @DiegoFelipe, a tabela extra do seu trabalho me parece estar em outro contexto, no de log operações. Na resposta sugerir ela para evitar um delete, então para trazer todas as noticias 'não excluidas' seria algo assim, `select * from noticias where excluida = false`
Comment: Ao invés desta coluna, em um trabalho de faculdade, fui orientado a fazer uma tabela extra de auditoria, onde eu salvaria tudo que fosse excluído. Essa sugestão também é boa?
Answer: Caso a tabela tenha uma chave primária com autoincrement você pode consultar o valor do próximo registro que será inserido com a consulta abaixo, pois o valor do AUTO_INCREMENT não é alterado por exclusões de linhas.
SELECT `AUTO_INCREMENT`
FROM information_schema.`TABLES`
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'noticias';

Essa maneira deve fazer o mesmo efeito do Nid+1 da pergunta.
Comment: ele retorna o valor da propriedade `AUTO_INCREMENT` da tabela, o campo não é mencionado na consulta porque depende da definição dos campos, no seu caso vai funcionar se o campo `Nid` tiver a propriedade `AUTO_INCREMENT` ativa
Comment: Parece a melhor opção :)
Comment: é bem isso mesmo vou testar aqui
Comment: mais aqui nao entendi como vou selecionar o campo que pego o ID pois em nenhum momento ai da sua consulta vc informou qual é o campo ( no meu caso Nid). SELECT `AUTO_INCREMENT`
FROM tabela esse Where serve para q?
Answer: pesquisando sobre o assunto achei esse metodo que é na minha opiniao o mais seguro pois ele pega diretamente o valor do auto_increment segue o codigo final do problema resolvido
try {
$sql = "SHOW TABLE STATUS LIKE 'nomedatabela' ";
$stmt = $DB->prepare($sql);
$stmt->execute();
$resultado = $stmt->fetch();
$proximoID = $results['Auto_increment'];  // a chave esta aqui
} catch (Exception $ex) {
 echo $ex->getMessage();
}
echo "$proximoID";

Feito isso mesmo que seja deletado o id ele vai sempre buscar o proximo a ser inserido
Comment: Quando disse seguro, disse em questao de realmente pegar o proximo ID da forma mais segura possivel pois é o proprio auto increment, quanto a consulta tambem pode se tornar mais seguro caso use outro metodo
Comment: Seguro não é, se tiver duas tabelas com nomes parecidos ...

Pegar ultimo key do auto increment PDO


doctrine2 ORM and naming strategy

Question: How doctrine has to be configured, to use underscore field name rather than camelCase?
a part of doctrine configuration in my symfony app:
doctrine:
    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        naming_strategy: doctrine.orm.naming_strategy.underscore

but doctrine:mapping:import generates everything camelCased
$qb->select('partial row.{ my_underscored_field })

throws "[Semantical Error] line 0, col 344 near 'my_underscored_field =': Error: Class My\Enity\Namespace has no field or association named my_underscored_field"
whilst
$qb->select('partial row.{ myUnderscoredField })

returns my expected result
Answer: In this call:
$qb->select('partial row.{ my_underscored_field })

my_underscore_field is the property of your entity.
The naming_strategy is used only for MySQL columns name, not in QueryBuilder or in class definition.
Comment: so, in order to use underscored names in queryBuilder i have to define entity property as my_undersoce_field, am i right?

doctrine2 ORM and naming strategy


Hensel's lemma & $p$-adic polynomial roots

Question: I want to determine the number of roots of $f(X) = X^3-5X+20$ in $\mathbb{Z}_p$ using Hensel's lemma (lemma is on the bottom). Unfortunately I am not very well trained to solve this. Take for example $p=2$. Then reducing the polynomial modulo 2 we get
\begin{align*}
f(X) &= X^3 - 5X+20 \equiv 0 \mod 2 \\
f'(X) &= 3X^2-5\equiv X + 1 \mod 2
\end{align*}
That would mean that only $0\in\mathbb{Z}/2\mathbb{Z}$ gives a solution such that $f(0) \equiv 0$ and $f'(0)\not \equiv 0$ modulo 2, which then by Hensel's lemma gives a unique root of $f$ in $\mathbb{Z}_2$, which is $0$ modulo $2$.
The problem is that I don't know the relation between finding a root via Hensel's lemma and there really being one. I have found one root via Hensel's lemma, but what guarantees me there aren't more?
Any help on how to proceed is most welcome. Thank you in advance.

The lemma I am using:
Let $K$ be a complete field with discrete valuation $v$, valuation ring $O$ and residue field $k$. Let $f(X)\in O(X)$. Suppose that the reduction $\bar{f}(X)\in k[X]$ has a simple root $\bar{a}$, ie $\bar{f}(\bar{a}) = 0$ and $\bar{f}'(\bar{a})\neq0$. Then there exists a unique $x\in O$ such that $f(x) = 0$ and $\bar{x}=\bar{a}$.
Comment: Are you lifting roots mod $p$ to roots mod powers of $p$?  That's what Hensel's Lemma is about. There are $2$ roots mod $2$, by inspection.
Comment: I added the lemma as I learned it. Could you explain how you determined there are two roots?
Comment: Plugged in.  But that may be because my meaning for $\mathbb{Z}_2$ is different from yours.
Comment: Ah my communication is so off.... I meant the $2$-adic number field.
Answer: This polynomial has a double root at $1$ in $\mathbb{F}_2$, so the out of the box Hensel's lemma won't apply.  However we can still apply a modified version by going one power of two up. In particular the double root at 1 splits into two simple roots mod 4 (at 1 and 3).  Now a modified Hensel's lemma (with essentially the same proof as the usual version) gives us that these each have a unique 2-adic lift.
One thing that is different about this modified Hensel's lemma from what you are used to is that if we want to solve $f(x) \equiv 0 \bmod 2^n$, the lifts we get will only be unique mod $2^{n-1}$.  In particular we get that $f(x) \equiv 0 \bmod 2^n$ has 5 solutions for all $n > 2$.
Comment: Thank you very much! I looked at the proof and was able to adjust the assumption to powers of $2$. One more question, is there a way to see whether a root splits in different ones in a higher power? Take $p=5$ for example. Then $f$ has a triple root at zero modulo 5, which (I believe) are no roots anymore when we look modulo 25. It would be exhausting (and impossible) to check all powers. Thank you.
Comment: I think the strategy is to increase the prime power one step at a time. If you ever have a root such that the derivative does not vanish then it automatically lifts to all higher powers by this modified Hensel's lemma.  If you reach a power such that there are no solutions lifting the one you are looking at then there are no p-adic solutions.  One thing to potentially be worried about is maybe you keep going up a prime power finding lifts but having the derivatives keep vanishing, but I think one can show that implies the polynomial has a double root.
Comment: For $p=5$, Eisenstein applies, since all coeffs are divisible by $5$ and the constant is not divisible by $25$. So irreducible, no roots in $\Bbb Z_5$.
Answer: Just for the number of roots in the $2$-adic integers $\Bbb Z_2$, I recommend forming $g(X)=f(X+5)=X^3+15X^2+70X+120$. Then I use something rather more powerful than plain old Hensel, namely the Newton polygon, in which you plot $(n,v_2(a_n))$ for every monomial $a_nX^n$ in your polynomial. The points you plot then are $(0,3)$, $(1,1)$, $(2,0)$, and $(3,0)$. You then take the "upper convex hull" of these points, which has segments of slope $-2$, $-1$, and $0$, each of width $1$. This says that in $\Bbb Z_2$, there are roots of $2$-adic valuation $2$, $1$, and $0$, three in all. You may be able to take $g$ and find the roots directly with your form of Hensel.


Main page content block doesn't move

Question: I'm currently building a Drupal 9 site.
I've built some custom blocks, which I display on a specific 'basic page'.
I want the 'Main Page Content' block to be above my custom blocks. So I moved the 'Main Page Content' block upwards. For some reason it didn't update, and my custom blocks are still on top.
Does anyone know why?
Comment: First, always clear the cache.  Next: Where are the custom blocks still on top?  In the layout builder UI?  Or are they on bottom in layout builder but on top in the actual page?
Comment: @PatrickKenny I already clear cache with every change. In the layout builder the main page content is on top. And on the actual page the custom content is on top.
Comment: What does the page source look like?  Check for the layout builder classes in the raw HTML, and consider updating your question here to show the actual markup that Drupal is outputting.
Comment: @PatrickKenny I managed to fix it. I only had main page content shown on a couple of specific pages. Added the needed page to it, and it directly got placed on the right direction. Altough that page wasn't in the list of pages before, I'm guessing that whenever 'Main page content' is enabled, that it automatically gets placed under all other blocks
Comment: If you solved the issue, please answer your own question instead of posting it in a comment.  That way you can later mark the question solved, so it won't show up as unanswered in the search results.
Answer: I managed to fix the issue. I only had main page content shown on a couple of specific pages. Added the needed page to it, and it directly got placed on the right direction. Although that page wasn't in the list of pages before, I'm guessing that whenever 'Main page content' is enabled, that it automatically gets placed under all other blocks

Main page content block doesn't move


Difference in success of mice imputation across variables

Question: I'm trying to use mice package to do imputations, I'm pretty new to all of this. I have 9 continuous variables and I used the following code to impute, using the 'cart' (Classification and regression trees) method:
imp <- mice(incomplete.data, method="cart", m=10, maxit=20, seed=456,
            pred=quickpred(incomplete.data, method="spearman"))

All imputations look fine, except two: '2' and '8' (see density plots below). I am puzzled why these two do not perform well and what could be done to improve them.
Initially I tried to use the default 'pmm' method for all of the imputations. However I got the following error code:
Error in solve.default(xtx + diag(pen)) : system is computationally singular:
 reciprocal condition number = 1.83676e-18'

This is why I switched to the method 'cart'. Any ideas how to improve this please? I'm stuck.
Comment: I understand, apologies.
'pmm' = predictive mean matching
'cart' = Classification and regression trees
Comment: Perhaps you could edit your question to tell us what the two methods you are using are?
Comment: Methods are in the code. The first method, that failed, was 'pmm'. Then I tried 'cart' which worked fine except for the two variables. Does that make sense?
Comment: No it does not. What does pmm involve? What does cart involve?  There is a danger that this would be closed as off-topic as asking about R code but you probably do have a statistical question too. The chances of someone answering that is improved if the question is thrown open to people who do not use MI by chained equation solely in R
Comment: Check the variables that you're using as predictors. I had this error when using complete (i.e. no missingness) as predictor that were not strictly continuous. Once I took them out the imputations ran fine
Answer: Based on your plots, you can not say which of the distributions are correct or wrong. These kind of plots are usually used to see whether the multiple imputations lead to similar imputed values (i.e. when the red lines of one plot would be completely different from each other, your imputation might be too unstable).
The difference of the red and blue lines in plot 2 and 8 might result from the response mechanism of your data. Consider the following example in R:
set.seed(3456789) # Reproducible example
N <- 1000 # Sample size

y_all <- rnorm(N, 10, 2) # Dependent variable
x1 <- y_all + rnorm(N, 0, 5) # Predictor 1
x2 <- y_all + rnorm(N, 20, 2) # Predictor 2
x3 <- y_all + rnorm(N, 5, 3) # Predictor 3
x4 <- y_all + rnorm(N, 15, 10) # Predictor 4
x5 <- y_all + rnorm(N, 10, 3) # Predictor 5

# Insert missing values according the response mechanism Missing At Random (MAR)
y_obs <- y_all
y_obs[(x1 + x2 + x3 + x4 + x5) > mean(x1 + x2 + x3 + x4 + x5)] <- NA # Observed values of Y
y_miss <- y_all[is.na(y_obs)] # Missing values of Y (for comparison)

# Plot observed and missing Values of Y
# This is only possible because we know the missing values - impossible in real life
plot(density(y_all),
     xlim = c(2.5, 17.5),
     ylim = c(0, 0.4),
     xlab = "Y",
     main = "Observed and Missing Values of Y",
     lwd = 2)
lines(density(y_obs[!is.na(y_obs)]),
      col = "cornflowerblue",
      lwd = 2)
lines(density(y_miss),
      col = "firebrick3",
      lwd = 2)
legend("topleft",
       c("All", "Observed", "Missing"),
       lty = 1,
       col = c(1, "cornflowerblue", "firebrick3"))

incomplete.data <- data.frame(y_obs, x1, x2, x3, x4, x5) # Create data.frame

# Impute data with your specifications
library("mice")

# Apply mice function
imp <- mice(incomplete.data,
            method = "cart",
            pred = quickpred(incomplete.data, method = "spearman"),
            m = 10,
            maxit = 20,
            seed = 456)

# Density plot of observed and imputed values
densityplot(imp, xlim = c(2.5, 17.5), ylim = c(0, 0.4))

In the second graphic you can see that the imputed values are very different from the observed values. However, they are much closer (i.e. much better) to the missing values that are indicated by the red line in the first graphic. In other words, by imputing different/higher values compared to your observed values, you are reducing bias (which is exactly what you want).
To get back to your example: You should evaluate whether it is logical from a theoretical viewpoint that the imputed values of plots 2 and 8 are higher than the observed values.
Furthermore, I would encourage you to investigate why your code has problems with pmm, since predictive mean matching is usually better than cart.


MySQL SUM() always returns a 0 value

Question: I have amounts stored as a varchar in my table.
When attempting to sum them it always returns 0.00.
Below is an example using only one record from the db.
SELECT col1, SUM(CAST(col2 AS DECIMAL(20,2))) derived1
                    FROM table
                    WHERE col3 = 'FIT'
                    AND col1 = '6211195'
                    GROUP BY col1

This returns one row with a 0 value.
By removing the SUM and CAST from the query, I can see that it is pulling the value as it should, but I can not sum or cast it, adding either of those breaks it and returns 0 again.
I have also tried converting the field to a decimal type and it just zeros all the values.
EDIT:
Ughh, I just ran a REGEX query to detect anything that isnt an alphanumeric value or a decimal point. It appears that there are non ascii characters in the field that I cant see, messing with the type casting. Will continue to update as I learn more.
Comment: what is the datatype of col2AS ?
Comment: Sorry, COL2 is a VARCHAR
Comment: Ughh, I just ran a REGEX query to detect anything that isnt an alphanumeric value or a decimal point. It appears that there are non ascii characters in the field that I cant see, messing with the type casting. Will continue to update as I learn more.
Answer: Show some sample values of col2.  A string that really is a number is treated as a number.  Hence '1' will become 1.
However, if the string does not start with a digit,  '-', or '+' (after leading spaces), then it will be 0 (in most cases).  So 'A1' will be 0.  And so on.  As will '$100'.
The lesson is:  If a column contains numbers, store them as numbers.  Really simple.

MySQL SUM() always returns a 0 value


Mounted VHD gives access denied during file copy

Question: I want to copy files from VM to another. I'm running windows 2008 on the VM Manager. Both machines are Server 2012. I tuned them both off. Mounted the VHDs to the manger pc. I can see the both files and folders, but cannot copy.
I want to move files from Windows/WinSxS to the other machine Winsxs.
<code>robocopy I:\Windows\WinSxs K:\Windows\WinSxs
</code>
After that it loops on access denied, waiting 30 seconds.
Why am I getting access denied. I'm using a domain admin account and administrator power shell.
Comment: Why are you copying stuff from WinSXS?
Comment: Now that is quite the story. in a nutshell, I'm attempting to get my fully updated server core to install the gui. The fact that it is updated seems to be the root issue.
Answer: You have to give permission to this folder and the sub-folders to copy files from this folder. If the destination folder is also a machines working folder then you have to give permission to that folder too. Read http://answers.microsoft.com/en-us/windows/forum/windows_7-files/i-need-to-copy-some-files-into-the-winsys-folder/8f6c3a6f-3483-e011-9b4b-68b599b31bf5?auth=1 , this will surely help


Example of distribution where large sample size is necessary for central limit theorem

Question: Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$.
I know this isn't enough for all distributions.
I wish to see some examples of distributions where even with a large sample size (perhaps 100, or 1000, or higher), the distribution of the sample mean is still fairly skewed.
I know I have seen such examples before, but I can't remember where and I can't find them.
Comment: Any sufficiently contaminated distribution will work, along the same lines as @Glen_b's example. *E.g.*, when the underlying distribution is a mixture of a Normal(0,1) and a Normal(huge value, 1), with the latter having only a tiny probability of appearing, then interesting things happen: (1) most of the time, the contamination does not appear and there is no evidence of skewness; but (2) sometimes the contamination appears and the skewness in the sample is enormous. The distribution of the sample mean will be highly skewed regardless but bootstrapping (*e.g.*) will usually not detect it.
Comment: @whuber's example is instructive, showing that the central limit theorem can, in theory, be arbitrarily misleading. In practical experiments, I suppose one needs to ask oneself whether there could be some huge effect that occurs very rarely, and apply the theoretical result with a little circumspection.
Answer:
Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$.

This common rule of thumb is pretty much completely useless. There are non-normal distributions for which n=2 will do okay and non-normal distributions for which much larger $n$ is insufficient - so without an explicit restriction on the circumstances, the rule is misleading. In any case, even if it were kind of true, the required $n$ would vary depending on what you were doing. Often you get good approximations near the centre of the distribution at small $n$, but need much larger $n$ to get a decent approximation in the tail.
Edit: See the answers to this question for numerous but apparently unanimous opinions on that issue, and some good links. I won't labour the point though, since you already clearly understand it.

I am wanting to see some examples of distributions where even with a large sample size (maybe 100 or 1000 or higher), the distribution of the sample mean is still fairly skewed.

Examples are relatively easy to construct; one easy way is to find an infinitely divisible distribution that is non-normal and divide it up. If you have one that will approach the normal when you average or sum it up, start at the boundary of 'close to normal' and divide it as much as you like. So for example:
Consider a Gamma distribution with shape parameter $α$. Take the scale as 1 (scale doesn't matter). Let's say you regard $\text{Gamma}(α_0,1)$ as just "sufficiently normal". Then a distribution for which you need to get 1000 observations to be sufficiently normal has a $\text{Gamma}(α_0/1000,1)$ distribution.
So if you feel that a Gamma with $\alpha=20$ is just 'normal enough' -

Then divide $\alpha=20$ by 1000, to get $\alpha = 0.02$:

The average of 1000 of those will have the shape of the first pdf (but not its scale).
Edit: a warning: very small shape parameters on the gamma can lead to numerical issues, so if you push it hard you'll get problems. Try the Poisson, perhaps, which is also infinitely divisible.
A convenient example that doesn't take advantage of infinite divisibility is the lognormal. Try $\sigma$ parameters in the vicinity of $3$ to $4$ for ones that need really large $n$ to work; it's usually not too difficult to pin down a $\sigma$ that gets close to the edge of some judgement of sufficiently close to normal at some given $n$.
If you instead choose an infinitely divisible distribution that doesn't approach the normal, like say the Cauchy, then there may be no sample size at which sample means have approximately normal distributions (or, in some cases, they might still approach normality, but you don't have a $\sigma/\sqrt n$ effect for the standard error).
@whuber's point about contaminated distributions is a very good one; it may pay to try some simulation with that case and see how things behave across many such samples.
Answer: In addition to the many great answers provided here, Rand Wilcox has published excellent papers on the subject and has shown that our typical checking for adequacy of the normal approximation is quite misleading (and underestimates the sample size needed).  He makes an excellent point that the mean can be approximately normal but that is only half the story when we do not know $\sigma$.  When $\sigma$ is unknown, we typically use the $t$ distribution for tests and confidence limits.  The sample variance may be very, very far from a scaled $\chi^2$ distribution and the resulting $t$ ratio may look nothing like a $t$ distribution when $n=30$.  Simply put, non-normality messes up $s^2$ more than it messes up $\bar{X}$.
Comment: This is a good point to make; it's often not actually the mean that people deal with but some function of it and other things. However it's not only $s^2$ that can be messed up - you also lose independence of numerator and denominator, and that can have some surprising effects in the tails.
Answer: You might find this paper helpful (or at least interesting):
http://www.umass.edu/remp/Papers/Smith&Wells_NERA06.pdf
Researchers at UMass actually carried out a study similar to what you're asking.  At what sample size do certain distributed data follow a normal distribution due to CLT? Apparently a lot of data collected for psychology experiments is not anywhere near normally distributed, so the discipline relies pretty heavily on CLT to do any inference on their stats.
First they ran tests on data that was uniform, bimodal, and one distibution that was normal. Using Kolmogorov-Smirnov, the researchers tested how many of the distributions were rejected for normality at the $\alpha = 0.05$ level.
Table 2. Percentage of replications that departed normality based on the KS-test.
 Sample Size
           5   10   15   20   25  30
Normal   100   95   70   65   60  35
Uniform  100  100  100  100  100  95
Bimodal  100  100  100   75   85  50

Oddly enough, 65 percent of the normally distributed data were rejected with a sample size of 20, and even with a sample size of 30, 35% were still rejected.
They then tested several several heavily skewed distributions created using Fleishman's power method:
$Y = aX + bX^2 +cX^3 + dX^4$
X represents the value drawn from the normal distribution while a, b, c, and d are
constants (note that a=-c).
They ran the tests with sample sizes up to 300
Skew  Kurt   A      B      C       D
1.75  3.75  -0.399  0.930  0.399  -0.036
1.50  3.75  -0.221  0.866  0.221   0.027
1.25  3.75  -0.161  0.819  0.161   0.049
1.00  3.75  -0.119  0.789  0.119   0.062

They found that at the highest levels of skew and kurt (1.75 and 3.75) that sample sizes of 300 did not produce sample means that followed a normal distribution.
Unfortunately, I don't think that this is exactly what you're looking for, but I stumbled upon it and found it interesting, and thought you might too.
Comment: @Glen_b: There are multiple sources of potential error here. If you read the document, you'll note that what is listed as "normal" here is actually normal random variates with mean 50 and standard deviation of 10 *rounded to the nearest integer*. So, in that sense, the test used is already using a misspecified distribution. Second, it *still* appears they have performed the tests incorrectly, as my attempts at replication show that for a sample mean using 20 such observations, the rejection probability is about 27%. (cont.)
Comment: (cont.) Third, regardless of the above, some software may use the *asymptotic* distribution and not the actual one, though at sample sizes of 10K this shouldn't matter too much (if ties had not been artificially induced on the data). Finally, we find the following rather strange statement near the end of that document: *Unfortunately, the properties of the KS-test in S-PLUS limit the work. The p-values for the present study were all compiled by hand over the multiple replications. A program is needed to calculate the p-values and make a judgment on them compared to the alpha level chosen.*
Comment: "*Oddly enough, 65 percent of the normally distributed data were rejected with a sample size of 20, and even with a sample size of 30, 35% were still rejected.*" -- then it sounds like they're using the test wrong; as a test of normality on completely specified normal data (which is what the test is for), if they're using it right, *it must be exact*.
Comment: Hi @Glen_b. I don't believe the rounding will reduce the rejection rate here because I believe they were testing against the *true* standard normal distribution using the *rounded* data (which is what I meant by saying the test used a misspecified distribution). (Perhaps you were, instead, thinking of using the KS test on a discrete distribution.) The sample size for the KS test was 10000, not 20; they did 20 replications at sample size 10000 each to get the table. At least, that was my understanding of the description from skimming the document.
Comment: @cardinal - you're correct, of course, so perhaps that could be the source of a substantial chunk of the rejections at large sample sizes. Re: "*The sample size for the KS test was 10000, not 20*" ... okay, this is sounding increasingly odd. One is left to wonder why they'd think either of those conditions was of much value, rather than say the other way around.
Comment: @cardinal thanks for that comprehensive list. The possibility of rounding (discreteness) occurred to me but that *reduces* the rejection rate. The asymptotic distribution one did occur to me too but even at $n=20$ the approximation is pretty good and in any case actually *reduces* the rejection rate. Consequently I knew neither of those could account for the problem. Indeed, the large observed rejection rate would actually be slightly *worse* if those two were in effect and were then corrected.
Comment: I feel that I should have mentioned the rounding thing on the first distributions. I did see it and forgot to mention it.  That's my fault.


Put SQL query within string with parameters

Question: Have some problems to place long sql query to command but i always have some problem - can you help me out with it?
This is my query i want to put within cmd with two parameters newvalue and oldvalue
query itself:
    UPDATE tbElemPics
    SET    PicturePath = Stuff(PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)
                                          + 1, Charindex('/', PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)
                                                                          + 1) - Charindex('/', PicturePath, Charindex('/', PicturePath) + 1) - 1, 'newvalue')
    WHERE  Substring(PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)
                                 + 1, Charindex('/', PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)
                                                                 + 1) - Charindex('/', PicturePath, Charindex('/', PicturePath) + 1) - 1) = 'oldvalue'

And method with cmd where above sql should be placed with two parameters:
 Public Sub ChangeMachineNames(OldMachName As String, NewMachName As String)
        Dim strcon = New AppSettingsReader().GetValue("ConnectionString", GetType(System.String)).ToString()
        Using con As New SqlConnection(strcon)
            Using cmd As New SqlCommand("here i want to put my sql", con)
                cmd.CommandType = CommandType.Text
                cmd.Parameters.AddWithValue("@newvalue", NewMachName)
                cmd.Parameters.AddWithValue("@oldvalue", OldMachName)
                con.Open()
                Dim rowsAffected As Integer = cmd.ExecuteNonQuery()
                con.Close()
            End Using
        End Using
    End Sub

Comment: If you explain what type of query you want to generate, probably you may get some more answers, may be new approach to solve your problem.
Answer: Just put it in a variable and then assign that variable to Command
Dim strCmd As String = ""
strCmd &= " UPDATE tbElemPics"
strCmd &= " SET    PicturePath = Stuff(PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)"
strCmd &= " + 1, Charindex('/', PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)"
strCmd &= " + 1) - Charindex('/', PicturePath, Charindex('/', PicturePath) + 1) - 1, @newvalue)"
strCmd &= " WHERE  Substring(PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)"
strCmd &= " + 1, Charindex('/', PicturePath, Charindex('/', PicturePath, Charindex('/', PicturePath)+1)"
strCmd &= " + 1) - Charindex('/', PicturePath, Charindex('/', PicturePath) + 1) - 1) = @oldvalue"

Using cmd As New SqlCommand(strCmd, con)

Comment: Don't know what you are trying to do here. It seems you are updating some paths stored in the database. Check your brackets. But the way to provide parameters is this. Put `@` sign before your param name and without quotes `( ' )`
Comment: The way to use parameters is with `@` but I have not tested it with Stuff function.
Comment: hi you sure there is no typo ? ^^
Comment: sorry... i updated my main post - how to change also your string to pass those two parameters - i marked them within query
Comment: exactly i want to pass two values into this query using parameters. Can you assure me its fine formatted? ^^
Comment: seems its ok like you posted - thx marked as answer. If you could just take a look last time as i am putting it to production would be apreaciated and give me feedback :)
Comment: sorry one more thing i forgot to mention i got something like that at the end of my query:   ) = N'oldvalue'    - as you see there additional N  , can i just put it into the string no more chars are needed for that means like this is enough?:  = N@oldvalue"
Comment: No need to put this additional N, you can just use it as = @oldValue, N is just a SQL Server way to explicitly mark a value as String. And regarding parameters in stuff: it should work as described here http://www.sql-server-helper.com/tips/tip-of-the-day.aspx?tkey=3934817c-1a03-4ac9-a0ba-55b2bfbaea0f&tkw=uses-of-the-stuff-string-function
Comment: i need it because if specific chars appear e.g ą ę ż etc.. then i need instead of    = 'oldvalue'     = N'oldvalue'       . What will be the diffrence in case of our string to attach this N?
Comment: Any values passed using parameters are automatically converted to the relevant column type. i.e. if column type is date, int, nvarchar then @param will be automatically converted to date, int and nvarchar respectively. So you need not specify N


truncate email field value before @ and assign to another field

Question: hopefully that makes sense.... basically I have a form and want to take the input for the "email" field and truncate/trim the value before the "@" symbol and give that value to another hidden field.
EDIT/ADDED: Also, if possible in the same function, it would be great to automatically generate and append a random number (between 01 and 99 would suffice) to ensure there are no "duplicates/matching "hiddenfield" values... for example if the email address is something nondescript like firstname.lastname@example.com, where "info" might be blocked if already in system but info46 and info07 would be fine.
IF it is not appropriate to add to the question like this I apologize and will edit it out, mark the question as answered based on original criteria and open a new question that hopefully can add to this.
So basically if someone entered "firstname.lastname@example.com", "JohnSmith26" (as an example) would be assigned as the value for another hidden form field.
example:
Email Address:<input type="text" id="email" name="email" value=""     size="30" maxlength="80"/>

    <input type="hidden" id="hiddenfield" name="hiddenfield" value="truncated email" />

I saw this thread but am not using coldfusion etc and am hoping to just find a simple way to do similar within the form: how do I trim an email input address so only the data before the @ is inputted into the database?
am looking for the best way using javascript/jQuery to assign the truncated value to the other field.
Comment: Check here https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split
Answer: First split the email address, then use regex to remove white spaces and then assign it to the hidden field.
var email = document.getElementById('email').value.split('@')[0],
    hidden = document.getElementById('hiddenfield'),
    trimmedValue = email.replace(/\s+/g, '');

hidden.value = trimmedValue ;

Comment: An e–mail address shouldn't have any white space, so you can use something like: `/([^\s@]+)@/` to get all non–whitespace characters before the @, e.g. `(emailAddress.match(/([^\s@]+)@/) || [])[1] || '';`
Answer: You probably want something like this
Javascript
var email = document.getElementById("email"),
    hidden = document.getElementById("hiddenfield");

function transferTruncated() {
    var target = this,
        name;

    if (target.value.indexOf("@") !== -1) {
        name = target.value.split("@")[0].trim();

        if (name && name.search(/\s/) === -1) {
            hidden.value = name;
        } else {
            hidden.value = "truncated email";
        }
    } else {
        hidden.value = "truncated email";
    }
}

email.addEventListener("change", transferTruncated, false);

On jsfiddle
Note: I removed the hidden type from the input field so that you can see it, and you may want to check that the name is further valid too before assigning it.
Comment: wow that was fast AND spot on! I unfortunately forgot to include another aspect which I have edited/added above now. But, if that is not acceptable to edit questions once worked on/answered I will remove it and mark this as solved based on your solution. Hoping that it can be added to this function though.
Comment: It can be easily done, just have another function that creates the random number and do this `hidden.value = name + randomNumber();`. If you don't know how to generate such a number it would probably be best to post a separate question (or just search SO I'm sure there are plenty of examples), based on the title of this current question.


Números vermelhos no Visual Studio

Question: Estava testando a biblioteca Colorama do Python e me ocorreu isso, alguém sabe o porquê? Não consigo rodar de jeito nenhum, aparece esse erro:
print (\033[33m'hello')
                          ^
SyntaxError: unexpected character after line continuation character

O que eu faço?
Answer: Estão faltando aspas para as sequências de escape ANSI. Tente o seguinte:
print('\033[33m' + 'hello')

Comment: muito obrigada!!

Números vermelhos no Visual Studio


How do I count the number of nodes in a View

Question: I would like to display the number of nodes in a particular view OUTSIDE OF THAT VIEW. For example, on the home page (page--front.tpl.php) I want to insert a simple php echo statement that pulls the number of nodes in "View X", and another php echo statement that pulls the number of nodes from "View Y", etc.
<code>function theme_get_node_count($content_type) {
     $query = "SELECT COUNT(*) amount FROM {node} n ".
              "WHERE n.type = :type";
     $result = db_query($query, array(':type' => $content_type))->fetch();
     return $result->amount;
}
</code>
The previous code does not work because I am not counting a content type. Is there a $view variable I can plugin into this function to achieve the same results?
Thanks
Answer: In an executed view object, you can access how many rows or results are returned by looking at:
<code>$count=count($view->results);
</code>
So, if your view returns 2 rows, $count would be 2, if 1, $count would be 1, etc. This can be accessed from a module that executed a view itself or from inside almost any of the views templates as well.


read mixed data into R

Question: I have a text file '\t' separated. First two columns are text and third one is in JSON format like {type: [{a: a1, timestamp: 1}, {a:a2, timestamp: 2}]}
How can i put it into DF correctly?
I would like to parse line like factor1\tparam1\t{type: [{a: a1, timestamp: 1}, {a:a2, timestamp: 2}]} into DF like
factor_column  param_column   a_column  ts_column
    factor1        param1        a1         1
    factor1        param1        a2         2

Answer: I have saved that one line of text you have provided into a file called 'parseJSON.txt'. You can then read the file in as per usual using read.table, then make use of library(jsonlite) to parse the 3rd column.
I've also formatted the line of text to include quotes around the JSON code:
factor1    param1  {"type": [{"a": "a1", "timestamp": 1}, {"a":"a2", "timestamp": 2}]}
library(jsonlite)

dat <- read.table("parseJSON.txt",
              sep="\t",
              header=F,
              quote="")

#parse 3rd column using jsonlite
js <- fromJSON(as.character(dat[1,3]))

js is now a list
> js
$type
   a timestamp
1 a1         1
2 a2         2

which can be combined with the first two columns of dat
res <- cbind(dat[,1:2],js$type)

names(res) <- c("factor_column", "param_column", "a_column", "ts_column")

which gives
> res
     factor_column param_column a_column ts_column
1       factor1       param1       a1         1
2       factor1       param1       a2         2

Comment: Thanks, tospig! Trouble is that my input file is huge and I can't just add quotes manualy as you did. Any suggestion how to parse 'almost-json' line?
Comment: I would first check how the data is generated and see if you can produce 'actual' JSON with the quotes around the `key` strings. If not there may be a way using regular expressions to put quotes around alphanumeric strings, but that's a whole other question
Comment: @crusoe [This question](http://stackoverflow.com/a/29550840/4002530) may be of use for extracting the text to put quotes around?

read mixed data into R


Object Required when trying to label y axis

Question: I am trying to use Excel Macros for the first time, but for some reason the corresponding VBA code doesn't like it when I try to label the y axis. I get a run time error 424, and then it says object required.
<code>Sub Graphplotz()
'
' Graphplotz Macro
'

'

    Range("A2:B15").Select
    ActiveSheet.Shapes.AddChart2(240, xlXYScatterSmooth).Select
    ActiveChart.SetSourceData Source:=Range("Plot!$A$2:$B$15")
    ActiveChart.ChartTitle.Select
    Selection.Format.TextFrame2.TextRange.Characters.Text = _
        "Stress vs. Deformation"
    With Selection.Format.TextFrame2.TextRange.Characters(1, 22).ParagraphFormat
        .TextDirection = msoTextDirectionLeftToRight
        .Alignment = msoAlignCenter
    End With
    With Selection.Format.TextFrame2.TextRange.Characters(1, 6).Font
        .BaselineOffset = 0
        .Bold = msoFalse
        .NameComplexScript = "+mn-cs"
        .NameFarEast = "+mn-ea"
        .Fill.Visible = msoTrue
        .Fill.ForeColor.RGB = RGB(89, 89, 89)
        .Fill.Transparency = 0
        .Fill.Solid
        .Size = 14
        .Italic = msoFalse
        .Kerning = 12
        .Name = "+mn-lt"
        .UnderlineStyle = msoNoUnderline
        .Spacing = 0
        .Strike = msoNoStrike
    End With
    With Selection.Format.TextFrame2.TextRange.Characters(7, 16).Font
        .BaselineOffset = 0
        .Bold = msoFalse
        .NameComplexScript = "+mn-cs"
        .NameFarEast = "+mn-ea"
        .Fill.Visible = msoTrue
        .Fill.ForeColor.RGB = RGB(89, 89, 89)
        .Fill.Transparency = 0
        .Fill.Solid
        .Size = 14
        .Italic = msoFalse
        .Kerning = 12
        .Name = "+mn-lt"
        .UnderlineStyle = msoNoUnderline
        .Spacing = 0
        .Strike = msoNoStrike
    End With
    ActiveChart.SetElement (msoElementPrimaryCategoryAxisTitleAdjacentToAxis)
    Selection.Format.TextFrame2.TextRange.Characters.Text = "Deformation"
    With Selection.Format.TextFrame2.TextRange.Characters(1, 11).ParagraphFormat
        .TextDirection = msoTextDirectionLeftToRight
        .Alignment = msoAlignCenter
    End With
    With Selection.Format.TextFrame2.TextRange.Characters(1, 11).Font
        .BaselineOffset = 0
        .Bold = msoFalse
        .NameComplexScript = "+mn-cs"
        .NameFarEast = "+mn-ea"
        .Fill.Visible = msoTrue
        .Fill.ForeColor.RGB = RGB(89, 89, 89)
        .Fill.Transparency = 0
        .Fill.Solid
        .Size = 10
        .Italic = msoFalse
        .Kerning = 12
        .Name = "+mn-lt"
        .UnderlineStyle = msoNoUnderline
        .Strike = msoNoStrike
    End With
    ActiveChart.SetElement (msoElementPrimaryValueAxisTitleAdjacentToAxis)
    ActiveChart.Axes(xlValue, xlPrimary).AxisTitle.Text = "Stress(Mpa)"
    Selection.Format.TextFrame2.TextRange.Characters.Text = "Stress(Mpa)"
    With Selection.Format.TextFrame2.TextRange.Characters(1, 11).ParagraphFormat
        .TextDirection = msoTextDirectionLeftToRight
        .Alignment = msoAlignCenter
    End With
    With Selection.Format.TextFrame2.TextRange.Characters(1, 11).Font
        .BaselineOffset = 0
        .Bold = msoFalse
        .NameComplexScript = "+mn-cs"
        .NameFarEast = "+mn-ea"
        .Fill.Visible = msoTrue
        .Fill.ForeColor.RGB = RGB(89, 89, 89)
        .Fill.Transparency = 0
        .Fill.Solid
        .Size = 10
        .Italic = msoFalse
        .Kerning = 12
        .Name = "+mn-lt"
        .UnderlineStyle = msoNoUnderline
        .Strike = msoNoStrike
    End With
    Range("L7").Select
End Sub
</code>
Comment: Please remove everything from your code that isn't relevant to the problem. And point out the line that causes the error. http://stackoverflow.com/help/mcve
Answer: Replace
    ActiveChart.SetElement (msoElementPrimaryValueAxisTitleAdjacentToAxis)
with
ActiveChart.SetElement msoElementPrimaryValueAxisTitleBelowAxis
The msoElementPrimaryValueAxisTitleAdjacentToAxis const does not appear to work.


Nylon strings for alternative tunings on hollowbody electric

Question: I have a hollowbody electric guitar on which I put some ball-end nylon strings as an experiment and tuned to by preferred tuning (G-D-d-f-g#-b).  I really liked the distinction in sound between the "boomy" wound strings and the "harp-like" unwound strings, but would like to have the fourth string sound match the upper four rather than the bottom two.
I've purchased a second set of ball-end strings and put the third string from that second set onto the guitar's fourth string (tuned down a minor third from its designed pitch) and it sounds good, but since it's tension is only about 2/3 that of the third string it's a bit "floppy".
Steel string sets publish the gauges of the strings included, and within a category of strings one can usually figure that tension is roughly proportional to the square of the gauge (wound strings of a given gauge are looser than unwound); I've not seen such figures published for nylon strings.
Are there any recommended approaches when selecting nylon strings for alternate tunings?  I would guess that combining strings from a few different sets (my guitar seems amenable to either ball-end or loop-end) would be the best approach, but I don't know different strings in different sets would compare, or whether the strings from different sets would have similar sounds.
Even though nylon strings aren't terribly expensive, they do have a significant break-in period, so it would be helpful to figure out a good combination of strings without having to try too many different ones.
I don't have a huge desire to file the nut or the bridge on my guitar, but even with a stock set of nylon strings I'm happier with the way it plays than I ever was with any other strings I tried it (for either standard tuning or my alternative one) so I don't particularly expect to go back to steel strings with it.  If the best unwound string for the fourth spot would be larger than an ordinary steel 4th string, I wouldn't mind committing the guitar to such usage.
Comment: What kind of pickups does the guitar have? Normal electric guitar pickups won't pick up the sound of nylon strings.
Comment: @BobBroadley: Like many hollowbody electrics, it has acoustical pickups.  The nylon strings are quieter than steel strings, but when using amplification it's not necessary that the guitar itself be very loud.
Answer: You should consult the D'Addario string company's extensive charts on different string gauges for alternate tunings. It covers steel strings and nylon strings, for guitar and more. Check it out.
D'Addario Catalog Supplement/ String Tension Specifications/
A complete technical reference for fretted instrument string tensions
Comment: Thanks. I'd seen the chart, but for whatever reason not noticed the nylon strings. The ball-end strings are given part numbers that look like steel-string gauges, but most nylon strings are given part numbers that look like a set designator plus the "normal" position (1-3 for unwound; 4-6 for wound) except that the "Rectified clear nylon" part of the chart has strings numbered consecutively (but I have no idea of the diameter).
Comment: None of the charts for unwound nylon go down to "d"; should I figure it will be a quarter the tension of "d'"?  The largest unit weight I see for unwound is .00004795 which is not hugely bigger than their ball-end sets, but might help a little.  Do you have any idea how black nylon compares in density with clear?
Comment: If there are no unwound nylon strings that go down to "d", then it's because the manufacturers have not found a satisfactory way to make one, and the "d" strings need to be wound.
Comment: I like the sound of the string I'm using for the "D" better than I like the sound of a wound fourth string; I would estimate that a J3103 would require a tension of about 7.2, which would be far less excessively loose than some other tensions the table gives.  If there aren't any unwound strings that are much heavier than what I'm using, I'd rather use what I have than use a wound string, but if I could find an unwound string that would be nice.  Normal tuning uses a 3+3 split I think to ensure that even a "D" chord will have a boomy-sounding bass, but...
Comment: ...my chords always combine one or both of the bottom two strings with the top four.  I wonder if there are any other instruments which use unwound nylon strings of a heavier gauge and suitable length which could be adapted to a guitar? I doubt fishing line would be sufficiently uniform to really work well, but it would seem possible; not sure how I'd select one, though.


Measure the uniformity of a distribution over weekdays

Question: I have a similar problem to the question asked here:
How does one measure the non-uniformity of a distribution?
I have a set of probability distributions over the days of the week. I want to measure how close each distribution is to (1/7,1/7,...,1/7).
At the moment I am using an answer from the above question; an L2-Norm, which has value 1 when the distribution has mass 1 for one of the days, and is minimised for (1/7,1/7,...,1/7). I am linearly scaling this so it lies between 0 and 1, then flipping it so 0 means perfectly non-uniform and 1 means perfectly uniform.
This works pretty well, but I have one issue with it; it treats each weekday equally as a dimension in 7-Dim space, so it doesn't account for the nearness of days; in other words, it gives the same score to (1/2,1/2,0,0,0,0,0) and (1/2,0,0,1/2,0,0,0) even though in some sense the latter is more "spread out" and uniform, and should ideally get a higher score. There is obviously the added complication that the ordering of days is circular.
How can I alter this heuristic to account for the nearness of days?
Comment: Your example of (1/2,1/2,0,0,0,0,0) and (1/2,0,0,1/2,0,0,0) are non-uniform in the same way, so it should not matter if you are only interested in testing for non-uniformity. So maybe you want to test something more that was not stated explicitly in your question? Btw,  [entropy](https://en.wikipedia.org/wiki/Entropy_%28information_theory%29) is a measure of uniformity.
Comment: Thanks Tim, I have tried using Entropy but I found the heuristic mentioned above worked better for my purposes. I'm not sure what to call the property of a probability distribution over weekdays that I am interested in, except that it should encapsulate the "spread out-ness" of the probabilities over the week.
Answer: The earth mover distance, also known as the Wasserstein metric, measures the distance between two histograms. Essentially, it considers one histogram as a number of piles of dirt and then assesses how much dirt one needs to move and how far (!) to turn this histogram into the other. You would measure the distance between your distribution and a uniform one over the days of the week.
This of course accounts for the nearness of days - it's easier to move "dirt" from Monday to Tuesday than from Monday to Thursday, so (1/2,0,0,1/2,0,0,0) would have a lower earth mover distance from the uniform distribution than a histogram that is concentrated on Monday and Tuesday.
What this does not do is consider the "circularity" of the week, i.e., that Saturday and Sunday are as close together as are Sunday and Monday. For that, you would need to look for an earth mover distance defined on circular probability mass distributions. This should be doable using a suitable optimization approach.

EDIT: In R, the emd package calculates earth mover distances between histograms.
You can address the "circularity" issue in a fairly simple (though ad-hoc) way.

Calculate an earth mover distance $d_1$ between your distribution and a uniform distibution on Monday through Sunday.
Calculate a distance $d_2$ against a uniform distribution on Tuesday through Monday.
Calculate a distance $d_3$ against a uniform distribution on Wednesday through Tuesday.
...
Finally, as the final distance, use the mean of $d_1, \dots, d_7$.

This takes care of the circularity at the expense of a couple of additional calculations.
2nd EDIT: this is not the circular earth mover distance as such. For that, you'd need to look through some of the literature a search will turn up. If the best way to move dirt between days involves moving it two days from Saturday to Monday, this will show up in five out of the seven $d_i$, but not in the remaining two (where the dirt will need to be moved five days).
However, I'd still consider this a potentially useful way to at least consider the circularity in some manner - certainly better than just using a single histogram and defining the week as going from Sunday to Saturday or in some other arbitrary manner. Plus, while some links above turn up implementations for the circular earth mover distance, I'm not aware of one for R, which is probably the most-used language here.
Comment: At first I thought that the latter example (mean of $d_1,\dots,d_7$) is an example of how to calculate the circular earth mover distance and was confused (because the result could be larger than some of $d_i$). Then I realized that this answer doesn't imply that anywhere. I don't know if others read this answer as I did, but it might be good to state it more clearly that the example is not the circular earth mover distance.
Comment: Many thanks, in fact I did manage to implement a circular earth mover distance in R with the emd package and emd2d function, by defining my own distance function, so did not need to use the hack that you mentioned. This is exactly what I was looking for! One other trifling matter: What should I call it? As Tim said above, I should not call this uniformity. What would be an appropriate name for this heuristic?
Comment: Well, you *are* testing for uniformity, so that term should be fine. What Tim is arguing about is what specific *departures* from uniformity you want to assess, so you may be looking for a more precise term than "*non*-uniformity". As you discussed, you are not looking for departures in an $L^2$ distance sense, but apparently in an EMD sense. I don't see a good name to call that baby. Perhaps you just want to sprinkle "EMD" across your prose. "Distribution A is more EMD-non-uniform than B." "A is more EMD-distant from uniformity than B." Doesn't sound too poetic, though. Sorry.
Comment: @JiK: good point, and one that also occurred to me after I lost connectivity yesterday. I clarified my answer to emphasize that this is a hack and not real circular earth mover distance.


What code should I use where I used db_query()?

Question: I want to execute SHOW TABLES in Drupal 8. In Drupal 7, I would use db_query("SHOW TABLES"), which is deprecated in Drupal 8.
What should I use for Drupal 8?
Comment: Note that this is pretty specific to MySQL.
Answer: As quick replacement for queries on the database Drupal uses by default, you can use the following code.
use Drupal\Core\Database\Database;

$db = Drupal::database();
$result = $db->query("SHOW TABLES");

Keep in mind that:

db_query() will not be removed before Drupal 9 goes off; you can still use it, even if it is preferable to refactor your code to be more object oriented
The code I shown should be used in legacy procedural code; when possible, you should consider refactoring it to be object oriented and use Drupal 8 services and Dependency Injection Container
Answer: You can still run arbitrary queries against the database. db_query is deprecated because it does not go well with the dependency injection pattern found elsewhere in D8.
All you need to do is getting the database object.
Default database
This should fit almost everyone, unless you are trying to run the query on a different connection.
$db = \Drupal::service('database');
$result = $db->query('SHOW TABLES'); // You can run the good old fetchAll(), etc on the $result now.

Other databases
If you want to run the query on a different database connection,  just get the service spawn, and you can run query() on it.
Examples
I could find several examples from the Backup and migrate module's MySQLiSource class.


can't add a file or application to /system/xbin of andriod

Question: i am having problem regarding to insertion or push anything from computer to the android AVD(install and build using AOSP) on Linux operating system.
the error i as follows.
error:Read only file system.
i have able to use adb shell and the adb su. but no luck.
Comment: Please read the guide on [how to write a good question](http://stackoverflow.com/help/how-to-ask) and update your question with more information.
Answer: Use this:
adb root
adb remount

there's no su program in Android.
Answer: By default, the system partition on Android is mounted as ro: Read-Only.You need superuser permissions(I find SuperSU nice).I had rooted my Nexus 4 and on that I had used this command(in a superuser shell):
mount -o remount,rw /system

You might want to install a terminal application(like Termux) for testing. Run su in a terminal to get root access(prompt should now look like blahblahblah #)
An app by Chainfire(SuperSU creator), with root access, gives adb shell auto-root access. You might want to find it quick, a warning had been given that CHinfire might make it paid-for at anytime.You shouldn't be able to do this to an Android emulator.


VBA: TextBox and ListBox

Question: On my UserForm I have a TextBox where is write down my Serial Numbers, something like 421012.
Below it I have a ListBox with options like "Tools, Bench, Wheel"
What I'm attemting to do:
After I type my Serial that I just typed in, I click on one of the options in the ListBox and it should put a value behind my Serial.
Something like this: 421012 + "selecting Tools" =42101215
the "Bench" would get an result of 42101245
My code so far:
Private Sub UserForm_Initialize()
With ListBox1
.AddItem "Tools"
.AddItem "Bench"
.AddItem "Wheel"
End With
End Sub

Somehow I have to tell VBA that Tools equals 15 or Bench equals 45 etc.etc.
Comment: [Possibly relevant](http://codereview.stackexchange.com/q/140995/23788)
Answer: Add a second column and then reference that when you're doing your concatenation of the value.  "42012" & me.listbox.column(1) should return the value of whatever is selected
with listbox
    .columncount = 2
    .additem
    .list(1,0) = "Tools"
    .list(1,1) = 15
    .list(2,0) = "Bench"
    .list(2,1) = 45
    .list(3,0) = "Wheel"
    .list(3,1) = 75
end with

Comment: I was just about to suggest this.  I'd personally set the column width of the "values" column to 0 though.
Comment: Keep getting Run time Error 381
Comment: I think the answer by @Comintern is much cleaner.  Didn't know about scripting dictionaries.  Very cool.
Answer: I use this much more frequently with objects than with intrinsic types, but another option is to store a Dictionary containing lookups keyed on your ListBox items:
'Module level variable
Private SerialLookup As Scripting.Dictionary

Private Sub UserForm_Initialize()
    Set SerialLookup = New Scripting.Dictionary
    With SerialLookup
        .Add "Tools", "15"
        .Add "Bench", "45"
        .Add "Wheel", "42"
    End With

    ListBox1.List = SerialLookup.Keys()
End Sub

Then you can retrieve whatever value you need by using it as an index into the Dictionary, i.e.:
TextBox1.Value = "421012" & SerialLookup.Item(ListBox1.Value)

Comment: One of my coolest answers involved using dictionary keys: [Is it possible to use autofilter or find on a dictionary?](http://stackoverflow.com/questions/39162841/is-it-ppossible-to-use-autofilter-or-find-on-a-dictionary/39177064#39177064)
Comment: @ThomasInzina - How do I always manage to forget that... Edited.

VBA: TextBox and ListBox


Analysing grade distributions: difference in proportions across years

Question: I have data on grades obtained in a course over three consecutive years. The grade distribution includes the number of students who received a fail, pass, credit, distinction and high distinction.
I need to work out if the proportion of students who obtain a pass (or fail, credit, etc.) is the same, or different across the three cohorts. How can I do this? What type of test should I use?
Comment: Hi @Bunya, welcome to CV. Could you give a little bit more detail in your question? An example of your dataset could help.
Comment: Hi Patrick, thanks! The data set includes data collected over three consecutive years. Each year data was collected on the grades achieved for one course.  The grade distribution includes the number of students who received a fail, pass, credit, distinction and high distinction.  I need to work out if the proportion of students who obtain a pass (for example) is the same, or different across the three cohorts.
Comment: I edited your question to reflect this new information.
Answer: Bunya, I know this is a really old question, and probably solved by now, but can you confirm what you ended up using?
I am working on a similar problem and I think the best solution is to use a Kruskal-Wallis test; it can compare % of grade outcomes one at at time between years. You might also try a multi-factor test that can look for sig. diffs between groups in multiple years. A repeated measure test (like Friendman's) lets you compare one group over multiple years.
I'm no statistician, but this is what I've come across in my own research.


How can I use ChefSpec during refactoring of environments/roles/nodes

Question: I have a Chef repo that needs some cleanup. Configuration data is scattered around into files in nodes, environments and roles folders.
My idea is to use ChefSpec to test against regressions during restructuring the data. As only roles appear to be supported directly by ChefSpec, how could I use existing nodes/environments in my ChefSpec tests?
Comment: It looks like you answered your own question. Could you please mark an answer as correct?
Answer: Answering myself, just in case someone else might find it useful:
I ended up with an example like this:
<code>it 'renders config file with proper data on node ABC in environment XYZ' do
  runner = ChefSpec::Runner.new

  node_attr = JSON.parse(File.open('nodes/ABC.json'))
  runner.node.consume_attributes(node_attr)

  env_attr = JSON.parse(File.open('environments/XYZ.json'))
  env = Chef::Environment.json_create(env_attr)

  runner.node.stub(:chef_environment).and_return('XYZ')
  Chef::Environment.stub(:load).and_return(env)

  runner.converge('cookbook::recipe')

  expect(runner).to render_file('/etc/cookbook.cfg').with_content('some data from ABC.json')
end
</code>
This works with the "monolithic chef repo" pattern, which I am using in this early phase of our chef adoption - in fact we are using chef-solo, but on a folder structure that tries to be compatible to a chef repo.
Moreover, I do not want to test for regressions during all the necessary restructuring without the speed of ChefSpec!
Answer: Most of the Chef tooling is designed to work with a single cookbook. You can use ChefSpec in the "monolithic Chef repo" pattern. ChefSpec does have support for mocking Environments and roles.
When you write ChefSpec tests (usually called "examples"), they are against a single cookbook, not a role or environment. If you wrote ChefSpec examples for all your cookbooks, then you could safely do this refactor, running the tests on each run. But there's no way to write tests against your entire Chef repository.
Sources:

I maintain it and wrote the current version
Comment: I do not fully understand 'there's no way to write tests against your entire Chef repository'. Say I am having two cookbooks which share a common pattern of resource usage. I want to factor this out to a definition in a third cookbook that is referenced by both. Would you say, this is out of scope to be tested by chefspec? (I hope not, since its speed is superb!)


I want a function to filter between pass and fail base on name and type

Question: I have a function that removes element with duplicate names and now i wanted to add additional logic to remove invalid types. example:
`
const values = ["SS.csv", "Account.csv", "Location.png"]
const supported_names = ["SS", "Account", "Location"]
const SUPPORTED_EXTENSION = 'csv'

const partitionFiles = (values: any, supported_names: any) => {
  let pass: String[] = [];
  let fail: String[] = [];
  for (const supported_name of supported_names) {
    for (let i = 0; i < values.length; i++) {
      const file = values[i];
      if (
        file.toLowerCase().includes(supported_name.toLowerCase())
      ) {
        pass.push(file)
        break
      }
    }
  }

  for (const value of values) {
    if (!pass.includes(value)) {
      fail.push(value)
    }
  }

  for(const name of values){
    if(!name.includes(SUPPORTED_EXTENSION)){
      fail.push(name)
    }
  }

  return [pass, fail]
}

const [validFiles, invalidFiles] = partitionFiles(values, supported_names);

console.log(validFiles)

console.log(invalidFiles)

`
Right now the code push the Location.png into the fail arraw, however, since the name match criteria is still in the pass array. I want to remove any element that does not match name and type... any help?
`
for(const name of values){
    if(!name.includes(SUPPORTED_EXTENSION)){
      fail.push(name)
    }
  }

`
this is the new line i added that adds value that doest match type to the fail array... how can i remove that element from the previous array?
Comment: Does this answer your question? [split an array into two based on condition using javascript](https://stackoverflow.com/questions/67333594/split-an-array-into-two-based-on-condition-using-javascript)
Comment: also: [Dividing an array by filter function](https://stackoverflow.com/questions/11731072/dividing-an-array-by-filter-function)
Answer: I created this sample:
const values = ["SS.csv", "Account.csv", "Location.png"]
const supported_names = ["SS", "Account", "Location"]
const SUPPORTED_EXTENSION = 'csv'

function checkIfValid(item) {
    let sliptted = item.split(".")
    return supported_names.includes(sliptted[0]) && SUPPORTED_EXTENSION.includes(sliptted[1])
}

for (let i = 0; i < values.length; i++) {
    console.log(checkIfValid(values[i]))
}

returns:
true
true
false

In your code this could be for example be used like this:
for (const value of values) {
    let sliptted = value.split(".")
    if (!(supported_names.includes(sliptted[0]) && SUPPORTED_EXTENSION.includes(sliptted[1]))) {
        let index = pass.indexOf(value);
        pass.splice(index, 1);
        fail.push(value)
    }
}

I hope it helped.


How to determine if short strings of text are closely related to a larger text?

Question: I have 1 short string of text (let's say it's a tweet, max 140 characters):
"A review of my beloved Roku 3 media player"
I also have a larger body of text (like a blog article, hundreds of words) which I know is related to the tweet:
"The Roku 3 media player is a great way to watch your favorite ...."
The tweet and blog article are both about the Roku 3 media player specifically. Same author, and they share many of the same phrases, words, collocations, etc. It's likely that the string "Roku 3" appears in the text, along with variations like ""Roku 3 streaming media player", "Roku 3 player" etc
I then have 10 other tweets, some which are related to the "Roku 3 media player", some of which are not (but very similar):
RELATED "A good review of the Roku 3 media player"
UNRELATED "The Roku 2 media player review"
RELATED "Roku 3 is amazing"
RELATED "The Roku 3 is better than the Roku 2 by far"
RELATED "The Roku version 3 streaming media player, fully reviewed"
UNRELATED "A comparison review of the top 3 media player boxes. Roku, Android, Toshiba"
RELATED "Roku 3 streaming media player reviewed"
Those are some examples, and I would have 10 in total. All of the tweets contain "Roku", some are about "Roku 2" and are unrelated, one uses "Roku version 3" and is related etc. Obviously, this is a very small data set.
What is the best method to classify each of the 10 tweets as relevant or not, in relation to the first tweet and blog article? What sort of features would be useful?
Comment: Just on a purely theoretical basis, would it not be possible to be exclusionary for some of the results? As in, exclude Roku results where Roku 2 is mentioned Roku 3 is not, but include results with both Roku 2 and Roku 3 mentioned.
However, I think on a broad, automated scale this is a very difficult task. I think this explains why so many search engines have difficulty with the concept of 'relevance'. I'd love to hear your feedback on this suggestion.
Comment: I think that part of the reason that this may be difficult is that you would need sets of related data. Using automated methods, I would think it would be quite difficult for an engine to determine that that are indeed two different Roku, 2 and 3 (and perhaps the original).
Answer: On a data set this size, it can be pretty tough to learn anything at all; as a baseline, it might be worth it to learn the N-Grams shared by the two examples, and run from there. For a training pair consisting of one short string and a longer article, it would be relatively easy to find the common N-Grams between the article and tweet, and then analyze the frequency with which those N-Grams appear in the test data. In the case you presented, the important N-Grams seem to be "Roku 3" and "Roku 3 Media Player" (a bigram and 4-gram, respectively), both of which are shared by the training data strings.