Android Application Keeps Crashing at XMl parsing

Question: My Application LogCat gives me this out put when running in Debug Mode:
I am trying to parse an XML document received from HTTPGet request (String result) result is the XML in string format. Just cant get it to put string values to test against for login.

02-20 19:39:48.326: E/AndroidRuntime(774): FATAL EXCEPTION: main 02-20
  19:39:48.326: E/AndroidRuntime(774): java.lang.ClassCastException:
  org.apache.harmony.xml.dom.ElementImpl cannot be cast to
  android.sax.Element 02-20 19:39:48.326: E/AndroidRuntime(774):    at
  com.testingapp.MainActivity$startLogin.onPostExecute(MainActivity.java:112)
  02-20 19:39:48.326: E/AndroidRuntime(774):    at
  com.testingapp.MainActivity$startLogin.onPostExecute(MainActivity.java:1)
  02-20 19:39:48.326: E/AndroidRuntime(774):    at
  android.os.AsyncTask.finish(AsyncTask.java:631) 02-20 19:39:48.326:
  E/AndroidRuntime(774):    at
  android.os.AsyncTask.access$600(AsyncTask.java:177) 02-20
  19:39:48.326: E/AndroidRuntime(774):  at
  android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:644)
  02-20 19:39:48.326: E/AndroidRuntime(774):    at
  android.os.Handler.dispatchMessage(Handler.java:99) 02-20
  19:39:48.326: E/AndroidRuntime(774):  at
  android.os.Looper.loop(Looper.java:137) 02-20 19:39:48.326:
  E/AndroidRuntime(774):    at
  android.app.ActivityThread.main(ActivityThread.java:5039) 02-20
  19:39:48.326: E/AndroidRuntime(774):  at
  java.lang.reflect.Method.invokeNative(Native Method) 02-20
  19:39:48.326: E/AndroidRuntime(774):  at
  java.lang.reflect.Method.invoke(Method.java:511) 02-20 19:39:48.326:
  E/AndroidRuntime(774):    at
  com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
  02-20 19:39:48.326: E/AndroidRuntime(774):    at
  com.android.internal.os.ZygoteInit.main(ZygoteInit.java:560) 02-20
  19:39:48.326: E/AndroidRuntime(774):  at
  dalvik.system.NativeStart.main(Native Method)

I was able to Focus the error into my onPostExecute AsyncTask:
@Override
              protected void onPostExecute(String result) {

                int z = 0;
                String xml = result;
                String KEY_ITEM = "Response";
                String KEY_SUCCESS = "Success";
                String KEY_FAIL = "Fail";
                String KEY_UID = "UID";

                  Document doc = getDomElement(xml);

                    if(xml != null)
                    {
                        NodeList nl = doc.getElementsByTagName(KEY_ITEM);
                        for (int i = 0; i < nl.getLength(); i++)
                        {
                        Element e = (Element) nl.item(i);
                        String successString = getValue(e,KEY_SUCCESS);
                        String UIDString = getValue(e,KEY_UID);
                        String failString = getValue(e,KEY_FAIL);

                        if(failString.equals("Fail")){
                            int duration = Toast.LENGTH_LONG;
                            Toast toast = Toast.makeText(MainActivity.this, "Please check username and password.", duration);
                            toast.setGravity(Gravity.TOP|Gravity.RIGHT, 0, 0);
                            toast.show();
                        }
                        else{
                            if(successString.equals("Success")){
                                Intent intent = new Intent (MainActivity.this, HomeActivity.class);
                                startActivity(intent);
                                }
                            }
                        }
                    }
                    else
                    {
                        Toast toast = Toast.makeText(MainActivity.this, "No Connection to Retrieve Data.", Toast.LENGTH_LONG);
                        toast.show();
                    }
        }

I don't understand what i am doing wrong?
Comment: Appears to be a ClassCastException... Can you please post your XML data?
Comment:

Success
801
Comment: the above is the output from the service.
Comment: Looks like you have a bad import. you should be importing.. org.apache.harmony.xml.dom.ElementImpl  (or a related class) instead of android.sax.Element.                                                   Can you please check and confirm?
Comment: all,using:import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
Comment: i am not able to import org.apache.harmony because once you get past org.apache it gives an error.
Comment: Try replacing 'import org.w3c.dom.Node' with 'import  org.w3c.dom.Element'.
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/24854/discussion-between-keeano-martin-and-gaurav-arora)
Answer: Here's the problem...
Change this
public String getValue(Element item, String str) {
    NodeList n = ((Document) item).getElementsByTagName(str);
    return getElementValue(n.item(0));
}

to
public String getValue(Element item, String str) {
    NodeList n = item.getElementsByTagName(str);  // Removed Typecast
    return getElementValue(n.item(0));
}

You are casting the Element.getElementsByTagName(str) is returning a NodeList object.
but you are casting it to a Document object, hence the Class Cast Exception.
Comment: Have been stuck on this for awhile and this helped out so much, thanks!!


book module navigation horizontal submenu

Question: I am using the book module navigation block to display links to book pages. I was wondering if it was possible to alter it so rather than the children links appearing below the parent, they would appear to the side. This is because I want a horizontal navigation
Thanks
EDIT:
For anyone wondering, the way to get book links to appear on the same level, you just need to make sure the child pages are pointing to the same parent, then just display that level of links to get a nav bar, styled this gives you a horizontal nav bar.
Answer: You can override node.tpl.php and customize where to render <code>$content['book_navigation']</code>.
Have also look at book-navigation.tpl.php and template_preprocess_book_navigation().


when click on row showing error IndexError: list index out of range

Question: test.py
import kivy

kivy.require('1.9.0')  # replace with your current kivy version !
import sqlite3 as lite
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
from kivy.properties import BooleanProperty, ListProperty, ObjectProperty,NumericProperty
from kivy.lang import Builder
from kivy.uix.dropdown import DropDown

from kivy.uix.recycleview.views import RecycleDataViewBehavior
from kivy.uix.button import Button
from kivy.uix.recyclegridlayout import RecycleGridLayout
from kivy.uix.behaviors import FocusBehavior
from kivy.uix.recycleview.layout import LayoutSelectionBehavior
from kivy.uix.popup import Popup
from kivy.core.window import Window
Window.size = (900, 500)

MAX_TABLE_COLS = 3

con = lite.connect('demo.db')
con.text_factory = str
cur = con.cursor()

class EditStatePopup(Popup):
    col_data = ListProperty(["?", "?", "?"])
    index = NumericProperty(0)

    def __init__(self, obj, **kwargs):
        super(EditStatePopup, self).__init__(**kwargs)
        self.index = obj.index
        self.col_data[0] = obj.rv_data[self.index]["Id"]
        self.col_data[1] = obj.rv_data[self.index]["Name"]
        self.col_data[2] = obj.rv_data[self.index]["Code"]

    def package_changes(self, stateName, stateCode):
        self.col_data[1] = stateName
        self.col_data[2] = stateCode

class SelectableRecycleGridLayout(FocusBehavior, LayoutSelectionBehavior,
                                  RecycleGridLayout):
    ''' Adds selection and focus behaviour to the view. '''

class SelectableButton(RecycleDataViewBehavior, Button):
    ''' Add selection support to the Button '''
    index = None
    selected = BooleanProperty(False)
    selectable = BooleanProperty(True)
    rv_data = ObjectProperty(None)
    start_point = NumericProperty(0)

    def __init__(self, **kwargs):
        super(SelectableButton, self).__init__(**kwargs)
        Clock.schedule_interval(self.update, .0005)

    def update(self, *args):
        self.text = self.rv_data[self.index][self.key]

    def on_press(self):
        popup = EditStatePopup(self)
        popup.open()

    def refresh_view_attrs(self, rv, index, data):
        ''' Catch and handle the view changes '''
        self.index = index
        return super(SelectableButton, self).refresh_view_attrs(rv, index, data)

    def on_touch_down(self, touch):
        ''' Add selection on touch down '''
        if super(SelectableButton, self).on_touch_down(touch):
            return True
        if self.collide_point(*touch.pos) and self.selectable:
            return self.parent.select_with_touch(self.index, touch)

    def apply_selection(self, rv, index, is_selected):
        ''' Respond to the selection of items in the view. '''
        self.selected = is_selected
        self.rv_data = rv.data
        #print("selection changed to {0}".format(rv.data[1]))

class RV(BoxLayout):
    data_items = ListProperty([])
    col1 = ListProperty()
    col2 = ListProperty()
    col3 = ListProperty()

    def __init__(self, **kwargs):
        super(RV, self).__init__(**kwargs)
        self.get_states()

    def update(self):
        self.col1 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Id'} for x in self.data_items]
        self.col2 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Name'} for x in self.data_items]
        self.col3 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Code'} for x in self.data_items]

    def get_states(self):

        rows = [(1, 'Yash', 'Chopra'), (2, 'amit', 'Kumar')]
        # create data_items

        i = 0
        for row in rows:
            self.data_items.append([row[0], row[1], row[2], i])
            i += 1
        self.update()

class CustDrop(DropDown):
    def __init__(self, **kwargs):
        super(CustDrop, self).__init__(**kwargs)
        self.select('')

class MainMenu(BoxLayout):
    rv = ObjectProperty(None)
    states = ObjectProperty(None)
    dropdown = ObjectProperty(None)

    def display_states(self):
        self.dropdown.dismiss()
        self.remove_widgets()
        self.rv = RV()
        self.states.add_widget(self.rv)

    def remove_widgets(self):
        for child in [child for child in self.states.children]:
            self.states.remove_widget(child)

    def update_states(self, obj):
        # update data_items
        # obj.start_point + 1 --- skip State_ID
        self.rv.data_items[obj.index] = [ obj.col_data[0], obj.col_data[1], obj.col_data[2], obj.index]
        self.rv.update()
        # update Database Table
        cur.execute("UPDATE m_state SET state_name=?, state_code=? WHERE state_id=?",
                (obj.col_data[1], obj.col_data[2], obj.col_data[0]))
        con.commit()

class TestApp(App):
    title = "test"

    def build(self):
        self.root = Builder.load_file('test.kv')
        return MainMenu()

if __name__ == '__main__':
    TestApp().run()

test.kv
#:kivy 1.10.0
#:import CoreImage kivy.core.image.Image
#:import os os

<EditStatePopup>:
    title: "Update State"
    size_hint: None, None
    size: 300, 300
    auto_dismiss: False

    BoxLayout:
        orientation: "vertical"
        GridLayout:
            cols: 2
            Label:
                text: "Id"
            Label:
                id: Id
                text: root.col_data[0]
            Label:
                text: "Name"
            TextInput:
                id: Name
                text: root.col_data[1]
            Label:
                text: "Code"
            TextInput:
                id: stateCode
                text: root.col_data[2]

            Button:
                size_hint: 1, 0.4
                text: "Cancel"
                on_release: root.dismiss()

            Button:
                size_hint: 1, 0.4
                text: "Ok"
                on_release:
                    root.package_changes(Name.text, Code.text)
                    #root.obj.update_states(root.start_point, root.max_table_cols, root.new_data)
                    app.root.update_states(root)
                    root.dismiss()

<SelectableButton>:
    # Draw a background to indicate selection
    canvas.before:
        Color:
            rgba: (0, 0.517, 0.705, 1) if self.selected else (0, 0.517, 0.705, 1)
        Rectangle:
            pos: self.pos
            size: self.size

<MyRV@RecycleView>:
    viewclass: 'SelectableButton'
    SelectableRecycleGridLayout:
        cols: 1
        default_size: None, dp(26)
        default_size_hint: 1, None
        size_hint_y: None
        height: self.minimum_height
        orientation: 'vertical'
        multiselect: True
        touch_multiselect: True

<RV>:
    BoxLayout:
        orientation: "vertical"

        GridLayout:
            size_hint: 1, None
            size_hint_y: None
            height: 25
            cols: 3

            Label:
                size_hint_x: .1
                text: "Id"
            Label:
                size_hint_x: .5
                text: "Name"
            Label:
                size_hint_x: .4
                text: "Code"

        BoxLayout:
            MyRV:
                size_hint_x: .1
                data: root.col1
            MyRV:
                size_hint_x: .5
                data: root.col2
            MyRV:
                size_hint_x: .4
                data: root.col3

<DropdownButton@Button>:
    border: (0, 16, 0, 16)
    text_size: self.size
    valign: "middle"
    padding_x: 5
    size_hint_y: None
    height: '30dp'
    #on_release: dropdown.select('')
    #on_release: app.root.test
    background_color: 90 , 90, 90, 90
    color: 0, 0.517, 0.705, 1

#<CustDrop>:
    #id: dropdown
    #auto_width: False
    #width: 150
    #DropdownButton:
        #text: 'Add State'
        #on_release: os.system("python m_State.py")

    #DropdownButton:
        #text: 'List State'
        #on_release: root.display_users()

<MenuButton@Button>:
    text_size: self.size
    valign: "middle"
    padding_x: 5
    size : (80,30)
    size_hint : (None, None)
    background_color: 90 , 90, 90, 90
    background_normal: ''
    color: 0, 0.517, 0.705, 1
    border: (0, 10, 0, 0)

<MainMenu>:
    states: states
    dropdown: dropdown

    BoxLayout:
        orientation: 'vertical'
        #spacing : 10

        BoxLayout:
            canvas.before:
                Rectangle:
                    pos: self.pos
                    size: self.size

            size_hint_y: 1

            MenuButton:
                id: btn
                text: 'Menu'
                size : (60,30)
                on_release: dropdown.open(self)

            CustDrop:
                id: dropdown
                auto_width: False
                width: 150

                DropdownButton:
                    text: 'user'
                    size_hint_y: None
                    height: '32dp'
                    #on_release: dropdown3.open(self)
                    on_release: root.display_states()

        BoxLayout:
            id: states
            size_hint_y: 9

        Label:
            size_hint_y: 9

When i click on menu then show sub menu user.when click on user then show list of user when i click on any row of user then show error IndexError: list index out of range.
update_states() function does not update in database
Comment: You have forgotten to import `Clock`: `from kivy.clock import Clock`
Answer:
1.When i click on menu then show sub menu user.when click on user then show list of user when i click on any row of user then show error like attached image.

You are iterating over rows, you have only two rows but 3 columns

update_states() function does not update in database

I don't think that the compilator terminates that function because there are few errors in it
To get your code works I had to make many changes
First prepare the data of the rvs in the python file
...

class RV(BoxLayout):
    data_items = ListProperty([])
    col1 = ListProperty()
    col2 = ListProperty()
    col3 = ListProperty()

    def __init__(self, **kwargs):
        super(RV, self).__init__(**kwargs)
        self.get_states()

    def update(self):
        self.col1 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Id'} for x in self.data_items]
        self.col2 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Name'} for x in self.data_items]
        self.col3 = [{'Id': str(x[0]), 'Name': x[1], 'Code': x[2], 'key': 'Code'} for x in self.data_items]

    def get_states(self):

        #cur.execute("SELECT * FROM `m_state` order by state_id asc")
        #rows = cur.fetchall()
        rows = [(1, 'Yash', 'Chopra'), (2, 'amit', 'Kumar')]
        # create data_items
        '''for row in rows:
            for col in row:
                self.data_items.append(col)'''
        i = 0
        for row in rows:
            self.data_items.append([row[0], row[1], row[2], i])
            i += 1
        self.update()

...

then in the kv replace the data of the rvs by col1, col2, col3 respectively
...

<RV>:
    BoxLayout:
        orientation: "vertical"

        GridLayout:
            size_hint: 1, None
            size_hint_y: None
            height: 25
            cols: 3

            Label:
                size_hint_x: .1
                text: "Id"
            Label:
                size_hint_x: .5
                text: "Name"
            Label:
                size_hint_x: .4
                text: "Code"

        BoxLayout:
            MyRV:
                size_hint_x: .1
                data: root.col1
            MyRV:
                size_hint_x: .5
                data: root.col2
            MyRV:
                size_hint_x: .4
                data: root.col3

...

To change automatically the text of the buttons of the rvs when you edit them I had to schedule a function
...

class SelectableButton(RecycleDataViewBehavior, Button):
    ''' Add selection support to the Button '''
    index = None
    selected = BooleanProperty(False)
    selectable = BooleanProperty(True)
    rv_data = ObjectProperty(None)
    start_point = NumericProperty(0)

    def __init__(self, **kwargs):
        super(SelectableButton, self).__init__(**kwargs)
        Clock.schedule_interval(self.update, .0005)

    def update(self, *args):
        self.text = self.rv_data[self.index][self.key]

...

    def on_press(self):
        popup = EditStatePopup(self)
        popup.open()

...

You have also to edit the EditStatePopup, I add the index attribute because It will be much easier to make change in the mainmenu with him
...

class EditStatePopup(Popup):
    col_data = ListProperty(["?", "?", "?"])
    index = NumericProperty(0)

    def __init__(self, obj, **kwargs):
        super(EditStatePopup, self).__init__(**kwargs)
        self.index = obj.index
        self.col_data[0] = obj.rv_data[self.index]["Id"]
        self.col_data[1] = obj.rv_data[self.index]["Name"]
        self.col_data[2] = obj.rv_data[self.index]["Code"]

Finally edit the updtate_states of the MainMenu class:
...

class MainMenu(BoxLayout):
...
    def update_states(self, obj):
        # update data_items
        # obj.start_point + 1 --- skip State_ID
        self.rv.data_items[obj.index] = [ obj.col_data[0], obj.col_data[1], obj.col_data[2], obj.index]
        self.rv.update()
        # update Database Table
        cur.execute("UPDATE m_state SET state_name=?, state_code=? WHERE state_id=?",
                (obj.col_data[1], obj.col_data[2], obj.col_data[0]))
        con.commit()

Please be careful when making those changes. I hope this helps
Comment: i have edited code.I don't know where i am making mistake. Please can you take a look?


Show/Hide data using AJAX, JQuery and HTML

Question: As part of a "Review" section, I have a button to read all reviews, with only a few on the page at the moment. I want the button to show the data from a .json file using AJAX and JQuery.
HTML..

*****
Great!!!

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

        <button id='button'>SHOW REVIEWS</button>
        <button id='button'>HIDE REVIEWS</button>

JavaScript...
    $(document).on('ready', function() {

        $('#button').on('click', function() {

            $.ajax('reviews.json', {
                type: 'GET',
                dataType: 'json',
                success: function(url) {
                    for(var i = 0; i < url.length; i++) {
                        $('.prod-review').append('
' + url[i].id, url[i].product_id, url[i].nickname, url[i].review, url[i].rating + '
');

                    }
                }
            });

            $('a.show').on('click', function(event) {
                event.preventDefault();
                var link = $(this);
                var list_item = link;
                var reviews = list_item.find('reviews.json');
                reviews.show();
                $(this).show();
                $(this).hide();
            });

            $('a.hide').on('click', function(e) {
                e.preventDefault();
                $(this).find('reviews.json').hide();
                $(this).show();
                $(this).hide();
            });

        });

    });

JSON Data..
[{"id":"1","product_id":"b-o","nickname":"Joe","review":"Great :3","rating":"5"}]

With this code, the data is appending to the page under the button when it is clicked, although showing all as one string, like this..
499
b-oJoeGreat :35
I am relatively new to JS and completely new to AJAX and JQuery, and I want to get an understanding of how it works.
Basically I need the data in the JSON file to appear under the button when clicked and then hide when the button is clicked again. Preferably with the data split as well, so it doesn't all appear on one line. I have spent a couple of days trying different things and this is the closest I got.
Any help would be much appreciated.
Edit. I split the objects using some simple code. It now appears how I want it to on screen when I click the button, but it won't hide it. It also appends the data everytime I click the button.
JavaScript..
$(document).on('ready', function() {

    $('#button').on('click', function() {
    $.ajax('reviews.json', {
        type: 'GET',
        dataType: 'json',
        success: function(data) {
            for(var i = 0; i < data.length; i++) {
                $('.prod-review').append('
' + 'User ID:' + ' ' + data[i].id + '
');
                $('.prod-review').append('
' + 'Product:' + ' ' + data[i].product_id + '
');
                $('.prod-review').append('
' + 'Nickname:' + ' ' + data[i].nickname + '
');
                $('.prod-review').append('
' + 'Review:' + ' ' + data[i].review + '
');
                $('.prod-review').append('
' + 'Rating:' + ' ' + data[i].rating + '
' + '
');
            }
        }
    });
    });

    $('a.show').on('click', function(event) {
        event.preventDefault();
        $(this).find('reviews.json').show();
      });

    $('a.hide').on('click', function(e) {
        e.preventDefault();
        $(this).find('reviews.json').hide();
    });
});

Comment: Start by using your template html to create each item in the ajax success. Instead of your `` just insert each property like `url[i].id` in the correct spot within that `` html string using same approach for example....  `"" + ulr[i].rating +""`
Comment: As for the button clicks the json file is irrelevant except for doing the ajax.... you want things like `$('.review').hide()`
Comment: Thanks for the suggestion. I am not 100% sure what you mean. If I put ```"" + ulr[i].rating +""``` where you suggested then it doesn't pick the data up.
Comment: OK but isn't that the structure you want....from the html at top of question? A `` for each product? Then inside that section a `` that includes a `` for each rating value in the data? You are creating a bunch of `` but don't seem to have an associated `` parent anywhere
Comment: I have accepted an answer which is showing the code exactly how I wanted it to. The  was just a placeholder in all honesty, I tried so many different things and gave up to ask the question. Appreciate the time to help.
Answer: I see your problem now is changing visibility of the review. Wrap the review inside div then use .toggle() to show or hide it.

$(document).on('ready', function() {

  $('#button').on('click', function() {
    var divReviews = $('#moreReviews');
    var btnReview = $(this);
    if (divReviews.text().length) {
      // reviews already downloaded, change button text
      // then show or hide it
      btnReview.text(btnReview.text() == 'SHOW REVIEWS' ? 'HIDE REVIEWS' : 'SHOW REVIEWS');
      divReviews.toggle();
      return
    }

    divReviews.text('Loading ajax content...');
    $.ajax({
      url: 'https://httpbin.org/get',
      type: 'GET',
      dataType: 'json',
      success: function(data) {
        divReviews.text('');
        // set dummy data
        data = [{"id":"1","product_id":"b-o","nickname":"Joe","review":"Great :3","rating":"5"}];
        for (var i = 0; i < data.length; i++) {
          var dd = '
User ID: ' + data[i].id + '
' +
            '
Product: ' + data[i].product_id + '
' +
            '
Nickname: ' + data[i].nickname + '
' +
            '
Review: ' + data[i].review + '
' +
            '
Rating: ' + data[i].rating + '
' + '
';
          divReviews.append(dd);
        }
        btnReview.text('HIDE REVIEWS');
      }
    })
  });
});

*****
Great!!!

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

  <button id="button" style="cursor: pointer;">SHOW REVIEWS</button>

Comment: This has worked perfectly, very much appreciated. I changed the URL to the json file, left the div empty and its working exactly as intended. Thanks again!


Python not accepting function variable

Question: Whenever I run the code, python comes up with the error message:
Traceback (most recent call last):
File ""/Users/jim/Desktop/Python/TextWindow.py", line 7, in module
Read(name)
NameError: name 'name' is not defined
def Writeline(string):
    print(string)

def Read(name):
    name = input()

Read(name)

Writeline(name)

Comment: [You may want to read up on Python functions](http://www.tutorialspoint.com/python/python_functions.htm), particularly variable scope :)
Comment: `name` is defined inside `Read` function and`name` has no meaning outside function
Comment: You haven't defined 'name' anywhere in your code other than inside a function. I think what you want is to remove name from being an argument to Read.

so:

def Read():

and then assign a variable to your call, so instead of Read(name), it would be:

name = Read()
Comment: I am voting to close based on Erica's comment.
Comment: @Zetys 3.x according to tag.
Comment: @Zetys. Added an edit. Hope you like it.
Answer: I am assuming that you want Read to read in a string that you will then pass to Writeline. In that case, Read has to return a value. Python strings are immutable, so you have to return the string you read to access it outside the function:
def Writeline(string):
    print(string)

def Read():
    return input()

name = Read()
Writeline(name)

Edit
Keep in mind that input() does different things in Pythons 2 and 3. In Python 3, it does what you appear to want. In Python 2 raw_input() is the function that reads in input. input() will attempt to evaluate anything you type as a line of Python code.
Answer: First, go through the link that @Erica provided as a comment.
There's a few things going wrong here.

You haven't actually defined the variable "name" on a scope that is able to be used.
You are passing in name to function Read(), when you probably don't want to pass in any variable to begin with.
I think you want raw_input(), not input() (This only applies to python 2.x, as I did not see the 3.x tag)

What you need to do is assign the variable "name" to be the return of the Read() function.
Such as the following:
def WriteLine(s):
    print(s)

def Read():
    r = raw_input()
    return r

name = Read()
WriteLine(name)

Python not accepting function variable


For loop iteration changing when I rerun the Python file

Question: <code>for i in range(1,5):
    print(i)
    a = input("Enter a Number")
</code>
If I don't Enter anything into the Input and Rerun the File again In VSCODE Terminal, The For loop iteration is advancing.
However I was expecting it to reset back to its initial value of 1.
Why is this happening and How do I fix It ?
Answer: It might be that you're still running the program on accident. Try clicking the play button in the top right corner and selecting "Run Python File" from the drop down menu. If you want to use the terminal, press ctrl + C to ensure the program stops running, then run your program from the terminal.
Comment: But Shouldn't it Automatically stop running and rerun when I press "Run Python File". Also When I press ctrl + C it throws a traceback "KeyboardInterrupt".
Comment: When it throws back the keyboard interrupt does it show  the line where the code got interrupted?
Comment: Yup. Here's the error:                                                  line 3, in
    a = input("Enter a Number")
KeyboardInterrupt
Comment: That means your program is definitely still running


Change folder preview image via powershell?

Question: Hello, is there any way to change a folder preview image via powershell script?
Comment: Please try to follow this guide when asking a question to get the best possible answer from the community, welcome! https://stackoverflow.com/help/how-to-ask
Answer: Based on this blog, the shortcut file can be modified by loading it as a shell object and then modifying the icon location.
https://devblogs.microsoft.com/scripting/how-can-i-change-the-icon-for-an-existing-shortcut/
For folders use these references
https://learn.microsoft.com/en-us/windows/win32/shell/how-to-customize-folders-with-desktop-ini
https://www.itdroplets.com/change-folder-icon-powershell/


Approaching serialization without common base class

Question: I am looking for some design advices for the following problem:
I am using boost geometry, I have a couple of custom geometry types compatible with boost geometry (via traits), but most of the types I am using are typedefs.
class MyPoint
{
  // custom stuff
};
// declare traits for MyPoint for use wih boost geometry here

class MyTaggedPoint : public MyPoint
{
  // more custom stuff
};
// declare traits for MyTaggedPoint for use wih boost geometry here

// example typedefs
typedef boost::geometry::model::polygon<MyPoint>        Polygon;
typedef boost::geometry::model::polygon<MyTaggedPoint>  TaggedPolygon;

My problem is when I want to serialize/deserialize my geometries.
Let's say all geometries are stored in a binary field in a database. If I would have a base geometry class, I would probably just write g->type() (4 bytes) and call g->save(some_outputstream) and write all of that to the binary field. Then when reading the binary field I would simply read the bytes and cast to appropriate geometry type.
But Boost geometries do not have a common base class.
How do you guys usually approach serialization when there are multiple types that can be stored as binary and you do not have a shared base class ?
I was thinking of maybe having a Serializer class, that returns a boost.Any and then the geometry can be casted afterward with the type that would be stored in the (de)serializer? But then the serializer would need a save method for each geometry types ? ex: Save(myPolygon), Save(myPoint)
Any ideas/experiences?
Answer: Boost's serialization supports non-invasive serialization if you do not wish to reimplement the wheel.  You may even be able to find library support for their geometry types somewhere.  The interface is somewhat complicated due to XML concerns unfortunately.
Comment: +1 for Boost.Serialization, especially since he's already using Boost.
Answer: To serialize objects to and from bytes, you ultimately need 2 functions for EACH type you have to support (primatives, objects, etc.). These are "Load()" and "Store()".
Ideally, you use a fixed interface for the bytes- a iostream, char*, some buffer object- etc.
For the sake of readability let's call it "ByteBuffer", since logically that's what its role is.
We now have something like template functions for the Serializable concept:
template<typename T>
ByteBuffer Store(const T& object) { // BUT, What goes here...? }

template<typename T>
T Load(const ByteBuffer& bytes);

Okay, this isn't going to work for anything other than the primitive types- even if we made these "visitors" or something they literally have to know every detail about the object's internals to do their job. Furthermore, "Load()" is logically a constructor (really, a FACTORY since it could easily fail). We've got to associate these with the actual objects.
To make Serializable a base class, we need to use the "curiously recurring template" pattern. To do this, we require all derived classes to have a constructor of the form:
T(const ByteBuffer& bytes);

To check for errors, we can provide a protected flag "valid" in the base class that derived constructors can set. Note that your object has to support factory-style construction anyway for Load() to work well with it.
Now we can do this right, providing "Load" as a factory:
template<typename T>
class Serializable // If you do reference-counting on files & such, you can add it here
{
protected:
    bool valid;

    // Require derived to mark as valid upon load
    Serializable() : valid(false) {}
    virtual ~Serializable() { valid = false; }

public:
    static T Load(const ByteBuffer& bytes); // calls a "T(bytes)" constructor

    // Store API
    virtual ByteBuffer Store() = 0;  // Interface details are up to you.
};

Now, just derive from the base class like so and you can pick up everything you need:
class MyObject : public Serializable<MyObject>
{
protected:
   // .. some members ...

   MyObject(const ByteBuffer& bytes)
   {
      //... Actual load logic for this object type ...
      // On success only:
      valid = true;
   }

public:
   virtual ByteBuffer Store() {
      //... store logic
   }
};

What's cool is that you can call "MyObject::Load()" and it'll do exactly what you expect. Futhermore, "Load" can be made into the ONLY way to build the object, allowing you clean APIs for read-only files and such.
Extending this to full File APIs takes a little more work, namely adding a "Load()" that can read from a larger buffer (holding other things) and "Store()" that appends to an existing buffer.
As a side note, do NOT use boost's APIs for this. In a good design serializable objects should map 1-to-1 to packed structures of primitive types on disk- that's the only way the resulting files are really going to be usable by other programs or on other machines. Boost gives you a horrible API that mostly enables you to do things you'll regret later.


Bootstrap Standard Errors: should I divide the sampling standard deviation by $\sqrt{n}$?

Question: Suppose I am bootstrapping an OLS regression and want the standard error of the coefficient $\beta_1$. I estimate the following regression on 1000 resamples of the data (where $B$ indexes the bootstrap resample): $$y^B_i=\beta^B_0+\beta^B_1 x^B_i+\varepsilon^B_i$$
Is the standard error of $\beta_1$ given by  $sd(\beta^B_1)$ or $\frac{sd(\beta^B_1)}{\sqrt{n-1}}$ (where sd is the standard deviation)?
It seems correct to divide by $\sqrt{n-1}$ since that is how we usually define a standard error, but I have seen many references that instruct me to only take the standard deviation without dividing by $\sqrt{n-1}$.
Comment: I have the similar confusion, did you figure this out? It looks like when I divide sd by ${\sqrt{n-1}}$, then the confidence interval is very narrow compared with other CI methods.
Answer: The standard error of the mean, with its $\sqrt{n-1}$ in the denominator, is a particular closed form of the standard error (SE), obtained exactly for the mean as a considered statistic. For any statistic, SE is defined as the standard deviation (SD) of its sampling distribution.
Thus, for the bootstrap, SE is the SD of statistic values obtained for bootstrap samples. In your case:
$$se=sd(\beta_1^B)$$ not the $$\frac{sd(\beta_1^B)}{\sqrt{n-1}}$$
Regarding the SE for mean, recall that $Var(aX) = a^2Var(X)$, where $a$ is some constant. Also, there is $Var(\sum_i X_i)$ = $\sum_i Var(X_i)$ for independent $X_i$. Therefore, we got:
$$Var(\bar X ) = Var(\frac{1}{n}\sum_i^n X_i) = \frac{1}{n^2}\sum_i^nVar(X_i) =\frac{1}{n^2}n\sigma^2 = \frac{\sigma^2}{n}$$
where $\sigma$ is the variance of population from each $X_i$ is drawn.
How is this related to sampling distribution?
We can safely stop here and state that whenever there is a sampling of population by taking $n$ elements of it and statistic of interest  is the mean of those elements, the variance of that mean is given by $\frac{\sigma^2}{n}$. So, the well-known form for the SE of mean is:
$$se_{mean}=\frac{\sigma}{\sqrt n}$$ and with the Bessel's correction: $$se_{mean}=\frac{\sigma}{\sqrt{n-1}}$$ .
But you also can find the following explanation. Let's denote vector of even sub-samples of the investigated population by $X_s = (X_{s_1}, X_{s_2}, ..., X_{s_k})$. If the population mean is estimated as total of all variates in $X_s$ divided by a total number of variates $n$, we got:
$$\bar X = \frac{1}{mk}\sum_{i,j}^{m,k} X_i,{s_j} = \frac{1}{n}\sum_{i}^{n} X^s_i$$
and reasoning is the same as for $Var(\frac{1}{n}\sum_i^n X_i$). However, this is a shortcut and conceptually is different from considering the sampling distribution of the $\bar X$. Also, this approach will get nasty for $Var(\bar X)$ if we estimate $\bar X$ as the mean of means from sub-samples: $\bar X =\frac{1}{n} \sum^n_i \bar X_{s_i}$.
Answer: According to Bradley & Efron if you want to estimate the standard error $se(\hat\theta)$ by the sample standard deviation of the $B$ replications you will take $$\hat{se}_{B} = \sum_{b=1}^{B}\left([\hat\theta^{\star}(b)-\hat\theta^{\star}(\cdot)]^2 / (B-1)\right)^{1/2}$$
where $\hat\theta^{\star}(\cdot) =\sum_{b=1}^{B}\hat\theta^{\star}(b)/B $
Doing so in R :
<code>
#Regression Data
#===============
n <- 27
x <- c(99, 152, 293, 155, 196, 53, 184, 171, 52, 376, 385, 402, 29, 76, 296, 151, 177, 209, 119, 188, 115, 88, 58, 49, 150, 107, 125)
y <- c(25.8, 20.5, 14.3, 23.2, 20.6, 31.1, 20.9, 20.9, 30.4, 16.3, 11.6, 11.8, 32.5, 32.0, 18.0, 24.1, 26.5, 25.8, 28.8, 22.0, 29.7, 28.9, 32.8, 32.5, 25.4, 31.7, 28.5)

#OLS Regression estimates
#========================
ones <- rep(1,n)
DM <- cbind(ones,x)
betas.OLS <- solve(t(DM)%*%DM)%*%t(DM)%*%y
betas.OLS

ehat.OLS <- y-DM%*%betas.OLS
ehat.OLS
sigma2.hat <- t(ehat.OLS)%*%ehat.OLS/(n-2)
sigma2.hat
sigma.hat <- sqrt(sigma2.hat)
sigma.hat
var.betas.OLS <- drop(sigma2.hat)*solve(t(DM)%*%DM)
var.betas.OLS
std.betas.OLS <- sqrt(diag(var.betas.OLS))
std.betas.OLS

betas.OLS
std.betas.OLS

#Nonparametric Bootstrap
#=======================
B<-10000
sample.betas<-NULL

for (i in 1:B)
{
  Boot.residual.sample <- sample(ehat.OLS,size=n,replace=T)
  Y.sample <- DM%*%betas.OLS+Boot.residual.sample
  Boot.betas <- solve(t(DM)%*%DM)%*%t(DM)%*%Y.sample
  sample.betas <- cbind(sample.betas,Boot.betas)
}
sampling.mean <- apply(sample.betas,1,mean)
sampling.mean
sampling.var <- apply(sample.betas,1,var)
sampling.s.e <- sqrt(sampling.var)
sampling.s.e

#Parametric Bootstrap
#====================
B<-10000
sample.betas<-NULL

for (i in 1:B)
{
  Boot.residual.sample <- rnorm(n,0,sigma.hat)
  Y.sample <- DM%*%betas.OLS+Boot.residual.sample
  Boot.betas <- solve(t(DM)%*%DM)%*%t(DM)%*%Y.sample
  sample.betas <- cbind(sample.betas,Boot.betas)
}
sampling.mean <- apply(sample.betas,1,mean)
sampling.mean
sampling.var <- apply(sample.betas,1,var)
sampling.s.e <- sqrt(sampling.var)
sampling.s.e
```
</code>
Comment: Is there an explicit answer to the question in the title lurking somewhere in this?
Comment: @BruceET read again my answer
Comment: Yes this is  the formula i was referring to. It is simply the std dev, not the std dev divided by $\sqrt{B-1}$. Any idea why we don't divide the std dev?
Comment: @lasoon because for any bootstrap sample $(x_{1}^{\star},x_{2}^{\star},\dots,x_{n}^{\star})$ you calculate the std of one sample.For $B$ total bootstrap samples you divide with $B-1$ as noted in the answer.If you don't divide with $B-1$ you have calculate only the std of one bootstrap sample
Comment: @Mr.Podilatis I don't follow. The definition of a std dev of a random variable $x$ is $$\sigma(x)=\sqrt{\frac{\sum_i^N (x_i-\overline{x})^2}{N}}$$ The standard error of the mean is then $$se(\overline{x})=\frac{\sigma(x)}{\sqrt{N}}=\frac{1}{\sqrt{N}}\sqrt{\frac{\sum_i^N (x_i-\overline{x})^2}{N}}$$ Notice that $\frac{1}{\sqrt{N}}$ appears twice in thh standard error--once when computing the standard deviation and second when dividing the sd by $\sqrt{N}$ What you have written is simply the standard deviation of the bootstrapped distribution of means with $\frac{1}{\sqrt{N}}$ appearing once.
Comment: My question is about the $\frac{1}{\sqrt{N}}$ that multiplies the standard deviation $\sigma(x)$. Why does this additional $\frac{1}{\sqrt{N}}$ not appear in the bootstrap standard error formula but it appears in other contexts when estimating the standard error of the mean?


Relearning C++ and a typedef issue

Question: So, it's been 5 or 6 years since I've used C++ for a large project, and about half a year since I last had to deal with small/trivial C++ programs and the C++ runtime in general. I've decided I want to re-learn the language, primarily because it might prove very relevant for an upcoming project at work.
I've been working mainly with C and Python, and I'm currently at a point where I'm not even at ease with the for loop syntax below:
for( int i(0); i != n; ++i) {}
although I admit it's not that hard to decipher.
Since I understand there's been a lot of additions to the language as well as libraries and of course idioms, patterns and styles, I'd like to ask for your opinions on good resources that are up to date. I'd like to avoid the tutorials that take the approach "This is C with a lot of extras". I still have my obligatory copy of "The C++ Programming Language" which I'm studying but I honestly don't know exactly where to focus next. Design patterns? Templates and STL/Boost? Something else?
I'm open to all suggestions!
Also, a more specific question regarding typedefs. Is the following:
typedef Type& TypeRef;
generally considered good practice when providing opaque types as part of an API? With a meaningful name, that is. Is it at all similar to the approach adopted by libraries like pthreads or libpcap, and if it is, is it preferable to using a pointer for the same job?
Thanks in advance.
Comment: @aaaa It is perfectly valid C++.
Comment: how about for(int i = 0; i < n; ++i)?
Comment: @mfukar: `int i(0)` is neither better nor worse than `int i = 0` in any objective way. Using `!=` rather than `<` and pre-increment rather than post-increment are good habits in C++, since they can give better code when looping with iterators.
Comment: Definitely learn to use the Standard Containers, Iterators and Algorithms libraries (historically known as STL), and understand how they unleash the power of generic programming. To me, this is what sets C++ apart from any other language.
Comment: Your for-loop example is some bad code. "i(0)" looks like a function call to a function named "i", not any kind of the usual initialization.
Comment: @Ed: Of course, they're equivalent, but I see the one I posted thrown around a lot lately. I've no idea whether it's somehow "better", anyway.
Answer: To answer your specific question, typedefs that hide the fact that something is a pointer or a reference are always a bad idea - the same is true in C. Consider for example the opaque type FILE - one still has to explicitly create FILE pointers in order to use it.
As for books, see The Definitive C++ Book Guide and List.
Comment: I just found a copy of Meyer's *Effective C++* at work, and since it's suggested, I'll start there. Thanks!
Answer: I never like typedefing away reference or pointer types.  It's not possible to effectively hide the fact that a type is a pointer vs. a reference since they have different calling semantics for clients, so this just leads to confusion in my experience.
Answer: The typedef policy taken by the Linux kernel for C is the best. Essentially the policy is to not use typedefs unless you are creating a new type abstraction. This is useful in cases where low-level types different among architectures, but kernel users want a common type. For example u64 is given a specific type, but the underlying type might change between builds or architectures.
typedef u64 unsigned long long;

However, it is highly likely that in C++ there are different idioms using typedefs that are perfectly acceptable, however, I would get comfortable with C before C++.
Perhaps the best way to get re-oriented with C++ is to write may one-off programs that exercise a specific feature. For example, write a template, use a library in Boost, create some threads, allocate some memory. The point isn't to learn all of this stuff, the point is to see and it so you aren't gasping for air later on when it's showtime.
Here is the Linux kernel typedef standard, taken from http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.33.y.git;a=blob_plain;f=Documentation/CodingStyle;hb=HEAD

  Chapter 5: Typedefs

Please don't use things like "vps_t".
It's a mistake to use typedef for
  structures and pointers. When you see
  a
vps_t a;
in the source, what does it mean?
In contrast, if it says
struct virtual_container *a;
you can actually tell what "a" is.
Lots of people think that typedefs
  "help readability". Not so. They are
  useful only for:
(a) totally opaque objects (where the
  typedef is actively used to hide
       what the object is).
 Example: "pte_t" etc. opaque objects that you can only access using
 the proper accessor functions.

 NOTE! Opaqueness and "accessor functions" are not good in themselves.
 The reason we have them for things like pte_t etc. is that there
 really is absolutely _zero_ portably accessible information there.

(b) Clear integer types, where the
  abstraction helps avoid confusion
       whether it is "int" or "long".
 u8/u16/u32 are perfectly fine typedefs, although they fit into
 category (d) better than here.

 NOTE! Again - there needs to be a _reason_ for this. If something is
 "unsigned long", then there's no reason to do

typedef unsigned long myflags_t;
 but if there is a clear reason for why it under certain circumstances
 might be an "unsigned int" and under other configurations might be
 "unsigned long", then by all means go ahead and use a typedef.

(c) when you use sparse to literally
  create a new type for
       type-checking.
(d) New types which are identical to
  standard C99 types, in certain
       exceptional circumstances.
 Although it would only take a short amount of time for the eyes and
 brain to become accustomed to the standard types like 'uint32_t',
 some people object to their use anyway.

 Therefore, the Linux-specific 'u8/u16/u32/u64' types and their
 signed equivalents which are identical to standard types are
 permitted -- although they are not mandatory in new code of your
 own.

 When editing existing code which already uses one or the other set
 of types, you should conform to the existing choices in that code.

(e) Types safe for use in userspace.
 In certain structures which are visible to userspace, we cannot
 require C99 types and cannot use the 'u32' form above. Thus, we
 use __u32 and similar types in all structures which are shared
 with userspace.

Maybe there are other cases too, but
  the rule should basically be to NEVER
  EVER use a typedef unless you can
  clearly match one of those rules.
In general, a pointer, or a struct
  that has elements that can reasonably
  be directly accessed should never be
  a typedef.
Comment: "I would get comfortable with C before C++" - most people would disagree with you.
Comment: the value of typedefs I think are easier to appreciate in C++ because having several functions that take `std::vector>>` becomes rapidly unreadable.
Comment: Why do you say that? Perhaps it is a matter of perspective. I credit learning C prior to C++ with both a deeper understanding of C++, as well as my deep hatred for C++ ;) I think if time is in your interested, then yeh, C++ first is fine, it isn't that difficult in comparison to C.
Comment: @Noah I say it because most people would disagree with you - this question has been asked a lot. Certainly, Dr. Stroustrup does. I learned C before C++ myself, but I wouldn't give that any credit for any deep understanding I might have - that would be due to learning assembler before C :-)
Comment: @Noah: I am aware of the Linux kernel coding style in general, and while I'm comfortable with C, I wouldn't want to extrapolate C++ idioms from C, without some input from more experienced C++ developers.
Comment: Not to mention that before C++0x becomes mainstream, you also have to remember to insert the extra spaces between the '`>`' characters...

Relearning C++ and a typedef issue


iOS playback RTMP with AVFoundation

Question: I need to implement an iOS app that playback RTMP with AV. Exists any library for this propose? I have bean searched on Internet but I didn't found nothing or any example.
I think that I must capture frames and decodec because iOS don't playback rtmp.
I haven't got idea to where start
Answer: There are few providers i had come across few months back.
You might want to refer them.

cine.io
VideoStream SDK for iOS
Brightcove Player SDK for iOS
RealTimeLibs

All of above are paid ones.
There are also some open source libs:

RTMP-Wrapper
MediaLibDemos

Hope this helps you.
Comment: MediaLibDemos is private paying library (not open source)

iOS playback RTMP with AVFoundation


Data Annotation not working with IE8 and telerik grid

Question: The problem
Using Telerik Grid and only with IE8, When I'm Editing a field of a grid that's customized with Data Annotations with wrong values I'm getting an error 500 because it's going to the server action with wrong parameters.
I was reading that's because some versions of jquery validation have a Bug, but I can't solve it updating plugins and libraries.
The question
I'll would like to know which version of each jquery I Should use in order to solve that problem, because maybe I'm not making a good plugin mixture.
The code
Plugins:

jQuery JavaScript Library v1.7.2
jQuery Validation Plugin 1.8.1
Unobtrusive Ajax support library for jQuery

The View:
<code> @(
Html.Telerik().Grid<ZoneData>()
    .Name("zoneDataModel")
    .ToolBar(commands => commands.Insert().Text("Add Zone"))
    .DataKeys(keys => keys.Add(param => param.Id))
    .HtmlAttributes(new { style = "width: 520px;" })
    .NoRecordsTemplate("No existen resultados...")

    .DataBinding(
        dataBinding => dataBinding.Ajax()
                           .Select("SelectGrid", Controllers.Zone, Model)
                           .Update("UpdateGrid", Controllers.Zone, Model)
                           .Insert("InsertGrid", Controllers.Zone, Model)
                           .Delete("DeleteGrid", Controllers.Zone, Model)
    )
    .Columns(columns =>
        {
            columns.Bound(param => param.Code);
            columns.Command(commands =>
                {
                    commands.Edit().ButtonType(GridButtonType.Image);
                    commands.Delete().ButtonType(GridButtonType.Image);
                }
                ).Width(100);
        })

    .Editable(editing => editing.Mode(GridEditMode.InLine)
    .Resizable(resizing => resizing.Columns(true))
    .Reorderable(reorder => reorder.Columns(true))
                  )
</code>
The Model:
<code>public class ZoneData
    {
        public string Id{ get; set; }

        [Range(0, 999,ErrorMessage = " ")]
        [Required(ErrorMessage = " ")]
        [StringLength(3,ErrorMessage = " ")]
        public string Code { get; set; }

    }
</code>
Answer: I solve that error with these configuration:

jQuery JavaScript Library v1.7.2
jQuery Validation Plugin 1.11.1
Unobtrusive Ajax support library for jQuery


LSTM time series forecasting accuracy

Question: EDIT3: [Solved] I experimented with the LSTM hyperparameters and tried to reshape or simplify my data, but that barely changed the outcome. So I stepped back from LSTM and tried a simpler approach, as originally suggested by @naive.
I still converted my data set, to introduce a time lag (best results were with 3 time steps) as suggested here. I fitted the data into a random forest classifier, and got much better results (accuracy up to 90% so far, with simplified data)
It looks like that either my dataset (~200k samples) was still too small, or the time windows I'm looking at are too short for the LSTM to shine. Or, I was simply to impatient and inexperienced with LSTMs.
So, I'm trying to perform time series forcasting using Keras. So far I get an accuracy of about 45%, and I'd like to know what I could try to improve that.
I've read through quite some LSTM examples on time series, and have done some tutorials on it, but now I have my own dataset and I think what I need is somewhat in between of those two examples:
https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/
https://github.com/keras-team/keras/blob/master/examples/lstm_text_generation.py
The former is about multivariate time series forecasting, but it's regression, and I want to do classification.
The latter is a text generation example, where the character (class) is predicted based on the previous x characters.
What I want to do is predict the type (class) for the next sample, based on multiple features (including the type) of the past samples.
I have multiple tables that look like this:
EDIT2 For clarification:
It's keylogging and eye tracking data from a person who translated a text from language A to language B. Now, the type of each activity (row) states whether the person is looking at text A or B, and if he is typing. The assumption we have is, that there can be some kind of pattern behind the process (e.g. "read A" - "read B" - "type some" - repeat). That's why I think the time series is relevant.
I normalized all data, except the time, which I don't use right now, and the Type, which I converted to 1-hot, so that it can be used for the classification.
Based on the text generation example, I converted my table into overlapping windows of e.g. 5 rows, and the corresponding label is the Type of the next row (again, 1-hot).
My model looks somewhat like this (tried with different LSTM dimensions, window widths and used features):
<code>model = Sequential()
model.add(LSTM(100,
      input_shape=(window_width, num_feats)))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy',
      optimizer="adam",metrics=['acc'])
model.fit(feats, labs,
      batch_size=batch_size,
      epochs=20,
      validation_data=(test_feats,test_labs))
</code>
Now, for the results I achieved this way, the accuracy, both training and validation, is around 45%. As you can see in this plot:

Simple guessing, would give a chance of 16% (6 classes).
I kind of hoped to reach a better accuracy, and I wonder if/how I could tune my LSTM to achieve improvements. I've read about under/overfitting, and how to improve in both cases, but I'm unsure what is applicable to mine, as the training and validation losses look somewhat strange:

I checked what happens for more epochs, and seems like the training loss keeps decreasing slightly, but the validation loss increases.
Also, I'm currently working on a set of 77 tables with a total of about 46000 samples. I could acquire more data, but do you think that could improve my model?
Probably it was a bad idea to use the sliding windows? Should I reshape my data differently? As you can see in my data example, each row is one event, with a certain duration. I think I've seen an example where the events were sampled at a fixed interval, could that make a model better, or should including the duration (as I do currently) perform similarly?
Alternatively, are there any simpler machine learning algorithms, applicable for that kind of problem? So far, most I found on time series forecasting was about LSTM.
Any thoughts on how to improve my accuracy would be appreciated :)
EDIT1: Oh, also, how can I interpret that the validation loss is kind of oscillating? It keeps jumping up and down between epochs.
Comment: I don't know if its okay to say it here but it's great that you acknowledged that one of the ideas worked which I had removed from the answer shortly after your edit. Awesome!
Answer:
Probably it was a bad idea to use the sliding windows?

I don't think so. Because giving time lags to LSTM is like giving it a chance to learn over a sequence, which is your objective. I bet it is a good idea.
So you have built a model. Looking at the loss curves it seems there is a room for improvement of the model.
Now what you are looking for is hyperparameter tuning. You want to do a search over the hyperparameter space. You can implement grid search or random search over the hyperparameters.
Tuning is an iterative process that takes a lot of effort and time, so you need to be patient with that. I would recommend some things that have worked for me:
1) Try different optimizers. Give Nadam a shot.
2) Try regularizers.
3) Try different weight initializers.
4) Try different network architectures. Change the units i.e. the dimensionality of the output space. I would recommend decreasing it from 100.
5) Try dropout.
6) Try recurrent dropout.
7) Try different batch sizes and different epochs after tuning the above mentioned parameters.
Comment: Do I have a reason to believe that there is a dependence of a feature's value at say time t on time t-1?
Yes, let me explain my data a bit more
It's keylogging and eye tracking data from a person who translated a text from language A to language B
Now, the type of each activity states whether the person is looking at text A or B, and if he is typing.
The assumption we have is, that there can be some kind of pattern behind the process (e.g. "read A" - "read B" - "type some" - repeat)
Which in my opinion couldn't be recognized if we only look at one sample, to predict the next
Comment: And, thanks for the list. I'm gonna check all those things out and see if I can make improvements :)
Comment: @KlausB have edited the answer accordingly. Thanks
Comment: Okay, I have to revise my old comment. Looks like you were right, I should have tried simpler approaches. I tried to tune different gears, and almost nothing changed. Now I adapted my dataset to feed it into a random forest classifier, while still using time lags (but only up to 5 or so). And I almost immediately reached 65% accuracy for the 6 classes.
I tried simplifying the classes (reduce to 3), which changed almost nothing for the LSTM, but boosted my random forest to almost 90% :)
Comment: That's great news!

LSTM time series forecasting accuracy


How to cast a STRING to a GUID

Question: I need cast a String to a Guid.
I am using this code but
string myUserIdContent = ((Label)row.FindControl("uxUserIdDisplayer")).Text;
Guid myGuidUserId = new Guid(myUserIdContent); // PROBLEM HERE
MembershipUser mySelectedUser = Membership.GetUser(myGuidUserId);

I receive this error

Exception Details: System.FormatException: Unrecognized
  Guid format.

What is the proper way to achieve the results?
Comment: Of course, you could correct the format...
Comment: I'm unable to reproduce your error just by using the string you've provided. Set a breakpoint in your code and MAKE SURE that `myUserIdContent` is really set to what you think it is.
Comment: What about using GuidConverter Class? Could you give me an example?
Comment: Please post the exact string you're using.
Comment: "9accaf71-fe30-48dd-b8a0-062459b4cdfc" string
Comment: The string you're showing us **will** convert to a Guid just fine.... either you're not really getting that string, or something else fails...
Answer: Use Guid.TryParse.
      string myUserIdContent = ((Label)row.FindControl("uxUserIdDisplayer")).Text;
      Guid myGuidUserId;
      if (Guid.TryParse(myUserIdContent, out myGuidUserId)
      {
          MembershipUser mySelectedUser = Membership.GetUser(myGuidUserId);
      }
      else
      {
          // throw exception and/or inform user
      }

Comment: Or, if you know the GUID is in a specific format: [`Guid.TryParseExact`](http://msdn.microsoft.com/en-us/library/system.guid.tryparseexact.aspx)
Answer: By the looks of that exception, your string isn't correctly formatted. Per MSDN:

A string that contains a GUID in one of the following formats ("d" represents a hexadecimal digit whose case is ignored):
32 contiguous digits:
dddddddddddddddddddddddddddddddd
-or-
Groups of 8, 4, 4, 4, and 12 digits with hyphens between the groups. The entire GUID can optionally be enclosed in matching braces or parentheses:
dddddddd-dddd-dddd-dddd-dddddddddddd
-or-
{dddddddd-dddd-dddd-dddd-dddddddddddd}
-or-
(dddddddd-dddd-dddd-dddd-dddddddddddd)
-or-
Groups of 8, 4, and 4 digits, and a subset of eight groups of 2 digits, with each group prefixed by "0x" or "0X", and separated by commas. The entire GUID, as well as the subset, is enclosed in matching braces:
{0xdddddddd, 0xdddd, 0xdddd,{0xdd,0xdd,0xdd,0xdd,0xdd,0xdd,0xdd,0xdd}}
All braces, commas, and "0x" prefixes are required. All embedded spaces are ignored. All leading zeros in a group are ignored.
The digits shown in a group are the maximum number of meaningful digits that can appear in that group. You can specify from 1 to the number of digits shown for a group. The specified digits are assumed to be the low-order digits of the group.
Comment: Guid is welle fomatted, I think is a problem with CAST. Maybe a Convert could solve it?
Comment: Please post the exact string you're using.
Comment: "9accaf71-fe30-48dd-b8a0-062459b4cdfc"

How to cast a STRING to a GUID


Godin Multiac Nylon : how well is that guitar adapted to classical guitar players?

Question: I would like to know if that guitar is totally replication the string length, distance and feel of a classical guitar.
And also get any opinion you may have on it.
Comment: 2 downvotes and no explanation ? I do not accept them.
Comment: Downvotes can mean a lot of things to people. They may think the question doesn't show effort, or is offtopic, or a range of other items. You kind of have to accept them though - that's the way a community system works :-) It is better when people comment, but it's not the end of the world. I think the question is a bit subjective and quite localised, myself - as individuals who have used this Godin may have differing opinions.
Comment: @DrMayhem: True :). But I did research the topic... Simply, I do Not have the classical guitar measurements in mind, so even checking Godin's specs in detail would not have helped.
Comment: I gave you a down-vote because you could have easily done a Google search or gone to the Godin website and looked up the information yourself. Most guitar manufacturers of all types publish measurements of their guitars and you could have easily compared the measurements of the Godin guitars with those of other guitar manufacturers yourself without posting a question here.
Comment: You may disagree with me, but I believe that any question that elicits the response "Why did he not do his own Google search rather than asking this question?" will result in down-votes.
Comment: Also, I don't think this site is a good place to ask for opinions about or experiences with a particular brand of guitar. That is more of a "shopping" question, which is off-topic here. There are many forums elsewhere for classical guitarists who discuss their instruments, for example, and such a question would be appropriate on one of those forums.
Comment: @SkippyFastol, please read a blog post of mine from 2009 where I discuss measurements of the guitar and how they affect playability according to the preference of the player. http://wheatwilliams.com/wordpress/?p=119
Answer: Godin make about eight different models of electric-acoustic nylon-string guitars for different purposes. Some have a wide, flat fingerboard and scale length and string spacing just like a standard classical guitar. Some have a narrower fingerboard with a radius to it and narrower string spacing. They have detailed specifications on their Web site, with all the essential measurements in millimeters (on the French pages) or inches (on the English pages), so you can go there and read them yourself.
Comment: Stupendous answer, as always.


How reliable is the Tomcat Web Server?

Question: In my past experience (Tomcat 4.x), the Tomcat developers recommended against using the Tomcat built-in web server, saying that it was not reliable and was not recommended for production environments. This was before lighttpd and before nginx - and before varnish, etc.
In these days of Tomcat 6.x et al, it appears that things have changed.
Can anyone enlighten me with detailed recommendations from the Apache Foundation and with their own personal experiences?
Comment: Not really the kind of answer you're looking for but I post the following (as a comment) since it might be of interest. The BuiltWith folks say they know of 211,577 sites using Apache Tomcat Coyote and that 3,729 of the Top Million Websites use it. See: http://trends.builtwith.com/websitelist/Apache-Tomcat-Coyote
Comment: And on this page: http://trends.builtwith.com/Web%20Server you can see that 184 or 219 (the graphic and the list at right don't seem to agree) of the top 10,000 sites use Apache Tomcat Coyote.
Answer: We use tomcat 6 in production with the HTTP connector, behind haproxy. It's been stable so far, even under high load (~ 100hits/s at tomcat level).
Haproxy has a pretty strict HTTP parser, if tomcat would be doing anything funky, I would see it in the logs.

How reliable is the Tomcat Web Server?


Python: better file I/0 using os.fork?

Question: My problem is quite simple: I have a 400MB file filled with 10,000,000 lines of data. I need to iterate over each line, do something, and remove the line from memory to avoid filling-up too much RAM.
Since my machine has several processor, my initial idea to optimize this process was to create two different processes. One would read the file several lines at a time and gradually fill a list (one element of the list being one line in the file). The other would have access to this same list and would pop() elements out of it and process them. This would effectively create a list that would grow from one side and shrink from the other.
In other words, this mechanism should implement a buffer that would constantly be populated with lines for the second process to crunch. But maybe this is no faster than using:
for line in open('/data/workfile', 'r'):

Comment: Correct me if I am wrong, but your algorithm appears to be I/O bound rather than CPU bound. That is, the amount of processing that needs to be done on each line is minimal, so your bottleneck is how fast the 400MB file can be read from disk. If this is indeed the case, don't expect much of a speed improvement for the time you'll be putting in.
Comment: Yes, my algorithm is definitely I/O bound. At the same time users are always impatient and anything I do will be called "too slow" by someone. That is the primary reason for me to start looking for something slightly more optimized than the built-in "for line in file".
Answer: Your proposed for line in open('/data/workfile', 'r'): will use a generator, so the entire file will not be read into memory.  I'd go with that until it actually turns out to be too slow.
Comment: I agree.  The proposal would probably hurt performance by (1) spamming the processor data cache, and (2) thrashing the Python global interpreter lock.
Answer: You're probably limited by the speed of your disk. Python already does buffering so reading it line by line is efficient.
Answer: Probably before blind optimizing would be better to profile your existing code and see where the bottle neck really is.
Even if processing really is I/O bound adding couple threads to program could help a bit.
This will allow another thread to wait IO while another is processing the previously read data.
Answer: The data structure you want to use is a Queue (it has the proper blocking mechanisms by example for concurrent writes), that is available in Multiprocessing module.
If you have no dependency between the processing of your data, you could MAP the line by line generator to a pool of processes with the functions in that module to multicore-enable the whole thing in a few lines.
See Also mapReduce approaches (it might be a little overkill however)

Python: better file I/0 using os.fork?


Como reduzir o espaçamento entre itens no BoxLayout?

Question: No meu exemplo, a classe Teste é um JPanel com BorderLayout.
Dentro da região central desse layout há um JScrollPane contendo outro JPanel com BoxLayout na vertical. A intenção é que os itens que eu adicionar a esse JPanel fiquem dispostos em lista na vertical.
Se você executar o exemplo e maximizar a janela eles fazem isso, mas ficam distribuídos igualmente dentro do JPanel e há espaços entre eles, que é o que não desejo. O que desejo é que eles ocupem somente o espaço necessário para caberem no layout e deixem uma sobra grande embaixo, como acontece se eu trocar as partes comentadas do código (trocar a adição dos JPanelOrderRow por JLabels).
Os itens que quero inserir na lista vertical (JPanelOrderRow) são JPanels com GridBagLayout.
Gostaria de entender por que isso está acontecendo e como faço para obter o efeito desejado.
Observação: Não posso usar JList no momento porque provocaria muito retrabalho na aplicação em que estou trabalhando.
Teste.java
import java.awt.BorderLayout;
import java.awt.EventQueue;

import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.JScrollPane;

public class Teste extends JPanel {

    private static final long serialVersionUID = 1L;

    private JPanel jPanelListaOrders = new JPanel();
    private JScrollPane jScrollPaneOrders = new JScrollPane(jPanelListaOrders);

    public Teste() {

        this.setLayout(new BorderLayout());

        this.add(jScrollPaneOrders, BorderLayout.CENTER);

        jPanelListaOrders.setBackground(new java.awt.Color(254, 254, 254));
        jPanelListaOrders.setLayout(new javax.swing.BoxLayout(jPanelListaOrders, javax.swing.BoxLayout.Y_AXIS));

        // Adicionando somente esses quatro JPanelOrderRow, eles ficam
        // espaçados por igual no layout, o que não quero.
        jPanelListaOrders.add(new JPanelOrderRow());
        jPanelListaOrders.add(new JPanelOrderRow());
        jPanelListaOrders.add(new JPanelOrderRow());
        jPanelListaOrders.add(new JPanelOrderRow());

        // Adicionando somente esses quatro JLabels, fica uma sobra embaixo
        // deles que é o que desejo.
        //jPanelListaOrders.add(new JLabel("Teste"));
        //jPanelListaOrders.add(new JLabel("Teste"));
        //jPanelListaOrders.add(new JLabel("Teste"));
        //jPanelListaOrders.add(new JLabel("Teste"));
    }

    public static void main(String[] args) {
        EventQueue.invokeLater(new Runnable() {

            @Override
            public void run() {
                JFrame f = new JFrame("Teste");
                f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                f.add(new Teste());
                f.pack();
                f.setLocationRelativeTo(null);
                f.setVisible(true);
            }
        });
    }
}

JPanelOrderRow.java
import java.awt.GridBagConstraints;
import java.awt.GridBagLayout;

public class JPanelOrderRow extends javax.swing.JPanel {

    private static final long serialVersionUID = 1L;

    public JPanelOrderRow() {
        initComponents();
        initializeData();
    }

    private void initializeData() {
    }

    private void initComponents() {

        setBackground(new java.awt.Color(254, 254, 254));
        setBorder(javax.swing.BorderFactory.createMatteBorder(0, 0, 1, 0, new java.awt.Color(237, 237, 237)));
        GridBagLayout gridBagLayout = new GridBagLayout();
        gridBagLayout.columnWidths = new int[] { 128, 0 };
        gridBagLayout.rowHeights = new int[] { 18, 0 };
        gridBagLayout.columnWeights = new double[] { 0.0, Double.MIN_VALUE };
        gridBagLayout.rowWeights = new double[] { 0.0, Double.MIN_VALUE };
        setLayout(gridBagLayout);
        lblName = new javax.swing.JLabel();

        lblName.setFont(new java.awt.Font("Ubuntu", 1, 14)); // NOI18N
        lblName.setText("Nome do cliente");
        GridBagConstraints gbc_lblName = new GridBagConstraints();
        gbc_lblName.anchor = GridBagConstraints.SOUTH;
        gbc_lblName.fill = GridBagConstraints.HORIZONTAL;
        gbc_lblName.gridx = 0;
        gbc_lblName.gridy = 0;
        add(lblName, gbc_lblName);
    }
    private javax.swing.JLabel lblName;
}

Comment: Onde está o espaçamento?
Comment: Desculpe, esqueci de falar. Tem que maximizar a janela para ver.
Comment: A causa é a mesma que eu disse na outra pergunta, é devido ao seu layout, você está misturando um monte de layouts diferentes.
Comment: Estou misturando porque imagino que o Swing trabalha assim, com layouts diferentes mesmo. Tenho BoxLayout para compor uma lista de elementos e cada elemento tem um layout GridBagLayout. Qual é o problema nisso? O que eu deveria fazer no lugar?
Comment: O boxlayout vai sempre redimensionar tudo dentro dele pra ocupar o maior espaço possivel até que todos estejam distribuidos dentro da janela, nao adianta nada definir parametros de gridbaglayout no JPanel interno, porque o layout mais externo(boxlayout) é que vai definir a distribuição e isso não é alteravel, a nao ser que reescreva o layout manualmente.
Comment: O mesmo não ocorre no JLabel porque ele tem definições que, a não ser que você defina tamanhos manuais relativos ao tamanho da tela, ele irá sempre ocupar o tamanho necessário apenas para exibir seu proprio conteudo, é uma caracteristica do componente JLabel, e o layout não se sobrepoe a isso.
Comment: Entendi! Experimentei trocar o `JLabel` por um `JPanel` com `JLabel` dentro e deu o mesmo problema que o `GridBagLayout`. É culpa do `BoxLayout` então. Eu até tentei adicionar cola (_vertical glue_) pra ver se ajudava mas não mudou nada. O que fazer então? Tirando `JList` não conheço nenhum outro layout que me ajude no lugar de `BoxLayout` para produzir uma lista vertical.
Comment: Você precizará definir um tamanho maximo para o JPanel, o BoxLayout respeita os tamanhos dados para os componentes, quando não há nada informado, o tamanho maximo é `Interger_MAX_VALUE` para largura e altura.
Comment: Entendi. Vou tentar. Faz uma resposta aí que eu aceito!
Comment: Já achei uma solução, é bem boba, mas faz o embasamento que expliquei fazer sentido
Comment: Se não se encomodar, eu vou converter os comentários na resposta, pra não ter que explicar de novo e evitar a fadiga :p
Comment: Fique à vontade :D
Answer: O BoxLayout vai sempre redimensionar tudo dentro dele pra ocupar o maior espaço possível até o tamanho máximo definido por cada componente dentro dele. Por isso os parâmetros de gridbaglayout no JPanel interno não alteram o tamanho dos componentes.
O mesmo não ocorre no JLabel porque ele tem definições que, a não ser que você altere manualmente os tamanhos dele, irá sempre ocupar o tamanho necessário apenas para exibir seu próprio conteúdo, é uma característica deste componente, e o BoxLayout não se sobrepõe a isso, pois respeita os tamanhos definidos pelo próprio componente através dos métodos setPreferredSize(), setMaximumSize() e setMinimumSize().
Você precisará definir um tamanho máximo para o JPanel, o BoxLayout respeita os tamanhos dados para os componentes, quando não há nada informado, o tamanho máximo é definido como Interger_MAX_VALUE para largura e altura, e enquanto houver espaço na janela, o layout tentará preencher cada componente o máximo possível até chegar no valor máximo definido por cada um.
A solução, retirada desta resposta do SOEn, parece até boba mas apenas explora esta característica do BoxLayout, definindo o tamanho máximo do JPanel para o tamanho preferido, que geralmente é o necessário para compor seu conteúdo, incluindo bordas e espaçamento interno. Apenas adicione a seguinte linha ao final do método que constrói o JPanel JPanelOrderRow e veja a mágica dos layouts acontecer:
setMaximumSize(getPreferredSize());

Resultado:


Is it better to use jQuery fadeIn or CSS3 animations?

Question: I am creating a simple gallery using some PHP and JavaScript and am trying to do a fade transition between images. Then I wondered if there is a performance difference between using a CSS animation, e.g.:
<code>@-webkit-keyframes fadeIn {
0%   { opacity: 0; }
100% { opacity: 1; }
}
</code>
and a jQuery fadeIn.
I would like to use the callback from the fadeIn but I also can just use a timer with the CSS I guess.
Are either of these likely to work better with large images? I can't see a difference, but wondered if it might affect slower computers.
Comment: @mgraph - jQuery may support IE for fades, but a lot of the time it looks awful because no matter how good jQuery is, it can't hide the fundamental limitations of IE.
Comment: For a better cross-browser compatibility, consider also using the unprefixed and `-moz-` etc. versions of this code.
Comment: of course, i just use webkit here to make the code shorter in the post. Thanks for the reminder though ;p
Answer: How about one with a fallback to the other? Use CSS3 transitions where possible, and use a feature detection library such as Modernizr to determine if the user's browser is capable of CSS3 transitions.
If AND ONLY IF the user's browser does not support native animations, only then should you use jQuery.
Native animations are GPU accelerated, resulting in less constraint on the CPU, and much smoother animations. Also, IT DOESN'T REQUIRE JAVASCRIPT nor does it take extra requests to pull off.
Comment: If the browser doesn't support native animations then simply don't animate as they will look like crap :)
Comment: 4 answers and 3 of them less than -4?! I would say this was a **very** good question!
Comment: CSS3 looks like the right place for clean animations code but many statistics prove that you are sometimes wrong when it comes to performance.
Comment: @netAction I'm not familiar with a case where jQuery outperforms native CSS animations.
Answer: Well, I think using CSS animation is a lot better when there is a supported browser and you should use jQuery only as a backup for unsupported browsers. As thoroughly explained on http://dev.opera.com/articles/view/css3-vs-jquery-animations, they conducted test of animating 300 divs at the same time with both CSS and jQuery, and noticed a huge difference between the performance statistics.
Statistics using CSS the animation were:

- Number of actions performed to finish the animation: 100
- Time taken to finish executing the animation: 2.9 seconds
- Memory consumed at the end of the animation: 1.5 MB

and Statistics using jQuery were:

- Number of actions performed to finish the animation: 2119
- Time taken to finish executing the animation: 5 seconds
- Memory consumed at the end of the animation: 6 MB

So, as you can see, there is a great difference between the performance. This is because the browser's CSS processor is written in C++ and native code executes very fast whereas jQuery (JavaScript) is an interpreted language and the browser can't predict JavaScript ahead in time very well, in terms of what event will occur next, which restricts browsers to optimize it for performance.
I hope that answers your question.


Erro "Argument of type is not assignable to" em função com parâmetros rest no TypeScript

Question: Tenho um tipo Role que deve ser uma chave do objeto roles e uma função que retorna todas as chaves:
export type Role = keyof typeof roles;

export function allRoles(): Role[] {
  return Object.keys(roles) as Role[];
}

E o seguinte decorador:
export const Roles = (...roles: Role[]) => setMetadata('roles', roles);

Porém, ao tentar usar o decorador como @Roles(allRoles()), o TypeScript informa que há erro nos tipos:

Argument of type '('Role1', 'Role2')' não é atribuível ao parâmetro do tipo 'Role1 | Role2'.
Comment: O que é essa função `Roles`? Poderia [edit] a pergunta para incluir sua assinatura?
Comment: Adicionei, a função Roles simplesmente adiciona o roles no metadata
Comment: O que você quer que `Roles` receba, de fato? Um array cujos elementos sejam as chaves de `roles`?
Comment: Exatamente, `Roles` deve receber uma array de Roles, os itens dessa array são chaves de um objeto, que no caso é o `keyof typeof roles`
Answer: O problema é que o tipo Role[] é diferente do tipo esperado pela construção (...roles: Role[]).
O tipo Role[] significa qualquer array cujos elementos sejam atribuíveis ao tipo Role.
Já (...roles: Role[]) é uma construção sintática que é usada para definir o tipo de uma função que aceita parâmetros rest, isto é, uma função com aridade variada.
Isso significa que a função não aceita um array no primeiro argumento (como você está passando), mas sim um número variado de argumentos do tipo Role que, em conjunto, formam um array do tipo Role[].
Então você precisa mudar o tipo da função, de modo que deixe de ser uma função variádica:

function Roles(...roles: Role[]) {
  // ...
}

Tornando-a uma função que aceita apenas um argumento – um array de Roles:
function Roles(roles: Role[]) {
  // ...
}

Veja funcionando no TypeScript playground.

Uma outra opção seria utilizar Function.prototype.apply ou o operador de espalhamento no retorno de getRoles(), mas semanticamente falando, não acredito que sejam corretas. Mudar o tipo esperado da função, nesse caso, parece-me o mais adequado.


Expected value of a iid random variables

Question: If $\hspace{0.2cm}X_1,\ldots,X_n\hspace{0.2cm}$ are iid random variables with $\hspace{0.2cm}\mathbb{E}(X_i)=\mu\hspace{0.2cm}$ and $\hspace{0.2cm}Var(X_i)=\sigma^2<+\infty$
Let $\bar{X_n}=\left(\frac{1}{n}\sum_{i=1}^nX_i\right)$ be the partial averages and define
$$S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar{X}_n)^2$$
How to prove that $\hspace{0.2cm}\mathbb{E}(S_n^2)=\sigma^2\hspace{0.2cm}$ ?
Comment: Please search the site. Does this answer your question? [Mean of $ \sum (X_i - \bar{X})^2$](https://math.stackexchange.com/questions/1701626/mean-of-sum-x-i-barx2) Check out the linked threads of the proposed duplicate for different approaches.
Answer: We have
$E(S^2)=E\left[\frac{1}{n-1}\sum_{i=1}^n (X_i-\overline X )^2\right]$
$=\frac{1}{n-1}E\left[\sum_{i=1}^n (X_i-\overline X)^2 \right] \quad $
Subtract and add $\mu$. This is a common 'trick'.
$=\frac{1}{n-1}E\left[\sum_{i=1}^n \left[(X_i-\mu)-(\overline X-\mu) \right]^2 \right] \quad$
multipliying out
$=\frac{1}{n-1}E\left[\sum_{i=1}^n \left[(X_i-\mu)^2-2(\overline X-\mu)(X_i-\mu)+(\overline X-\mu)^2 \right]\right] \quad$
writing for each summand a sigma sign
$=\frac{1}{n-1}E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\sum_{i=1}^n(X_i-\mu)+\sum_{i=1}^n(\overline X-\mu)^2 \right] \quad$
$=\frac{1}{n-1}E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\color{blue}{\sum_{i=1}^n(X_i-\mu)}+n(\overline X-\mu)^2 \right] \quad$

transforming the blue term
$\sum_{i=1}^n(X_i-\mu)=n\cdot \overline X-n\cdot \mu$
Thus $2(\overline X-\mu)\color{blue}{\sum_{i=1}^n(X_i-\mu)}=2(\overline X-\mu)\cdot (n\cdot \overline X-n\cdot \mu)=2n( \overline X- \mu)^2$

$=\frac{1}{n-1}E\left[\sum_{i=1}^n (X_i-\mu)^2-2n( \overline X- \mu)^2+n(\overline X-\mu)^2 \right] \quad$
$=\frac{1}{n-1}E\left[\sum_{i=1}^n (X_i-\mu)^2-n( \overline X- \mu)^2\right] \quad$
$=\frac{1}{n-1}\left[\sum_{i=1}^n E\left[(X_i-\mu)^2\right]-nE\left[( \overline X- \mu)^2\right]\right] \quad$
It remains to know what $E\left[(X_i-\mu)^2\right]$ and $E\left[( \overline X- \mu)^2\right]$ is. Are all steps up to here comprehensible?


Consul containers are creating as privileged containers, how to change that?

Question: With the azure policy addon enabled(as per organization policy), we can't create privileged containers on the aks, azure kubernetes.
Our application is set to security context as below.
    securityContext:
        allowPrivilegeEscalation: false
        runAsNonRoot: true
        runAsUser: 999

So our app can create without privileged access.
But, when linked with consul(through annotations), the consul init containers are failing to create.
 Warning  FailedCreate  6s (x15 over 90s)  replicaset-controller  Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [azurepolicy-psp-container-no-privilege-esc-30132221bc21e5b724da] Privilege escalation container is not allowed: envoy-sidecar
[azurepolicy-psp-container-no-privilege-esc-30132221bc21e5b724da] Privilege escalation container is not allowed: consul-sidecar
[azurepolicy-psp-container-no-privilege-esc-30132221bc21e5b724da] Privilege escalation container is not allowed: consul-connect-inject-init

Answer: The way the Azure policy for Kubernetes works, you need explicitly declare for each container the allowPrivilegeEscalation value being set to false. It is not enough for the container not to require it, this must be set in the manifest.
So, you need to amend your console deployment to make sure that this is being set on the sidecar pods. I am not all that familiar with Consul, but if it is being deployed with Helm, have a look at the options in the values file to see if you can set this.
Comment: I didn't find much here, but there is one property enablePodSecurityPolicies . But it is with  applying the policies only.No entry related to the ProvilegeEscalation. https://www.consul.io/docs/k8s/helm


Writing Greek text in Rmarkdown

Question: Hello guys I'm trying so hard to make a document in greek with Rmarkdown, cause I dont want to learn all the Latex commands and it's very easy to actually make a document that seems like a Latex document with Rmarkdown. Really my only problem is the language. I've got so many errors and I've managed to handle them until now. Bellow I give you a simple example of what Rstudio produces. The problem is that it doesn't print any  greek letter but no error occured. I really dont know what's going on.
---
title: "Title"
author: "Me"
fontsize: 12pt
header-includes:
   - \usepackage{fontspec}
   - \usepackage[english,greek]{babel}
   - \newcommand{\en}{\selectlanguage{english}}
   - \newcommand{\gr}{\selectlanguage{greek}}

output:
   pdf_document:
      latex_engine: xelatex
---

{\gr{Αυτά είναι ελληνικά}}

I'm using tinytex in Rstudio with session info :
R version 3.5.0 (2018-04-23)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service
Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=Greek_Greece.1253
LC_CTYPE=Greek_Greece.1253
LC_MONETARY=Greek_Greece.1253
[4] LC_NUMERIC=C
LC_TIME=Greek_Greece.1253

attached base packages:
[1] stats     graphics  grDevices utils
datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_3.5.0  backports_1.1.2 magrittr_1.5
rprojroot_1.3-2 htmltools_0.3.6
[6] tools_3.5.0     yaml_2.2.0      Rcpp_0.12.18
stringi_1.2.4   rmarkdown_1.10
[11] knitr_1.20      stringr_1.3.1   digest_0.6.15
evaluate_0.11

Answer: It is crucial that you use a font that actually contains Greek glyphs.
The following works for me when the input document is saved in UTF-8:
---
header-includes:
   - \usepackage[english,greek]{babel}
   - \newcommand{\en}[1]{{\selectlanguage{english}#1}}
   - \newcommand{\gr}[1]{{\selectlanguage{greek}#1}}
mainfont: GFS Didot
output:
   pdf_document:
      latex_engine: xelatex
---

\gr{Αυτά είναι ελληνικά}   \en{The gick brown fox jumps over the lazyy dog.}

I removed \usepackage{fontspec}, since that is done automatically when  xelatex is used. I have also updated your language switching commands to take an argument, since pandoc does not like the {\command text} style.
However, it would be more conventional to combine xelatex with polyglossia instead of babel. That's what pandocdoes when you use the lang feature:
---
lang: el
mainfont: GFS Didot
output:
   pdf_document:
      latex_engine: xelatex
---

Αυτά είναι ελληνικά [the quick brown fox jumps over the lazy dog]{.class lang="en"}

Αυτά είναι ελληνικά

::::: {.class lang="en"}

Here is a paragraph.

And another.

:::::

Αυτά είναι ελληνικά

Note that you might have to update fontspec/mathspec/polyglossia to avoid this problem.
Answer: First of all thank you for your immediate response, and your advices are much appreciated. Well I tried to update the packages you said via the tlmgr_update() command (from tinytex package), I copy-pasted your last code and the error below is produced:
processing file: untitled.Rmd
output file: untitled.knit.md

! Package fontspec Error: The font "GFS Didot" cannot be
found.

Error: Failed to compile untitled.tex. See untitled.log for
more info.
Execution halted

I even did tlmgr_update("GFS") cause I saw that the font you used was from this package.
Comment: Anyway my friend it worked! I just used the Arial font and everything is ok! Thank you so much!

Writing Greek text in Rmarkdown


Erro ao rodar projeto react native "A problem occurred configuring project ':app'" no Linux manjaro

Question: eu estava instalando o react native seguindo os passos do facebook, e aí quando fui testar com o comando react-native run-android aconteceu esse erro.
já configurei o android SDK no android studio, mas pelo jeito não funcionou, não sei oque aconteceu, qualquer ajuda eu agradeço.
meu SO é o linux manjaro, sou iniciante qualquer erro me perdoem.
[bruno@bruno-pc AwesomeProject]$ sudo react-native run-android
info JS server already running.
info Installing the app...

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> SDK location not found. Define location with an ANDROID_SDK_ROOT environment variable or by setting the sdk.dir path in your project's local properties file at '/home/bruno/AwesomeProject/android/local.properties'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

error Failed to install the app. Make sure you have the Android development environment set up: https://facebook.github.io/react-native/docs/getting-started.html#android-development-environment. Run CLI with --verbose flag for more details.
Error: Command failed: ./gradlew app:installDebug -PreactNativeDevServerPort=8081

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':app'.
> SDK location not found. Define location with an ANDROID_SDK_ROOT environment variable or by setting the sdk.dir path in your project's local properties file at '/home/bruno/AwesomeProject/android/local.properties'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

    at checkExecSyncError (child_process.js:639:11)
    at execFileSync (child_process.js:657:15)
    at runOnAllDevices (/home/bruno/AwesomeProject/node_modules/@react-native-community/cli-platform-android/build/commands/runAndroid/runOnAllDevices.js:75:39)
    at buildAndRun (/home/bruno/AwesomeProject/node_modules/@react-native-community/cli-platform-android/build/commands/runAndroid/index.js:137:41)
    at then.result (/home/bruno/AwesomeProject/node_modules/@react-native-community/cli-platform-android/build/commands/runAndroid/index.js:103:12)
    at processTicksAndRejections (internal/process/task_queues.js:86:5)

Answer: Para resolver este problema de "SDK location not found" existe duas opções. Uma delas seria criar um arquivo local.properties dentro da pasta "Android" que está na raiz do seu projeto, e inserir o caminho para o SDK. Exemplo:
sdk.dir = /Users/USERNAME/Library/Android/sdk

Porem, a forma recomendada é editar o arquivo .bash_profile. Exemplo:
export ANDROID_HOME=~/Android/Sdk
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/platform-tools

Para instalar, dê uma olhada neste tutorial em português:
https://docs.rocketseat.dev/ambiente-react-native/android/linux
Comment: é realmente eu não tinha feito isso ainda, mas ainda não funcionou :/
Answer: inserir a descrição da imagem aqui


MySQL unexpectedly returning results for "WHERE integer_column = 'string_value'"

Question: I'm querying an INT column with a string, like this:
SELECT id
FROM (`navigation_links`)
WHERE `navigation_links`.`navigation_link_id` =  'anyRandomString'

(originally, this was by accident)

Expected results: 0, because navigation_link_id is an INT
Actual results: It seems to return every row whose value is 0

This returns the exact inverse:
SELECT id
FROM (`navigation_links`)
WHERE `navigation_links`.`navigation_link_id` !=  'anyRandomString'

Here is the table, stripped down:
CREATE TABLE `navigation_links` (
    `id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
    `navigation_link_id` INT(10) UNSIGNED NULL DEFAULT '0',
    `navigation_group_id` INT(10) UNSIGNED NULL DEFAULT '0',
    PRIMARY KEY (`id`)
)

Any idea what this is all about?
Answer: When comparing values of different types, MySQL will attempt to convert one of the values so comparison is possible. In the case of strings and numbers, MySQL will convert the string by interpreting any leading digits as a number. No leading digits corresponds to a numeric value of 0, which makes sense in a couple of ways:

leading zeros can often be dropped; '042' is the same as '42'. Thus '0' and '' are the same.
When converting a numeric string to a number, the initial value for the number is 0. The string is iterated over, adding the digits to the number and multiplying by the base:
n = 0
while str[i] is a digit character:
    n += str[i] as a digit
    ++i
return n

If there are no digit characters, the value is 0.
An empty string is nothing, so the numeric string at the beginning of a string that starts with no digits is nothing. Under this way of thinking, NULL could also be the value for a string with no leading digits interpreted as a number. This latter option was not picked by the MySQL developers, possibly to make implementation slightly simpler.
Comment: I see, so it sort of boils down to `0` after each "digit" is checked then?. I noticed that `where int = '3x'` gets cast to `3`, but `where int = 'x3'` gets cast to `0`. I guess the conversion can make sense sometimes, but I'm surprised at this behavior.
Comment: @Madmartigan: note that only leading digits get converted, rather than the first contiguous digit substring. This is the same behavior as the C functions `atoi` and `strtol`; MySQL might use one of these (a quick perusal of the source will tell you whether this is so). You can use the `CAST` and `CONVERT` functions to see what number MySQL will convert a string to.


limiting behaviour of the heat kernel on the real line

Question: I am lost with the following exercise that is posed in Steve Rosenberg's book "The Laplacian on a Riemannian Manifold":
Show that for a continuous function f,
\begin{equation}
\lim_{t \to 0} \quad \frac{1}{\sqrt{4 \pi t}} \, \int_\mathbb{R} \exp \left(-\frac{(x - y)^2}{4t}\right) f(y) \, dy = f(x)
\end{equation}
I am not sure how to tackle this - using L'Hopital's Rule doesn't really simplify the limit .. any hint would be a huge help, many thanks !!
Comment: Here are some hints for a fairly straightforward approach. The idea is that as $t \to 0$ the kernel is very large when $x$ is close to $y$ and very small otherwise, so the integral asymptotically only picks up contributions for fixed $x$ when $y$ is very close to $x$.
Comment: @QiaochuYuan: Hm .. by looking at the integral I thought the kernel becomes small as $t \to 0$, since the t stands in the denominator of a negative fraction of the exponential. Another thing that confuses me is  we are integrating y over the whole line, so what do you exactly mean by saying that a fixed x is close to y ? Sorry for these dumb questions, and thanks for your patience! Your comment sounds very interesting as it might help me understanding the impact of the kernel better, if you could elaborate a bit more that would be great !
Answer: I like the probabilistic approach.
The integral on the left is $\mathbb{E}(f(X_t))$, where $X_t$ is a normal random variable with mean $x$ and variance $2t$. Since $X_t\to x$ in distribution as $t\to0$, we get $$\mathbb{E}(f(X_t))\to f(x).$$ We have assumed that $f$ is a bounded,  continuous function.$ $ $ $ $ $
Comment: What could be said, if the function is not bounded, but merely $\mathbb E(f(X_t))$ is finite for small enough $t$?
Answer: From the distribution point of view let's observe that
$\delta_a(d)=\dfrac 1{a\sqrt{\pi}}e^{-\dfrac {d^2}{a^2}}$ admits the limit $\delta(d)$ as $a \to 0$ (see the animation at the right).
Let's set $a:=2\sqrt{t}$ and $d:=x-y$ then
$\delta_t(x-y)=\dfrac 1{\sqrt{4\pi t}}e^{-\dfrac {(x-y)^2}{4t}}$ will have the limit $\delta(x-y)$ as $t\to 0$ and
$$\lim_{t \to 0} \quad \frac{1}{\sqrt{4 \pi t}} \, \int_\mathbb{R} \exp \left(-\frac{(x - y)^2}{4t}\right) f(y) \, dy =\int_\mathbb{R} \delta(x-y) f(y) \, dy = f(x)$$
Of course justifications are required in the previous steps. You may find them in introductions to distributions like Zemanian's 'Distribution Theory and Transform Analysis' or in the chap. 6 of Olver's 'Generalized Functions and Green's Functions'.
Hoping this helped,
Answer: If we assume $f(x)$ to be analytic around $x$, then we can expand it in Taylor series at $x$ and we obtain
$$
\lim_{t \to 0} \quad \frac{1}{\sqrt{4 \pi t}} \, \int_\mathbb{R} \exp \left[-\frac{(x - y)^2}{4t}\right] \sum_n f^{(n)}(x) (y-x)^n \, dy
= f(x) .
$$
Comment: I completely forgot to specifiy that f can only assumed to be continuous. I have now added this to the post. But in any case, thanks for this initial answer!