Export nodes from Drupal 4?

Question: I am looking into updating a Drupal 4 site. The Node Export module does not seem to have a release suitable for Drupal 4. Can anyone recommend an alternative module or procedure?
The site is running Drupal 4.6.3.
Answer: I'm assuming you're on a shared server. Here are some options.
Option 1 - Downgrade PHP.
Although it might not be the safest choice, as both the PHP and Drupal is probably no longer supported, you can do the following.

Some hosts allow to specify a different PHP version in php.ini.
Rent a cheap VPS, which will allow you to set the PHP version.
Some shared hosts specialize with older PHP versions, make a switch.

Option 2 - Use backup and migrate modules.
There are a number of backup and migrate drupal modules which will make the migrating process easier. This would be a safer choice, but the downside, is that you'll loose your theme and you might have to upgrade to ->5, ->6 and only then to ->7. Here are a few modules to look at

Backup and Migrate
Backup

Option 3 - Start from scratch, import in bulk.
It's both safe and probably the best choice. Set up a new site and a theme. Export the data from the old site, using feeds, or other Drupal modules and import it into the new site using modules like Migrate.
I would advise to upgrade to D7, since the support for D6, not alone D5, will be dropped soon.
Answer: I don't go back as far as 4, but if it's just content you're after, I think you can disable all the modules, then do an upgrade to 5. Once that's working, do an upgrade to 6, then to 7. I think if you're not worried about modules, the upgrades work just fine, or at least this was my experience going from 5 to 6. Unfortunately, I'm a bit ignorant of going 4 to 5.
Comment: Hi Jonathan. The upgrade would be to Drupal 7. I'll look into the approach you suggested - thanks.
Answer: Even i faced same problem. After trying few export modules, decided to write my own php code to move content from Drupal 4 to Drupal 7. Here i am sharing the link, someone may get some help out of this: http://goo.gl/GPmG9F

Drupal having a huge support with version 6 > but no support with 4.X and limited support to 5.x.
  Initially I install few module to import node/content with CSV/XML, but they are not working as expected. I need to import content first which include mainly title, description images (more than one) and attachments (more than 1) for each content. I checked feed, taxanomy csv import, migrate and couple of more but they are able to upload content but not images and attachments. So I thought of writing our own script which will read each node from Drupal 4.x and insert directly into Drupal 7.x database.
Comment: Whilst this may theoretically answer the question, [it would be preferable](http://meta.stackoverflow.com/q/8259) to include the essential parts of the answer here, and provide the link for reference.

Export nodes from Drupal 4?


Installing Team Foundation Server 2010 with SharePoint Foundation 2010

Question: Is it possible to install TFS 2010 with SharePoint Foundation 2010? If yes is there any installation guide?
UPDATE (05. February 2010):
I found some useful help in Internet. For example this one. The problem is that I can't use the standard port 80 for Web Application because this one is already assigned to my web page. So what is to do to use other port? Can I use other port or should I use bindings?
Best Regards
Anton Kalcik
Comment: There is information on [MSDN Forums](http://social.msdn.microsoft.com/Forums/en-US/tfssetup/thread/f79769d4-9c3d-4303-9c85-78a70f4ded6c), which is the second hit from a quick [Google](http://www.google.com/search?q=%22tfs+2010%22+%22sharepoint+2010%22&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a).
Comment: I ask explicit about SharePoint Foundation 2010 (it is not the same as SharePoint 2010) and searching also for installation guide.
Answer: Yes it is possible to install TFS 2010 with SharePoint Foundation 2010. There isn't out any official installation guide, but there are some out there (for example this one). It is also possible to use some of the installation guides which describes TFS integration with SharePoint 2010. You can get also more information in my cross post on MSDN.
Answer: Yes, it is possible. I created a video on my blog that walks you through it:
EDIT: Ah, wait. Foundation. My video is for MOSS 2010. Should still work. For port 80 issue, I would not use a non 80 port. If you do, all your users will have to add the port number to URLs. Use 80 or setup host headers.


Table not creating while installing a custom module in Drupal 7

Question: I am creating a simple module. In that module I need to save the values in a table and I have created an .install file to create the table when the module is installed.
While installing the module, it is not creating the table and it gives a PDOException Error. I am unable to find the issue with my code.
The error message:

PDOException: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'unsigned NOT NULL COMMENT ' AMOUNT', rate unsigned NOT NULL COMMENT 'R' at line 3: CREATE TABLE {sampletest} ( id INT unsigned NOT NULL COMMENT 'My unique identifier', amount unsigned NOT NULL COMMENT 'AMOUNT', rate unsigned NOT NULL COMMENT ' RATE', time unsigned NOT NULL COMMENT '  TIME', PRIMARY KEY (id) ) ENGINE = InnoDB DEFAULT CHARACTER SET utf8 COMMENT ' table'; Array ( ) in db_create_table() (line 2717 of H:\xampp\htdocs\drupal7\includes\database\database.inc) ;

Content of the .install file :
<?php
/**
* Implements hook_install().
*/
function sampletest_install() {
 drupal_install_schema('sampletest');
}
/**
* Implements hook_install().
*/
function sampletest_uninstall() {
  drupal_uninstall_schema('sampletest');
}
/**
* Implements hook_schema().
*/

function sampletest_schema() {
  $schema['sampletest'] = array(
    'description' => t('EMI table'),
    'fields' => array(
      'id' => array(
        'description' => t('My unique identifier'),
        'type' => 'int',
        'unsigned' => true,
        'not null' => true,
      ),
      'amount' => array(
        'description' => t('AMOUNT'),
        'type' => 'bigint',
        'unsigned' => true,
        'not null' => true,
      ),
      'rate' => array(
        'description' => t('RATE'),
        'type' => 'bigint',
        'unsigned' => true,
        'not null' => true,
      ),
      'time' => array(
        'description' => t('TIME'),
        'type' => 'bigint',
        'unsigned' => true,
        'not null' => true,
      ),
    ),
    'primary key' => array('id'),
  );
  return $schema;
}

Answer: I can see 2 problems with in your code:

You should not call drupal_install_schema() from hook_install() in Drupal 7. hook_schema() will be called and the table will be automatically created/dropped on installing/uninstalling. See here: https://drupal.org/node/224333#install-schema
As @Ollie also stated, there's no field type named bigint. You should use int and set its size to big.

Like following:
'amount' => array(
  'description' => t('Amount'),
  'type' => 'int',
  'size' => 'big',
  'unsigned' => TRUE,
  'not null' => TRUE,
),

Answer: I think the problem is in using 'bigint' as the type. Use 'type'=>'int', 'size'=>'big', instead.
Comment: And are you sure your 'id' shouldn't be of type serial (auto-increment)?


Example of functions satisfying certain little-oh condition

Question: Let $f:\mathbb{Z}_{>0} \rightarrow \mathbb{C}$ be a function. Now require $f$ to satisfy the condition
$$
|f(n)|\leq n^{o(1)}
$$
where $o(1)$ is any function $g(n)$ satisfy $\frac{g(n)}{1}\rightarrow 0$ as $n\rightarrow\infty$.
Can someone give some example of such $f(n)$ besides constant functions?
Answer: $f(n)=\log n, n \ge 2$ and $g(n)=(\log n)^{-1/2}=o(1), n \ge 2$; $n^{g(n)}=e^{(\log n)^{1/2}}$ and we have $(\log n)^{1/2} \ge \log \log n$ so $n^{g(n)} \ge e^{\log \log n}=\log n$
(if needed of course we can use a small constant and take $f(n)=C\log n$ to take care of small cases but I think the inequality $(\log n)^{1/2} \ge \log \log n$ holds for all $n \ge 2$ not only for large $n$ where it is trivial)
Answer: Other more interesting examples are the divisor function and $n/\phi(n)$ where $\phi$ is the Euler function. Not so interesting examples are constant functions.


What can I use as default value for a datetime form element?

Question: I need to create a custom contact form in Drupal 8. I wrote all the code to build the form. I just need to set the default value for a datetime form element.
The code I am using to add that form element is the following one.
$form['start_date'] = [
  '#type' => 'datetime',
  '#title' => t('Start date'),
  '#default_value' => time(),
];

The value I am using for the default value doesn't show the expected value. What should I use instead of time()?
Answer: use Drupal\Core\Datetime\DrupalDateTime;
and in form
$form['start_date'] = array(
  '#type' => 'datetime',
  '#title' => t('Start Date'),
  '#default_value' => DrupalDateTime::createFromTimestamp(time()),
);

Comment: just new DrupalDateTime() also defaults to the currrent time.
Comment: But for the value, use: `'#value' => DrupalDateTime()->format('Y-m-d H:i:s')`
Answer: The #default_value datetime field element expects a DrupalDateTime object, not a date string.
use Drupal\Core\Datetime\DrupalDateTime;

$form['start_date'] = [
       '#type' => 'datetime',
       '#title' => t('Start Date'),
       '#description' => date($date_format, time()),
       '#default_value' => new DrupalDateTime('1978-11-01 10:30:00', 'Europe/Berlin'),
     ];

For more information see Datetime::processDatetime() & Datetime in the API docs.
Comment: The code is using the wrong value for *#description*.
Answer: Also using Drupal 8.  This discussion raises an important point - ensure that your date format matches.
I wanted to display the date as d/m/Y but that is not the default format used by the date field:
$today = date("Y-m-d");
$form['general']['endDate'] = ['#type' => 'date','#date_format' => 'd/m/Y','#title' => t('End date'),'#default_value' => $today,];

The clue was in the generated HTML:
data-drupal-date-format="Y-m-d"

The discussion states that the default could be changed but I did not validate that proposition.
Answer: For Drupal 8, the following piece of code works:
// Form element inside buildForm().
$form['start_date'] = [
      '#type' => 'datetime',
      '#title' => t('Start Date'),
      '#default_value' => DrupalDateTime::createFromTimestamp(strtotime($config->get('start_date')))
    ];

// Inside submitForm().
->set('start_date', (string) $form_state->getValue('start_date'))


I am trying to create user registration form in js which beautiful and flexible

Question: I am using drupal 7. I am trying to create user registration form with javascript which beautiful and flexible. Something like this:

How can I achieve this?
Answer: You can do it by using these modules :
Fancy Login

Fancy Login adds javascript based, lightbox-style login functionality
  to your site. When the user clicks on any link that links to
  /user/login, they screen dims and a login box is created in the center
  of the screen.

Better Login

Styling Drupal login/password/registration forms are a mission most of
  the time because they form part of the theme... This module removes
  that issue, it is slightly inspired by the way Wordpress login forms
  work, and uses three page templates to change the style of the forms.
  It is very simple to use, just install the Better Login module and it
  starts working immediately.

Model Form

Modal forms make use of the modal feature in the ctools module to open
  some common forms in a modal window.
Answer: Call User Login block in colorbox. Use colorbox module.
For Forgot Password and Need and Account? Sign Up Link You can use form_alter i.e. Drupal Form API
If you totally want to do it with JS then also you can use colorbox library.
<code>jQuery("#id-of-link-which-on-click-will-open-colorbox").colorbox({
   inline:true,
   href:'#id-of-block-to-open-in-colorbox',
   width:"1000px",
   height:"600px",
   transition:"fade",
   scrolling: false
});
</code>
For Forgot Password and Sign Up Link You can use append / prepend HTML accordingly.


sum the values from Datagridview1(Table1) and save it to Datagridview2(Table2) Using Linq and MySql

Question: I am trying to sum the values from Datagridview1(Table1) and save it to Datagridview2(Table2)
both DGVs are Databound and both have the same 2 columns Name, Quantity.
On Childform1(DGV1) I have a Textbox1 and a Post button.
The user must input a Name on Textbox1 and click Post, then all the Quantity on DGV1 must  be summed up that has the same Name as entered on Textbox1.
ex:
Table1
Name | Quantity
 a       10
 b        5
 a       10
 b        5

After pressing POST button
Table 2
Name | Quantity
 a        20
 b        10

Codes
    Dim occurences As New Dictionary(Of String, Double)
    Dim oTempObjects As New List(Of DataGridViewRow)

    MasterForm.oTransferRows = oTempObjects

    For Each xRow As DataGridViewRow In Datagridview1.Rows

        If (Not xRow.Cells("GVName").Value Is Nothing AndAlso xRow.Cells("GVName").Value.ToString() = TextBox1.Text) Then

            oTempObjects.Add(xRow)
        End If
    Next

    Dim _Name As New List(Of String)

    For Each xRows In MasterForm.oTransferRows

        _Name.Add("'" & xRows.Cells("GVName").Value.ToString() & "'")

        Dim inClause1 As String = String.Join(",", _Name.ToArray())
        Dim _sqlUpdate As String = String.Format("UPDATE tested SET Posted = @Posted WHERE Name IN ({0})", inClause1)

        Using _conn As New MySqlConnection("Server = localhost; Username= root; Password =; Database = test")
            Using _commm As New MySqlCommand()
                With _commm

                    .CommandText = _sqlUpdate
                    .Connection = _conn
                    .CommandType = CommandType.Text
                    .Parameters.AddWithValue("@Posted", "Yes")

                End With
                Try

                    _conn.Open()
                    _commm.ExecuteNonQuery()

                Catch ex As MySqlException

                    MsgBox(ex.StackTrace.ToString)

                End Try

                _conn.Close()

            End Using
        End Using

        If (occurences.ContainsKey(xRows.Cells(1).Value.ToString())) Then

            occurences(xRows.Cells(1).Value.ToString()) = Double.Parse(occurences(xRows.Cells(1).Value.ToString()).ToString()) + Double.Parse(xRows.Cells(4).Value.ToString())

        Else

            occurences.Add(xRows.Cells(1).Value.ToString(), Double.Parse(xRows.Cells(4).Value.ToString()))
        End If

    Next

    For Each KVP As KeyValuePair(Of String, Double) In occurences
        oChildForm2.DataGridView2.Rows.Add(New Object() {KVP.Key, KVP.Value})
    Next

    Dim xlist As List(Of KeyValuePair(Of String, Double)) = occurences.ToList

    For Each pair As KeyValuePair(Of String, Double) In occurences

        Dim oUpdateString = String.Format("Insert Into testing (Name, Quantity) VALUES (@iName, @iQty)")
        Using _conn As New MySqlConnection("Server = localhost; Username= root; Password =; Database = test")
            Using _commm As New MySqlCommand()
                With _commm

                    .CommandText = oUpdateString
                    .Connection = _conn
                    .CommandType = CommandType.Text
                    .Parameters.AddWithValue("@iName", pair.Key)
                    .Parameters.AddWithValue("@iQty", pair.Value)
                End With
                Try

                    _conn.Open()
                    _commm.ExecuteNonQuery()

                Catch ex As MySqlException

                    MsgBox(ex.StackTrace.ToString)

                End Try

                _conn.Close()

            End Using
        End Using
    Next

This code only sums the same Name on the DGV1 on the first click and saves it on DGV2.
but for example I add another data on DGV1 with the same name,
I want the QUANTITY of the newly added data  to add to the current sum of the existing data on DGV2.
A little help with this too would be really awesome.**
Ex:
TABLE 2
Name | Quantity
 a       20
 b       10

Then I add data on Table 1
Table1
Name | Quantity
  a       20
  b       10

After Pressing Post
Table2
Name | Quantity
 a        40
 b        20

Comment: Here's what I've done so far it now adds and saves to the database the problem is when I close my Childform2(DGV2), I get an error No row can be added to a DataGridView control that does not have columns. Columns must be added first.
Comment: I'll take a look at it when I get a chance, shouldn't be too big of a problem to fix
Comment: How does the code look in the ChildForm.Closing sub? How does the code look around where you open ChildForm? Is it using ShowDialog or Show? Can you supply this code. We can't really find the error in a part of the code that seems to have nothing to do with the issue :/
Comment: @WozzeC
 I open Childform2 with this 2 lines  ` oChildForm2.MdiParent = MasterForm` and   `oChildForm2.Show()`
Comment: I looked through the code and didn't see anything that would handle the forms being closed (that would also cause that error that is). You mind posting up your most recent project somewhere for me to see if it differs from mine, as I deleted some stuff last night as we talked.
Comment: @KreepN sorry for the late reply  i had lunch... it's pretty much the same except for the added codes trying to create a condition that if a data being added to DGV2 is already existing, it will just update it, if not it inserts a new one. but sure I'll post it on fileswap
Comment: @KreepN http://www.fileswap.com/dl/4Xo6Mn1yQT/AlexLuthorLINQ_(1).rar.html
Comment: @KreepN okay okay I got that closing error now... thanks :D hmmm, maybe a little help with the other problem? :D and the if I get that other problem down, I'm 90% done with the proj, and all the rest that remains are basic ones... :D
Comment: I'd assume you could use the DGV1_RowAdded event handler to get the row and add it to the DGV2.Rows collection. You would want to see if that name exists in DGV2 first and if it does, sum the quantity, or if it is now just add it.
Comment: @KreepN row added event and transfer the codes I used on how to insert or update the data?
Comment: Since DGV2 is the summation of DGV1, you would do a few things. First, check if the name is in DGV1, if so you know it will be an update and all you need to do is add that new quantity to the sum found in the SUM table. If it is not present in DGV1, it must not exist in the SUM table and therefore can be inserted, much like you are already doing.
Comment: @KreepN I cant do checking names on DGV1 because it would have similar names which aren't supposed to be posted yet, so I need to do the checking on the DGV2...
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/20153/discussion-between-kreepn-and-alex-luthor)
Answer: Check the Posted column when iterating through your rows to see if a row has been posted or not:
For Each xRows In MasterForm.oTransferRows.Where(Function(x) x.Cells("Posted").Value.ToString() = "No")

Then use the following psuedo-logic to write your code:
 'code to select data - select name from testing where name = 'Product1'
 'if(dr.Read())
 'update goes here
 'Else
 ' insert goes here
 'End if

Comment: hi there's an error on the update codes... I'll post it on chat.
Comment: @AlexLuthor Just got home, glad to hear you got it :)!
Comment: Probably wouldn't be able to do it without someone teaching me all these logic and syntax..
Comment: btw..... remember on my insert query we are only inserting 2 values.... what if I need to insert more than those 2 values(Key, Value) what if I have more than 2 data that needs to be inserted?
Comment: or should I use tuples now? instead of dictionary?
Comment: Continue to use dictionary and make the value an object that has all the properties you could ever wish to reference. Then use the key to get the object and use all the object's properties.
Comment: ohh ok so im going to like `Dim Object = "value1", "Value2", "value3"?
Comment: ok ok thanks... Ill try doing that, if this thing i'm doing right now doesn't work... :D
Comment: Hello I kind of need help again, it's about that object thing... I kind of need to have another value... but I'm quite confused on how to do this set the object on the dictionary and how to call what value I need from the object. I'll edit the codes on my question...
Comment: I just posted it as another question since this one is quite crowded. http://stackoverflow.com/questions/13677717/using-objects-in-a-dictionary-in-vb-net
Comment: @AlexLuthor You caught me as I was off to bed, I know what you're looking at doing and I'll point ya in the right direction. Make a class like you see in the above screenshot. You can then create objects like you see me doing and store those in the dictionary.
Comment: yup I know, I get the pic above, but how do I call the values needed in that object? ex: I have 2 values on an object, quantity and description. how do I get description from that object?


Prevent HTML form action from being changed

Question: I have a form on my page where users enter their credit card data. Is it possible in HTML to mark the form's action being constant to prevent malicious JavaScript from changing the form's action property? I can imagine an XSS attack which changes the form URL to make users posting their secret data to the attacker's site.
Is it possible? Or, is there a different feature in web browsers which prevents these kinds of attacks from happening?
Comment: How would the malicious JavaScript get into the form in the first place? Does the payment form allow user submitted content to be displayed (XSS)? That would be an unusual payment form.
Answer: This kind of attack is possible, but this is the wrong way to prevent against it.  If a hacker can change the details of the form, they can just as easily send the secret data via an AJAX GET without submitting the form at all.  The correct way to prevent an XSS attack is to be sure to encode all untrusted content on the page such that a hacker doesn't have the ability to execute their own JavaScript in the first place.

More on encoding...
Sample code on StackOverflow is a great example of encoding.  Imagine what a mess it would be if every time someone posted some example JavaScript, it actually got executed in the browser.  E.g.,

Were it not for the fact that SO encoded the above snippet, you would have just seen an alert box.  This is of course a rather innocuous script - I could have coded some JavaScript that hijacked your session cookie and sent it to evil.com/hacked-sessions.  Fortunately, however, SO doesn't assume that everyone is well intentioned, and actually encodes the content.  If you were to view source, for example, you would see that SO has encoded my perfectly valid HTML and JavaScript into this:
<script type="text/javascript">alert('foo');</script>
So, rather than embedding actual < and > characters where I used them, they have been replaced with their HTML-encoded equivalents (< and >), which means that my code no longer represents a script tag.
Anyway, that's the general idea behind encoding.  For more info on how you should be encoding, that depends on what you're using server-side, but most all web frameworks include some sort of "out-of-the-box" HTML Encoding utility.  Your responsibility is to ensure that user-provided (or otherwise untrusted) content is ALWAYS encoded before being rendered.
Comment: What exactly do you mean by 'encode'?  Sounds great but I'm just not sure what technologies this would involve.
Comment: I know that the best way of preventing XSS attacks is encoding everything. But it's hard, you can always forget about a field. It's not so easy to send the secret data using AJAX because of the same origin policy. I can still see some advantages of having a feature like that.
Comment: @Michal - It's pretty easy to send data to a cross-origin server.  All you have to do is insert a script tag that points the script src=xxx" at some other server and puts the data as query parameters to the script tag URL.   This is how JSONP works, though JSONP is not needed here to merely send data to some other server.
Comment: @Michał - it's actually *very* easy to send secret data via AJAX using a `GET` request. It wouldn't be very "correct" in terms of RESTful semantics, but I don't think the hacker will lose much sleep on that point. It would be very ill-advised to try and prevent XSS the way that you requested. It's kind of like asking "How do I prevent my stolen car from being driven to Vegas?" Once you have that solved, you have to keep them from taking it to Seattle, and then LA, and so on. You're much better off simply learning how to properly lock the doors (i.e., prevent XSS entirely) :p
Comment: @jmar777 - You're right. I agree that preventing XSS is the best way to go. I'm not trying to say that the feature I proposed should be used to protect against that kind of attacks. What I had in mind is that this might be a nice feature to prevent that attack from happening when you accidentally forgot to encode the JS. Just a double-check.
BTW. Why do we need the same origin policy if it's still so easy to call other domain's URLs :)
Comment: @Michał - I can understand your motivation in wanting to provide some "insurance", in the event that you accidentally forgot to encode some content.  I'd just really want to impress upon you that any hacker that is capable of embedding an XSS attack in your page is also capable of getting around any client-side protections put in place. *(cont...)*
Comment: @Michał - regarding the same-origin policy, yes, it's absolutely needed. You have to remember that the same-origin policy isn't designed to protect data on the page that the user is *currently* on, but rather to protect data on pages that the user has *previously* been on. If it weren't for the same-origin policy, for example, you could create a script that performs secured actions against that user's banking site, as long as they still had a valid session. Provided that the bank is diligent about requiring POST's for secured actions, however, the user will be protected.
Comment: See the OWASP XSS Prevention Cheat Sheet for information on how to encode: https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet
Answer:
Is there a different feature in web browsers which
  prevents these kinds of attacks from happening?

Your concern has since been addressed by newer browser releases through the new Content-Security-Policy header.
By configuring script-src, you can disallow inline javascript outright.  Note that this protection will not necessarily extend to users on older browsers (see CanIUse ).
Allowing only white-labeled scripts will defeat most javascript XSS attacks, but may require significant modifications to your content.  Also, blocking inline javascript may be impractical if you are using a web frameworks that relies heavily on inline javascript.
Answer: Nope nothing to really prevent it.
The only thing I would suggest to do is have some server side validation of any information coming to the server from a user form.
As the saying goes: Never trust the user
Comment: I think this question is more about protecting the user.
Comment: Unwittingly. They can think they post the data to the same site, but the data is sent somewhere else.
Comment: Protecting the user? from what?
Comment: @MichałFronczyk There is nothing I know that you can do to prevent that.


What is an 'async_generator' & how is it different from a 'coroutine' in Python 3.6?

Question: I suspect this has something to do with the differences b/w yield from & await.
However, & aside from the new object's designation as an async_generator, I'm unclear about consequences regarding the differences between it and a coroutine.
(I'm not sure how else to ask the question other than the one I put in the title...)
import asyncio

async def async_generator_spits_out_letters():
    yield 'a'
    yield 'b'
    yield 'c'
    yield 'd'
    await asyncio.sleep(0)

async def coroutine_prints_messages():
    while True:
        print('hi')
        await asyncio.sleep(2)

def test_it():
    print(type(async_generator_spits_out_letters))
    print(type(coroutine_prints_messages))
    # This is how I choose to do newlines....it's easier for me to read. :[
    print(); print()

    print(type(async_generator_spits_out_letters()))
    print(type(coroutine_prints_messages()))

This gives:
<class 'async_generator'>
<class 'coroutine'>

<class 'function'>
<class 'function'>

I can't make heads or tails of this...
Comment: A generator `yield`s, a coroutine does not.
Comment: Is that the only difference b/w the two?
Answer: In order for an async_generator-producing function to run in an eventloop, its output must be wrapped in a coroutine.

This is to prevent the async_generator from yielding values directly INTO the event-loop.

import asyncio

# This produces an async_generator
async def xrange(numbers):
    for i in range(numbers):
        yield i
        await asyncio.sleep(0)

# This prevents an async_generator from yielding into the loop.
async def coroutine_wrapper(async_gen, args):
    try:
        print(tuple([i async for i in async_gen(args)]))
    except ValueError:
        print(tuple([(i, j) async for i, j in async_gen(args)]))

Loops only like tasks & futures.
If a loop receives an integer or string or..anything not derived from a future from one of its tasks, it will break.
Therefore coroutines must either:

produce futures (or subclasses of future,)
OR NOT pass any values back into the loop.

Here's main():
def main():
    print('BEGIN LOOP:')
    print()
    loop = asyncio.get_event_loop()
    xrange_iterator_task = loop.create_task(coroutine_wrapper(xrange, 20))
    try:
        loop.run_until_complete(xrange_iterator_task)
    except KeyboardInterrupt:
        loop.stop()
    finally:
        loop.close()
    print()
    print('END LOOP')
    print(); print()
    print('type(xrange) == {}'.format(type(xrange)))
    print('type(xrange(20) == {}'.format(type(xrange(20))))
    print()
    print('type(coroutine_wrapper) == {}'.format(type(coroutine_wrapper)))
    print('type(coroutine_wrapper(xrange,20)) == {}'.format(type(coroutine_wrapper(xrange, 20))))
if __name__ == '__main__':
    main()

Here's the output:
BEGIN LOOP:

(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)

END LOOP

type(xrange) == <class 'function'>
type(xrange(20)) == <class 'async_generator'>

type(coroutine_wrapper) == <class 'function'>
type(coroutine_wrapper(xrange,20)) == <class 'coroutine'>


How can I check if my time series data is zero mean, stationary and independent identically distributed?

Question: I have time series data which is international monthly tourist arrival to Malaysia (N=264).
My objective is to forecast tourist arrival for 6 months ahead. After analyzing my data pattern, I found that using Box-Jenkins method gives me the best forecasting model.
The question is that, at such preliminary stage, when using Box-Jenkins methodology time series data must follow certain criteria: mean is zero, variance is constant and the data is stationary.
I used graphical methods to check if my time series data is stationary, has zero mean and constant variance.
However, how to check whether my time series data is independent and identically distributed (iid)?
What method should i used?
Answer: The errors from the model should have a zero mean or a mean that is not significantly different from zero everywhere. (1) In practice this means no Pulses, no Level/Step shifts , no seasonal pulses and no local time trends. (2) The variance of the errors from the final model should be constant which means no structural shift in error variance or dependency on the level of the original series. (3) The parameters of the model must be invariant over time. The appropriate tests for (1) is available via Intervention Detection Tests ( Tiao , Tsay and others ). The appropriate test for (2) is both the Tsay Test for constant error variance AND the Box-Cox test for transformations. The test for (3) is the Chow test . In case you don't want to program these tests yourself as they are not freely available you might want to look at AUTOBOX. All of these have been seamlessly integrated into a piece of software that I helped write , available from http://www.autobox.com. Hope this helps.
Answer: Box and Jenkins suggest used the autocorrelation and partial autocorrelation functions to identify the model.  The general Box Jenkins models are seasonal ARIMA modles which allow  for nonstationary components (periodic components and polynomial trend).  The rule for testing for nonstationarity is to compute the autocorrelation function and if the correlations are large and drop off slowly that is an indication of nonstationarity.  This is somewhat subjective but IrishStat does this in a more formal automated way with AUTOBOX. As he mentioned level shifts and pulses can also be indications of nonstationary behavior which the AUTOBOX procedures will detect and incorporate into the model. Once you fit the model the diagnostic checking looks at the residuals to determine if they are independent with 0 mean and constant variance.  Independence cannot be tested for but you can test for significant autocorrelation in the residuals and such correlation is an indication of lack of independence. The Ljung-Box test is a general test for correlation in the residual series.  If zero autocorrelation is rejected higher order AR and or MA terms may be needed.  If the residual series also looks nonstationarity in terms of slowly decaying autocorrelation then differencing of the series 1st 2nd or 3rd for linear quadratic or cubic trends respectively can be tried. If the appear to be periodic components remaining in the residual series then seasonal differencing can be used.


Correct my proof : Reflexive, transitive, symetric closure relation

Question: Let R be a relation on a set A. True or False ? If true give a proof else give a counter proof.
A) Let S be a reflexive closure of R, T is the symetric closure of S, U is the transitive closure of T. Thus U is an equivalence relation. True or False.
My answer : False.
$A = \left\{ {a,b,c} \right\}$
Let R be  $\left\{ {(a,c) , (b,c)} \right\}$
$S = \left\{ { (a,c), (b,c),(a,a), (b,b), (c,c) } \right\} $ (reflexive)
$T = \left\{ { (a,c), (b,c),(a,a), (b,b), (c,c),(c,a), (c,b) } \right\} $ (symetric)
Since (a,b) is not in my final set, it is not transitive.
Is this right ?
B) Let V be a reflexive closure of R, W is the transitive closure of V, X is the symetric closure of W. Thus X is an equivalence relation. True or False.
My answer : True.
$A = \left\{ {a,b,c} \right\} $
Let R be  $\left\{ {(a,c) , (b,c)} \right\}$
$V = \left\{ { (a,c), (b,c),(a,a), (b,b), (c,c) } \right\} $ (reflexive)
$W = \left\{ { (a,c), (b,c),(a,a), (b,b), (c,c), (a,b) } \right\} $ (transitive)
$X = \left\{ { (a,c), (b,c),(a,a), (b,b), (c,c), (a,b), (b,a) (c,a), (c,b)   } \right\} $ (symetric )
Not so sure how to finish this one... I would just go and say it works... because it seems to work. haha....  ...
Answer: In a) you forgot $U={(a,c),(b,c),(a,a),(b,b),(c,c),(c,a),(c,b),(a,b),(b,a)}$
And this is equivalence. In a, you have reflexivity and transitivity granted. Only
you need to check if all relation added by transitivity closure have symetry. And they
really have, because if there is path $((x_1,x_2),(x_2,x_3),\dots, (x_{n-1}
,x_{n})$ in S, there is path $((x_{n},x_{n-1}),\dots, (x_{2}
,x_{1})$ thanks to symetry closure and so the transitive cover inserts relation in both ways.I hope it is clear.
b)I do not think this will work try $A=(a,b,c)$
 and $R=((a,b),(c,b)$. transitivity cover adds nothing. Symmetry cover adds (c,b) (b,a) but you dont have (a,c) and X is not transitive then


Proving $|a| + |b| \leq |a+b| + |a-b|$ using the Triangle Inequality

Question:
Prove
$$|a| + |b| \leq |a+b| + |a-b|$$
using the Triangle Inequality.

Struggling a bit with this question, here's the little progress I made.
$|a+b| \leq |a| + |b|$
$|a-b| \leq |a| + |b|$
$|a+b| + |a-b| \leq 2|a|$
Answer: $$|a| = \left|\dfrac{a+b}{2} + \dfrac{a-b}{2}\right|\leq\dfrac{1}{2}(|a+b| + |a-b|)$$
and you can do a similar thing for $|b|$ to obtain your claim.


Qual o Local adequado para se alocar as bibliotecas Javascript e CSS no CakePHP 3

Question: Qual o Local adequado para se alocar as bibliotecas e plugins Javascript e CSS no CakePHP 3, pois houve um grande mudança na estrutura de diretórios desta versão para anterior.
Todos os arquivos .js e .css tem um local especifico e separados.
Todas as chamadas a arquivos são feitas desta forma ():
<?= $this->Html->css('style.css') ?>
<?= $this->Html->script('main.js',['defer' => true]) ?>

Logo é possível notar que há presunção do diretório onde os arquivos estão.
Estrutura de diretórios (Tirei prints para verem 100% a estrutura e o que tem dentro de cada):

OBS: Plugins e Bibliotecas como Jquery, Jquery-ui e Bootstrap possuem pastas e subpastas, arquivos de fontes, icones, imagens, javascript e css.
Comment: @ricardo não conheço os métodos `Html->css() ` do cake, quando estiver com um tempinho aqui vou pesquisar nos docs. O que eu faço é mapear no apache a pasta webroot como root e nos arquivos referenciar o caminho relativo, que seria css/default.css
Comment: Geralmente o front-end vai no public, www, webroot e assim vai.
Comment: @gmsantos e como ficaria as referencias?
Comment: O cakephp faz isso reescrevendo pelo `.htaccess`. Provavelmente, `css/default.css` apontaria para `app/webroot/css/default.css`
Answer: Você deve colocar na pasta webroot da sua aplicação, levando em consideração que o caminho apontado para o css ou js seria declarado sem o "webroot"
Exemplo:
No html css/default.css aponta para o diretório webroot/css/default.css
Veja o código .htaccess do Cakephp 3, para ter mais certeza disso:
<IfModule mod_rewrite.c>
    RewriteEngine on
    RewriteRule    ^$    webroot/    [L]
    RewriteRule    (.*) webroot/$1    [L]
</IfModule>

https://github.com/cakephp/app/blob/master/.htaccess
Comment: e como ficaria a declaração por exemplo: Html->css('style.css') ?> esta aponta para um arquivo solto dentro da pasta css que esta dentro do webroot, como ficaria para lingar um css da lib bootstrap?


Saving a .txt file dynamically in Android sdcard0 when everytime applicaiion opens

Question: I'm getting voice records from some people. I want to give them id's. I'm trying to save a .txt file in Android sdcard0 which contains new id value.
I mean, I open the application for new person. Program reads last id value from txt file. And then add +1 value to the new person's id. And update the .txt file content.
Later I close the application. And then, I open the application again, and read the last id value, and save the another person's voice with person id +1. I want to update .txt file id's content in Android sdcard0 memory with dynamically every time the application opens.
How can I do that? Please help me. Here is my simple code.
enter cod private String Load() {
String result = null;;
String FILE_NAME = "counter.txt";

    String baseDir = Environment.getExternalStorageDirectory().getAbsolutePath() + "/" + "Records";
    File file = new File(baseDir, FILE_NAME);

   int counter = 0;
    StringBuilder text = new StringBuilder();

    try {
        FileReader fReader = new FileReader(file);
        BufferedReader bReader = new BufferedReader(fReader);
        //.....??....

        }
        result = String.valueOf(text);
    } catch (IOException e) {
        e.printStackTrace();
    }

return result;

}
Comment: if all you want is saving your counter value, why not use the android sharedPreference?
Comment: how can I do sharedPreference? I'm new Android eclipse IDE.Please some more hints. thanks.
Answer: if I understand correctly you want to add lastid+1 to your textfile every time you open the application. And you want to store and edit this file on your Sd card as well!
There are 3 steps to try and accomplish this:

Read from the file
Find the last added id
Write the new id to the text file

//Find the directory for the SD Card using the API
//*Don't* hardcode "/sdcard"
File sdcard = Environment.getExternalStorageDirectory();

//Get the text file
File file = new File(sdcard, "counter.txt");

//Read text from file
StringBuilder text = new StringBuilder();

try {
    BufferedReader br = new BufferedReader(new FileReader(file));
    String lastLine = "";

    while ((sCurrentLine = br.readLine()) != null)
    {
        lastLine = sCurrentLine;
    }

    br.close();
    //Parse the string into an actual int.
    int lastId = Integer.parseInt(lastLine);

    //This will allow you to write to the file
    //the boolean true tell the FileOutputStream to append
    //instead of replacing the exisiting text
    outStream = new FileOutputStream(file, true);
    outStreamWriter = new OutputStreamWriter(outStream);
    int newId = lastId + 1;

    //Write the newId at the bottom of the file!
    outStreamWriter.append(Integer.toString(newId));
    outStreamWriter.flush();
    outStreamWriter.close();
}
catch (IOException e) {
    //You'll need to add proper error handling here
}

Writing to an external storage like a SD card requires a special permission in your Android Manifest, just add
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

And that should do it!
For Reference material have a look at these links:
How can I read a text file in Android?
how to read last line in a text file using java
android saving to SD card as text file
append text to the end of the file
Answer: if all you want is persistent primitive data such as save/load an int value,
you should use android shared preferences mechanism:
shared preference example


$L^2$-Norm Convergence in Probability Measure

Question: Suppose $f_n\to f$ almost everywhere for all $x\in X=(0,1)$ and for some fixed $M<\infty$, we have: $$\sup_n |{f_n}|_{L^2(X)}\leq M.$$
(a) Is it true that $\lim_{n\to \infty}|{f_n- f}|_{L^2(X)}=0$? If so, prove it.
(b) Now, suppose that $\lim_{n\to \infty}|{f_n}|_{L^2(X)} = |{f}|_{L^2(X)}$. Does the limit in (a) hold?
I think part (a) is false, with the following counter example: Let $\{f_n\}$ on $X = (0, 1)$ be a sequence of functions such that each $f_n$ is a moving bump that gets narrower and taller as $n$ increases but always integrates to 1 (I think this is the Dirac Deta function). This sequence can converge almost everywhere to the zero function (since each point is eventually outside the support of all bumps), but the $L^2$ norm of $f_n - f$ where $f$ is the zero function will not converge to 0. This is because the integral of the square of each bump does not vanish despite the bumps becoming arbitrarily narrow.
I'm not too sure about part (b) and how it would affect the answer for (a).
Comment: For $(b)$, check [this](https://math.stackexchange.com/q/51502/624428).
Answer: For a), an explicit counter-example can be $f_n(x)=\sqrt n\mathbf{1}_{(0,1/n)}(x)$. For each $x\in (0,1)$, $f_n(x)$ vanishes for $n$ larger than say $1+1/x$ hence $f_n\to 0=f$ almost everywhere but $\lVert f_n -f\rVert_2=1$.
For b), this is a special case of this thread.


Why does android.content.ContentResolver.delete return an integer?

Question: Just wondering, is there any other reason than "because the developer decided to do so" for android.content.ContentResolver.delete to return an integer?
Shouldn't it return a long, in case one deletes more entries than an integer can store?
Answer: I read somewhere that 2,147,483,647 (which is also the maximal number for an int) is the maximal number an auto increase primary key column in SQLite can store. Thus, I assume that the android developers assumed that there is no scenario in which a database would store more entries.


Reduce disk writes in sendmail system

Question: I am using sendmail in a farm of ~10 servers processing 200-500 messages a minute. I have found that after moving to a virtual environment, my largest bottleneck is disk write, and specifically the sendmail log output, and mailqueue.
I do not have control over the hardware to modify anything there.
I have two questions:
1) Is it possible to output sendmail logs compressed onto disk? This would not only reduce total disk writes but remove the need to compress during log rotation.
2) How might I set up sendmail to not write temp files to mailqueue when possible?
Edit: To provide more information:
I do not have access to information on the hardware or hypervisor (it is my understanding that it is running on a VMWare Enterprise platform though).
The filesystem is ext4.
The purpose of the servers is to act as a gateway system providing light virus and spam filtering prior to the mailserver.
Comment: Much of what sendmail does is to implement "no mail loss" integrity. It might seem there are places to speed it up, but even if you could, that exposes it to mail loss. Other mail server software has a lot of the same issue. Push the problem to whoever does have control over the hardware.
Comment: what file system?
Comment: what hypervisor?
Comment: What kind of SMTP servers do you run? Incoming MX with local mailboxes / outgoing SMTP AUTH / Email gateways between Internet and internal email system. Some tuning tips are type of load dependent.
Comment: You could also setup a log server and not log locally
Comment: Chopper3: Filesystem is ext4
Comment: Alex: I don't have access to information on the Hypervisor unfortunately
Skaperen: Thank you for the input. Unfortunately I'm stuck with the hardware for now, and the solutions are to let it be slow (which the people in charge of the hardware are OK with) or try to tune it in software.
Andrzej: These are email gateways between internet and internal systems that provide milters.
adamo: Excellent suggestion! I hadn't thought of it. That would solve a number of problems at once. Thank you.
Answer: Reducing sendmail log I/O load
1) Have you turned off fsync of log files after every every log entry?
In /etc/syslog.conf preped log file name with -
mail.* -/var/log/mail

2) You may consider using single/centralized log server foe email events [see adamo comment]
rsyslogd supports TCP based forwarding - http://linux.die.net/man/5/rsyslog.conf
Comment: After reading up on this, it seems like a great solution. It had, unfortunately, already been implemented (seems one of my predecessors tried to track down the same issue).

We are running rsyslog.
Answer: Reducing I/O load on "Internet to internal" gateway
You may use "double interactive" delivery mode to reduce I/O operations. It would delay sending ACK to "the final dot" after performing delivery attempts. It is worth to consider with high availability of internal mail servers.
define(`confDELIVERY_MODE',`interactive')
define(`confSAFE_QUEUE',`interactive')

"Sendmail Performance Tuning" by Nick Christenson (section 3.2 "Synchronization") additionally suggests considering increasing XF and DF buffers when using "double interactive" delivery mode.
define(`confXF_BUFFER_SIZE',`16384')dnl   16KB; default=4KB
define(`confDF_BUFFER_SIZE',`102400')dnl 100KB; default=4KB

Most likely your milters I/O demand can be fine tuned too.

Reduce disk writes in sendmail system


In IIS, what is the rational for shutting down the worker processes after 20 minutes of idle time?

Question: In IIS, the default is to shut down the worker process after 20 minutes of inactivity?  What is the rational behind it?  It seems like people go to lunch, then come back forcing a reload of everything.
Specifically in regards to a .NET 4, ASP.NET MVC, C# app.  Is there any reason to have the idle timeout enabled?
Answer: To save server resources.
If the server is dedicated to only serving up a single site, then it's not that big of a deal, but if you are hosting many sites, and some sites are rarely hit, then there's no reason to have an application pool that is constantly idling, and taking up valuable resources.
Another good reason is that there's always the possibility the code can have a memory leak. If it does and the process is never recycled, then it will eventually bring down the entire server.
I'm sure there are other reasons as well, those are the first ones that come to mind.
Answer: That option is designed especially for a scenario where you have many applications running on your IIS server and some of them are used infrequently.  If you have just a handful of applications then the idle timeout isn't of much use.


Evaluating $\lim_{n\rightarrow\infty}\left(\frac{\pi}{2^n}\left(\sum_{j=1}^{2^n}\sin\left(\frac{j\pi}{2^n}\right)\right)\right)$

Question:
Evaluate
$$\lim_{n\rightarrow\infty}\left(\frac{\pi}{2^n}\left(\sum_{j=1}^{2^n}\sin\left(\frac{j\pi}{2^n}\right)\right)\right)$$

I separated the expressions and also evaluated the summation. My answer is coming $0$. Can anybody review$?$ The software wolframalpha is showing nothing.
Any help is greatly appreicated.
EDIT
Let $$S=\sum_{j=1}^{2^n}\sin\left(\frac{j\pi}{2^n}\right)$$
Considering $\theta=\displaystyle\frac{\pi}{2^n}$ and using the formula I got the value of $S$ $$S=\frac{\displaystyle\sin\left(\frac{2^n+1}{2}\cdot\displaystyle\frac{\pi}{2^n}\right)}{\displaystyle\sin\left(\frac{\pi}{2^n\cdot2}\right)}\cdot\sin\left(\frac{\pi\cdot2^n}{2\cdot2^n}\right)$$
Now, re-writing the limit as $$\pi\lim_{n\rightarrow\infty}S\cdot\lim_{n\rightarrow\infty}\frac{1}{2^n}$$
Now, the value of the last limit is $0$ so the final answer is $0$.
Comment: Does it make a difference if you replace $2^n$ with $n$? Or is it just easier to deal with?
Comment: @courageousmartingale i think it won't make a difference but even in that case my answer is coming $0$. Since there is a $\frac{1}{\infty}$ term the answer will always be $0$ in my opinion.
Comment: Do you know about Riemann sums?
Comment: @StefanLafon oh no no...im just an amateur...but if you have a solution using that...I'll appreciate that
Comment: "Now, the value of the last limit is 0 so the final answer is 0." Only if the first limit exists and is finite.  "Since there is a 1∞ term the answer will always be 0 in my opinion."  What about $1 = \lim_{n\to \infty} 1 =\lim_{n\to \infty}\frac nn = \lim_{n\to \infty} n\cdot \frac 1n = \lim_{n\to \infty} n\cdot \lim_{n\to \infty} \frac 1n = \lim_{n\to\infty} n\cdot 0 = 0$.  Do you see what (2 things) are wrong with that?
Answer: You have wrongly assumed that $\lim_{n\to\infty} S$ is finite. Hence, you cannot use limit laws to write it like in your last line. Instead, \begin{align*}
\lim_{n\rightarrow\infty}\left(\frac{\pi}{2^n}\left(\sum_{j=1}^{2^n}\sin\left(\frac{j\pi}{2^n}\right)\right)\right) &= \pi \left(\lim_{n\to\infty} \sin\left(\frac{2^n+1}{2}\cdot\displaystyle\frac{\pi}{2^n}\right)\right)\left(\lim_{n\to\infty}\frac{1}{2^n\sin\left(\frac{\pi}{2^{n+1}}\right)}\right).
\end{align*}
For the first term, you can take the limit inside because sine is a continuous function to get $\sin(\pi/2)=1$.
For the second term, I assume you already know $\lim_{x\to0} \sin(x)/x=1$. It should be easy to show that the second term gives $2/\pi$ using that.
Thus, the final answer is 2.
As one of the comments suggested, you can use integration to solve this problem. For a continuous function $f$ on the interval $[a, b]$, we have \begin{align*}
\int_a^b f(x)dx &= \lim_{n\to\infty} \frac{b-a}{n} \sum_{j=1}^n f\left(a+j\frac{b-a}{n}\right).
\end{align*}
So, your problem becomes $\int_0^{\pi} \sin(x)dx = \cos(0)-\cos(\pi) = 2$.


Saber quando uma linha do arquivo começa com determinada string

Question: Tenho um arquivo como esse:
# Blender v2.69 (sub 0) OBJ File: 'CUBO.blend'
# www.blender.org
mtllib cube.mtl
o Cube.022_Cube.030
v 0.450000 -1.450000 0.550000
v 0.450000 -1.450000 1.450000
v -0.450000 -1.450000 1.450000
v -0.449999 -1.450000 0.550000
v 0.450000 -0.550000 0.550000
v 0.449999 -0.550000 1.450000
v -0.450000 -0.550000 1.450000
v -0.450000 -0.550000 0.550000
usemtl Material
s off
f 5 8 7 6
f 1 5 6 2
f 3 7 8 4
f 5 1 4 8
usemtl Material.026
f 2 6 7 3
usemtl Material.047
f 1 2 3 4

Como faço para identificar quando a linha começar com o v usemtl ou f? Pois dependendo disso eu devo adicionar o conteúdo da linha em uma matriz, atualmente faço isso assim:
float **readVertices(char *filename)
{
    int lines = fileLines(filename), li;
    float **matriz = createArrayFloat(lines, 3);

    FILE *file;

    file = fopen(filename, "r");

    if(file == NULL)
            printf("Erro, nao foi possivel abrir o arquivo\n");
    else {
        int count = -1;
        do {
            count ++;
        }while(fscanf(file, " v %f %f %f", &matriz[count][0], &matriz[count][1], &matriz[count][2]) != EOF);
    }
    fclose(file);
    return matriz;
}

Mas quero que dentro do while verifique qual tipo de linha estou lendo e armazene em uma matriz diferente, por exemplo, quando a linha começa com v eu armazeno na matrizV, quando começa com f eu armazeno na matrizF.
Answer: Você pode ler a string inicial e, dependendo do conteúdo, tomar uma ação diferente.
char aux[16];
fscanf(arq, " %s", aux);
if      (strcmp(aux,      "o") == 0) { ... }
else if (strcmp(aux, "usemtl") == 0) { ... }
...


Programmatically subscribing user to a organic group

Question: I need to expose OG memberships within API functions calls and wonder how to programmatically subscribe a user to a group (pending state) and then i need to approve him programmatically and change from pending to member. Im using OG 7.2.
function ask_membership($uid, $gid){
  ????
}

function approve_membership($uid, $gid){
  ????
}

Any help appreciated
Answer: Ok got it working, its still the og_group() function.
function group_membership($type, $gid, $uid){
  $group_type = 'node'; //All our/my groups are of type node
  $entity_type = 'user'; //All our/my members are type users
  $values = array(
    'entity_type' => 'user',
    'entity' => $uid,
    'field_name' => FALSE,
    'state' => 1,
  );
  $return = FALSE;

    switch ($type){
        case 'subscribe':
          $values['state'] = 2;
          og_group($group_type, $gid, $values);
        $return = 'SUBSCRIBED TO TEAM';
          break;
      case  'approve':
          $values['state'] = 1;
          og_group($group_type, $gid, $values);
        $return = 'APPROVED TO TEAM';
        break;
      case 'delete':
        og_ungroup($group_type, $gid, $entity_type, $uid);
        $return = 'REMOVED FROM TEAM';
        break;
      default:
        $return = 'FALSE TYPE';
        break;
    }

  print json_encode($return);
}

Not that pretty but it works for now.
Comment: Thanks for sharing this, I think it's great overall. I don't understand the purpose of $url, so I'm going to try and run it without that param.
Comment: Thx, think i removed some code that wasn't useful and it might be a left over :)


A sequence of measurable sets

Question: I want to find a sequence of measurable sets $A_k$.
such that $A_k \subset [0,1]$, $\lim \lambda(A_k) =1$, but $\liminf A_k = \varnothing$.
There are some examples on function such as $\sin x \over x$ , but I can't apply on a set, $A_k$.
Please give me a simple example.
Answer: One technique you can use is a "running window". The sequence of intervals  $I_1=[0,1/2),I_2=[1/2,1],I_3=[0,1/3),I_4=[1/3,2/3),I_5=[2/3, 1],I_6=[0,1/4),\ldots$ has the property that their measure goes to $0$ but every element of $[0,1]$ is in infinitely many of these intervals. So you can simply let $A_k=[0,1]\backslash I_k$.
Comment: Thank you for your answer Michael, but I don't know in your example does there exist a limit of $I_k$? I think there exists not $lim$ but only sub limit of $A_k$. And I think that $liminf(A_k)$ is not $\varnothing $ but $[0,1]$.. because $lim(I_k)=\varnothing$. If I'm confusing something please notice me that. Thank you.
Comment: I'm not sure what you mean. The sequence $(\lambda(A_k))$ is nondecreasing and bounded and certainly has a limit. The sequence $A_k$ does not have a limit in the sense that $\lim\inf A_k=\emptyset$ but $\lim\sup A_k=[0,1]$. This follows from the characterizations that $\lim\inf A_k$ is the set of points that are in all but finitely many of the $A_k$ and $\lim\sup A_k$ is the set of points that are in infitely many of the $A_k$. If any sequence $(A_k)$ of measurable sets has a limit $A$, then $\lim\lambda(A_k)=\lambda(A)$, so this is unavoidable.
Comment: $\lim(I_k)\neq\emptyset$ since $\lim\sup(I_k)=[0,1]$. Every point in $[0,1]$ is in infinitely many thin intervals. I have to leave now, I can respond again in a few hours.
Comment: Oh. I got it. Thank you very much Then I wanna prove next question above. Can I have some intution for proving that?

A sequence of measurable sets


MySQL Datetime - Gap identification

Question: I am designing a MySQL database that will contain minute by minute data collected from a remote source.
I plan for the data table to hold a datetime field and as such I might have 3 months of data each row representing the values at each minute.
How can I identify gaps in the data easily? Should I even be using datetime?
I will be using PHP5 for the application itself.
Thanks,
Jim
Comment: if you are building a logging application then consider using TIMESTAMP data type
Comment: In what situations would there be gaps? Lack of connection to remote source?
Comment: Yes, where the connection to the remote service is unavailable.
Comment: (or my cron to check the data source isn't running for some reason)
Answer: I'd give each row a unique sequential reference (1,2,3...) then do a self join on the table
...from mytable t1 join mytable t2 where (t1.ref+1)=t2.ref

Depending on how you store the dates, you can subtract them (one way or another) and determine if there is a gap. Wrap the whole thing in a query that extracts the rows where there is a gap.
You might want to consider storing the dates as seconds from a particular start date.
Answer: Identifying the gaps is an interesting problem. The best approach will depend on the size of the gap, but here is another way to tackle it, and one which might be better if the gaps are reasonably large compared to the number of records you have.
Use a MySQL aggregation function in a query to count the number of records for a set of buckets. The buckets need to be similar in size to the kinds of gaps you are interested in. Assuming you're interested in gaps approximating a day or so, I'd do something like this:
SELECT TO_DAYS(my_timestamp), COUNT(*)
FROM my_table
GROUP BY TO_DAYS(my_timestamp)

This will return an association between days and timestamp counts. I'd do the rest in a language like Perl or Java (or even R, see later) where I can process the data.
The technique I'd use would be a test of the difference between the observed frequency (the count) and the expected frequency, which will be the total number of records, divided by day range. The expected frequency for each day would be something like:
SELECT (SELECT COUNT(*) FROM my_table) /
        ((SELECT TO_DAYS(MAX(my_timestamp)) FROM my_table) -
         (SELECT TO_DAYS(MIN(my_timestamp)) FROM my_table) + 1)

Now, for each bucket (remembering that in the first result, completely missing days will just be not returned, not returned as a count of zero -- you need to treat them as if they are zero, you can use a statistical test, the chi square test, to estimate the probability of this being chance (for details, see: http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test). The calculation is, basically ((expected - observed)^2 / expected). This is an estimate of the likelihood of deviation.
If you need to work out which buckets are low in samples, set a reasonable threshold on this calculated value, and look for buckets where the value exceeds the threshold. It may take a little experimentation to devise an appropriate value, but this is a sound way of determining gaps.
Comment: Why R? Well, it has some very good built-in stuff for statistical calculations like the chi square tests

MySQL Datetime - Gap identification


How to compare the results of two hierarchical classification results?

Question: How could I compare two classification results from two implementations of e.g. using different methods/features for pairwise similarity calculation? If one method classified e.g. 3000 data points into 180 groups and another into 150 groups:

Can I consider one method results as standard of truth?
Can I build a ROC, AUC, or something similar?
How could true negative/false negative, etc., be defined?
Comment: Welcome to CV. All exploratory approaches to classification suffer from an absence of pre-existing groupings. This means that there is no "ground truth" against which to validate the results. Given that, statistical methods should be applied which evaluate various properties of the solution(s). For instance, cross-validation techniques would tell you the extent to which the competing partitions are misclassified -- the higher the rate of misclassification, the less desireable the solution. Another consideration is the utility of the results over time -- which solution is more "actionable?"
Comment: Related, unclear if duplicate: http://stats.stackexchange.com/questions/15548/validation-of-clustering-results
Comment: Pretty sure this question is about hierarchical clustering and not classification.
Answer: As @DJohnson stated, "ground truth" is difficult in such situations - so there is no simple "absolute reference" to compare to. But you could directly compare 2 results that were composed over multiple classes if this helps in your case.
I assume you are able to obtain a confusion matrix for both results:

For each result, the true/false positive/negative rate, AUC, EER, etc., would be calculated for each individual class. Consequently, this leaves you will all those rates for all individual classes.
For comparing your results: look at the distribution of those values, over both results. The distribution gives you an idea how well the results are for the classes within (e.g. average performance + performance spread). For direct comparison, you could compare the numeric values of mean/median and sd/mad performance - but looking at and comparing e.g. two boxplots generated for each results might be easier.
Comment: Hey, the 1st results was from manually curation of biologist, so they only have a list of groups that biologist think should put together, so true positive is clear, but true negative is kind of non-clear.    i might should consider any combination of one item from group i with group j (i!=j), but based on their classification, each group may have various numbers.(some may have 5 items, some 3 items, some 2 items...)
Comment: When i used computational method results to compare a manually curation, again, true positive is clear, i.c. the items puts into a the same group as man did one. what should i consider true negative? false negative etc....
Comment: @user1830108 I struggle to imagine how this data looks like. Could you show a small example in your question? If I understand correctly that you don't have certain combinations of features in your data which are realistic/important and which you would consider being negative, please reflect this is the sample too.


Index was outside of bounds of the array while comparing images

Question: I am trying to figure this one out. I am comparing images in my program. For the most part when I try and compare the image I have no issues. Although when I use a few different combinations I get this error. I originally thought it was because the image formats were different. Although that is not the case.
If I take each image and compare them against something else it works fine. Just not when compared against eachother. I have marked below where the error appears.
public Color this[int x, int y]
{
    get
    {
        int index = (x + (y * image.Width)) * 4;
        return Color.FromArgb(rgbValues[index + 3], rgbValues[index + 2], rgbValues[index + 1], rgbValues[index]); //This line is where the index out of bounds of the array error happens.
    }

    set
    {
        int index = (x + (y * image.Width)) * 4;
        rgbValues[index] = value.B;
        rgbValues[index + 1] = value.G;
        rgbValues[index + 2] = value.R;
        rgbValues[index + 3] = value.A;
    }
}

///
/// Width of the image.
///
public int Width
{
    get
    {
        return image.Width;
    }
}

///
/// Height of the image.
///
public int Height
{
    get
    {
        return image.Height;
    }
}

///
/// Returns the modified Bitmap.
///

I would appreciate any help. I am so confused to why it only happens when the images are used in combination.
In case you were wondering how I load the image I do it in vb.net. The C# code is in a dll. Here is my vb.net code to load the bitmap to the memory. Also the image size is: 431x253
Dim bm As Bitmap = Bitmap.FromFile(Label1.Text)
Dim bm2 As Bitmap = Bitmap.FromFile(Label2.Text)
' Dim pnt As Point = ImageFinder.Contains(bm, bm2)
Dim ir As New ImageChecker(bm, bm2)
Dim imageContains As Boolean = ir.findimageboolean()
MessageBox.Show(imageContains)

EDIT: Here is the full code of the DLL
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Linq;
using System.Text;
using System.IO;

namespace ImageRecognition
{
    public class LockedFastImage
    {
        private Bitmap image;
        private byte[] rgbValues;
        private System.Drawing.Imaging.BitmapData bmpData;

        private IntPtr ptr;
        private int bytes;

        public LockedFastImage(Bitmap image)
        {
            this.image = image;
            Rectangle rect = new Rectangle(0, 0, image.Width, image.Height);
            bmpData = image.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite, image.PixelFormat);

            ptr = bmpData.Scan0;
            bytes = Math.Abs(bmpData.Stride) * image.Height;
            rgbValues = new byte[bytes];
            System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);
        }

        ~LockedFastImage()
        {
            // Try to unlock the bits. Who cares if it dont work...
            try
            {
                image.UnlockBits(bmpData);
            }
            catch { }
        }

        ///
        /// Returns or sets a pixel of the image.
        ///
        /// <param name="x">x parameter of the pixel</param>
        /// <param name="y">y parameter of the pixel</param>
        public Color this[int x, int y]
        {
            get
            {
                int index = (x + (y * image.Width)) * 4;
                return Color.FromArgb(rgbValues[index + 3], rgbValues[index + 2], rgbValues[index + 1], rgbValues[index]);
            }

            set
            {
                int index = (x + (y * image.Width)) * 4;
                rgbValues[index] = value.B;
                rgbValues[index + 1] = value.G;
                rgbValues[index + 2] = value.R;
                rgbValues[index + 3] = value.A;
            }
        }

        ///
        /// Width of the image.
        ///
        public int Width
        {
            get
            {
                return image.Width;
            }
        }

        ///
        /// Height of the image.
        ///
        public int Height
        {
            get
            {
                return image.Height;
            }
        }

        ///
        /// Returns the modified Bitmap.
        ///
        public Bitmap asBitmap()
        {
            System.Runtime.InteropServices.Marshal.Copy(rgbValues, 0, ptr, bytes);
            return image;
        }
    }

    public class ImageChecker
    {

        private LockedFastImage big_image;
        private LockedFastImage small_image;
        ///
        /// The time needed for last operation.
        ///
        public TimeSpan time_needed = new TimeSpan();

        ///
        /// Error return value.
        ///
        static public Point CHECKFAILED = new Point(-1, -1);

        ///
        /// Constructor of the ImageChecker
        ///
        /// <param name="big_image">The image containing the small image.</param>
        /// <param name="small_image">The image located in the big image.</param>
        public ImageChecker(Bitmap big_image, Bitmap small_image)
        {
            this.big_image = new LockedFastImage(big_image);
            this.small_image = new LockedFastImage(small_image);
        }

        ///
        /// Returns the location of the small image in the big image. Returns CHECKFAILED if not found.
        ///
        /// <param name="x_speedUp">speeding up at x achsis.</param>
        /// <param name="y_speedUp">speeding up at y achsis.</param>
        /// <param name="begin_percent_x">Reduces the search rect. 0 - 100</param>
        /// <param name="end_percent_x">Reduces the search rect. 0 - 100</param>
        /// <param name="begin_percent_x">Reduces the search rect. 0 - 100</param>
        /// <param name="end_percent_y">Reduces the search rect. 0 - 100</param>
        public Point bigContainsSmall(int x_speedUp = 4, int y_speedUp = 4, int begin_percent_x = 0, int end_percent_x = 100, int begin_percent_y = 0, int end_percent_y = 100)
        {
            /*
             * SPEEDUP PARAMETER
             * It might be enough to check each second or third pixel in the small picture.
             * However... In most cases it would be enough to check 4 pixels of the small image for diablo porposes.
             * */

            /*
             * BEGIN, END PARAMETER
             * In most cases we know where the image is located, for this we have the begin and end paramenters.
             * */

            DateTime begin = DateTime.Now;

            if (x_speedUp < 1) x_speedUp = 1;
            if (y_speedUp < 1) y_speedUp = 1;
            if (begin_percent_x < 0 || begin_percent_x > 100) begin_percent_x = 0;
            if (begin_percent_y < 0 || begin_percent_y > 100) begin_percent_y = 0;
            if (end_percent_x < 0 || end_percent_x > 100) end_percent_x = 100;
            if (end_percent_y < 0 || end_percent_y > 100) end_percent_y = 100;

            int x_start = (int)((double)big_image.Width * ((double)begin_percent_x / 100.0));
            int x_end = (int)((double)big_image.Width * ((double)end_percent_x / 100.0));
            int y_start = (int)((double)big_image.Height * ((double)begin_percent_y / 100.0));
            int y_end = (int)((double)big_image.Height * ((double)end_percent_y / 100.0));

            /*
             * We cant speed up the big picture, because then we have to check pixels in the small picture equal to the speeded up size
             * for each pixel in the big picture.
             * Would give no speed improvement.
             * */

            //+ 1 because first pixel is in picture. - small because image have to be fully in the other image
            for (int x = x_start; x < x_end - small_image.Width + 1; x++)
                for (int y = y_start; y < y_end - small_image.Height + 1; y++)
                {
                    //now we check if all pixels matches
                    for (int sx = 0; sx < small_image.Width; sx += x_speedUp)
                        for (int sy = 0; sy < small_image.Height; sy += y_speedUp)
                        {
                            if (small_image[sx, sy] != big_image[x + sx, y + sy])
                                goto CheckFailed;
                        }

                    //check ok
                    time_needed = DateTime.Now - begin;
                    return new Point(x, y);

                CheckFailed: ;
                }

            time_needed = DateTime.Now - begin;
            return CHECKFAILED;
        }
         public Boolean findimageboolean(int x_speedUp = 1, int y_speedUp = 1, int begin_percent_x = 0, int end_percent_x = 100, int begin_percent_y = 0, int end_percent_y = 100)
        {
            /*
             * SPEEDUP PARAMETER
             * It might be enough to check each second or third pixel in the small picture.
             * However... In most cases it would be enough to check 4 pixels of the small image for diablo porposes.
             * */

            /*
             * BEGIN, END PARAMETER
             * In most cases we know where the image is located, for this we have the begin and end paramenters.
             * */

            DateTime begin = DateTime.Now;

            if (x_speedUp < 1) x_speedUp = 1;
            if (y_speedUp < 1) y_speedUp = 1;
            if (begin_percent_x < 0 || begin_percent_x > 100) begin_percent_x = 0;
            if (begin_percent_y < 0 || begin_percent_y > 100) begin_percent_y = 0;
            if (end_percent_x < 0 || end_percent_x > 100) end_percent_x = 100;
            if (end_percent_y < 0 || end_percent_y > 100) end_percent_y = 100;

            int x_start = (int)((double)big_image.Width * ((double)begin_percent_x / 100.0));
            int x_end = (int)((double)big_image.Width * ((double)end_percent_x / 100.0));
            int y_start = (int)((double)big_image.Height * ((double)begin_percent_y / 100.0));
            int y_end = (int)((double)big_image.Height * ((double)end_percent_y / 100.0));

            /*
             * We cant speed up the big picture, because then we have to check pixels in the small picture equal to the speeded up size
             * for each pixel in the big picture.
             * Would give no speed improvement.
             * */

            //+ 1 because first pixel is in picture. - small because image have to be fully in the other image
            for (int x = x_start; x < x_end - small_image.Width + 1; x++)
                for (int y = y_start; y < y_end - small_image.Height + 1; y++)
                {
                    //now we check if all pixels matches
                    for (int sx = 0; sx < small_image.Width; sx += x_speedUp)
                        for (int sy = 0; sy < small_image.Height; sy += y_speedUp)
                        {
                            if (small_image[sx, sy] != big_image[x + sx, y + sy])
                                goto CheckFailed;
                        }

                    //check ok
                    time_needed = DateTime.Now - begin;
                    return true;

                CheckFailed: ;
                }

            time_needed = DateTime.Now - begin;
             return false;
        }
    }
}

Comment: Your `public Color this[int x, int y]` indexer getter implementation appears to be 0-index based; are you _sure_ that you do not ask for the pixel at `Width` and/or `Height`? Are you sure the highest values you hit are at `Width - 1` and/or `Height - 1`?
Comment: well when you Debug the code ..what is the value of index at this line
int index = (x + (y * image.Width)) * 4; x=? y=? what is the image.Width as well..?
Comment: Where is the array `rgbValues` declared and initialized?
Comment: What pixel format do you use when locking the images?
Comment: I have edited the question to include the full code, the image size and how I load them.
Comment: DJ Kraze- The index at this point is: 329284
Comment: You should also forcefully set the `PixelFormat` so that they are always the same for every image when locking them..
Comment: Simon, where should I do that? Can I do that in the vb.net code that opens the image? OR does this need to be in C# code.
Answer: If Stride is negative you need to copy image bytes in a different way, so I made some modifications to your constructor:
    public LockedFastImage(Bitmap image)
    {
        this.image = image;
        Rectangle rect = new Rectangle(0, 0, image.Width, image.Height);
        bmpData = image.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);

        ptr = bmpData.Scan0;
        bytes = Math.Abs(bmpData.Stride) * image.Height;
        rgbValues = new byte[bytes];
        if (bmpData.Stride < 0)
        {
            int lines, pos, BytesPerLine = Math.Abs(bmpData.Stride);
            for (lines = pos = 0; lines < image.Height; lines++, pos += BytesPerLine)
            {
                System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, pos, BytesPerLine);
                ptr = (IntPtr)(ptr.ToInt64() + bmpData.Stride);
            }
        }
        else
            System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, bytes);
    }

If you are storing pixel data as ARGB, you need to lock pixels using the corresponding PixelFormat.
I also modified your accessors:
    public Color this[int x, int y]
    {
        get
        {
            int index = y * (image.Width << 2) + x;
            return Color.FromArgb(rgbValues[index + 3], rgbValues[index + 2], rgbValues[index + 1], rgbValues[index]);
        }

        set
        {
            int index = y * (image.Width << 2) + x;
            rgbValues[index] = value.B;
            rgbValues[index + 1] = value.G;
            rgbValues[index + 2] = value.R;
            rgbValues[index + 3] = value.A;
        }
    }

By the way, I haven't tested this code!

Strictly speaking, you should save BytesPerLine as a private field and then use it to calculate index in your accessors. So you should change this (image.Width << 2) to BytesPerLine.
Also you can try changing your get accessor to the following (not sure if it's faster, but you can try it if you wish):
    get
    {
        return Color.FromArgb(BitConverter.ToInt32(rgbValues, y * (image.Width << 2) + x));
    }


Renaming y axis ticks ggplot

Question: I create the following plot: enter image description here
These ticks are generated automatically.
I'd like to rename the Y axis ticks with :
80000 to 1:20:000,
84000 to 1:24:000,
88000 to 1:28:000,
92000 to 1:32:000
This is my code:
seby_time<-ggplot(tempi_seb,aes(lap,milliseconds),color=position) +
geom_point() +
geom_line(color="red")

I tried varius solutions but no one work.
Geom_line() or other type of "line functions" are required for this job.
Thanks
Comment: See https://ggplot2.tidyverse.org/articles/articles/faq-axes.html#label-formatting-and-customization
Answer: Use scale_y_continuous. Here is a minimal example
library(ggplot2)

ggplot(iris, aes(x = Petal.Length, y = Sepal.Length)) + geom_point() +
  scale_y_continuous(breaks = c(5, 7), labels = c("5.0", "7.0"))

Comment: Thank you so much ,you give me a really big help!

Renaming y axis ticks ggplot


Can't point my subdomain address to CloudFront distribution (using S3 + CF + Route 53)

Question: [Jump to bottom to see updates and latest attempts]
First of all, I apologize if I misuse terminology here. I work as FE developer and this area on networking and setting up domains and aws services is really not my strongest. I took this task as a good chance for learning.
I'm hosting an application on S3 buckets. The idea is to have a bucket for analytics.gleechi.io, another one for staging-analytics.gleechi.io where I will use as staging. gleechi.io on itself has no content, so user just redirects to analytics.gleechi.io
The domain name gleechi.io was previously on a different registrar (godaddy) and I transferred it to aws.
After much struggle I was able to create hosted zones and CloudFront distributions for it and link them to my S3 buckets.
The CF works fine and if I open its link d10yu7tqracz7m.cloudfront.net my application loads correctly.
What is missing now is to point analytics.gleechi.io address to that CF.
I can't figure out what am I doing wrong in here as I followed aws guides already so many times and everything seems to be in place.
I have a few clues that something is wrong with DNS and perhaps it has to do with the fact that everything was registered somewhere outside aws before and I perhaps need to do something in these places?

I login into godaddy but I see absolutely nothing there but when I open gleechi.io it shows me a page that is clearly from godaddy.
Running the command dig +short gleechi.io gives me the address from 1) but running dig +short analytics.gleechi.io gives me nothing
To add to the confusion when I run  whois gleechi.io it gives me Registrar WHOIS Server: whois.gandi.net
Registrar URL: http://www.gandi.net
why gandi? Shouldn't it be aws, since it is where it was transferred? Or if anything...godaddy then. Is it possible that maybe only the domain was registered in godaddy but maybe DNS was setup in gandi ? (I didn't do the setup myself, but a former employee years ago, so I might be dealing with unknown scenario here)

Here is how I setup my Route 53:

I created a hosted zone analytics.gleechi.io and in there I created a record A that points to my CF d10yu7tqracz7m.cloudfront.net.
I copied the 4 NS values from there
I added the 4 NS values from the analytics.gleechi.io together with the already 4 existent NS values in gleechi.io
Under Domains -> Registered Domains -> gleechi.io I have the 4 NS originally created by Route 53 on gleechi.io (but not the 4 from analytics.gleechi.io) , which I assume is the right way to do

Here is the aws guide I followed https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-routing-traffic-for-subdomains.html
I also tried the 1st option (Create records in the hosted zone for the domain) but that didn't work out either.
Here are screenshots of the configs and values I have on Route 53

UPDATE after comments and answers:
I deleted the host zone for analytics.gleechi.io and added that as an A record under gleechi.io . Still no success
Trying to lookup gleechi.io or analytics.gleechi.io on https://lookup.icann.org/lookup throws me this error:
Failed to perform lookup using WHOIS service: TLD_NOT_SUPPORTED
Below screenshots of my updated configuration
I'm running out of ideas here
Answer: There are two things that you have to do:

create an A DNS record (of type Alias) for analytics.gleechi.io and point it to the CloudFront distribution in the gleechi.io hosted zone.
set analytics.gleechi.io as an alternate domain for your CloudFront Distribution (Using custom URLs by adding alternate domain names (CNAMEs).

You can get rid of the secondary hosted zone you have created for analytics.gleechi.io and under Domains -> Registered Domains, leave only the 4 original name servers for the gleechi.io hosted zone.
Comment: omg you are right! I love u man! Screw the formal etiquette on forums. You have no idea how much time I wasted on this. Thank you so much!
Comment: I did this. Guess I'll have to wait a bit to see if it worked. Thanks
Comment: I updated the question with this new attempt by the end
Comment: It works for me ;)
Answer: your hosted zone should have only 4 DNS records. You have 8, which will cause the DNS to function incorrectly. What you have to do is to create hosted zone only for the main domain gleechi.io. Then you can create the DNS records (Alias A records or CNAME records) for the two subdomains  analytics and staging-analytics in the same hosted zone itself.
Comment: It has 8, because the other 4 are from the subdomain as described in the 2nd method here https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-routing-traffic-for-subdomains.html  . I tried your solution before (1st method in the link) and that did not work out
Comment: The documentation was not understood correctly. See point no.4. When hosted zone for subdomain is created, the main domain has to be told about it and hence separate nameservers should be specified. The solution specified will work above for sure. But since you misconfigured the DNS zones, it will not work. So get rid of the current hosted zones.  only 4 unique NS records should be shown for the main domain. Then the above solution will start working automatically. Also did you specify the subdomain as the Alternate domain name in the cloudfront settings?
Comment: Im sorry, I'm still confused about "When hosted zone for subdomain is created, the main domain has to be told about it and hence separate nameservers should be specified." . That is exactly why the 8 NS on the main domain. Originally it had just 4. Once I created the hosted zone for subdomain, then I added these 4 from the subdomain to the main domain. What am I missing here? I'll try your solution for sure later, but I still need to be able to understand because I believe you, likely it's me misreading the guide somewhere
Comment: if the separate hosted zone for analytics.gleechi.io is created, then the nameservers should be specified for analytics.gleechi.io in the main domain's DNS records. But you have specified them for gleechi.io. This is the misinterpretation of the documentation i was referring to.
Comment: no, I have specified for both. Each one contains 4. But anyway, I updated my question with the other approach still no success :(
Comment: yes, you need to wait before DNS propogates.
Comment: it worked. You are a legend Sir!


The right way to use Machine Learning to predict latitude and longitude

Question: There are some simple ML techniques that can be used to easily predict latitude/longitude co-ordinates, such as predicting the latitude and longitude separately using two different models. However, I get the sense that this is a simple hack that doesn't give the best results. To quote another paper:

Most regression
  methods assume either that either only one real number is to be
  predicted, or if multiple real numbers are to be predicted that they
  are independent. The problem of predicting a point on the surface of a
  sphere is more complicated as the latitudes and longitudes involved
  are not independent.

Unfortunately, the authors of the linked paper just side-step the issue by using kNN. I'd like to use supervised learning with some non-geographical inputs (strings, numbers, etc...) to predict a latitude/longitude co-ordinate, and I'd like to approach it using "best practices" rather than a simple hack. How should I go about it? Any links to any papers or blog posts would be much appreciated. Thanks!
Comment: [Here's a question](http://stats.stackexchange.com/questions/172792/) with more detail on von Mises and so on.
Comment: von Mises distribution is a good fit for modeling data on circle and on sphere by extension.
Answer: The problem isn't just potential interdependence of latitudes and longitudes; it's that the scales wrap around. On a circle 359 degrees and 1 degree are quite close. A general term for this type of problem is directional statistics.
One way to start with analysis of spatial data would be to go over the CRAN Task View on that topic. That page details the many R packages available for handling spatial data, analyzing point patterns, doing spatial regression, etc. Documentation for R packages that seem related to your specific interests will typically include helpful references to related literature.
Comment: A little additional info that might be helpful: according to [another question](http://stats.stackexchange.com/questions/36145/linear-regression-and-spatial-autocorrelation?rq=1), there is a [book](http://www.springer.com/us/book/9780387781716) about spatial data analysis in R. This book might be helpful for those hoping to learn about this kind of thing.
Comment: Both answers have some useful information, but I've ultimately decided to mark this answer as the correct one. I think that when trying to do machine learning/regression with a lat/lon as the output (rather than the input), this answer is more relevant.
Comment: As if things weren't complicated enough! Now I have to deal with spatial autocorrelation AND the scales wrapping around! :)


Naive single output Q Network: how to combine state/action for input?

Question: Most often in (deep) Q network implementations are made using DeepMind's approach of having the state as the network's input and the individual Q-values for each output [1].
My question is related is its naive construction, this is, having the state/action pair as input and the Q value as the output [1]: how would you combine the state (say, represented by a stack of images) with a given action, if order to feed the pair to the network?

[1] Naive (left) and optimised (right) q network architectures: https://www.nervanasys.com/wp-content/uploads/2015/12/Screen-Shot-2015-12-21-at-11.27.12-AM.png
Answer: I've eventually found one possible answer to this.
When approximating the Q-function with a network which accepts both the state and action as inputs, if we use a we use a convolutional network ending in a fully connected layer for example, then one option would to add the action as an additional input on this fully connected layer.