<commit_before># }!/usr/bin/python

# adds episodes from an external source, like a json file or url.

"""
fields:
id - uniquie ID of item (used to update item if details change)
name - title of talk
room - "room1" if there is only one room.
start - datetime in some parsable format 
duration in minutes, or HH:MM:SS 
authors - comma seperated list of people's names.
contact - email(s) of presenters.
released - permission to release.
license - CC license (13 is safe)
description - used as the description of the video (paragraphs are fine)
conf_key - PK in source database - should be uniquie across this file
conf_url - URL of talk page
tags - comma seperated list 
"""

"""
NOTE: In general it is better to build the export as simple as posible, 
even at the expense of deviatng from the above fields.  Exporting extra
fields is just fine.  They will be ignored.
For instance, if you store start and end in the database but not duration, 
export end, I can can calculate duration.  
Given historic problems with duration, I woujldn't mind seeing both end 
and duration so that I can verify my transformations.  
"""

"""
There is a datetime format issue here because json doesn't define a date format.  Do whatever makes the server side code smallest and easiest to code. easy to read data is good too.  Do not write extra server side code to try and make it easier to parse.  That has lead to data loss, which means trying to debug a ssytem that starts with the event's data input and ends with veyepar's database, which is not fun.  

Here is PyCon 2010's impemtation:
datetime objects are represented as a time tuple of six elements:
    (year, month, day, hour, min, sec) 
        "start":      [2010, 2, 19, 9, 30, 0],
        "duration":   30, # in min
http://us.pycon.org/2010/conference/schedule/json/
Easy to code, hard to read.

OSDC2010: easy to read, harder to parse/assemble into start durration.
http://2010.osdc.com.au/program/json
# Day: "Tue 23 Nov"
# Time: "09:00 - 17:00"

Again, keep the server side code simple.
"""

# FireFox plugin to view .json data:
# https://addons.mozilla.org/en-US/firefox/addon/10869/

import datetime 
import urllib2,json
# from csv import DictReader
# from datetime import timedelta
from dateutil.parser import parse

import gdata.calendar.client
import gdata.calendar.service

import pw # for google calandar

import process
import lxml.etree


from main.models import fnify, Client, Show, Location, Episode

class add_eps(process.process):

    def addlocs(self, schedule, show):
      """ pycon 2010 
      seq=0
      locs=d['rooms']
      for l_id in locs:
          l = locs[l_id]
          seq+=1<commit_msg>less blury providor guidelines<commit_after># }!/usr/bin/python

# adds episodes from an external source, like a json file or url.

"""
fields:
id - uniquie ID of item (used to update item if details change)
name - title of talk
room - "room1" if there is only one room.
start - datetime in some parsable format 
duration in minutes, or HH:MM:SS 
authors - comma seperated list of people's names.
contact - email(s) of presenters.
released - permission to release.
license - CC license (13 is safe)
description - used as the description of the video (paragraphs are fine)
conf_key - PK in source database - should be uniquie across this file
conf_url - URL of talk page
tags - comma seperated list 
"""

"""
NOTE: In general it is better to build the export as simple as posible, 
even at the expense of deviatng from the above fields.  Exporting extra
fields is just fine.  They will be ignored, or maybe I will use them in 
a future version.
For instance, if you store start and end in the database but not duration, 
export end, I can can calculate duration.  
Given historic problems with duration, I woujldn't mind seeing both end 
and duration so that I can verify the transformations.  
"""

"""
datetime and json:
There is a issue here because json doesn't define a date format.  Do whatever makes the server side code smallest and easiest to code. easy to read data is good too.  

Here is PyCon 2010's impemtation:
datetime objects are represented as a time tuple of six elements:
    (year, month, day, hour, min, sec) 
        "start":      [2010, 2, 19, 9, 30, 0],
        "duration":   30, # in min
http://us.pycon.org/2010/conference/schedule/json/
Easy to code, kinda hard to read.
I parse it with 
          start = datetime.datetime(*row['start'])
good.

This is also good:
    json: Start: "2011-06-09 19:00:00"
    parser:  datetime.datetime.strptime( x, '%Y-%m-%d %H:%M:%S' )

OSDC2010: easy to read, harder to parse/assemble into start durration.
http://2010.osdc.com.au/program/json
# Day: "Tue 23 Nov"
# Time: "09:00 - 17:00"
but if that is how it is stored on the server, don't try to transform it.

Again, keep the server side code simple.
I can fix my consumer easier than I can get someone else's website updated.
"""

# FireFox plugin to view .json data:
# https://addons.mozilla.org/en-US/firefox/addon/10869/

import datetime 
import urllib2,json
# from csv import DictReader
# from datetime import timedelta
from dateutil.parser import parse

import gdata.calendar.client
import gdata.calendar.service

import pw # for google calandar

import process
import lxml.etree


from main.models import fnify, Client, Show, Location, Episode

class add_eps(process.process):

    def addlocs(self, schedule, show):
      """ pycon 2010 
      seq=0
      locs=d['rooms']
      for l_id in locs:
          l = locs[l_id]
          seq+=1
{'fraction_non_alphanumeric': 0.07959479015918958, 'fraction_numerical': 0.02206946454413893, 'mean_word_length': 3.4733009708737863, 'pattern_counts': {'":': 4, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2840318', 'n_tokens_mistral': 1772, 'n_tokens_neox': 1624, 'n_words': 780}