<jupyter_start><jupyter_text>The script assumes that the training/test data consists of a list of tables. Each table corresponds to a week. The rows of table show the days in the week, and the columns are the periods in the day.The paper assumes 30-minute periods, whihch leads to 48 periods per day. Here I create a toy dataset with 2 weeks and 3 periods per day:<jupyter_code>d1 = matrix(c(1:(3*7)*3), nrow=7, ncol=3)
d2 = matrix(c(1:(3*7)*5), nrow=7, ncol=3)
d = list(d1, d2)<jupyter_output><empty_output><jupyter_text>`table_pr` gives labels to the columns and rows of a table. The rows are named by the days of week (*Monday*, ..., *Sunday*), and the columns are named *Period1*, ... *Periodx*.<jupyter_code>table_pr = function(m) {  
  rownames(m) <- weekdays(as.Date(4,"1970-01-01",tz="GMT")+0:6)
  colnames(m)<-colnames(m, do.NULL = FALSE, prefix = "Period")
  m=as.table(m)
  return(m)
}
table_pr(d[[1]])<jupyter_output><empty_output><jupyter_text>The function `calculsum` computes a sum over multiple weeks. It takes the day as argument, and iterates over the periods in the day.<jupyter_code>calculsum = function(z,n,liste) {                   
  # I changed the number of periods from 48 to 3  
  sapply(1:3,function(y) sum(sapply(1:n, function(x)liste[[x]][z,y])))
}<jupyter_output><empty_output><jupyter_text>By applying `calculsum` across the days, we will get a single matrix that sums the matrices in training data:<jupyter_code>sum_past = lapply(1:7, function(x) calculsum(x, 2, d))
sum_past<jupyter_output><empty_output><jupyter_text>Do type conversion and put labels using `table_pr`:<jupyter_code># I changed the number of periods from 48 to 3
sum_past <- table_pr(matrix(unlist(sum_past), ncol = 3, byrow = TRUE))
sum_past<jupyter_output><empty_output><jupyter_text>To compute the average, create a matrix populated with number of weeks. The divide the sum matrix by this one:<jupyter_code># 48 to 3
nbr_week=table_pr(matrix(2, ncol = 3, nrow = 7))
p_1=round(sum_past/nbr_week)
p_1<jupyter_output><empty_output><jupyter_text>Let's create a toy dataset:<jupyter_code>t1 = matrix(round(c(1:(3*7)*1.5)), nrow=7, ncol=3)
t2 = matrix(round(c(1:(3*7)*2.5)), nrow=7, ncol=3)
t3 = matrix(round(c(1:(3*7)*3.5)), nrow=7, ncol=3)
t = list(t1, t2, t3)<jupyter_output><empty_output><jupyter_text>Now the script does something weird. For each period in test, it replace that period with a period in the train, and recomputes the sum.<jupyter_code># I did rename things a bit
n1 = 3
sum_past_list=lapply(1:(n1-1),function(x) sum_past-d[[x]]+t[[x]])
sum_past_list<jupyter_output><empty_output><jupyter_text>Then the poisson list is computed. It is a modification of the averages that we had already computed, by adding replacing the past week with a week in the sum and recomputing the average.<jupyter_code>poisson_list=lapply(1:n1,function(x) if(x==1){p_1}else{round(sum_past_list[[x-1]]/nbr_week)})
poisson_list<jupyter_output><empty_output><jupyter_text>Finally, the poisson rates themselves are used as predictions:<jupyter_code>pred_p=unlist(lapply(1:n1, function(x) as.vector(t(poisson_list[[x]]))))
pred_p<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12988250238170848, 'fraction_numerical': 0.03016830739917434, 'mean_word_length': 6.429245283018868, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '591611', 'license': 'Unlicense', 'n_tokens_mistral': 1226, 'n_tokens_neox': 1165, 'n_words': 326, 'path': 'R_scripts/Poisson.ipynb', 'repo_name': 'bhrzslm/notebooks', 'size': '3149'}