<jupyter_start><jupyter_text>Check data from hubs Testing the 29 limit proximity scansBadges - 32 badges with older versionOne badge with new version5 beacons (with old version, doesn’t matter)I started the hub, activated the badges, then marke all old version badges as active and restarted the hub (so that there is no issue with pulling data)Had some issues with other badges around. Had to restart the whole thing and change the project id to 13tests: ~14:16- - 32 badges, 5 beacons, all very close. Expecting to see at least 4 beacons in a scan~14:22  - 32 badges very close, 5 beacons further away. Expecting to see only 4~14:24 ~ - moved them even further away. Expecting to see only 4~14:27~ - Only 2 beacons very close, 3 are turned off. Expecing to see 2~14:30~ - turn off allThe badge that was used for the data itself:FA:6F:8C:0C:39:0D, E92D598WEX<jupyter_code>#import sys
#!{sys.executable} -m pip install sklearn
from __future__ import absolute_import, division, print_function

import pandas as pd
import numpy as np

import pytz
import gzip
import os

# Import the data analysis tools
import openbadge_analysis as ob
import openbadge_analysis.preprocessing
# Matplotlib for additional customization
from matplotlib import pyplot as plt
%matplotlib inline

# Seaborn for plotting and styling
import seaborn as sns
# Settings
time_zone = 'US/Eastern' # Your local timezone
hublog_time_zone = 'UTC'
log_version = '2.0'
time_bins_size = '15S' # For proximity scans

folder = '../data/raw/20181121_test_fw3_proximity/'
proximity_filename = folder+'proximity_archive.txt.gz'
audio_filename = folder+'audio_archive.txt.gz'
hublog_filename = folder+'hub.log.gz'<jupyter_output><empty_output><jupyter_text>Load data<jupyter_code>with gzip.open(proximity_filename, 'r') as f:
    m2badge = ob.preprocessing.member_to_badge_proximity(f, time_bins_size, tz=time_zone)
m2badge = m2badge.reset_index()
m2badge.head()<jupyter_output><empty_output><jupyter_text>Calc data<jupyter_code>m2badge['is_beacon'] = m2badge.apply(lambda row: 1 if row.observed_id >= 16000 else 0  ,axis=1)
m2badge.groupby('datetime')['is_beacon'].sum().plot()<jupyter_output><empty_output><jupyter_text>plot number of beacons and RSSIs<jupyter_code>#rssi_means = m2badge.groupby(['datetime','is_beacon'])[['rssi']].mean()
#rssi_means
#ax = sns.lineplot(rssi_means)
ax = sns.lineplot(x="datetime", y="rssi",
    hue="is_beacon", style="is_beacon", data=m2badge)<jupyter_output>/home/orenled/.virtualenvs/openbadge-analysis-test-badge/local/lib/python2.7/site-packages/scipy/stats/stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval
{'fraction_non_alphanumeric': 0.09896907216494845, 'fraction_numerical': 0.029553264604810996, 'mean_word_length': 6.031400966183575, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '491666', 'license': 'MIT', 'n_tokens_mistral': 1056, 'n_tokens_neox': 991, 'n_words': 299, 'path': 'notebooks/20181121_test_fw3_proximity.ipynb', 'repo_name': 'OrenLederman/openbadge-analysis-test-badge', 'size': '2910'}