split a column into N groups by value differences (timestamp)

Question: Sample data in .csv format
| No.|   IP     |      Unix_time     |    # integer unix time
| 1  | 220.127.116.11  |     1563552000     |    # equivalent to 12:00:00 AM
| 2  | 188.8.131.52  |     1563552030     |    # equivalent to 12:00:30 AM
| 3  | 184.108.40.206  |     1563552100     |    # equivalent to 12:01:40 AM
| 4  | 18.104.22.168  |     1563552110     |    # equivalent to 12:01:50 AM
| 5  | 184.108.40.206  |     1563552180     |    # equivalent to 12:03:00 AM
| 6  | 184.108.40.206 |     1563552120     |

Here's the current working code using pandas groupby( ) and get_group( ) functions:
data = pd.read_csv(some_path, header=0)
root = data.groupby('IP')

for a in root.groups.keys():
    t = root.get_group(a)['Unix_time']
    print(a + 'has' + t.count() + 'record')

You will see the results below:
184.108.40.206 has 5 record
126.96.36.199 has 1 record

Now, I want some improvement based on above code.
For the same IP value (e.g., 184.108.40.206), I want to make further sub-groups based on a maximum time interval (e.g., 60 seconds), and count the number of elements in each sub-group. For example, in above sample data:
Start from row 1: row 2 Unix_time value is within 60 seconds, but row 3 is beyond 60 seconds.
Thus, row 1-2 is a group, row 3-4 is a separate group, row 5 is a separate group. In other words, group '18.104.22.168' has 3 sub-groups now. The result should be:
188.8.131.52 start time 1563552000 has 2 record within 60 secs
220.127.116.11 start time 1563552100 has 2 record within 60 secs
184.108.40.206 start time 1563552150 has 1 record within 60 secs
22.214.171.124 start time 1563552120 has 1 record within 60 secs

How to make it?
Comment: It seems to me like row 5 should be grouped with rows 3 and 4 according to the rules you are describing since rows 3, 4, and 5 all occur within the 100 second window starting at 1563552100. What have I misunderstood?
Comment: @moormanly: yes, i make a mistake. i make revision.
Answer: You can use pd.Grouper:
df['datetime'] = pd.to_datetime(df['Unix_time'], unit='s')
for n, g in df.groupby(['IP', pd.Grouper(freq='60s', key='datetime')]):
    print(f'{n[0]} start time {g.iloc[0, g.columns.get_loc("Unix_time")]} has {len(g)} records within 60 secs')

Output:
22.214.171.124  start time 1563552000 has 2 records within 60 secs
184.108.40.206  start time 1563552100 has 2 records within 60 secs
22.214.171.124  start time 1563552150 has 1 records within 60 secs
184.108.40.206 start time 1563552120 has 1 records within 60 secs

Using "root" and integers:
root = df.groupby(['IP',df['Unix_time']//60])

for n, g in root:
     print(f'{n[0]} start time {g.iloc[0, g.columns.get_loc("Unix_time")]} has {len(g)} records within 60 secs')

Output:
126.96.36.199  start time 1563552000 has 2 records within 60 secs
188.8.131.52  start time 1563552100 has 2 records within 60 secs
220.127.116.11  start time 1563552150 has 1 records within 60 secs
126.96.36.199 start time 1563552120 has 1 records within 60 secs

Comment: in my case, df is root.get_group(a)?
Comment: is it possible not use to_datetime? if i just want to read them as integers
Comment: it works for my sample data only. But if the time are e.g., 1563557387 and 1563557418. they are within 60 seconds, but their // 60 values (26059289 & 26059290) are different, so don not fall in the same group
Comment: Yeah it is better to convert to datetime
Comment: i tested the datetime method, it always starts to count from :00 second of the minute, instead of the real start time of each group.


How to properly use JdbcTemplate.update with KeyHolder?

Question: I have a method which save the user in DB and return auto generated id:
public int save(User user) {
    KeyHolder holder = new GeneratedKeyHolder();
    jdbcTemplate.update(SAVE_USER, user.getParams(), holder);
    return holder.getKey().intValue();
}

public Object[] getParams() {
        return new Object[]{
                email, // String email
                password}; // String password
}

SAVE_USER = INSERT INTO user (email, password) VALUES (?, ?)
When I trying to execute a method, I'm getting an exception:
Request processing failed; nested exception is org.springframework.dao.TransientDataAccessResourceException: PreparedStatementCallback;
SQL [INSERT INTO user (email, password) VALUES (?, ?)]; Invalid argument value: java.io.NotSerializableException; nested exception is java.sql.SQLException: Invalid argument value: java.io.NotSerializableException

Comment: Does this help? http://stackoverflow.com/a/15436696/1325237
Answer: Looks like you're using Spring. Depending on the version, the JdbcTemplate interface varies a bit. Still, in no version I know of is there a method signature that matches what I think your intended call is:
update(String sql, Object[] args, KeyHolder keyHolder)

As of Spring 3, there's only:
update(PreparedStatementCreator psc)
update(PreparedStatementCreator psc, KeyHolder generatedKeyHolder)
update(PreparedStatementCreator psc, PreparedStatementSetter pss)
update(java.lang.String sql)
update(java.lang.String sql, java.lang.Object[] args, int[] argTypes)
update(String sql, PreparedStatementSetter pss)

/* and this is the one I think you're matching */
update(java.lang.String sql, java.lang.Object... args)

Unfortunately, even though your call matches that last method's signature, it doesn't do what I think you're looking for. It tries to apply each argument as an individual argument to the SQL statement. Your KeyHolder is out of place in that circumstance, and the params array would have to be split out as individual parameters. Basically, that method won't work for what you're doing.
If you want to use the KeyHolder, I think you need to change your approach to use this method:
update(PreparedStatementCreator psc, KeyHolder generatedKeyHolder)


Continuity of Fourier series with decay condition on coefficients

Question: Let $\theta \in \ell^2(\mathbb{N})$ satisfy
$$
\sum_{j=1}^n \theta_j^2 a_j^2 \leq Q
$$
for some $Q > 0$ where
$$
a_j = \begin{cases}
j^\beta & \text{$j$ even}\\
(j-1)^\beta & \text{$j$ odd}
\end{cases}
$$
for some $\beta > 1/2$. Let $\{\varphi_j\}_{j=1}^\infty$ denote the trigonometric basis of $[0,1]$, i.e. $\varphi_1(x) = 1$, $\varphi_{2k}(x) = \sqrt{2} \cos(2\pi k x)$ and $\varphi_{2k+1}(x) = \sqrt{2} \sin(2\pi k x)$ for $k \in \mathbb{N}$. Show that $f = \sum_{j=1}^\infty \theta_j \varphi_j$ is continuous.
I know that $\varphi_j$ is Lipschitz with constant $\pi\lfloor j+1 \rfloor$ but I don't think that helps very much. Any ideas?
Answer: Note first that the hypothesis is just a silly way of writing $$\sum_{j=1}^\infty\theta_j^2j^{2\beta}<\infty.$$
Proof by Looking it Up
The hypothesis says precisely that $f$ is in the Sobolev space often denoted $H^\beta$; since $\beta>1/2$ the Sobolev Embedding Theorem shows that $f$ is continuous.
Actual Proof
$$\sum|\theta_j|=\sum|\theta_j|j^\beta j^{-\beta}\le\left(\sum\theta_j^2j^{2\beta}\right)^{1/2}\left(\sum j^{-2\beta}\right)^{1/2}<\infty,$$since $2\beta>1$. So the series converges uniformly, hence $f$ is continuous.
Proof of a Stronger Result
Say $x<y$ and $y-x$ is small. Choose an integer $N$ with $1/2\le N(y-x)\le 2$. Now $$f(y)-f(x)=\left(\sum_{j=1}^N+\sum_{j=N+1}^\infty\right)\theta_j(\phi_j(y)-\phi_j(x)):=I+II.$$Now the Lipschitz constant for $\phi_j$ shows that $$I\le c(y-x)\sum_{j=1}^N\theta_jj^\beta j^{1-\beta}\le c(y-x)\left(\sum_{j=1}^N\theta_j^2j^{2\beta}\right)^{1/2}\left(\sum_{j=1}^Nj^{2-2\beta}\right)^{1/2}\le c(y-x)N^{\frac32-\beta}\le c(y-x)^{\beta-\frac12}.$$To estimate $II$ you just use the fact that $|\phi_j|\le1$: $$II\le 2\sum_{j=N+1}^\infty\theta_j j^\beta j^{-\beta}\le c\left(\sum_{j=N+1}^\infty j^{-2\beta}\right)^{1/2}\le cN^{\frac12-\beta}\le c(y-x)^{\beta-\frac12}.$$Since $\beta-\frac12>0$ this shows that $f\in Lip_{\beta-\frac12}$.
Note
It's worthwhile spending some time understanding this argument, because the outline "use a trivial uniform estimate for the high-order terms and the Lipschitz condition on the low-order terms" comes up a lot in this sort of thing.


Ending company IT Admin relationship

Question: We have been using a company to administer our (small office) IT infrastructure.  We don't have complete records of what has been done or hasn't been done and don't know what we need to ask for in order to pick this up ourselves.  Is there a good "checklist of things to make sure you get" for people in this situation?  (Windows OS product keys, installation media(?), domain controller admin password, etc.)
Comment: You might consider having the old IT company overlap the new IT company for a month, if they will do it.
Comment: This actually happened a year ago :-) Asking for future reference and figured it would be good to have this answer in the forum!
Answer:
All passwords (for all devices, applications and accounts).
All records relating to software, licensing and media (including purchase order history/proof).
All media (including installation media and live data backups).
All documentation, including,

Server, hardware, network (including IP addresses) and operating system configurations.
Details of processes and procedures (e.g. adding users, create new mailboxes, etc.)
Information on any automated / manually triggered jobs (backups, housekeeping, etc.)
Review the documentation beforehand and have them make improvements if you have questions or find missing/old information.

Details of any third party contracts they might have taken out for support that you may need to take on / take out yourself (e.g. hardware maintenance).
Any physical / VPN / secure access items (badges, keys, tokens, fobs, etc.)
Information about telecom accounts
Logins to any websites you might need. (Download software, open support cases.. )
Account information related to domain name registrations, details of registrars used, etc.
Copies of any security certificates, and the relevant key phrases.  Ensure the old supplier also destroys their copies.

And once this is all done - change all the passwords.
Comment: Badge, keys, tokens, fobs for VPN/NOC/Building access.
Comment: I've also seen a lot of companies install a customer remote desktop/support software on the company computers that you may want to have them uninstall. Sometimes these are customized VNC programs that were scripted on and can be clunky to manually remove.
Comment: Shamelessly copying bullet points from the other answers to this one, since this seems to be the favored answer so far. (also, IMHO, the other "in addition" answers should have probably been edits to this anyway)
Comment: No, I was not directing that at you, I was commenting on my own action of shamelessly adding. :)
Comment: @ErikA my apologies for being over sensitive.
Comment: You could probably add: 11) Document all scheduled automated and non-automated procedures such as backups and 12) document all ad-hoc procedures, such as creating new users, opening mailboxes ETC
Comment: Can I ask for an edit to the answer to list out devices to ask for passwords for?  (e.g. "Domain controller, managed switch, ...") Great list!
Comment: @ErikA it's a collaborative answer site, feel free to edit the answer and make it better.  I don't recall shamelessly copying bullet points from other answers, but I did incorporate information from some of the comments to my answer (and I upvoted the relevant comment).
Comment: @albiglan I don't think you should restrict the question for passwords to specific devices, I think you should ask the IT company all passwords.  I actually think you should have a solid contract in place before you engage the company so that this doesn't come up at the end - but we're past that point.
Comment: @EightBitTony - yes, I know. That's why I added those bullets to your answer. I've been around the block a few times here. :)
Comment: I know you have which is why your claim of me shamelessly adding bullet points seems a little hollow.  I've tidied the list up, merging duplicates.  I don't understand #7 and #9 if someone else wants to clarify them.
Comment: @EightBitTony  Was more thinking it would be helpful to have the list o stuff for people.  Some people might not know/remember switches, WAPs, PBX systems, etc. all have PW's  also ErikA... Really love the collaboration on the answer!
Answer: In addition:
Account information related to domain registrations.
Comment: +1 - this is a good one. I've just been through the situation of wrangling ownership of a domain back from a rogue IT person who registered it on behalf of a company.
Answer: In addition to those that you mentioned...

static IPs of all devices/servers (so you know what devices you need to manage)
information about telecom accounts
all other passwords for all devices and services (the domain admin account isn't the only one)
Answer: in addition:
Logins to any websites you might need. (Download software, open support cases.. )
Answer: All of the answers so far have been awesome.
Ensure that you have purchase records for Licenses.  Some that you may not think of include Small Business Server (SBS) CALs, SQL Server Licenses, Terminal Services Licenses.  Also if your servers have Out-Of-Band Management (HP's iLO, Dell's DRAC, etc), ensure that you are provided with those license keys.
Ensure that you have any SSL Certs for websites, email, logins, etc.
Be prepared to do a password reset fire-drill.  I would audit services and scheduled tasks that don't run as a builtin account (such as Local System, Network Service accounts).  (If I remember, I'll dig up the audit tools that I've written at a previous gig.)
Also, it behooves you to audit the licenses.  Ensure that they are registered to your company, not the IT Service Provider.  Also important is ensuring that they are appropriate licenses for the use; I am validating that MSDN SQL licenses are not being used in Production environments.  Action Packs, Microsoft Subscriptions, Gold Reseller memberships and such give great benefits such as free licenses, but they are restricted as to where and how they can be used.

By way of enabling you to behoovulence, here's some help:
Here is a script that I've had sitting around that I've released that will report services running on machines that are not running under a builtin service account.
GitHub:gWaldo:ListServices
Comment: Upvoted for usage of "behooves" ;-)
Comment: Nice addition!  Like the license audit one especially...
Comment: Many IT service providers will shamelessly register *everything* that belongs to their clients to themselves instead. Definitely something to watch out for.
Comment: Not just Certificates, but the Private Keys corresponding to them, too.
Answer: PIN/Password for phone accounts if you have a corporate account with a cellular carrier.
If you have remote management tools for tablets/phones, software and passwords to the management console.

Ending company IT Admin relationship


HDFS NFS locations using weird numerical username values for directory permissions

Question: Seeing nonsense values for user names in folder permissions for NFS mounted HDFS locations, while the HDFS locations themselves (using Hortonworks HDP 3.1) appear fine. Eg.
➜  ~ ls -lh /nfs_mount_root/user
total 6.5K
drwx------. 3 accumulo  hdfs    96 Jul 19 13:53 accumulo
drwxr-xr-x. 3  92668751 hadoop  96 Jul 25 15:17 admin
drwxrwx---. 3 ambari-qa hdfs    96 Jul 19 13:54 ambari-qa
drwxr-xr-x. 3 druid     hadoop  96 Jul 19 13:53 druid
drwxr-xr-x. 2 hbase     hdfs    64 Jul 19 13:50 hbase
drwx------. 5 hdfs      hdfs   160 Aug 26 10:41 hdfs
drwxr-xr-x. 4 hive      hdfs   128 Aug 26 10:24 hive
drwxr-xr-x. 5 h_etl   hdfs   160 Aug  9 14:54 h_etl
drwxr-xr-x. 3    108146 hdfs    96 Aug  1 15:43 ml1
drwxrwxr-x. 3 oozie     hdfs    96 Jul 19 13:56 oozie
drwxr-xr-x. 3 882121447 hdfs    96 Aug  5 10:56 q_etl
drwxrwxr-x. 2 spark     hdfs    64 Jul 19 13:57 spark
drwxr-xr-x. 6 zeppelin  hdfs   192 Aug 23 15:45 zeppelin
➜  ~ hadoop fs -ls /user
Found 13 items
drwx------   - accumulo   hdfs            0 2019-07-19 13:53 /user/accumulo
drwxr-xr-x   - admin      hadoop          0 2019-07-25 15:17 /user/admin
drwxrwx---   - ambari-qa  hdfs            0 2019-07-19 13:54 /user/ambari-qa
drwxr-xr-x   - druid      hadoop          0 2019-07-19 13:53 /user/druid
drwxr-xr-x   - hbase      hdfs            0 2019-07-19 13:50 /user/hbase
drwx------   - hdfs       hdfs            0 2019-08-26 10:41 /user/hdfs
drwxr-xr-x   - hive       hdfs            0 2019-08-26 10:24 /user/hive
drwxr-xr-x   - h_etl    hdfs            0 2019-08-09 14:54 /user/h_etl
drwxr-xr-x   - ml1        hdfs            0 2019-08-01 15:43 /user/ml1
drwxrwxr-x   - oozie      hdfs            0 2019-07-19 13:56 /user/oozie
drwxr-xr-x   - q_etl hdfs            0 2019-08-05 10:56 /user/q_etl
drwxrwxr-x   - spark      hdfs            0 2019-07-19 13:57 /user/spark
drwxr-xr-x   - zeppelin   hdfs            0 2019-08-23 15:45 /user/zeppelin

Notice the difference for users ml1 and q_etl that they have numerical user values when running ls on the NFS locations, rather then their user names.
Even doing something like...
[hdfs@HW04 ml1]$ hadoop fs -chown ml1 /user/ml1

does not change the NFS permissions. Even more annoying, when trying to change the NFS mount permissions as root, we see
[root@HW04 ml1]# chown ml1 /nfs_mount_root/user/ml1
chown: changing ownership of '/nfs_mount_root/user/ml1': Permission denied

This causes real problems, since the differing uid means that I can't access these dirs even as the "correct" user to write to them. Not sure what to make of this. Anyone with more Hadoop experience have any debugging suggestions or fixes?

UPDATE:
Doing a bit more testing / debugging, found that the rules appear to be...

If the NFS server node has no uid (or gid?) that matches the uid of the user on the node accessing the NFS mount, we get the weird uid values as seen here.
If there is a uid associated to the username of the user on the requesting node, then that is the uid user that we see assigned to the location when accessing via NFS (even if that uid on the NFS server node is not actually for the requesting user), eg.

[root@HW01 ~]# clush -ab id ml1
---------------
HW[01,04] (2)
---------------
uid=1025(ml1) gid=1025(ml1) groups=1025(ml1)
---------------
HW[02-03] (2)
---------------
uid=1027(ml1) gid=1027(ml1) groups=1027(ml1)
---------------
HW05
---------------
uid=1026(ml1) gid=1026(ml1) groups=1026(ml1)
[root@HW01 ~]# exit
logout
Connection to hw01 closed.
➜  ~ ls -lh /hdpnfs/user
total 6.5K
...
drwxr-xr-x. 6 atlas     hdfs   192 Aug 27 12:04 ml1
...
➜  ~ hadoop fs -ls /user
Found 13 items
...
drwxr-xr-x   - ml1        hdfs            0 2019-08-27 12:04 /user/ml1
...
[root@HW01 ~]# clush -ab id atlas
---------------
HW[01,04] (2)
---------------
uid=1027(atlas) gid=1005(hadoop) groups=1005(hadoop)
---------------
HW[02-03] (2)
---------------
uid=1024(atlas) gid=1005(hadoop) groups=1005(hadoop)
---------------
HW05
---------------
uid=1005(atlas) gid=1006(hadoop) groups=1006(hadoop)

If wondering why I have, user on the cluster that have varying uids across the cluster nodes, see the problem posted here: How to properly change uid for HDP / ambari-created user? (note that these odd uid setting for hadoop service users was set up by Ambari by default).
Answer: After talking with someone more knowledgeable in HDP hadoop, found that the problem is that when Ambari was setup and run to initially install the hadoop cluster, there may have been other preexisting users on the designated cluster nodes.
Ambari creates its various service users by giving them the next available UID of a nodes available block of user UIDs. However, prior to installing Ambari and HDP on the nodes, I created some users on the to-be namenode (and others) in order to do some initial maintenance checks and tests. I should have just done this as root. Adding these extra users offset the UID counter on those nodes and so as Ambari created users on the nodes and incremented the UIDs, it was starting from different starting counter values. Thus, the UIDs did not sync and caused problems with HDFS NFS.
To fix this, I...

Used Ambari to stop all running HDP services
Go to Service Accounts in Ambari and copy all of the expected service users name strings
For each user, run something like id <service username> to get the group(s) for each user. For service groups (which may have multiple members), can do something like grep 'group-name-here' /etc/group. I recommend doing it this way as the Ambari docs of default users and groups does not have some of the info that you can get here.
Use userdel and groupdel to remove all the Ambari service users and groups
Then recreate all the groups across the cluster
Then recreate all the users across the cluster (may need to specify UID if nodes have other users not on others)
Restart the HDP services (hopefully everything should still run as if nothing happend, since HDP should be looking for the literal string (not the UIDs))

For the last parts, can use something like clustershell, eg.
# remove user
$ clush -ab userdel <service username>
# check that the UID you want to use is actually available on all nodes
$ clush -ab id <some specific UID you want to use>
# assign that UID to a new service user
$ clush -ab useradd --uid <the specific UID> --gid <groupname> <service username>

To get the lowest common available UID from each node, used...
# for UID
getent passwd | awk -F: '($3>1000) && ($3<10000) && ($3>maxuid) { maxuid=$3; } END { print maxuid+1; }'
# for GID
getent passwd | awk -F: '($4>1000) && ($4<10000) && ($4>maxuid) { maxuid=$4; } END { print maxuid+1; }'

Ambari also creates some /home dirs for users. Once you are done recreating the users, will need to change the permissions for the dirs (can also use something like clush there as well).
* Note that this was a huge pain and you would need to manually correct the UIDs of users whenever you added another cluster node. I did this for a test cluster, but for production (or even a larger test) you should just useKerberos or SSSD + Active Directory.


chaging width of one column css

Question: Here's my css properties for a table. I want to fix the width of a certain column and hide the overflow. I tried on all the columns but i can't seem to make it work.
.celltable2 table {
table-layout:fixed;
font-family: Helvetica, Arial, sans-serif;
width: 640px;
border-collapse:
collapse; border-spacing: 0;
}

table.celltable2 td, th {

border: 1px solid transparent;
width: 30px;
height: 30px;
transition: all 0.3s;
overflow: hidden;
}

table.celltable2 td {

text-align:center;
    vertical-align:middle;
}

Is there something i might be missing?. I tried defining colgroup and colspan but no success.
Comment: please add your html aslo.
Comment: Your CSS would indicate you're targeting a *table* and *not* a column within a table. But.. without HTML it's impossible to be certain.
Answer: The problem is with your this css
table.celltable2 td, th {
  border: 1px solid transparent;
  width: 30px;
  height: 30px;
  transition: all 0.3s;
  overflow: hidden;
}

In this you have targeted the width of  but not the  as after , you need to define like this table.celltable2 th instead of just th.
This is correct
table.celltable2 td, table.celltable2 th {
  border: 1px solid transparent;
  width: 30px;
  height: 30px;
  transition: all 0.3s;
  overflow: hidden;
}

chaging width of one column css


How can I generate an array of random numbers that fluctuate by 1 in javascript?

Question: I want to be able to generate the following array (or something like it) in javascript:
[
  52, // random number 0-100
  53, // random +1 or -1
  54, // random +1 or -1
  53, // random +1 or -1
  52, // random +1 or -1
  53, // random +1 or -1
  52, // random +1 or -1
  51, // random +1 or -1
  50, // random +1 or -1
  51, // random +1 or -1
  // etc., etc., etc.
]

How can I do that?
I've tried this, but I always get a random number followed by 1's and -1's only:
Array(50).fill(0).map((v, i, a) => i !== 0 ? (Math.round(Math.random()) ? a[i-1] + 1 : a[i-1] - 1) : Math.floor(Math.random() * 101))

Comment: Is that ± 1 from the previous value? Or ± 1 from an initial, randomly generated value?
Comment: The question is not clear. What exactly are you tring to do?
Comment: @ItoPizarro It's from the previous value, however the `array[0]` is randomly generated
Comment: @odedbartov I'm trying to generate an array, starting with a random number from 0 - 100, for example 50, followed by either 50 + 1 (51) or 50 - 1 (49) (chosen randomly). For the next one, if 49 is chosen, it is followed by either 48 (49 - 1) or 50 (49 + 1), chosen randomly.
Comment: i posted an answer, check it out
Answer: This might help:

function randomGenerator(size) {
    let result = [];
    let firstValue = Math.round(Math.random() * 100);
    result.push(firstValue);

    for (let i=0;i<size-1;i++){
        firstValue += Math.random() > 0.5 ? 1:-1;
        result.push(firstValue)
    }
    return result;
}

console.log(randomGenerator(10));
console.log(randomGenerator(13));

or if you prefer to go functional you can take advantage of default values
and comma operator:

const randGenerator = (size = 13,init = Math.round(Math.random() *100)) =>
Array(size).fill(0).map(e => (init += Math.random() > 0.5 ? 1:-1,init))

console.log(randGenerator(10))
console.log(randGenerator(13))
Answer: Simple solution:

function rand(min, max) {
  return min + Math.floor((max + 1 - min) * Math.random())
}

function generate(size) {
  let last = rand(0, 100)
  const array = []
  for (let i = 0; i < size; i++) {
    array.push(last)
    last += rand(0, 1) ? 1 : - 1
  }
  return array
}

console.dir(generate(50))
Answer: Here is how you can do it with a recursive function, you can set the initial min and max for the random number and also specify the final array length:

const arr = [];

function getRandomInt(min, max) {
  min = Math.ceil(min);
  max = Math.floor(max);
  return Math.floor(Math.random() * (max - min + 1) + min);
}

function updateArr(min, max, arrLength, arr) {
  if (arrLength) {
    let resultNum = getRandomInt(min, max);
    if (resultNum !== arr[arr.length - 1]) {
      arr.push(resultNum);
      updateArr(resultNum - 1, resultNum + 1, arrLength - 1, arr);
    } else updateArr(min, max, arrLength, arr);
  }
}

updateArr(0, 100, 10, arr);

console.log(arr);
Answer: arr = [Math.random()*100]
for (let i = 0; i < 49; i++){
arr.push(arr[i] + (Math.random() > 0.5? 1 : -1))
}

Is this working for you?


How to create a langchain doc from an str?

Question: I've searched all over langchain documentation on their official website but I didn't find how to create a langchain doc from a str variable in python so I searched in their GitHub code and I found this :
  doc=Document(
                page_content="text",
                metadata={"source": "local"}
            )

PS: I added the metadata attribute
then I tried using that doc with my chain:
Memory and Chain:
memory = ConversationBufferMemory(memory_key="chat_history", input_key="human_input")
chain = load_qa_chain(
    llm, chain_type="stuff", memory=memory, prompt=prompt
)

the call method:
  chain({"input_documents": doc, "human_input": query})

prompt template:
template = """You are a senior financial analyst analyzing the below document and having a conversation with a human.
{context}
{chat_history}
Human: {human_input}
senior financial analyst:"""

prompt = PromptTemplate(
    input_variables=["chat_history", "human_input", "context"], template=template
)

but I am  getting the following error:
AttributeError: 'tuple' object has no attribute 'page_content'

when I tried to check the type and the page content of the Document object before using it with the chain I got this
print(type(doc))
<class 'langchain.schema.Document'>
print(doc.page_content)
"text"

Answer: This worked for me:
from langchain.docstore.document import Document

doc =  Document(page_content="text", metadata={"source": "local"})

Comment: It seems that importing from `langchain.schema.document` also works, and that latter module is the one specified in the [API documentation](https://api.python.langchain.com/en/latest/schema/langchain.schema.document.Document.html#langchain.schema.document.Document).
Answer: First, some context. From what I've learned so far, a Document is a list of Document objects. If you run type(doc[0]) you get langchain.schema.document.Document. This Document object is a dictionary made of two keys: one is page_content: which accepts string values, and the second key is metadata: which only accepts dictionaries. {page_content: str, metadata: dict}. By default (don't quote me on this: it's been lots of trial and error and, as you mentioned, there is no documentation), an "empty" Document contains the two mentioned keys, and a single dictionary in its metadata: with one key: {source:} that only accepts strings. You can create a multiple "page" Document object by creating a list of Document objects like so:
First, you must have a list of string texts: text_list below, and a list of dictionaries for the metadata: text_list below. You must ensure both lists are the same length.
from langchain.docstore.document import Document

document =  []

for item in range(len(text_string)):
    page = Document(page_content=doc_text_splits[item],
    metadata = metadata_string[item])
    doc.append(page)

Additionally, you can also create Document object using any splitter from LangChain:
from langchain.text_splitter import CharacterTextSplitter

doc_creator = CharacterTextSplitter(parameters)

document = doc_creator.create_documents(texts = text_list, metadatas = metadata_list)

Comment: Useful info above regarding the text splitter, thanks.  In addition to the create_documents method, TextSplitter.split_documents() is also useful, as it creates 'bite size' documents that can be ingested by the LLM.
Answer: I had a similar issue and I noticed the API calls for a list, so try
    doc = Document(page_content=input,
            metatdata={
                "source": "userinput"
            }
        )
    #db.add_documents(doc)
    db.add_documents([doc])

Answer: Langchain's API appears to undergo frequent changes. While @Rahul Sangamker's solution remains functional as of v0.1.11, it may encounter compatibility issues due to the recent restructuring – splitting langchain into langchain-core, langchain-community, and langchain-text-splitters (as detailed in this article). However, it's worth noting that these changes are backwards compatible.
Here's an updated solution, reflective of the v0.1 style, now importing from langchain_core.documents.
from langchain_core.documents import Document

doc = Document(page_content="text", metadata={"source": "local"})

Answer: this is the best that I could come with
def str_to_doc(text,name):
   folder_name = 'docs'
   if not os.path.exists(folder_name):
       os.makedirs(folder_name)
   file_name = name+'.txt'
   path = os.path.join(folder_name, file_name)
   with open(path, "w") as file:
        file.write(text)
   loader = TextLoader(path)
   return loader.load()

Comment: Thank you for contributing to the Stack Overflow community. This may be a correct answer, but it'd be really useful to provide additional explanation of your code so developers can understand your reasoning. This is especially useful for new developers who aren't as familiar with the syntax or struggling to understand the concepts. **Would you kindly [edit] your answer to include additional details for the benefit of the community?**


Show that f is differentiable at every point in R, and find the derivative f'

Question: I have been stuck in this problem for a while, its practice question for my exam in my real analysis calculus class. Any help would be great! Thank you!
Define $f: \Bbb R \to \Bbb R$ by
$$f(x) =   \left\{\begin{array}{l}x^2\ \text{if}\ x \ge 0\\-x^2\ \text{if}\ x < 0\end{array}\right. $$
(a) Show that $f$ is differentiable at every point in $\Bbb R$, and find the derivative $f'$.
(b) Show that $f'$ is not differentiable at $0$. Hence there are differentiable functions
that are not twice differentiable.
Comment: and the derivative $f'$...?
Comment: sorry i just edited it, find the derivative
Answer: How about this? It helps to see where the problem points may be.
 Do you know that every function of the type $x^n$ is differentiable? If so, $x^2$ and $-x^2$ are differentiable everywhere, but there may be a problem at $x=0$ , because of the definition of $f$. For (b), calculate $f''(x)$
Answer: For $x\ge 0$, $f'(x)=\frac{\text dx^2}{\text dx}=2x$, while if $x<0$, $f'(x)=\frac{\text d(-x^2)}{\text dx}=-2x$, so $$ f'(x)=\left\{\begin{array}{l}2x\ \forall x\ge0\\-2x\ \forall x<0\end{array}\right.=|2x| $$
It should be straightforward to show $|2x|$ is not differentiable.
Answer: Find the potential problem points, they are mostly obvious, like in your case, it is $x=0$. You have to prove that $\lim_{x \to 0^{=}}f'(x) = \lim_{x \to 0^{=}}f'(x) = f'(0)$. If that condition is fulfilled, that means that the function is differentiable in $x=0$.
After that, b) use the same condition, if $\lim_{x \to 0^{=}}f''(x) = \lim_{x \to 0^{=}}f''(x) = f''(0)$ is not fulfilled, that means that the function is not differentiable in $x=0$, thus it is not differentiable.
Hope this helps. Cheers!


Error: "warning C4005: 'SWIGTEMPLATEDISAMBIGUATOR': macro redefinition"

Question: I'm trying to compile a SWIG project, and it keeps on giving errors like this:

swig_wrap.cpp(55): warning C4005: 'SWIGTEMPLATEDISAMBIGUATOR': macro
  redefinition

And errors like this:

fatal error C1010: unexpected end of file while looking for precompiled
  header. Did you forget to add '#include "stdafx.h"' to your source?

Compiler

Visual Studio 2015+Update 3.
SWIG v3.0.12
Answer: Add the following to your .i file:
%begin %{
#include "stdafx.h"
%}

This section injects code into the generated wrapper at the top of the file and you won't need to disable pre-compiled headers.
Comment: Absolutely brilliant! I had a decent go at trying to find a solution like this, but missed finding the `begin` keyword in the docs. This is by far the most elegant answer, it works well for both Debug and Release without fiddling with Visual Studio settings. Thanks!
Answer: In Visual Studio, right click on the swig-generated .cpp wrapper file, and select "Not Using Precompiled Headers".
In my case, the wrapper file that swig auto-generated was swig_wrap.cpp.
You can leave precompiled headers on for the entire project.
For more info, see nabble.com and Fatal error C1010: unexpected end of file while looking for precompiled header. Did you forget to add '#include "stdafx.h"' to your source?

Appendix A: SWIG could be fixed to avoid this!
Another way to fix this is to manually add the line "stdafx.h" to the start of this file, but unfortunately, every time swig is run it will remove this fix! There is a way to fix this, but its not at all obvious (see the other answer).


Como faço para remover os milésimos de um valor?

Question: Por exemplo
number=10.123456

como faço para remover os milésimos de number para que fique apenas 10.12?Me ajudem por favor
Answer: Tente também string.format("%.2f",number).
Answer: Uma solução possível é criar uma função para arredondamento, utilizando a função math.floor:
function arredonda(num, numCasasDecimais)
   local mult = 10^(numCasasDecimais or 0)
   return math.floor(num * mult + 0.5) / mult
end

Após a execução:
> a=10.123456
> arredonda(a, 2)
10.12
>

A função acima é apenas um exemplo. Para implementações mais eficientes e/ou precisas, consulte a referência (em inglês): Simple Round


issue installing twilio module in python

Question: In my Mac I have installed Python 2.7.12 and it works correctly if I enter the command "python" in the terminal it promp for python commands. Also I have Idle installed and it is working correctly too.
Now I would like to install the Twilio module, I did [all the steps][1] I run "sudo easy_install twilio" and this is the result:
<code>Searching for twilio
Best match: twilio 6.3.dev0
Processing twilio-6.3.dev0-py2.7.egg
twilio 6.3.dev0 is already the active version in easy-install.pth

Using /usr/local/lib/python2.7/site-packages/twilio-6.3.dev0-py2.7.egg
Processing dependencies for twilio
Finished processing dependencies for twilio
</code>
I guess that everything went ok, but the problem is when I ran the command "import twilio" it said `ImportError: No module named twilio`
I also did the process using pip and the result is the is the same.
Any help on this will be appreciate.
Comment: I really don't know why someone without trying to help downvote the question, at least let everybody know why the downvote...
Answer: Perhaps you have more than one version of python installed in your machine try to check the directory that it installed to
Comment: Okay try to re install online installation is better I guess
Comment: Thanks for your answer, I downloaded the package and I installed manually with `python setup.py install` and now it is working, thanks again
Comment: I used `which python` command and I got this: /Library/Frameworks/Python.framework/Versions/2.7/bin/python, so I guess I don't have to python version installed


JQM ajax page reloading multiple times

Question: Wierd one here...
I am setting up my application, and I am loading in another page via the ajax method. Between these pages I have a simple forward/back method.
On the firing of the back method, I have placed a console log, and I can see that the first time i click back, it shows once, if i then go back into that page, and click back again, it then shows two, and so on.
This is resulting in wierd page reloading issues. (Clicking back and it transitions in the page I just clicked back from).
Using the data-rel="back", or a changePage method, the issue is the same.
I've had a google, and could not find anything.
Any help would be greatly appreciated.
Answer: Your problem comes from multiple event binding. Because you didn't provide your code I cant tell you what specific line to change, but I can give you a link to my other answer.
There you will find multiple solutions with examples: jQuery Mobile: document ready vs page events. Just search for a topic: Prevent multiple event binding/triggering
In your case, some event is probably been applied each time a page is visited again. In case you are using jQuery Mobile page events and if it is possible for you an easiest solution would be to use a pageinit event.
Comment: Ah, now that would make sense. Currently, I have a global scripts file with calls to pagecreate inside. What is a better solution? Individual scripts loaded in on each page?
Comment: I prefer individual scripts in each page but you will need to be careful even with this solution. Take a look at my other answer: http://stackoverflow.com/a/15806954/1848600 to find out why. Basically if you have several HTML files and you want to have a separate js files in each HTML, those js files will need to be placed in BODY.
Comment: OK Sounds great. My HTML files currently are just data-rel="page", there is no other content. Is this wrong? Or can i just put my JS file call inside the page content?
Comment: Had a look and your solution to kill the live() event sorted my issue. Your posts are very helpful - and I have bookmarked for future reference. Thanks for your assistance!

JQM ajax page reloading multiple times


Use libphonenumber w/ typescript and JSPM

Question: https://www.npmjs.com/package/google-libphonenumber
I installed that with JSPM. i tried a straight import 'google-libphonenumber';
I also tried a d.ts
declare module "google-libphonenumber" {
    import libphonenumber = require('google-libphonenumber');
    export var PhoneNumberUtil: typeof libphonenumber;
}

import PhoneNumberUtil = require("google-libphonenumber");

I'm just trying to use the format function in my typescript (ES6 style modules) and JSPM. I tried with and without any definitions.
import libphonenumber from 'google-libphonenumber'; // module not found.

import 'google-libphonenumber'; // module foun
declare var libphonenumber: any; // needed for TS
const phoneUtil = libphonenumber.PhoneNumberUtil.getInstance();

phoneUtil and libphonenumber is null.
using the code
   var phoneNumber = phoneUtil.parse(input, 'US');
   return <string> phoneUtil.format(phoneNumber);

Comment: And the problem you encountered was what?
Comment: It worked in typescript before I implemented JSPM and ES6 imports.
Comment: var libphonenumber = require('google-libphonenumber'); TS doesn't like it
Answer:
var libphonenumber = require('google-libphonenumber'); TS doesn't like it

Use declare var require:any.
Better
declare module "google-libphonenumber" {
    var foo:any;
    export = foo;
}

in a file global.d.ts
Then you can do :
import libphonenumber = require('google-libphonenumber');


Hook_menu_alter does not load correct page

Question: I'm trying to check if a user has permission(registered by a module) before he adds/edits fields on a node type registered by the module.
The approach is to use hook_menu_alter and add an access callback.
The URL i am trying to add this to is
admin/structure/types/manage/applx-application/fields

Where 'applx-application' is the node type.
The problem is after the callback has been added via the hook, the page falls to
admin/structure/types/ instead of admin/structure/types/manage/applx-application/fields
I'm not sure what i'm doing wrong. Oh and hook_menu_alter is called by hook_hook_info. If that is of any concern.
    <?php
/*
    Implements hook_menu_alter
*/
function applx_menu_alter(&$items){
    $items['admin/structure/types/manage/applx-application/fields']['access callback'] = 'applx_permission_check';
    $items['admin/structure/types/manage/applx-application/fields']['access arguments'] = array('field_management');
}

My only guess is that it falls back to structure/types because the system module's menu hook expects a wildcard in place of applx-application i suppose.
What is the right way to do this, if not this way?
Answer: Digging a little bit more i found out that the right way to do this is to use
hook_field_access
Comment: May you expand a little your answer? Saying you found another hook to use doesn't help future users, which is what Stack Exchange aims to do.


Stored Procedure to display all the records in a table in PL/SQL

Question: I really need your help. I'm trying to create a stored procedure in PL/SQL to count student by Location
I've done it using Cursor,even if it may sound stupid somehow. I don't know to do it using stored procedure.
So here is the code I used to count student by location using cursor
// Codes to count by location using cursor
declare
s_location VARCHAR(15);
cnt NUMBER;
CURSOR curlocation IS
SELECT address,COUNT(address) AS cnt
FROM student
GROUP BY address;
BEGIN
OPEN curlocation;
LOOP
FETCH curlocation INTO s_location,cnt;
EXIT WHEN curlocation%NOTFOUND;
IF cnt=0 THEN
dbms_output.put_line('No Students For'||s_location);
ELSE
dbms_output.put_line('--------------------------------------------');
dbms_output.put_line(cnt ||'  '||'Students For '|| s_location);
END IF;
END LOOP;
CLOSE curlocation;
END;

As you can imagine the output will be something like::
1  Students For ...
2  Students For ...
5  Students For ...
So I want to do something like this using stored procedure.
Thanks guyz
Comment: Can you be little more specific? What should be input to your procedure, and what should be the result? Just some prints?
Comment: Thanks for your answer... So for being more specific I would like to to create a stored procedure which will do exactly what that cursor is doing. I should count the student(Because am dealing with their table) based on their Location
Comment: So When I execute the procedure. I would like to get the Result like   You have 4 students in USA ,     You have 3 Students in Canada, etc
Answer: You can create a procedure and use your cursor like before:
create procedure count_and_print_students as
  l_location varchar(15);
  l_count number;

  cursor c_student_locations
  is
    select address, count(address) as cnt
    from student group by address;
begin
  open c_student_locations;
  loop
    fetch c_student_locations into l_location, l_count;
    exit
  when c_student_locations%notfound;
    if l_count = 0  then
      dbms_output.put_line('No Students For'||l_location);
    else
      dbms_output.put_line('--------------------------------------------');
      dbms_output.put_line(l_count ||'  '||'Students For '|| l_location);
    end if;
  end loop;
  close c_student_locations;
end;

and run it as:
DECLARE
BEGIN
count_and_print_students();
END;

Moreover, you can create the procedure for counting students per single location as argument like this:
create procedure count_students_per_location(p_location in varchar2, x_count out number) as
begin
  select count(address)
  into x_count
  from student
  where address = p_location;
end;

So, it can be used in following way:
DECLARE
  l_count number;
BEGIN
  count_students_per_location('LOCATION_X',l_count);
  dbms_output.put_line(l_count ||'  '||'Students For LOCATION_X');
END;

Comment: Thanks for your response... I will Use the Cursor and the procedure!! :-)
Answer: There's not much point in creating a procedure that returns a single OUT value. That should be a function instead:
CREATE OR REPLACE FUNCTION student_count_per_location (p_location IN VARCHAR2)
   RETURN NUMBER
   AUTHID DEFINER
AS
   x_count   NUMBER;
BEGIN
   SELECT COUNT (address)
     INTO x_count
     FROM student
    WHERE address = p_location;

   RETURN x_count;
END;
/

BEGIN
   DBMS_OUTPUT.put_line (
         student_count_per_location ('LOCATION X')
      || '  '
      || 'Students For LOCATION_X');
END;
/

Comment: Thanks for your reponse :-), but the problem is that I don't want to return a single value.But I think the best option I have is to integrate my cursor in a procedure as @mkuligowski Did!
Comment: Makes sense Carlos. My response was actually a reaction to mkuligowski's submission.


Solving recursion (Gambler's ruin)

Question:
Let $M_{i}$ denote the mean number of games until the gambler either goes broke or reaches a fortune of $N$, given that he starts with $i = 0,1,\ldots,N$.

I have shown that
$M_{0} = M_{N} = 0$ and $M_{i} = 1 + pM_{i+1} + qM_{i-1}$ for $i=1,\ldots,N-1$
I want to solve these equations to obtain
$$M_{i} = i(N-i)$$if $p=1/2$
or
$$M_{i} = \frac{i}{q-p} - \frac{N}{q-p}\cdot\frac{1 - (q/p)^{i}}{1- (q/p)^{N}}$$
if $p \neq 1/2$.
I've tried verifying by direct substitution, but I don't think I'm substituting correctly. How can I solve these difference equations to show the desired result?
Answer: Let $X_0=i$ and $\{X_n\}$ be iid with $\mathbb P(X_1=1)=1-\mathbb P(X_1=-1)=p$. Let $$S_n = \sum_{k=0}^n X_k $$ be the gambler's wealth at time $n$ and $$\tau=\inf\{n: S_n\in\{0,N\}\} $$ be the time until he stops. Define $$Z_n = \left(\frac qp\right)^{S_n}. $$
Then
\begin{align}
\mathbb E[Z_{n+1}\mid \mathcal F_n] &= \mathbb E\left[\left(\frac qp\right)^{S_n+X_{n+1}}\mid\mathcal F_n \right]\\
&= \left(\frac qp\right)^{S_n} + \left(\frac qp\right)^{-1}\mathbb P(X_{n+1}=-1) + \left(\frac qp\right)^1\mathbb P(X_{n+1}=1)\\
&= Z_n + \left(\frac pq\right)q + \left(\frac qp\right)p\\
&= Z_n + p + q\\
&= Z_n,
\end{align}
so that $Z_n$ is a martingale. Then by the optional stopping theorem we have
$$\mathbb E[Z_\tau] = \mathbb E[Z_0] = \left(\frac qp\right)^i. $$ It follows that
\begin{align}\mathbb E[Z_\tau] &= \left(\frac qp\right)^0\mathbb P(Z_\tau=0) + \left(\frac qp\right)^N\mathbb P(Z_\tau=N)\\
&= \mathbb P(Z_\tau=0) + \left(\frac qp\right)^N (1-\mathbb P(Z_\tau=0) )\\
&= \left(\frac qp\right)^i,\end{align}
and so
\begin{align} \mathbb P(Z_\tau=0) &= \frac{\left(\frac qp\right)^i - \left(\frac qp\right)^N}{1-\left(\frac qp\right)^N},\\
\mathbb P(Z_\tau=N) &= \frac{1 - \left(\frac qp\right)^i}{1-\left(\frac qp\right)^N}.\\
\end{align}
Therefore the gambler's expected total winnings are
\begin{align}
\mathbb E\left[\sum_{n=1}^\tau X_n \right] &= \mathbb E[S_\tau] - S_0\\
&= \mathbb E[S_\tau\mid Z_\tau=0]\mathbb P(Z_\tau=0) + \mathbb E[S_\tau\mid Z_\tau=N]\mathbb P(Z_\tau=N) - i\\
&= N\left(\frac{1 - \left(\frac qp\right)^i}{1-\left(\frac qp\right)^N}\right)-i
\end{align}
By Wald's identity, we have
$$\mathbb E\left[\sum_{n=1}^\tau X_n \right] = \mathbb E[\tau]\mathbb E[X_1], $$
and so $$\mathbb E[\tau] = \frac1{p-q}\left[N\left(\frac{1 - \left(\frac qp\right)^i}{1-\left(\frac qp\right)^N}\right)-i\right], $$
when $p\ne q$.
Answer: The recurrence relation
$$pM_{i+1} - m_i +qM_{i-1} = -1 \qquad\qquad (1)$$
is a non-homogeneous, second-order, linear recurrence relation with constant coefficients ($p,-1,q$) and can be solved by finding its "characteristic equation" as described here or here or in most texts on discrete mathematics.
Initially, consider the homogeneous relation
$$pM_{i+1} - m_i +qM_{i-1} = 0. \qquad\qquad (2)$$
The characteristic equation is $pr^2 - r + q = 0$ giving roots: $\;r=1 \text{ or } q/p$.
$$\\$$
Case $p=q$:
We have a double root $r=1$ so the general solution to $(2)$ is
$$M_i = A + Bi\qquad\qquad (3)$$
The particular solution for $(1)$ would normally have form $M_i=C$ because RHS $(1)$ is constant, but in view of $(3)$ we instead try $M_i = Ci^2$ and substitute this into $(1)$ to give
$$-1 = pC(i+1)^2 - Ci^2 + qC(i-1)^2 = C \qquad\text{using $p=q=1/2$}.$$
So our particular solution is $M_i=-i^2$ and our general solution to $(1)$ is $M_i = A+Bi-i^2$.
$M_0 = 0 \implies 0=A.$
$M_N = 0 \implies 0 =BN-N^2\;$ so $\;B=N$.
So the solution to $(1)$ is
$$M_i = Ni-i^2.$$
$$\\$$
Case $p\neq q$:
We have distinct roots $r=1,\; q/p\;$ so the general solution to $(2)$ is
$$M_i = A + B\left(q/p\right)^i \qquad\qquad (4)$$
Here we try the particular solution $M_i = Ci$ and substitute this into $(1)$ to give
$$-1 = pC(i+1) - Ci + qC(i-1) = C(p-q).$$
Therefore, $\; C=1/(q-p)\;$ and our general solution to $(1)$ is $M_i = A + B\left(q/p\right)^i + \dfrac{i}{q-p}$.
$M_0 = 0 \implies 0=A + B\;$ so $\;A=-B.$
$M_N = 0 \implies 0 = -B + B\left(q/p\right)^N + \dfrac{N}{q-p}\;$ so $\;B=\dfrac{N}{q-p}\cdot \dfrac{1}{1-\left(q/p\right)^N}$.
So the solution to $(1)$ is
\begin{align}
M_i &= - \dfrac{N}{q-p}\cdot \dfrac{1}{1-\left(q/p\right)^N} + \dfrac{N}{q-p}\cdot \dfrac{\left(q/p\right)^i}{1-\left(q/p\right)^N} + \dfrac{i}{q-p} \\
& \\
&= \dfrac{i}{q-p} - \dfrac{N}{q-p}\cdot \dfrac{1-\left(q/p\right)^i}{1-\left(q/p\right)^N}.
\end{align}

Solving recursion (Gambler's ruin)


Retornar consulta sem registro no banco de dados

Question: Olá. Tenho um sistema em PHP com MySQL de uma empresa de frota de caminhões. Nesse sistema cada usuário realiza um checklist do veículo antes de utilizá-lo. Esse checklist é cadastrado em uma tabela no banco de dados.
Meu cliente está pedindo uma tela de consulta dos checklist que não foram realizados. Exemplo: ele informa o período de data e o sistema retorna os checklists que não foram feitos naquele período.
Como vou retornar uma consulta de algo que não está cadastrado no banco?
Answer: Então EltonRodrigo, primeiramente bem-vindo! A resposta simples é não. A resposta completa é: você provavelmente modelou o banco sem fazer uma análise mais apurada da regra de negócio. Nessa situação eu teria feito uma cadastro de checklist, onde eu indicaria o veículo, se o checklist foi realizado ou não (um campo booleano status), quem realizou o checklist e data de realização desse checklist. Daria para obter essa informação pela simples filtragem dos checklists com status true.
Comment: Obrigado pela resposta Silva Júnior. Sim, eu tenho uma tabela com todos os veículos e tabela com os condutores, elas estão com chave estrangeira para a tabela de checklist. Na tabela de checklist também tem data de realização. A dúvida é seguinte: não existe nenhum cronograma para a realização dos checklist e nem data específica, qualquer condutor pode pegar o veículo quando e onde quiser.
Comment: Por exemplo, pode pesquisar pela data na tabela checklist. Suponhamos que o relatório seja mensal, veja quantos checklists foram feitos e subtraia de de 30 ou 31, conforme o mês. Nesse caso, tu usaria a função COUNT do SQL.
Comment: Você pode adapatar o seu código com base nesse post;
Comment: Deu certo!  Eu vinculei a tabela frotas com a tabela checklist utilizando o left join como mostra nesse link desse post que você me passou.  Eu tive que adicionar uma coluna "realizado" do tipo booleana e no Select utilizei a cláusula WHERE com valor "is null". Muito obrigado.


Regex expression to find value in string that has a static position but variable amount of characters

Question: I am reading in data from a PDF, and the output is a block of text. The goal is to pull out one value from that block of text. The value I need to pull is in a static position, but the amount of characters can change. What is regex expression to find the string in question.
The block of text I am using is as follows:
Residue mmbtu 3,787.11 100.00% 3,787.11 $1.623163 $6,147.10
Total 3,787.11 3,787.11 $6,147.10

I need to find the dollar amount after the line with "Total" which in this case is 6,147.10
The expression that has yielded the best so far is (?<=\$)([^\s]+), however this gives me the numbers after the dollar sign excluding spaces. I need to to only find the last value though
(?<=\$)([^\s]+)

The expected result would be to find the dollar amount that comes after the line with the word "Total" knowing that any PDF that I read in will have a block of text similar to what is posted above. The example above would give me a value of 6,147.10
Comment: Maybe `(?<=\$)\S+$`?
Comment: Do you mean like this using a capturing group? `^Total.* \$(.+)$`
Comment: Perhaps [`^Total[^$]+(.+)`](https://regex101.com/r/kP11Je/1/) ?
Comment: Hey Wiktor, that expression doesn't bring back any value. Thank you for the suggestion though!
Comment: See https://regex101.com/r/bI9hzF/1 - it returns what you need.
Comment: @WiktorStribiżew Interesting that the program you posted does actually recognize the expression correctly. Jan works as well. However, I am using https://regexr.com/ to test expression becasue my program (UiPath) apparently recognizes that format instead, and that site does not show the right answer.
Comment: In UiPath, I have to assign a string variable and use an expresstion like this system.Text.RegularExpressions.Regex.Match(str_ResVolMCF,"(?<=Res Return )([^\s]+)").ToString  to get the program to understand
Comment: Ok i see the problem! There is an ending space after the last value. When it is removed, the expressions everyone has posted works, but when there is a space, then it fails. Any way to get around this?
Comment: Then just use `(?<=\$)\S+(?=\s*$)`
Answer: You could use
^Total[^$]+(.+)

See a demo on regex101.com.


Como manipular ordem de Event Listeners de um elemento DOM

Question: Tenho um link:
go to foo

Primeiro event handler (main.js)
document.querySelector('.foo').addEventListener('click', function( event ) {
   console.log(1);
});

Em outro script, adiciono o segundo event handler (script.js)
document.querySelector('.foo').addEventListener('click', function( event ) {
  console.log(2);
});

Quando executado, tenho o seguinte log:
1 (main.js)

2 (script.js)

A respeito, tenho duas dúvidas:

Como manipular essa ordem?
Como congelar todos os event listeners do link e dispará-los apenas se alguma condição for satisfeita?
Comment: Boa tarde, a resposta resolveu o seu problema?
Answer: Pergunta 1.
Os eventos serão disparados por ordem de declaração. Eventos declarados primeiro executam primeiro, pois desta maneira o segundo evento click faz um override do primeiro, concatenando os dos métodos como se fossem um só.
Pergunta 2.
A maneira mais simples é criar uma variável auxiliar e executar a função caso ela for verdadeira, assim você pode flexibilizar o código
//main.js
var executar1;
//atribuir true à variável executar1 para a condição desejada
if (true)
   executar1 = true;
document.querySelector('.foo').addEventListener('click', function( event ) {
   if(executar1)
   console.log(1);
});

//script.js
var executar2;
//Não quero executar a dois, então vou deixá-la nula, valendo como false

document.querySelector('.foo').addEventListener('click', function( event ) {
   if (executar2)
     console.log(2);
   else
     console.log("não quis executar o 2");
});

Caso queira remover os eventos definitivamente, use o método $().off() para remover esta função
Comment: A resposta para a **pergunta 1** foi esclarecedora. Obrigado.
Answer: Não tem como manipular a ordem até aonde eu sei, o que você pode fazer é criar um array com todas funções, ficaria algo como:

var funcoes = [
    function() {
        console.log("foo");
    },
    function() {
        console.log("Oi");
    },
    function() {
        console.log("Olá mundo!");
    },
    function() {
        console.log("Stack Overflow");
    },
    function() {
        console.log(1);
    },
    function() {
        console.log(2);
    }
];

/*
 * Reordenamento da array, coloquei uma ordem aleatória
 * Fonte: http://stackoverflow.com/a/2450976/1518921
 */

function shuffle(array) {
  var currentIndex = array.length, temporaryValue, randomIndex ;

  // While there remain elements to shuffle...
  while (0 !== currentIndex) {

    // Pick a remaining element...
    randomIndex = Math.floor(Math.random() * currentIndex);
    currentIndex -= 1;

    // And swap it with the current element.
    temporaryValue = array[currentIndex];
    array[currentIndex] = array[randomIndex];
    array[randomIndex] = temporaryValue;
  }

  return array;
}

shuffle(funcoes);

var foo = document.getElementById("foo");
var j = funcoes.length;

for (var i = 0; i < j; i++) {
    foo.addEventListener('click', funcoes[i]);
}
<button id="foo">Testar</button>

Como congelar todos os event listeners do link e dispará-los apenas se alguma condição for satisfeita?

Pra "congelar" você terá que terá que criar um evento separado que é disparado no momento desejado, ficaria algo como:
var emEspera = [];

function AdicionarEspera(callback) {
    emEspera.push(callback);
}

var funcoes = [...];

shuffle(funcoes);

var foo = document.getElementById("foo");
var j = funcoes.length;

for (var i = 0; i < j; i++) {
    foo.addEventListener('click', function() {
        AdicionarEspera(funcoes[i])
    });
}

foo.addEventListener('click', function() {
    var copia = emEspera, j = emEspera.length;

    emEspera = [];//Limpa eventos em espera

    for (var i = 0; i < j; i++) {
        copia[i]();
    }
});

var test = document.getElementById("test");

html:
<button id="foo">Adicionar a lista</button>
<button id="test">Executar eventos em espera</button>

No momento que clicar em test então ele vai executar todos eventos em espera, após clicar ele limpa os eventos


Swift enums complex rawvalues

Question: basically, what I wish to do would be
enum MyEnum : MyObject {

case first = MyObject(value1, value2, value3)
...

}

I've looked the documentation and tried to code something like that but it looks like it's not possible as the rawvalue type is complex.
My questions are: so is this complex type of enum can be done?
If not, which design would you recommend to do that (as I would like to avoid making huge switch cases)?
Thanks a lot!
Comment: I assume that what are you looking for is similar to [this answer](https://stackoverflow.com/questions/41922097/enum-of-structs-in-swift-3-0/41922262#41922262).
Answer: Here is an example of achieving this
enum CroudInfo: String {
    case bitkom = "Битком"
    case free = "Свободно"
    case avarage = "Загружено"

    func values() -> (title: String, image: UIImage, textColor: UIColor, fullImage: UIImage) {
        switch self {
        case .bitkom:
            return (self.rawValue, UIImage(named: "redValue")!, UIColor(red: 208 / 255, green: 19 / 255, blue: 0 / 255, alpha: 1), UIImage(named: "redImage")!)
        case .free:
            return (self.rawValue, UIImage(named: "greenValue")!, UIColor(red: 84 / 255, green: 175 / 255, blue: 98 / 255, alpha: 1), UIImage(named: "greenImage")!)
        case .avarage:
            return (self.rawValue, UIImage(named: "yellowValue")!, UIColor(red: 247 / 255, green: 157 / 255, blue: 0 / 255, alpha: 1), UIImage(named: "yellowImage")!)
        }
    }
}

Swift enums complex rawvalues


Basic question about simplifying a square root

Question: I just wanted to know how to get from $\sqrt{12}$ to $2\sqrt{3}$
Because my buddy was teaching me math the other day and gave me a list with some basic exercises to do, one of which is to solve $f(x)=\sqrt{x^2 + 2x + 4}$ for $x=2$
The result I found was $\sqrt{12}$ but WolframAlpha says it is $2\sqrt{3}$ and I don't know how to get there. Is it wrong to just leave it as the first one?
Comment: $\sqrt{12}=\sqrt{4\cdot3}=\sqrt{4}\,\sqrt{3}=2\sqrt{3}$
Comment: And no, it is not wrong.
Answer: As pointed out by egreg in the comments, $\sqrt{12} = \sqrt{4\times 3} = \sqrt{4}\times\sqrt{3} = 2\times\sqrt{3} = 2\sqrt{3}$.
If the number $\sqrt{12}$ comes up, it is not wrong to leave it as $\sqrt{12}$ instead of $2\sqrt{3}$ (they are equal after all). If a question asks you to give the final answer in simplest terms, then you are expected to use $2\sqrt{3}$ instead of $\sqrt{12}$.


Pros / cons of using password-less OpenVPN client keys

Question: I'm setting up an OpenVPN server for my organization, and I'm reading up on the different ways  that the client side keys work. I'm still a little unfamiliar with all of the concepts behind these client keys and account auth.

What are the real benefits of using the ./build-key-pass vs the ./build-key to generate the client keys? Is this just similar to password protecting an SSH key? Not all of our employees who require an account are tech savvy, so is it worth introducing another set-up step?
I see now that there is an option to not require typical username / password authentication when using client keys. My plan was to do a useradd on the server for each client. If I don't use the user-pass auth settings then how would I revoke access to a specific client?
How can I securely deliver these client certificates to all of our employees who need an account? I believe that the client.key files are supposed to be private, and emailing the files seems like it would be insecure.
Answer: To respond to each of your points:
1 - You are correct in analogizing the password protection of OpenVPN keys to password protection of SSH keys.
2 - Without using any additional authentication method OpenVPN relies only on the verification of the client certificate by the server (and ideally verification of the server certificate by the client) for authentication of the client. This makes revoking an individual client's access a matter of either adding the client's certificate to a certificate revocation list (CRL) (supported by versions 1.5 and up of OpenVPN) or removing the key material from the client (or switching up the certificates on all your other clients). If you're not using an additional authentication method you need to have a CRL to allow for client access revocation.
Keep in mind that password protecting the keys doesn't help you at all re: additional authentication. That password just "unlocks" the key on the client device-- it doesn't alleviate the problem of additional authentication of the user on the client to the server computer (and the access revocation problem).
3 - You should generate the private / public key pairs on the clients themselves, as opposed to transmitting them over the wire. You can generate the certificate request on the client, send the public key (in a Certificate Signing Request) up to your CA for signing, and install the signed certificate on the client. This could all be scripted, and I'm sure somebody has already done it (and I'd hope that the commercially licensed OpenVPN product probably has some of that functionality built-in).
I find this My Certificate Wizard project that was written just for this purpose, but I'd go for scripting the entire thing on the client using the OpenSSL command-line tools in an attempt to make the entire process mostly invisible to the user.
Comment: Great, thanks! I'm going to go with the user-password based authentication on top of the client certificate/key pair. As far as your response to question 3, is this just a suggestion or would security be fatally flawed if we transmitted password-protected keys over the wire?
Comment: If you have to send the keys over the wire they need to be password protected, at which point you've moved the key distribution problem from the private keys to the passwords protecting the private keys. I'd look for a way to generate the keys client-side, personally.
Comment: I tend to slightly disagree with (2): denying access to a particular client has to be done by revoking its certificate--by adding its serial number to the CRL maintained by the CA, and the OpenVPN server should have access to that CRL--each time a client connects, the server checks the CRL.  The upside of this approach is that it's easy to deal with cases like stolen devices.
Comment: ...or, at my $dayjob, we twice had to deny the access to folks which have been fired and hence we just hadn't legal access to their devices with their private keys.  For this reason we keep eash issued certificate (but not their matching private keys) near the CA for easy revocation just in case.
Comment: @kostix - I will grant you that-- a properly-working PKI, completely with a CRL, would make revocation possible. More often than not, though, people knock-together a glue-and-tape PKI just for OpenVPN and don't spend the time putting together niceties like a CRL. Good point, though-- I'll drop on an edit.
Comment: Well, we're doing just that--a script in the spirit of `CA.pl`. I mean, it's just `openssl` doing all the stuff and a simple text file containing the CRL synced to the OpenVPN server.


Add comments to custom node

Question: I have a module which shows content with the page callback. Here is my page callback (just the relevent code):
function my_page_callback() {
    $build=array();
    $build[] = array(
        '#theme' => 'my_theme',
        '#marker' => 'foo',
        '#data' => ...,
        '#contextual_links' => ...,
    );
    $comment=comment_node_page_additions(node_load(getTargetNodeId()));
    $build[]=$comment['comment_form'];
    return $build;
}

However the function getTargetNodeId() retuns a valid id of a node which is generated unter some conditions with this code:
$node = new StdClass();
$node->nid=42;
$node->vid=42;
$node->uid=1;
$node->type = 'my_type';
$node->status = 1;
$node->title = "Fresh generated";
$node->content['mapping'] = 4711; // some internal magic
$node->comment='2';
$node->language = LANGUAGE_NONE;
$node->body[$node->language][0]['value'] = "This text should nobody read.";
node_save($node);

Even if in the code above the nid and nid are fixed I take care of it the ids are unique.
And here is my code where I render my node:
function module_node_view_alter(&$build) {
    if($build['#bundle']=='my_type') {
        $p=my_page_callback;
        $build['body']=$p[0];
    }
}

However the the comment header is generated but the input field is missing. What is the problem?
Answer: If my_type is a content type defined in the Drupal site, then I would check in admin/structure/types/manage/my_type/comment/fields that you didn't delete the comment body.

If this is what happened then the node page changes from what shown in the following first screenshot to what shown in the second screenshot.

The screenshots has been taken using the user #1 for my test site, which means what shown happens to every user.
The same thing should happen when the content type has been deleted, or it has never existed on the Drupal site.
There could be other reasons:

The only input format available in that site is not accessible from the user who is visiting the page your code is outputting. This is not clearly the case if you are using the user #1 to visit the page.
It also means the site doesn't have the default input format, which is not possible to disable from the user interface. It could be the default input format has been deleted from the database; if this is the case, users would have problems also editing nodes, as the input formats used for comments are the same used for nodes, even if a comment for a node using (for example) full HTML could use plain text, or filtered HTML.
The content type has been created programmatically, in a way that didn't cause hook_node_type_insert() to be invoked. If this is the case, then the Comment module doesn't have the possibility of adding the comment body field, which is done using the following function.
function comment_node_type_insert($info) {
  _comment_body_field_create($info);
}

Add comments to custom node


How is the poisson distribution a distribution? It seems more like a formula

Question: I just watched this video: https://www.youtube.com/watch?v=Fk02TW6reiA
It shows a formula to calculate an answer for the following problem:

There are 2 customers expected every 3 minutes in a store
Therefore there are 6 customers expected every 9 minutes
What is the likelihood of there being 4 or less in the store in 9 minutes?
Answer is: P(0;6)+...+P(4;6) which is about 0.28

This makes sense and is well described. However, numpy treats the poisson distribution basically like a random number generator: http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.poisson.html
We can specify lambda as say 5 and how many numbers are desired (the second argument) and get a big list of integers:
>>> import numpy as np
>>> s = np.random.poisson(5, 10000)
>>> s
array([2, 4, 4, ..., 3, 4, 3])
>>> len(s)
10000

These seem like two totally different things. How do you get from using the Poisson formula to calculate the possibility of a certain number of events in a timeframe, to a list of seemingly random integers?
Comment: You'd probably like the **Poisson** videos over at Khan Academy--check them out.
Answer: The formula $f$  is the probability mass function for the Poisson distribution. That formula, as explained in the video, can be used to calculate the probability of a given value under the assumed distribution. The related cumulative distribution function $F$ can be used to generate random numbers following the distribution:

Use the CDF to partition the interval $(0,1)$ into subintervals: $(0, F(x_1))$, $(F(x_1), F(x_2))$, $etc...$
Generate random numbers on the interval $(0,1)$ and see which bin they fall into.

More in this tutorial, which goes through a Poisson example using R. The Poisson PMF and CDF are available in scipy.
Comment: This makes sense, I watched the youtube "Engineer Clearly" videos on PMF and CDF which are great: http://www.youtube.com/watch?v=bGS19PxlGC4 . However, why bother putting the random numbers through the poisson if they are already random?
Comment: @LittleBobbyTables Do you mean, why would one use the PMF to calculate the probability of random Poisson-generated numbers? Generally, randomly generated numbers are used in simulation; PMFs are used in many contexts, including p-values and computing posterior distributions. Offhand, I can't think of a context in which one would randomly sample, only to immediately calculate the probability of the randomly generated data.
Answer: The function you link to is a random number generator. It does not return the Poisson distribution, but returns random numbers from a Poisson distribution.
That is, it does exactly what its name suggests - gives you random Poisson variates, not the distribution.
The Poisson probability function is of the form $P(X=x) = \frac{e^{-\lambda} \lambda^x}{x!}\,,\quad 0,1,2,\ldots$,
while the distribution function is $P(X\leq x) =\sum_{i=0}^x \frac{e^{-\lambda} \lambda^i}{i!}\,,\quad 0,1,2,\ldots$.

There are a variety of methods for generating random numbers from this distribution, which will (almost always) begin with a source of uniformly distributed random numbers on $[0,1)$ (notionally continuous, but in practice limited to at best the accuracy with which numbers are represented by the particular implementation on computers).
The scipy function will use one of those methods; which one will be discernable by examining the code (which you'd be better placed to locate than me). However, if I am looking at the right underlying C code that numpy uses (source here), then it uses two different algorithms, depending on the Poisson parameter:
long rk_poisson(rk_state *state, double lam)
{
    if (lam >= 10)
    {
        return rk_poisson_ptrs(state, lam);
    }
    else if (lam == 0)
    {
        return 0;
    }
    else
    {
        return rk_poisson_mult(state, lam);
    }
}

The code for those two functions (rk_poisson_ptrs and rk_poisson_mult)  is in the same file, immediately above the quoted code.
Answer: I generally use R so my answer here is based on a quick web search. It looks like numpy supports generating random samples from a Poisson distribution and doesn't have functions for computing the probability mass function (PMF) described by the Poisson formula to which you refer. Generating random samples from a distribution can be very useful but as you point out is not the same as computing the PMF which is what you'd need to do to solve the "customer" problem.
It seems like you should be looking at scipy which seems to support the generation of PMF's for a large variety of distributions including Poisson.
Comment: My point was that numpy does not seem to have a function for computing the PMF of a Poisson but scipy does. I am not sure how numpy generates its random numbers and not clear on why that's a consideration here.
Comment: So you reckon that numpy is using another random function to pick particular n ontop of an existing distribution? https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/distributions.c is a bit mysterious as to what is happening lol


How to set the time period for django model

Question: I want to keep the time period in Django model.
like 60 sec, 10 min.
so I choose TimeField class for this purpose
 my_period = models.TimeField()

Is it correct practice???
Because when I try to edit this value in admin page, there shows am or pm.
I think I should just ignore it though....

Thanks to the comment!
I updated the code as below.
import datetime
class GlobalParam(models.Model):

    my_period = models.DurationField()
    #my_period = datetime.timedelta(days=20, hours=10) #where should I put...??

    def __str__(self):
        return my_period

However it shows error...
 File "/Users/whitebear/anaconda3/lib/python3.7/site-packages/django/utils/duration.py", line 44, in duration_microseconds
    return (24 * 60 * 60 * delta.days + delta.seconds) * 1000000 + delta.microseconds

Answer: For your case, DurationField is better.
model.my_period = datetime.timedelta(days=20, hours=10)

Comment: Thanks its quite helping but your example doesn't use `DurationField`. In my model class  `my_period = models.DurationField()` it made the column in DB but it says, `AttributeError: 'int' object has no attribute 'days'`


The provided token is malformed or otherwise invalid

Question: latestAll S3 bucket file is displayed, but when I upload file then error is generate.
I have ARN and instance profile.
use Aws\Credentials\CredentialProvider;
use Aws\Credentials\InstanceProfileProvider;
use Aws\Credentials\AssumeRoleCredentialProvider;
use Aws\S3\S3Client;
use Aws\Sts\StsClient;

$profile = new InstanceProfileProvider();
$ARN = ""; // MY ARN
$sessionName = "s3-access-example";

$assumeRoleCredentials = new AssumeRoleCredentialProvider([
    'client' => new StsClient([
        'region' => "ap-east-1",
        'version' => "latest",
        'credentials' => $profile
    ]),
    'assume_role_params' => [
        'RoleArn' => $ARN,
        'RoleSessionName' => $sessionName,
    ],
]);

$provider = CredentialProvider::memoize($assumeRoleCredentials);

$this->s3hd = S3Client::factory([
    'credentials' => $provider,
    'version' => "latest",
    'region' => "ap-east-1"
]);

public function upload($name, $file, $type, $Bucket = false)
{
    if (! $Bucket) {
        $Bucket = $this->bucket;
    }
    $result = $this->s3hd->putObject([
        'Bucket' => $Bucket,
        'Key' => $name,
        'SourceFile' => $file,
        'ContentType' => $type,
        'ACL' => 'public-read'
    ]);
    $this->s3hd->waitUntil('ObjectExists', [
        'Bucket' => $Bucket,
        'Key' => $name
    ]);
    return $result;
}

Message: Error executing "PutObject" on error file url here; AWS HTTP error: Client error: PUT error file url here resulted in a 400 Bad Request` response: InvalidTokenThe provided token is malformed or other (truncated...) InvalidToken (client): The provided token is malformed or otherwise invalid. - InvalidTokenThe provided token is malformed or otherwise invalid.
Comment: Which line is generating the error? I can't see a `PutObject` command in your code. It appears that your code is very similar to [Using a Credential Provider - AWS SDK for PHP](https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_provider.html). Not sure if this is related to your error, but the example on that page has values for `version` and `region` whereas your code does not.
Comment: @JohnRotenstein I updated code.
Comment: @JohnRotenstein it is already working before with access key and secret key. But now i have to use IAM credential ARN instead if access / secret keys.
Comment: I'm sorry, I don't understand what you mean. What did you change for Access Key vs Role? And why do you not have a `region` and `version` in `new StsClient()`?
Comment: Yes i have region and version, I have to access s3 bucket using ARN & Instance Profile (IAM credential) instead of access / secret keys.
If i used access / secret keys then it's already working, but now i have to access it without keys using IAM credential.
Comment: Where is your code running (eg on an EC2 instance)? To call `AssumeRole`, you need a set of credentials to identify who you are, so that AWS can confirm that you are entitled to assume the role. Thus, you still need a way to provide those credentials. If your code is running on an EC2 instance, then you can assign the role directly to the instance and the SDK will automatically find and use those permissions.
Comment: If i call $this->s3hd->listBuckets() it return `Message: is_readable(): open_basedir restriction in effect. File(/usr/share/httpd/.aws/config) is not within the allowed path(s)`, But i have no access/secreat keys then why i am set this config.
Answer: In my case. I was trying through CLI.

I configured AWS cli by running

aws configure

Inside ~/.aws/credentials, look for session_token. If you have the session_token. Then update it.
Otherwise remove session_token and try again.

And Voila It worked!
Comment: Thank you, this worked for me when nothing else was working.
Comment: I deleted both "credentials" and "config" files found under the "~/.aws/" directory. Then I re ran the "aws configure" and it resolved the issue.
Comment: Thats what worked for me. Thanks.
Answer: You're getting this error because Hong Kong (ap-east-1) is not enabled by default. You need to enable it in the AWS console if you have the correct permissions, then try again.
See the AWS Docs for instructions on how to do that.
Answer: I am using s3cmd and based on the above answer I was able to solve by deleting the .s3cmd file and reconfiguring (s3cmd --configure), which cleared the access_token:
diff .s3cfg previous_s3cmd

< access_token =
---
> access_token = vkp1Jf\etc...

Also increased the chunk sizes:
< recv_chunk = 65536
---
> recv_chunk = 4096

< send_chunk = 65536
---
> send_chunk = 4096

And I guess I had previously opted for https requests, as opposed to the (current) default:
< use_https = True
---
> use_https = False

Success
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Answer: In my case, I had to update the aws configuration file.
You can follow the following steps.
In local command line terminal:

open ~./aws

Open the credentials file and update the values for the following 3 entries:

aws_access_key_id
aws_secret_access_key
aws_session_token

You can get these values from AWS console.
Hope it helps!!


Can a melodic minor chord be a pivot chord?

Question:

The image above shows, from Bach 371 Chorales, No. 3 "Ach Gott vom
Himmel sieh darein" in E Minor.

I think my analysis is totally wrong, but I just wrote it down on the image to try it out.
1. I can't figure it out why F#(G) in the first red box is used
I think that this F# is not for a secondary dominant, secondary diminished, borrowed chord, or modulation.  Can you use a chromatic passing tone like that? I think not.
So I don't know why that F# came out
(As a reference, 'F natural' didn't appear before.)
2. Second red box
I believe there is modulation in this part, but I cannot be sure.
If there is no modulation, I can't explain the progression in the last measure.
If there is modulation, can you use A melodic minor chord as a pivot chord?
Because there seems to be only the A melodic minor chord that can be the pivot chord.
(for exactly 'ascending melodic minor')
I can't figure it out.  So I would be very grateful if you could help.
Comment: @Tim harmonic analysis of Bach chorales is bread and butter of music theory.  You're right that not everything fits nicely into common practice tonal harmony, but it's not correct to say that Bach didn't write chordally -- rather, the Bach's chordal theory was somewhat different (principally, it didn't include the modern concept of inverted chords, so D-F-A and F-A-D were considered to be distinct chords).  I'll post an answer in a little while.
Comment: It may help to follow *each* line of music (satb) and decide why Bach wanted those notes to follow.
Comment: Partly based on the chat for the accepted answer, I have reason to believe that the Bach 371 Chorales, No. 3 "Ach Gott vom Himmel sieh darein" is, in fact, *not* in E minor as claimed, but in A minor with the "Dorian" key signature picked (see the Toccata and Fugue in D minor, BWV 538, "Dorian" for an infamous example of this tendency).
Comment: Bach didn't write chordally. He wrote separate lines, some of which do translate that way, but trying to dissect his work harmonically won't work too succssfully.
Comment: @tim lot of people here say like that and i kinda get it.. But I'd like to know if there's a way to proceed that I'm not aware of.
Comment: My understanding is that Bach chorales are arrangements of pre-existing melodies, not wholly original compositions.  So while Bach most definitely used functional tonal harmony, most/all(?) of those melodies predate it, so some of the weirdness is a result of harmonizing modal melodies.  IIRC.
Comment: @ibonyun that's all arrangement? i thought somehow I got a little bored this chorales. thanks to give a information
Comment: @Dekkadeci According to bach-chorales.com, you are correct.  (It's BWV 153.1)
Answer:
I can't figure it out why F#(G) in the first red box is used

There should be no accidental, since there's an F♯ in the key signature.  Both the Bach-Gesellschaft edition and the Neue Bach-Ausgabe print this F♯ without an accidental.  (Both are available at IMSLP; note that this chorale, unusually, appears at the beginning of the cantata.)
F♯ is diatonic to E minor, so this is just a regular passing tone.
For the note to be F-double-sharp, it would have to be a double-sharp sign, which looks like a small x.  Accidentals are not additive; this sharp sign is just a notational redundancy that doesn't change the music.

Second red box...

I would analyze the entire second phrase, starting on the fourth beat in the example, in A minor.  Lutheran chorales often have each phrase cadencing on a different degree of the scale, and it's frequently useful to consider the entire phrase relative to that scale degree.  The first chord of that phrase comprises the pitches G♯-B-D-F♮, so it is vii°7/iv.
Analyzing the phrase relative to A minor means that the E major chords are not i (which they can't be because i is minor) nor even I, but V/iv.  Similarly, the chord comprising B-D-F♮ is ii°/iv.  The E in the bass is an accented passing tone; the structural root is D.  This leads us to an analysis of the cadence, ignoring suspensions and other contrapuntal details, as ii♯6/iv - V/iv - iv, which can hardly be more standard.
If you're looking for a pivot chord, you might consider that the I chord at the end of the first phrase (beat 3 in the example) is reinterpreted as V/iv.
Finally, a word on terminology.  "Melodic minor" is a scale.  There's no such thing as a melodic minor chord, melodic minor key, or melodic minor tonality.  "Melodic minor" arises from the tendency for the sixth and seventh scale degrees to vary in minor keys -- sometimes one sees the major sixth or seventh and sometimes the minor.
It is also interesting to note that the melody here is in B phrygian, even though the harmonization is in E minor.  This provides another explanation for the frequent appearance of E major.
Comment: @guss2222 1: yes, the E and C are passing tones that you can ignore for the harmonic analysis. 2: ♯6 indicates that the sixth is raised. I was a bit confused when I added it, though; it's a major sixth but it is diatonic, so it isn't raised and the figure just be a plain 6. I will take it out. 3: ii/iv is followed here by V/iv, but yes, I am suggesting that it's better to analyze the entire 8-beat phrase using `X/iv` symbols; alternatively you can use `iv:` and then analyze in A minor.  4: there could be a source somewhere with F♯, though I can't imagine that it's anything other than a ...
Comment: @guss2222 ... mistake.  The only question is where the mistake first occurred.  It doesn't change the music; it's just redundant.  Accidentals are not additive, so it is not an F-double-sharp and therefore not a G.  5: but we don't talk about chords being "in a scale"; they may be "in a key."  So E minor is a chord in the key of A minor, and E major is also a chord in the key of A minor.  The same is true of D minor, D major, G major, and G♯ diminished.  All of these chords could be found in pieces in A minor for centuries before the melodic, harmonic, and natural minor scales were invented.
Comment: @guss2222 my point is that we don't speak of "pivot with A melodic minor scale."  A pivot chord is a chord that exists in two keys.  There's no "melodic minor" about it anywhere.
Comment: so you mean, (EDF♮B-DDF♮C-DDF♮D) is 'ii♯6/iv', (EG#EC) is V/iv, (EAEC) is iv64?
Comment: what is '#' in ii♯6/iv?
Comment: but if ii/iv come out, isn't will come out next is iv? or you talking modulation is happened from 'I' in first phrase?
Comment: and F# is just mis writed? that's shocking to me . i actually download that sheet in youtube.
Comment: and i know melodic minor is a scale but i mean just If the chord from E minor is pivot chord in A melodic minor scale, that can be modulation to A minor?
Comment: all thank you! but last answer. so you mean 'A melodic minor scale' is actually in 'A minor key', right? so if some another key's chord is pivot with A melodic minor scale (include F# and G#), that chord can modulation to A minor key right? that's my question
Comment: so how should say that? chord include F# and G#. is just pivot with A minor key ?
Comment: that's actually just exhausting question for you.. i get it you mean. and F#(G) mistake issue is if Without you, I would never have solved it. bless you!
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/136458/discussion-between-guss2222-and-phoog).
Answer: This section is more easily analyzed if one works backward from the end. The piece ends on an A minor chord preceded by a V, i64, and V yielding a common perfect cadence (as your analysis shows). While the notes are offset with each other (the first eighth note in the bass causes a bit of rhythmic displacement.) The final bar is V-i64-V-i; the i64 I a double suspension of a V7 and this pattern occurs quite often. (As a matter of terminology, "melodic minor" isn't used to describe a key, only a scale pattern within the minor-key complex.)
Because the piece is primarily contrapuntal in texture, there are lots of chord changes happening in a hurry. No real modulation occurs. (To have a "true" modulation, there needs to be an extended section that is clearly in another key with significant use of notes outside of A minor.) So the whole piece should be analyzed using A minor as the tonic. The music consists of four independent voices (as noted in the comments), a harmonic analysis may look rather complicated. It's not wrong though and may be helpful in understanding the way Bach used combinations of melodies to produce harmonic effects.
Note that analyzing the piece in A minor simplifies things though. The first measure starts with a V chord followed by V7 (the eighth note D creating a seventh chord.) The chords up to the first fermata are E6, E65, a, D43, E. In Roman Numerals, V6, V7, i, IV43, V. The sixth and seventh scale steps in minor are mutable; in the key of a minor, the lower forms F and G or upper forms F# and G# may be used as long as good voice-leading (i.e., "sounds good" in this style) is followed. Thus the harmonic analysis isn't affected by the D major chord instead of a D minor (IV43 rather than iv43.) Similarly for the VII to i63 that follows.
There are a couple of places (left as an exercise) where the movement in eighth notes is best analyzed as using a non-harmonic tone, usually creating a suspension.
One more notational funny (at least I did a double-take). The piece is in 4/4 (probably written as C) time but he last measure with the repeat sign has only 3 beats. It seems as if this is the (older?) notational style wherein the last measure is shortened if there is a pick-up measure. I'm guessing this piece starts with a 1-beat pickup measure.
Comment: you talking the modulation happened in the first measure? Or do you not know this Chorales in 'E Minor?'
Comment: The cadence shown in the example is not the end of the piece; it's the end of the "A" section -- the chorale is in [bar form](https://en.wikipedia.org/wiki/Bar_form).
Comment: I don't know the chorale. I just analyzed the posted section from the cadence backwards. If this is the tail of an E minor composition; I'd go with the A minor from the last beat of the first (posted) measure, as suggested above.
Comment: maybe i posted all wrong in the first place . actually this chorale start 'EEG#B'. first i saw this i think that just secondary dominant, but maybe just start as a A minor.


Administration menu affects theming

Question: I'm using the Admin menu in Drupal 7. I recently started developing a Zen sub-theme for my website and discovered something annoying.
My theme's heading background is a simple 200px high image (a brown rectangle) that repeates horizontally:
body {
  background: transparent url(../images/background.gif) repeat-x center top;
}
I created the theme while I was logged in and achieved this (first image on website). Happy as I was, I logged out and noticed with anger that my lay-out had changed (second image on website)!
I suspect that the admin menu 'pushes' all the content on my website to the bottom, while the background stays put. Is there a way to 'push the background to the bottom' aswell, so that I don't have to log out every time to see what happened when I changed my CSS theme code?
Answer: I also faced same problem with Zen theme and I think there is no professional fix for that.
Although, I have used the attributes not-logged-in and logged-in class'es (which will be available in body tag) to differentiate whether or not the user is logged and placed/adjusted my css accordingly.
Answer: Most themes provide div wrapper for their main content. In Skeleton Theme it's
 Simply use it (or create it if you don't have one yet) to attach backgrounds and position elements, and tools that needs space outside it would not interfere.
Comment: I more or less solved my problem by disabling 'Adjust top margin' in the section Configuration > Administration > Administration Menu. The upper side of my icon is now covered by the toolbar, but that's not a very big deal.

Administration menu affects theming