Python regex - split alphanumeric characters but remove and combine hyphenated words

Question: I am parsing over two million text files for a project and need your help with regex make it faster. I need to split the words so I am able to do it using:
text_file = re.sub(r'\W+',' ', text_file)

However, for hyphenated words, I would like to remove the hyphen but combine the word (i.e. non-profit would be nonprofit instead of "non", "profit"). Currently, I am able to do this with two search and replace (not ideal at all). The end output would be a list of words that will used by collections.Counter() object.
def mod1(text_file):
     text_file = re.sub('\s*-\s*', '', text_file) # handling hyphens :( O(n)
     text_file = re.sub(r'\W+',' ', text_file)
     text_file = text_file.lower()
     return(text_file)

Any faster processing tips to do this beyond regex would also be much appreciated. Thanks.
Comment: So you don't want to split, you only want to replace non-words characters with a space, (except for hyphen separated words). What is the expected result for `abc123 abc#123-123abc   -123abc-abc`? And do you want to obtain in fine a list or a string with each "words" separated by a space? Please edit your question with clear examples with expected results.
Comment: I need to obtain a list of words. Will edit it. Thanks.
Comment: Where are the clear examples?
Answer: After using lower(), you can use re.sub(r'([a-z])-([a-z])', r'\1\2', s) to remove hyphens.
Then re.split(r'[^a-z]'), to split the words.
s = s.lower()
s = re.sub(r'([a-z])-([a-z])', r'\1\2', s)
s = re.split(r'[^a-z]', s)

This solution may (or may not) be faster depending on the text it is used on. It does have some probable speed ups though:

Ignoring numbers allows usage of [a-z] instead of \w.
lower() being called first, allows re.sub() to search for half the letters; that is [a-z] instead of [A-Za-z].
Assuming no spaces before and after hyphens (since usually that should be the case) allows replacing of '\s*-\s*' with a faster [a-z]-[a-z]
Comment: Thanks. I am already using timeit to check and compare processing times. It seems like split and combine using regex has to be called twice regardless. Will see if there are more other solutions. Cheers!
Comment: @everestbaker Let me know of its speed. Also, my solution assumes you have no spaces in hyphenated words. Is that assumption incorrect?
Comment: Using the same list of sample text files, processing time went down to 0.064 secs compared to 0.081 for the original solution. Much faster. Thanks. Will wait and see if there are any other suggestions.
Comment: but this solution uses both re.split and re.sub
Comment: Agreed Avinash. I'm looking into it as I might have to use a much bigger sample size.
Comment: @AvinashRaj I am assuming his original solution is 0.064 now, because `\w` would check for numbers which is probably not needed. Also, i assume `'\s*-\s*'` would be much slower than `'([a-z])-([a-z])'`.
Comment: what if the spaces exists before and after to hyphens?
Comment: I used a Google ngram text file (~70 mb) and the processing time with @user5061's solution was faster by more than five seconds (21 secs vs 15.6 secs). Must add though ngrams are not the best sample text files. Will look into nltk corpora datasest to see if I find better ones.
Comment: @everestbaker Will you be having any `'word-\nword'` cases? If so you ll have to use `r'([a-z])-\n?([a-z])'` instead. Also, (even though i did check it) make sure my solution _does_ give the expected words.
Comment: Unfortunately, can't say for sure. I'll just have to make sure it captures most of the cases.
Comment: @user5061 **re.split(r'[^a-z]', s)** splits word starting with letters a-z but ignores numbers. Maybe we should change your answer to separate and include numbers as well i.e. 1985-apple gives "1985", "apple"
Comment: @everestbaker You said you need words, so i excluded numbers to make it faster. You can change that by making `re.split(r'[^a-z]', s)` to `re.split(r'\W+', s)`. Cases like `1984-apples` will still be separated though (but i dont think you'd need them united anyway).
Comment: Agreed, I should have been clearer. Adding \W+ does not help as I tested it out I changed it to re.split(r'[^0-9a-z]', s) for now. I will accept your answer. Thanks for your help.
Answer: You could do something like this,
>>> s = 'non-profit foo:?bar/buzz'
>>> ''.join(re.sub(r'\W+', ' ',i.strip()) for i in s.split('-'))
'nonprofit foo bar buzz'

OR
You could define a function in the re.sub's replacement part.
>>> s = 'non-profit foo:?bar/buzz'
>>> re.sub(r'\W+', lambda m: re.sub(r'.*', '', m.group(0)) if re.match(r'\s*-\s*$', m.group(0)) else re.sub(r'.*', r' ', m.group(0)), s)
'nonprofit foo bar buzz'

OR
Keep it simple...
>>> re.sub(r'\W+', lambda m: '' if re.match(r'\s*-\s*$', m.group(0)) else ' ', s)
'nonprofit foo bar buzz'

This code will do the replace for every match founded. That is, after each single match, function m is called. \W+ matches one or more non-word  characters. If a match is found, it will return an empty string if the matchobject.group(0) satisfies this \s*-\s*$ condition else it would return a single space string. That is the match was replaced with the return value.
Comment: I believe `split` complexity is O(n), so your solution has exactly the same complexity as the OP approach right?
Comment: Thanks Avinash. Your solution works but is much slower. For a list of text file (combined file size ~ 400kb), the read time was around 9.6 secs compared to 0.08 secs for calling regex twice.
Comment: Thanks. It's much faster than your previous solution (0.3 secs compared to 9.6 secs) but it's slower than the original approach (0.08 secs). It seems like split and combine are two different tasks and will need to be called twice. Let's see if we are wrong. Cheers!
Answer: You can use str.translate method , that is more faster than regex:
>>> import string
>>> s ='aslkdf -  sadlk#gjf'
>>> s.translate(str.maketrans(string.punctuation,32*' ','-'+string.whitespace))
'aslkdfsadlk gjf'

Answers benchmark :
#Avinash Raj
:~$ python3 -m timeit "import re;s='aslkdf -  sadlk#gjf';re.sub(r'\W+', lambda m: '' if re.match(r'\s*-\s*$', m.group(0)) else ' ', s)"
100000 loops, best of 3: 7.96 usec per loop
#Kasra
:~$ python3 -m timeit "import string;s='aslkdf -  sadlk#gjf';s.translate(str.maketrans(string.punctuation,32*' ','-'+string.whitespace))"
100000 loops, best of 3: 4.75 usec per loop
#user 5061
:~$ python3 -m timeit "import re;s='aslkdf -  sadlk#gjf';s=s.lower();s = re.sub(r'([a-z])-([a-z])', r'\1\2', s);re.split(r'[^a-z]', s)"
100000 loops, best of 3: 8.46 usec per loop

Comment: @Kasra Your tests are flawed. The string you are using to conduct tests is not representative of the "usual text". It has no upper case, no numbers etc. I tried timing some text from wikipedia; my solution is about 20% faster with some texts, and about 1% faster with other text. However since both of us don't know the actual text used our tests cant be that accurate.
Comment: Thanks Kasra. The processing speed is similar to the picked solution. Definitely food for thought in my next project to further explore str.translate. Cheers!
Comment: I am using large sample files so I feel confident with the proposed solution. Also, just to put everything in context, I'm building a word frequency program and calling 's' again in collections.counter provides a the count of single characters (A-Z) in your case in my output text file.
Comment: @Kasra I m not saying my solution is better. I m saying that only the asker could test and get reliable results and determine which one is faster.
Comment: Thanks for the str.translate tip. Will look into it further later because I am getting a TypeError: translate() takes exactly one argument (2 given).
Comment: @everestbaker ahan sorry i miss that you are in python3, fixed! check out the edit!
Comment: @everestbaker Welcome, I edited the code its now more faster! you don't need `zip_longest`!!!! ;)
Comment: @everestbaker anyway if you find an answer useful you can tell this to community with voting!;)


Solve $x=C \log(C \log(x+A)+B)$

Question: Is it possible to resolve an equation of the type
$$x=C\log{(C\log{(x+A)}+B)}$$
(where $A$, $B$, and $C$ are real-valued parameters) for $x$?
As far as I can see, the function on the right hand side is concave for positive $C$ (for negative $C$ it may be partly convex, partly concave) and the equation has two real solutions if $C$ is large enough (consider $C=10$, $A=1$, $B=1$), otherwise none. (For negative $C$, the equation may have one real solution.)
My attempts to resolve the equation for $x$ were not very successful and Wolframalpha does not give a solution either (though it computes the numeric solutions for any specific parameter set). From what I gathered, even the solution of a similar function with a single logarithm (not the double one as above) involves the transformation into a form with Lambert's W function which is then impossibly difficult to work with (to further resolve the argument if you have another logarithm in there...) and which may even be impossible to derive if there is another logarithm in the intended argument (?).
Is there another possibility to solve the above equation or does this mean, that the solution is either currently or generally not possible? If so, is there scientific literature dealing with this problem (either saying why this or a similar problem is impossible to resolve or detailing what would have to be done in order to be able to solve it)?
Answer: This answer is for solving the equation by closed-form solutions. A closed-form function is a function from a given set of allowed functions.
The elementary functions according to Liouville and Ritt are those functions which are obtained in a finite number of steps by performing only algebraic operations, exponentials and/or logarithms. A Liouvillian function is an elementary function or (recursively) the integral of a Liouvillian function, e.g. the nonelementary integral of an elementary function.
Assume an ordinary equation $F(x)=c$ is given where $c$ is a constant and $F$ is a function. Isolating $x$ on one side of the equation only by operations to the whole equation means to apply a suitable partial inverse function $F^{-1}$ of $F$: $\ x=F^{-1}(c)$.
The problem of existence of elementary inverses of elementary functions is solved in Ritt, J. F.: Elementary functions and their inverses. Trans. Amer. Math. Soc. 27 (1925) (1) 68-90. Unfortunately your equation doesn't seem to be able to be brought into the form of an equation which is solvable through elementary functions.
The elementary functions form a differential field, and the Liouvillian functions form also a differential field. But you also can take other differential fields. The problem of solving a given equation by solutions from a differential field is solved in Rosenlicht, M.: Liouville's theorem in Differential Algebra. Publications Mathématiques de l'IHÉS. 36 (1969) 15-22. Unfortunately your equation doesn't seem to be able to be brought into the form of an equation which is solvable through elementary functions.
For applying only Lambert W and elementary functions, your equation should be in the form
$$f_1(f_2(x)e^{f_2(x)})=c,\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (1)$$
where $c$ is a constant and $f_1$ and $f_2$ are elementary functions with a suitable elementary partial inverse. Unfortunately your equation doesn't seem to be able to be brought into this form.

Solve $x=C \log(C \log(x+A)+B)$


Discrete mathmatic how many ways

Question: In how many ways can 13 people be placed at 13 (distinguishable) chairs around
a round table, without Suzy and Johnny (which is among the 13) sits next
each other?
I am confused should i use bioniomial theorem or?
Comment: Do the problem with 12 "people", one of which is the two (Suzy,Johnny) thought of as one "person", which afterwards goes in one of the two possible orders.
Answer: There are $13$ ways to seat the first person, $12$ for the second, etc. So a priori we have $13!$ ways to seat everyone. However we have included the illegal arrangements where John and Suzy are next to each other. We should count the cases where they are together and subtract that from $13!$.
We can place Suzy in any of $13$ chairs. Then John must be either to her left or right, so he has $2$ possibilities. Then there are $11!$ ways to seat the rest of the people.
There will be $13!-13\cdot 2\cdot 11!$ possibilities.
Comment: We counted $13!$ ways to seat them if we ignore the requirement that John and Suzy can't be together. So we count the number of "illegal" arrangements and delete them from the total number of ways to seat everyone.
Comment: Why must we subtract from 13! Elliot G??
Answer: Here is an alternative solution:
We first choose where Suzy sits, of which there are $13$ possibilities.
Then we choose where Johnny sits. Johnny can't sit where Suzy is sitting and he can't sit to the left or right of Suzy, so there are $10$ possibilities.
Then we seat the remaining $11$ people, of which there are $11$! ways.
Thus, there are $13 * 10 * 11!$ possibilities.

Discrete mathmatic how many ways


Google BigQuery Node SDK Load Job failing with no error

Question: We are attempting to use the node.js google_cloud sdk to load data into BigQuery from Google Cloud Storage. Here is a snippet of the Load code:
<code>//init Google Cloud storage
var gcs = require("@google-cloud/storage")({
  projectId: bq_projectId,
  keyFilename: "./" + keyfile
});

//init Google BigQuery
var bq = require("@google-cloud/bigquery")({
  projectId: bq_projectId,
  keyFilename: "./" + keyfile
});

const datasetId = "my-dataset";
const tableId = "data_load";
const fileName = "data-20170518-082238.csv";

bq
  .dataset(datasetId)
  .table(tableId)
  .import(gcs.bucket(bucketName).file(fileName), (err, job, apiResponse) => {
    if (err){
      throw err
      return;
    }

    console.log(apiResponse);
    console.log(`Job ${job.id} started.`);
  });
</code>
The job kicks off and returns a jobId, no error is thrown and the apiResponse is:
<code>{ kind: 'bigquery#job',
  etag: '"TcVKUQ2ft7DS9Q8U3noJdmpEDQ4/tFIJTWM2yuacXB5EvzWR1ffuKig"',
  id: 'my-project:job_FTO4_Jb5ctr2oEy2IsDSAUCWrgw',
  selfLink: 'https://www.googleapis.com/bigquery/v2/projects/my-project/jobs/job_FTO4_Jb5ctr2oEy2IsDSAUCWrgw',
  jobReference:
   { projectId: 'my-project',
     jobId: 'job_FTO4_Jb5ctr2oEy2IsDSAUCWrgw' },
  configuration:
   { load:
      { sourceUris: [Object],
        schema: [Object],
        destinationTable: [Object],
        sourceFormat: 'CSV' } },
  status: { state: 'RUNNING' },
  statistics: { creationTime: '1495151832686', startTime: '1495151833523' },
  user_email: 'service-account' }
</code>
The exact same file loads fine from the BigQuery interface into the specified table.
Comment: the job is in RUNNING state. you need to use jobid that you've got and then periodically check for status of that job - till when you will get state DONE. when it is done you will see errors if any!
Comment: Ah yes, I just included the initial output, we are certainly waiting for the job to complete before checking the table. It completes successfully.
Answer: It turns out that errors in the BigQuery load process do not throw via callback or promise.catch.
In this case, you need to read the job.status.errors object in order to determine if there were errors in the load, here is some sample code:
<code>bq
  .dataset(datasetId)
  .table(tableId)
  .import(gcs.bucket(bucketName).file(fileName),{
    skipLeadingRows: 1
  })
  .then(results => {
    console.log("initial results:")
    console.log(results);
    const job = results[0];
    console.log(`Job ${job.id} started.`);
    return job.promise();
  })
  .then(results => {
    console.log("completed results:")
    console.log(results);

    const job = results[0];

    // determine if there are any errors in the completed job
    if(job.status.errorResult || (job.status.errors && job.status.errors.count > 0)) {
      console.error(JSON.stringify(job.status.errors));
      return;
    }

    console.log(`Job ${job.id} completed successfully`);
  })
  .catch(err => {
    console.error(err);
  });
</code>


Menu item path saved as node/nid instead of alias

Question: An alias supplied to the [Path] attribute of a Menu Link is converted to node/nid when saved, (along with a nice message: "The menu system stores system paths only, but will use the URL alias for display".)   This is implemented in core: function menu_edit_item_validate of menu.admin.inc.   (Drupal 7.22)
When moving nodes around between instances of the same site with node-export and uuid, node/nid is no longer valid, and results in menu links being dropped, or pointing to the wrong node.
A similar question was raised earlier:
How do I use URL aliases in menus?
But the solutions I think are for different use cases.  The suggestion of adding the alias and menu item together on node creation does not seem to work.   Our concern has nothing to do with how the node url is displayed.
UUID_path or maybe UUID_features seem to imply a solution, but if there is a resolution there, I don't understand it, and will appreciate any insight into how to handle this.
In the end, we need to have exportable menu links with path attributes that, one way or another stay, linked to an alias, regardless of nid.
Answer: Both nid and aliases are not guaranteed to be stable and unchanged, nid being a bit less prone to mutation. What you seem to really need, are UUID Links. It should allow you to store links by the unchangeable UUID, not localised nid.
There was also a sandbox project UUID Menu for just the case, but it seems it was never properly initiated.
The other approach is to export both content and menu as a Feature. This problem is in Features' issue queue and in UUIDs for Menu Links issue queue, unresolved yet. But in the second one patches exists.
Comment: project/uuid_Links covers a different use case, does not handle with menu_links.   We're pretty well committed to content export with node_export, not features.
Comment: project/uuid_Links is a different use case: does not handle menu_links.   We're committed to using node_export, not features, for content export.  Workaround so far: exporting the affected *menu* with features (not content), then editing the feature, replacing the nids with aliases.  Still, the menu_link parent is not respected when the feature is enabled, so we're having to specifically lookup and set the parent (menu_link_save) after the menu feature is enabled, despite setting plid.  In our case, our menu can be expected to be stable, can survive kludging, but wondering, what would you do?
Comment: I would find a way to store uuid links in menu. Write a module or hack for it if I have to. Portability is the very reason of uuid existence and there is no better approach at the moment.
Answer: This turned out to be pretty easy, once I finally understood what's going on.   We implemented our menu in an install hook, where we resolve uuid to normal path (node/nid):
<code>function _get_normal_path_for_uuid($uuid) {
  $nid = db_select('node', 'n')
    ->fields('n', array('nid'))
    ->condition('n.uuid', $uuid, '=')
    ->execute()
    ->fetchField();
  return 'node/' . $nid;
}
</code>
and here, and example of a single menu_link,
<code>$normal_path = _get_normal_path_for_uuid('8674c2c0-9ee4-4628-8666-e7eee4167250');
$menu_link = array(
  'menu_name' => 'products-menu',
  'link_path' => $normal_path,
  'router_path' => 'node/%',
  'link_title' => 'Resources',
  'options' => array(
    'attributes' => array(
      'title' => '',
    ),
  ),
  'module' => 'menu',
  'hidden' => 0,
  'external' => 0,
  'has_children' => 0,
  'expanded' => 0,
  'weight' => -48,
  'plid' => $some_plid,
);
menu_link_save($menu_link);
</code>
Now our menu links are pinned to specific nodes via uuid, in code, and content is fully portable between various site instances.   Whenever we import node_export content from one site instance to another (which results in the "normal path" changing), we simply reinstall this menu module.


Subscales (not items) as indicators of latent variables in SEM

Question: I have a sample (n=200) that I have collected questionnaire data from. Each participant will complete 5 questionnaires that capture different behaviours and all of my 5 questionnaires include between 10 and 27 items.
I am planning to use SEM to construct a model that investigates whether scores on 4 of my questionnaires (IVs) predict scores on the 5th questionnaire (DV). The IVs and DV will be treated as latent variables in the model, each with a number of indicators attached.
However, because my questionnaires have so many items I want to engage some sort of process of reduction so that I have less indicators in my SEM. I'm wondering how my rationale holds, as I'm new to this.
What I am considering at the moment is conducting an exploratory factor analysis and then confirmatory factor analysis on each of the 5 questionnaires separately to extract how many factors are captured by the items (lets say for example, that magically each questionnaire has 3 underlying factors). I would then sum the scores of the items that are associated with each of the 3 factors separately, so each participant now has 3 scores from each questionnaire rather than the 1 total sum score.
The 3 scores would then form 3 indicators used in the final SEM predictive model. In my predictive SEM model, I would have 5 'latent variables', each loading on 3 indicators.
I guess the central question is: If you have in excess of 10-items associated with a latent variable, is it acceptable to use derived factor scores as indicators in SEM rather than the original items?
Answer: Yes, is the short answer.
This is frequently done in my field (psychology). Your model complexity will be too high otherwise, and it will (probably) not converge.  You're not testing the assumptions of things like unidimensionality that you'd like to test - but that's OK, it might be the case that you're not even interested in that (or that it's irrelevant - your indicators might be causal).
An approach that's sometimes recommended (it's recommended far more than it's used) is to use your total score as a single indicator, and constrain the loading to be the reliability.
Answer: A cool decade later and I'm currently working on a similar analysis. If you happen to see this, I'm curious what you ended up doing and how you ended up justifying the use of summed scores?
Comment: This does not really answer the question. If you have a different question, you can ask it by clicking [Ask Question](https://stats.stackexchange.com/questions/ask). To get notified when this question gets new answers, you can [follow this question](https://meta.stackexchange.com/q/345661). Once you have enough [reputation](https://stats.stackexchange.com/help/whats-reputation), you can also [add a bounty](https://stats.stackexchange.com/help/privileges/set-bounties) to draw more attention to this question. - [From Review](/review/late-answers/320186)


Twitter Oauth Failed due to signature and token

Question: I am trying to crate a application to authenticate from twitter but am getting the following error
:
Failed to validate oauth signature and token

Here is my code for app:
public class TwitCons {

    String sign_method ="HMAC-SHA1";

    public static String consumer_key ="^^^^^^^^^^^^^^^^^^^^";
    public static String consumer_secret="^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^";
    String request_uri = "https://api.twitter.com/oauth/request_token";
    String oauth_callback="http://127.0.0.1:8080/twitter/getToken";

    String getNonce(){
        Random ran = new Random();
        long nonce1 = ran.nextLong();

        Long nonce = Math.abs(new Long(nonce1));

        return nonce.toString();
    }

    String getTimestamp(){
        long time = System.currentTimeMillis();

        return String.valueOf(time/1000);
    }

    String getSignature(String request_uri ,String request_token,String nonce) throws UnsupportedEncodingException
    {
        String base="oauth_callback=http://127.0.0.1:8080/twitter/getToken&oauth_consumer_key="+consumer_key+"&oauth_nonce="+nonce+"&oauth_signature_method="+sign_method+"&oauth_timestamp="+getTimestamp()+"&oauth_token=&oauth_version=1.0";

        String baseString="POST&"+URLEncoder.encode(request_uri , "UTF-8")+"&"+URLEncoder.encode(base , "UTF-8");

        String signKey = consumer_secret +"&"+ request_token ;
        System.out.println(signKey +"\n"+baseString);

        try {
            SecretKeySpec signingKey = new SecretKeySpec(signKey.getBytes(),"HmacSHA1");
            Mac mac = Mac.getInstance(signingKey.getAlgorithm());
            mac.init(signingKey);
            byte[] rawHmac = mac.doFinal(baseString.getBytes());
            String result = new String(Base64.encodeBase64(rawHmac));
            System.out.println(URLEncoder.encode(result , "UTF-8"));
            return URLEncoder.encode(result , "UTF-8");
            //return result;

        } catch (GeneralSecurityException e) {
            return "";
        }

    }

}

and
public class Twitter extends TwitCons{

    public void getRequest() throws IllegalArgumentException, HttpException, IOException
    {

        HttpClient client = new HttpClient();
        PostMethod post = new PostMethod("https://api.twitter.com/oauth/request_token");
        String nonce=getNonce();
        String header = "oauth_callback=\""+URLEncoder.encode(oauth_callback ,"UTF-8")+"\","+
            "oauth_consumer_key=\""+consumer_key +"\","+
            "oauth_nonce=\""+nonce + "\","+
            "oauth_signature=\""+getSignature(request_uri,"",nonce) + "\","+
            "oauth_signature_method=\""+sign_method + "\","+
            "oauth_timestamp=\""+getTimestamp() + "\","+
            "oauth_version=\"1.0\""  ;

        post.addRequestHeader("Authorization", "OAuth "+header);
        client.executeMethod(post);

        System.out.println(post.getResponseBodyAsString()+"\n"+post.getRequestHeader("Authorization"));

    }

Class has method getRequest for making the twitter request for getting request_token and token_secret
and second class TwitCons have different methods and constants to get request token
Answer: In Twitter.getRequest(), it seems that your header looks wrong. The value of each parameter has to be surrounded by doubles quotes (") :
StringBuilder headerBuilder = new StringBuilder("OAuth ");

headerBuilder.append("oauth_callback=\"").append(oauth_callback).append("\", ");
headerBuilder.append("oauth_consumer_key=\"").append(consumer_key).append("\", ");
headerBuilder.append("oauth_nonce=\"").append(nonce).append("\", ");
headerBuilder.append("oauth_signature=\"").append(getSignature(request_uri,"",nonce)).append("\", ");
headerBuilder.append("oauth_signature_method=\"").append(sign_method).append("\", ");
headerBuilder.append("oauth_timestamp=\"").append(getTimestamp()).append("\", ");
headerBuilder.append("oauth_version=\"1.0\"");

String header = headerBuilder.toString();

See the corresponding page in the Twitter API for more details : https://dev.twitter.com/docs/auth/authorizing-request
EDIT : code fixed because there is not any spaces before commas.
Comment: i have chnages the code and added " ...as you said..but still not working
Comment: After reading how to make the header again, there's no spaces before commas. This is my fault. I edited the code.
Answer: You're missing the oauth_token in your String base. See https://dev.twitter.com/docs/auth/3-legged-authorization for how to get one (if you do not already know this and have not just forgotten putting it in your base string ;)).
It could be that your signature is calculated wrong as your including &oauth_token= in your base string, but that key is not part of your request. The OAuth 1.0 specification just says:

The Signature Base String is a consistent reproducible concatenation of the request elements into a single string.

Could be worth a try to just remove that part of the string.
UPDATE:
You probably have to urlencode your callback URL in the Authorization header, have alphabetical ordering and put the parameters in quotes as @air-dex recommended. Try this:
String oauth_callback = "http://127.0.0.1:8080/twitter/getToken";
String oauth_callback_encoded = URLEncoder.encode(oauth_callback, "UTF-8");

then
String header = "oauth_callback=\"" + oauth_callback_encoded + "\", " +
            "oauth_consumer_key=\"" + consumer_key + "\", " +
            "oauth_nonce=\"" + nonce + "\", " +
            "oauth_signature=\"" + getSignature(request_uri, "", nonce) + "\", " +
            "oauth_signature_method=\"" + sign_method + "\", " +
            "oauth_timestamp=\"" + getTimestamp() + "\", " +
            "oauth_version=\"1.0\"";

and
String base = "oauth_consumer_key=" + consumer_key +
            "&oauth_nonce=" + nonce +
            "&oauth_signature_method=" + sign_method +
            "&oauth_timestamp=" + getTimestamp() +
            "&oauth_version=1.0";

Comment: Wrong. The `oauth_token` is required for all the authenticated requests but the `POST https://api.twitter.com/oauth/request_token`. It is logical since you are asking for an OAuth token. ;-)
Comment: Damn it, I confused it with the OAuth2 flow where you first redirect the user and then request the access token ;) I will update the post with another suggestion...
Comment: I updated my answer another time with a new shot on your problem ;)
Comment: Is the code in your question your current code? Because the base string there is still not right... Is your callback entry correct in the Twitter App Settings? Further try [checking your system clock](http://blainegarrett.com/2009/07/14/failed-to-validate-oauth-signature-and-token-on-twitter-oauth-check-your-cloc/)
Comment: @Jan Gerlinger : right for the percent encoding. I saw it reading again how to make this header on the Twitter API documentation.
Comment: There's still a `oauth_callback=http://127.0.0.1:8080/twitter/getToken` and a `&oauth_token=` too much in the `base` string. I also missed the spaces before the commas ;)


finding residues for poles

Question: I'm struggling to find the residues of the equation
$$
\frac{-z\ln(z)}{(z^2+a^2)(2-z)},
$$
with poles at $z=\pm ai$ and $z=2$.
I have the residue for $z=2$ as
$$
\frac{-2\ln2}{4+a^2},
$$
but I am struggling to find the residues for $\pm ai$.
Could anyone help me?
For $+ai$ I have
$$
\frac{-ai\ln(ai)}{2-ai},
$$
and for $-ai$, I have
$$
\frac{ai\ln(-ai)}{2+ai}
$$
but this doesn't give me the result I need.
Comment: Since $z^2+a^2=(z-a i)(z+a i)$ you are missing a term $2a i$ in the denominator for the residue at $a i$. Assuming $a i\ne \pm 2.$ Similarly, missing a term $-2a i$ in the residue at $-a i.$
Comment: which branch of $\log$ did u choose?
Comment: the branch is along the negative real axis, using a keyhole contour
Comment: ok, and which result do u expect?
Comment: yes that is exactly what i'm missing thank you!!!!!! :)
Comment: ................................................... trying to delete duplicate comment of mine!
Answer: The problem with finding residues at $z=\pm i$ for this function is that the result depends on the branch cut chosen for the log.  For example, if you are using residues for evaluating an integral, the contour used in the contour integral will dictate the branch cut.  For example, if you were using a keyhole contour about the positive real axis, you would set $i=e^{i \pi/2}$ and $-i = e^{i 3 \pi/2}$.  In that case,
$$\log{(i a)} = \log{a} + i \frac{\pi}{2} $$
$$\log{(-i a)} = \log{a} + i \frac{3 \pi}{2}$$
On the other hand, if you are using a keyhole contour about the negative real axis, then you would set $-i= e^{-i \pi/2}$.

finding residues for poles


how to wait for a specified time until a command gives a desired output

Question: I am writing a small python script to wait for a max of 1200 secs for devices to come up ,what is the best way to accomplish this?
command to run
device.exe list

Output when no devices attached:
List of devices attached

Output When devices attached
List of devices attached
device#1
device#2
device#3

Answer: You might be interested in the retrying library:
https://pypi.python.org/pypi/retrying
You will likely need some combinations of stop_max_delay and retry_on_result keyword arguments with the @retry decorator.


Whats the best way to securely publish a site post build?

Question: So, in your experience, whats the best way? Is there a secure way that's also scriptable/triggerable in a build automation tool?
Edit: I should mention this is windows/.net and I'll be deploying to iis6
Answer: For some projects I use Capistrano to push out to live. It is built on top of ruby and makes deploy script writing super easy and uses ssh.
On other projects I have a tiny deploy app that uses bash to do an svn export to a temporary directory and then rsync it over to the live server. You can make rsync use ssh.
I greatly prefer the Capistrano method, even if your project isn't in ruby/rails.
Answer: This seems like the sort of thing that could be done easily with SFTP. Take a look at PuTTY (psftp and pscp) or WinSCP for Windows, or rsync and OpenSSH for Unixes.
Answer: Make a copy of your live site directory, use rsync to update that copy with your latest version, then rename the live and updated directories so that the updated version is now live.
In bash:
#!/bin/bash

set -e
cp -R /var/livesite /var/newversion
rsync user@devserver:/var/readytogolive /var/newversion
mv /var/livesite /var/oldlivesite
mv /var/newversion /var/livesite

Viola!
Edit: @Ted Percival - That's a good idea. I didn't even know about "set -e". Updated script. Edit: updated again at Ted's suggestion (although I think it would still work if somehow the cp command failed, and if cp fails you probably have more serious problems.)
Answer: @Neall, I'd add a set -e on the second line, because you don't want the live site being replaced if the rsync fails for any reason. set -e causes the script to exit if any of its commands fail.
Edit: The set -e should be the first thing in the script, right after #!/bin/bash.
Answer: I'll second the recommendation for Capistrano, though if you're looking for a GUI-based solution you could try the Webistrano front end.  Clean, ssh-based, sane deployment and rollback semantics and easy scripting and extensibility via ruby.
Answer: You could always write a small client/server app that encrypts at the source, pushes the files, and then decrypts at the destination. That's a little bit of work, but probably a trivial amount. And it's scriptable as long as your automation tool supports executing something in the file system (which I think all do).
The only downside is that you may not be able to get meaningful error messages on failure in your integration environment without a bit more work on your part (though depending on your setup, this could be as simple as sending error messages to stdout).
Answer: hm, around here we use a staging "server" for testing purposes on the live environment (actually, its an apache virtual host on the production server) and araxis merge (a really smart line-by-line file comparison tool) to sync development and staging.
once its tested, just; replace the files on the production webroot :)
/mp
Answer: On a freelance job I did, we set up three seperate enviroments.

A Dev server, that ran continues builds using CruiseControl. Any check-in would trigger a build. QA Testing was done here.
A Test Server, that user acceptance testing was done on.
Production.

The workflow was as followed:

Developer checks in changes to SourceControl.
CruiseControl builds and deploys the build to Dev.
Dev is QA'ed
After passing QA, a robocopy script is ran that deploys the Dev build to Test.
Test is UAT'ed
After Test passes, a robocopy script is ran that deploys Test to PRD.


error message when installing drupal 8

Question: When I was installing Drupal 8 (Macos, nginx, php7.0, mysql5.6), the following error occurred, Can someone help me?
An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /core/install.php?langcode=zh-
hans&profile=standard&continue=1&id=1&op=do_nojs&op=do
StatusText: OK
ResponseText:
Warning:  Declaration of
Drupal\Core\DependencyInjection\ContainerBuilder::callMethod($service,
$call) should be compatible with Symfony\Component\DependencyInjection\ContainerBuilder::callMethod($service, $call, SplObjectStorage $inlinedDefinitions) in /Users/xile/code/drupal/core/lib/Drupal/Core/DependencyInjection/ContainerBuilder.php on line 22
Drupal\Core\Config\UnmetDependenciesException: Configuration objects (block.block.bartik_help, block.block.bartik_search,
block.block.seven_help, contact.form.feedback,
core.entity_form_display.node.article.default,
core.entity_form_display.node.page.default,
core.entity_form_display.user.user.default,
core.entity_view_display.node.article.default,
core.entity_view_display.node.article.teaser,
core.entity_view_display.user.user.compact,
core.entity_view_display.user.user.default,
field.field.node.article.field_image,
field.field.user.user.user_picture, field.storage.node.field_image,
field.storage.node.field_tags, field.storage.user.user_picture,
rdf.mapping.comment.comment, rdf.mapping.node.article,
rdf.mapping.node.page, rdf.mapping.taxonomy_term.tags,
taxonomy.vocabulary.tags) provided by standard have unmet dependencies in Drupal\Core\Config\UnmetDependenciesException::create()(line 89 of /Users/xile/code/drupal/core/lib/Drupal/Core/Config/UnmetDependenciesException.php).

Answer: Install drupal console and then try to install the site using drupal console
https://drupalconsole.com/articles/how-to-install-drupal-console
drupal si

error message when installing drupal 8


Get OData or Atom Service feed for SSRS 2008 R2 Shared Dataset

Question: Is there anyway to get a shared dataset to simply dump it's contents in an Atom or OData feed that can then be consumed somewhere else?  In our scenario we want to build just the datasets and let users consume them from PowerPivot or elsewhere.
Thanks,
Brent
Answer: You might find this useful:
http://technet.microsoft.com/en-us/library/ee240754.aspx
Comment: I know I can get the feeds from a report.  I was curious as to whether or not there is a way to run just the dataset to get the data.  It appears there is not.
Comment: I was just looking for this also see: http://www.odata.org/producers "Microsoft SQL Server 2008 R2 Reporting Services can expose data from reports as OData."
Answer: It does not look like SSRS can easily consume OData sources but that they are considering it.
http://connect.microsoft.com/SQLServer/feedback/details/559570/ssrs-should-be-able-to-use-odata-as-a-data-source
In the following article Mattias suggests a workaround that might meet your needs by using the XML data source and embedding your OData query in the URL
http://mskold.blogspot.com/2011/05/tfs-odata-services-as-data-source.html
Comment: I don't want to consume OData from a report... I want to expose it from a shared data set.


Affinity Propagation (sklearn) - strange behavior

Question: Trying to use affinity propagation for a simple clustering task:
from sklearn.cluster import AffinityPropagation
c = [[0], [0], [0], [0], [0], [0], [0], [0]]
af = AffinityPropagation (affinity = 'euclidean').fit (c)
print (af.labels_)

I get this strange result:
[0 1 0 1 2 1 1 0]
I would expect to have all samples in the same cluster, like in this case:
c = [[0], [0], [0]]
af = AffinityPropagation (affinity = 'euclidean').fit (c)
print (af.labels_)

which indeed puts all samples in the same cluster:
[0 0 0]
What am I missing?
Thanks
Answer: I believe this is because your problem is essentially ill-posed (you pass lots of the same point to an algorithm which is trying to find similarity between different points). AffinityPropagation is doing matrix math under the hood, and your similarity matrix (which is all zeros) is nastily degenerate. In order to not error out, the implementation adds a small random matrix to the similarity matrix, preventing the algorithm from quitting when it encounters two of the same point.


Optifine causes crash: java.lang.IllegalStateException: Lock is no longer valid

Question: Recently, I have started a modded survival with a decent number of mods. When I loaded the world, everything was A Ok. That is until I tried to rejoin, giving out the error java.lang.IllegalStateException: Lock is no longer valid. I've looked on other websites and it seems that Optifine is what is causing the crash. I depend on Optifine when it comes to modded Minecraft.
How could I possibly fix this problem while using Optifine also?
Here is the crash report:
java.lang.IllegalStateException: Lock is no longer valid
at net.minecraft.world.storage.SaveFormat$LevelSave.func_237301_i_(SaveFormat.java:296) ~[?:?] {re:mixin,pl:accesstransformer:B,re:computing_frames,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.world.storage.SaveFormat$LevelSave.func_237298_f_(SaveFormat.java:360) ~[?:?] {re:mixin,pl:accesstransformer:B,re:computing_frames,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
at net.minecraft.server.MinecraftServer.func_184109_z(MinecraftServer.java:769) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:common.MixinMinecraftServer,pl:mixin:APP:imm_ptl_mixins.json:common.portal_generation.MixinMinecraftServer_P,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.alternate_dimension.MixinMinecraftServer_A,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.altius_world.MixinMinecraftServer_A,pl:mixin:A}
at net.minecraft.server.MinecraftServer.func_184106_y(MinecraftServer.java:763) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:common.MixinMinecraftServer,pl:mixin:APP:imm_ptl_mixins.json:common.portal_generation.MixinMinecraftServer_P,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.alternate_dimension.MixinMinecraftServer_A,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.altius_world.MixinMinecraftServer_A,pl:mixin:A}
at net.minecraft.client.renderer.GameRenderer.func_195458_a(GameRenderer.java:699) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,xf:OptiFine:default,xf:fml:xaerominimap:xaero_gamerenderer_resetprojectionmatrix,xf:fml:xaerominimap:xaero_gamerenderer_renderworld,pl:mixin:APP:imm_ptl_mixins.json:client.block_manipulation.MixinGameRenderer_B,pl:mixin:APP:imm_ptl_mixins.json:client.render.MixinGameRenderer,pl:mixin:A}
at net.minecraft.client.Minecraft.func_195542_b(Minecraft.java:976) [?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:computing_frames,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:client.MixinMinecraftClient,pl:mixin:APP:imm_ptl_mixins.json:client.block_manipulation.MixinMinecraftClient_B,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.client.Minecraft.func_99999_d(Minecraft.java:607) [?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:computing_frames,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:client.MixinMinecraftClient,pl:mixin:APP:imm_ptl_mixins.json:client.block_manipulation.MixinMinecraftClient_B,pl:mixin:A,pl:runtimedistcleaner:A}
at net.minecraft.client.main.Main.main(Main.java:184) [?:?] {re:classloading,pl:runtimedistcleaner:A}
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_51] {}
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_51] {}
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_51] {}
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_51] {}
at net.minecraftforge.fml.loading.FMLClientLaunchProvider.lambda$launchService$0(FMLClientLaunchProvider.java:51) [forge-1.16.4-35.1.37.jar:35.1] {}
at net.minecraftforge.fml.loading.FMLClientLaunchProvider$$Lambda$508/435034854.call(Unknown Source) [forge-1.16.4-35.1.37.jar:35.1] {}
at cpw.mods.modlauncher.LaunchServiceHandlerDecorator.launch(LaunchServiceHandlerDecorator.java:37) [modlauncher-8.0.9.jar:?] {}
at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:54) [modlauncher-8.0.9.jar:?] {}
at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:72) [modlauncher-8.0.9.jar:?] {}
at cpw.mods.modlauncher.Launcher.run(Launcher.java:82) [modlauncher-8.0.9.jar:?] {}
at cpw.mods.modlauncher.Launcher.main(Launcher.java:66) [modlauncher-8.0.9.jar:?] {}

A detailed walkthrough of the error, its code path and all known details is as follows:

Thread: Render thread
Stacktrace:
    at net.minecraft.world.storage.SaveFormat$LevelSave.func_237301_i_(SaveFormat.java:296) ~[?:?] {re:mixin,pl:accesstransformer:B,re:computing_frames,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
    at net.minecraft.world.storage.SaveFormat$LevelSave.func_237298_f_(SaveFormat.java:360) ~[?:?] {re:mixin,pl:accesstransformer:B,re:computing_frames,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B}
    at net.minecraft.server.MinecraftServer.func_184109_z(MinecraftServer.java:769) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:common.MixinMinecraftServer,pl:mixin:APP:imm_ptl_mixins.json:common.portal_generation.MixinMinecraftServer_P,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.alternate_dimension.MixinMinecraftServer_A,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.altius_world.MixinMinecraftServer_A,pl:mixin:A}
    at net.minecraft.server.MinecraftServer.func_184106_y(MinecraftServer.java:763) ~[?:?] {re:mixin,pl:accesstransformer:B,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:common.MixinMinecraftServer,pl:mixin:APP:imm_ptl_mixins.json:common.portal_generation.MixinMinecraftServer_P,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.alternate_dimension.MixinMinecraftServer_A,pl:mixin:APP:imm_ptl_peripheral_mixins.json:common.altius_world.MixinMinecraftServer_A,pl:mixin:A}
-- Affected level --
Details:
    All players: 1 total; [ClientPlayerEntity['Pale_Gray'/158, l='ClientWorld minecraft:overworld', x=8.50, y=65.00, z=8.50]]
    Chunk stats: Client Chunks (ImmPtl) 0
    Level dimension: minecraft:overworld
    Level spawn location: World: (8,64,8), Chunk: (at 8,4,8 in 0,0; contains blocks 0,0,0 to 15,255,15), Region: (0,0; contains chunks 0,0 to 31,31, blocks 0,0,0 to 511,255,511)
    Level time: 0 game time, 0 day time
    Server brand: ~~NULL~~
    Server type: Integrated singleplayer server
Stacktrace:
    at net.minecraft.client.world.ClientWorld.func_72914_a(ClientWorld.java:617) ~[?:?] {re:mixin,re:classloading,xf:OptiFine:default,xf:fml:xaerominimap:xaero_clientworldclass,xf:fml:xaeroworldmap:xaero_wm_clientworldclass,pl:mixin:A}
    at net.minecraft.client.Minecraft.func_71396_d(Minecraft.java:2024) [?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:computing_frames,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:client.MixinMinecraftClient,pl:mixin:APP:imm_ptl_mixins.json:client.block_manipulation.MixinMinecraftClient_B,pl:mixin:A,pl:runtimedistcleaner:A}
    at net.minecraft.client.Minecraft.func_99999_d(Minecraft.java:628) [?:?] {re:mixin,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:computing_frames,pl:accesstransformer:B,pl:runtimedistcleaner:A,re:classloading,pl:accesstransformer:B,pl:mixin:APP:imm_ptl_mixins.json:client.MixinMinecraftClient,pl:mixin:APP:imm_ptl_mixins.json:client.block_manipulation.MixinMinecraftClient_B,pl:mixin:A,pl:runtimedistcleaner:A}
    at net.minecraft.client.main.Main.main(Main.java:184) [?:?] {re:classloading,pl:runtimedistcleaner:A}
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_51] {}
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_51] {}
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_51] {}
    at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_51] {}
    at net.minecraftforge.fml.loading.FMLClientLaunchProvider.lambda$launchService$0(FMLClientLaunchProvider.java:51) [forge-1.16.4-35.1.37.jar:35.1] {}
    at net.minecraftforge.fml.loading.FMLClientLaunchProvider$$Lambda$508/435034854.call(Unknown Source) [forge-1.16.4-35.1.37.jar:35.1] {}
    at cpw.mods.modlauncher.LaunchServiceHandlerDecorator.launch(LaunchServiceHandlerDecorator.java:37) [modlauncher-8.0.9.jar:?] {}
    at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:54) [modlauncher-8.0.9.jar:?] {}
    at cpw.mods.modlauncher.LaunchServiceHandler.launch(LaunchServiceHandler.java:72) [modlauncher-8.0.9.jar:?] {}
    at cpw.mods.modlauncher.Launcher.run(Launcher.java:82) [modlauncher-8.0.9.jar:?] {}
    at cpw.mods.modlauncher.Launcher.main(Launcher.java:66) [modlauncher-8.0.9.jar:?] {}

-- System Details --
Details:
    Minecraft Version: 1.16.4
    Minecraft Version ID: 1.16.4
    Operating System: Windows 10 (amd64) version 10.0
    Java Version: 1.8.0_51, Oracle Corporation
    Java VM Version: Java HotSpot(TM) 64-Bit Server VM (mixed mode), Oracle Corporation
    Memory: 1435255904 bytes (1368 MB) / 4261412864 bytes (4064 MB) up to 4294967296 bytes (4096 MB)
    CPUs: 4
    JVM Flags: 9 total; -XX:HeapDumpPath=MojangTricksIntelDriversForPerformance_javaw.exe_minecraft.exe.heapdump -Xss1M -Xmx4G -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:G1NewSizePercent=20 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=50 -XX:G1HeapRegionSize=32M
    ModLauncher: 8.0.9+86+master.3cf110c
    ModLauncher launch target: fmlclient
    ModLauncher naming: srg
    ModLauncher services:
        /mixin-0.8.2.jar mixin PLUGINSERVICE
        /eventbus-4.0.0.jar eventbus PLUGINSERVICE
        /forge-1.16.4-35.1.37.jar object_holder_definalize PLUGINSERVICE
        /forge-1.16.4-35.1.37.jar runtime_enum_extender PLUGINSERVICE
        /accesstransformers-3.0.1.jar accesstransformer PLUGINSERVICE
        /forge-1.16.4-35.1.37.jar capability_inject_definalize PLUGINSERVICE
        /forge-1.16.4-35.1.37.jar runtimedistcleaner PLUGINSERVICE
        /mixin-0.8.2.jar mixin TRANSFORMATIONSERVICE
        /OptiFine_1.16.4_HD_U_G6.jar OptiFine TRANSFORMATIONSERVICE
        /OptiForge-MC1.16.4-0.4.6.jar optiforge TRANSFORMATIONSERVICE
        /forge-1.16.4-35.1.37.jar fml TRANSFORMATIONSERVICE
        /OptiForge-MC1.16.4-0.4.6.jar FakeOptiFine TRANSFORMATIONSERVICE
        /MixinBootstrap-1.0.5.jar mixinbootstrap TRANSFORMATIONSERVICE
        /OptiForge-MC1.16.4-0.4.6.jar optiforgewrapper TRANSFORMATIONSERVICE
    FML: 35.1
    Forge: net.minecraftforge:35.1.37
    FML Language Providers:
        firstname.lastname@example.com
        minecraft@1
    Mod List:

Answer: Try logging into the world without Optifine, walking away from where the creeper exploded, then save the world, then come back into with Optifine loaded. I'm no Java developer, so I'm not sure what this error message means, but it sounds something like it ran out of video memory. Optifine is really buggy around particles in 1.16, so it may be something like that. If I use shaders, I have issues with particles. And I have an RTX 2070 Super. Oh, and be sure you're running the latest version of Optifine for 1.16.4.
Comment: Note: if this does not solve the issue, check out this forum post: https://www.minecraftforum.net/forums/support/java-edition-support/1863330-java-lang-illegalstateexception. If you are not familiar with the .minecraft directory, PLEASE DO NOT TRY THIS METHOD! You may accidentally delete something of importance. And, no offense, don't assume you know what you're doing. Please ask for help if you need it. I'm here to help.
Comment: I have figured out that the Scaling Health mod + Optifine was causing the creeper crash, but the optifine crash ```lock is no longer valid``` is still occurring even with mods that have already been on the world and new ones. I have the recent optifine 1.16.4 and forge 35.1.36... I really have no idea what the problem is. I cannot and repeat CANNOT use mc without optifine especially with mods because my fps will go from good to poop. And yes @AgentLoneStar007 I have no idea what I am doing
Comment: I found that a temporary solution is to delete optifine 1.16.4 G6 from the mods folder, put in an invalid or old optifine version, load minecraft and let it crash, then load mc with 1.16.4 G6 again and it works. I guess this is just how it is going to be for now... unless Optifine 1.16.4 gets a new update
Comment: I found a permanent solution was to keep the minecraft window in windowed mode, not maximized, then make it windowed (maxmized) once it is done loading
Comment: Dude, I do that all the time.


Drupal Page Process

Question: Help me understand the best practice on how basic site building should work in Drupal 8.
Say I have a section on a website that is all about teaching. I'm going to create a basic page content type that has a title, body and various paragraphs elements that a user can add to the page. I then create the page called Teaching and say to add it to the main menu as this is going to be a main section of my website and then use URL alias to make the URL website.com/teaching.
Now, I want to add another page to my website which should fall under the Teaching branch of the site and call it Learning Cafe. Would I just go and use the same Basic Page content type and add all the info to it, and then add it to the main menu except as a child of the Teaching menu item? If I was to do this, wouldn't I just end up with a URL of website.com/learning-cafe instead of what I really want, website.com/teaching/learning-cafe?
How do I tell Drupal that this new page should be a type of "sub-page" in the Teaching section of the website? And how do I use URL aliases to automatically create the URL as a sub-section of /teaching instead of manually doing this for every page I create? I mean, yes I could do it, but when I hand the site off to non technical people, I wouldn't expect them to know to do this.
Also, if I do it this way, the breadcrumbs don't recognize this as being a sub page of Teaching. Instead of showing Home / Teaching / Learning Cafe, I just see Home / Learning Cafe.
Am I going about this the wrong way? Am I overthinking? If I could get a thorough explanation on how this should be done properly, that would be great. I feel like I know quite a bit about how to use everything else in Drupal but the site building portion of it seems to be confusing me.
Also, if more information is needed, let me know and I'll try to answer it.
Answer: If you add the node under another node in the Menu section of the node form, that page is the child of the page you set it to in the menu.
You need Pathauto to configure automatic aliases to get what you want, URL alias wise. You may also want Menu Breadcrumb to get the breadcrumb to behave a bit better and follow the menu.
As far as the 'page' content type - this is a fine approach. The differences in content types comes down to their purpose and field set. If you are creating five nodes, and theres virtually no difference between them or anything unique (like say, Article vs Product vs Job listing vs Page where fields may be different) - being a Page is fine. To understand your needs better, you should probably flesh out a content wireframe or information architecture / sitemap document. It will help you identify content, and their differing types, which will help you plan their fields and other configurations.
Comment: Thanks @Kevin, this makes sense but how would I set up Pathauto to understand what I want done? Like what if I created a basic page to go under another heading (say Curriculum), how would Pathauto know that this Basic Page needs the /curriculum URL instead of /teaching?  Would it make more sense to create a Basic Page content type for each menu item that I planned on having?
Comment: No don't do that. Install Pathauto and configure it. The docs and issue queue have examples of how to use menu parent tokens to setup automatic aliasing. It will know what to do. I don't have the exact token offhand, but the most common one is something like `[node:menu-parent:join:/]/[node:title]` which creates an alias based off the parent menu path, then attaches the current node title. If no parent (i.e. teacher), then it is just the title.
Comment: Excellent! Thanks Kevin! I have used Pathauto before but didn't know that it had tokens that could be used that way. This makes more sense to me now. The only other question I would have then would be: When a user goes and creates a new Basic Page, they would always be given the permission to select where in the menus this page should go, correct? Like there's no automatic way for Drupal to know where these Basic Pages should go otherwise then?
Comment: Yes they should be able to select wherever in the menu structure they feel the page should live. If you also add on the Redirect module, it will correct mistakes and 301 old urls to the new one if they move nodes around the menu. Basically, creating content types that have no differing aspects is kind of a waste and just more 'things' to maintain. The rest is just really good content entry training.
Comment: Makes sense. Thanks again for everything. You've definitely enlightened me when it comes to this aspect of content building.
Comment: The fact that you are using paragraphs basically grants you a lot of flexibility to using fewer content types. Try to think concretely about the type of content and if a new type is needed at a higher level - like Article, Page, Product, Event, Job Listing, Portfolio Item, things like that that would dictate specific fields or displays or other purposes... it will make more sense.

Drupal Page Process


Difference between x and &x in c

Question: What is the difference between a pointer to an array and a pointer to the first element of an array? In most cases they would be same. Please specify an example when they are not the same. Thank you.
Comment: I just read somewhere they are not but couldn't understand the explanation.
Comment: Good grief, if you "read somewhere" something that is confusing you should tell us what you read and where you read it. Otherwise we're shooting in the dark here..
Comment: It depends, what is your definition of an array? An array is a contiguous block in memory. A pointer to that will, 9/10 be a pointer to the first element in that block of memory. If you mean a pointer to a pointer, then they're different types alltogether
Comment: @RogerRowland Different sizes. According to the explanations given, sizeof(x) contains the size of the entire array whereas sizeof(&x) contains the size of only the first element.
Answer: The type is different:
int x[1] = {0};

&x[0] is a pointer to the first element of x array and is of type int *.
&x is a pointer to x array and is of type int (*)[1].
But their value is the same because there is no padding in arrays:
(int *) &x == x  /* in a value context the expression evaluates to 1 */

Comment: Additionally, the representation and size may be different. In a C implementation for a word-based machine, a pointer to an array of `char` could be the address of the word where the array starts while a pointer to the first `char` of the array would need the address of the word and the offset of the byte. So a pointer to a `char` could have a larger size than a pointer to an array of `char`. The value of these representations would only appear to be equal in the C computation model, when they are suitably converted to a common pointer type.
Comment: You could also point the type of `x` which is of type `int [1]`
Comment: Thanks..! Took me some time to grasp..but happy now!
Answer: The identity of an object is given by the pair (address, type). Different objects can have the same address as long as their types are different, in which case one is a subobject of the other.
This is the case with arrays: The array is an object, and the array elements are objects, and the array elements are subobjects of the array. The first element happens to have the same address as the array itself. Something similar is true for structs and the first struct member.
So if you have an array T a[N], then the type of a is T[N] and the type of a[0] is T, and so the address of the array is
T (*array_addr)[N] = &a;

and the address of the first element is
T * elem_addr = &a[0];

Since a naked array expression decays to a pointer to the first element under certain conditions, the last line could also be written as T * elem_addr = a;, which has the exact same meaning.
Answer: The main difference is that when you obtain the reference to an array, it decays to a pointer and sizeof can't be used anymore on it to obtain the array size, eg:
int array[10];

printf("%d\n", sizeof(array)/sizeof(array[0]));
printf("%d\n", sizeof(&array)/sizeof(array[0]));

This prints on x64:
10
2

Of course you will lose any reference to the size of the array even if you convert to a pointer without any use of the & operator:
printf("%d\n", sizeof((int*)array)/sizeof(array[0])); // prints 2

Answer: Pointer to an array and pointer to the first element of an array is represented as q=&a and p=a. When you increment the p as p++ it will point to the next element in array but when you increment q++ it will give to the next location to the last element in that array. Thus pointer to an array represents to whole array while pointer to first element in array represents only to the first element in array.
Comment: p=a and q=&a[0] both are same. Array a is pointer to first element in a while &a is pointer to complete array. For example a[5]={1,2,3,4,5} and memory location for a is 1002 then in the case of p=a and q=&a[0] both p++ and q++ will give the value 1004 if int takes 2 byte while for p=a and q=&a, the value for p++ will be 1004 and q++ will be 1012.
Comment: @user3138755 your first line of the answer should be as "Pointer to an array and pointer to the first element of an array is represented as q=&a and p=a." This would solve the confusion. Your explanation was correct though. Thank you.
Comment: @RogerRowland a and &a[0] is the same!! Pointers to the first element of the array. But a and &a (&a[0] and &a) are different in types and sizes but equal in values.
Answer: you can print the a and &a, than print a++ and &a++. Through the result, you will know the answer, just try it. a means a element address, and &a means the array address. int a[] = {1, 2 };

Difference between x and &x in c


Running MySQL on Amazon EC2 with EBS (Elastic Block Store) and Elastic Beanstalk

Question: I'm new to Elastic Beanstalk, but I created an environment with a container running 64bit Amazon Linux running PHP 5.3.  I want to get MySQL set up on EBS, then install phpMyAdmin and import my databases.
However, I don't know how to do this because the documentation is not working for me:  Running MySQL on Amazon EC2 with EBS (Elastic Block Store)
Since that guide was last updated on March 23, 2010 I guess it might be out of date.
Here's what I did:

Set up the EC2 instance, attached EBS volume, connected to the instance via ssh, and got root access with the command "sudo su -" (so far so good).
According to the guide, I should now run the "sudo apt-get update && sudo apt-get upgrade -y" command, but the apt-get command is not found.  No problem, I ran yum update and yum upgrade.
Similarly, "sudo apt-get install -y xfsprogs mysql-server" didn't work, so I ran "sudo yum install -y xfsprogs mysql-server" and this installed MySQL.
The guide says to create an XFS file system on /dev/sdh, but my EBS volume is attached to /dev/sda1 (this is the default when you use Elastic Beanstalk and it cannot be changed) and there already is a file system on it (I guess Elastic Beanstalk automatically creates one), so it doesn't let me run the "sudo mkfs.xfs /dev/sdh" command (I changed it to "sudo mkfs.xfs /dev/sda1" because that's where my volume is attached).
The next 3 commands in the guide seemed to execute (no error messages), so now I've got a my EBS volume mounted as /ebsvol.  However, I noticed that /ebsvol has a copy of the root directory structure in /ebsvol.  However, /ebsvol/ebsvol is an empty directory.  At this point I'm a bit worried, but I carry on.
I stop the MySQL server using the command "/sbin/service mysqld stop" because the command in the guide (sudo /etc/init.d/mysql stop) did not work.
Now I have to move the existing database files to the EBS volume and point MySQL to the EBS volume.  This was a disaster because the mysql files are not located where the guide says they should be.  Now I can't restart the MySQL server.

Help!

Is there an updated guide that will configure MySQL to use an EBS volume on Elastic Beanstalk?  Searching Google, stackoverflow, serverfault result in a couple of guides from 2008/2009, so they haven't been tested with Elastic Beanstalk. (EDIT: is there something like the Eric Hammond documentation, but for CentOS/RHEL?)
How can I undo all the mount binds which are preventing MySQL from starting?  Should I just delete the instance and volume and start over?
Am I wasting my time trying to set up MySQL on EBS on Elastic Beanstalk?  Based on the lack of information available, it's either a) trivially easy to do and no guide is needed (therefore making me an idiot); or, b) nobody sets up MySQL on Elastic Beanstalk because it's unnecessary/redundant (therefore I shouldn't bother).

EDIT:
Thank-you for your comments and answers. I'm trying to figure out the best way to set up a PHP/MySQL website on AWS, so Elastic Beanstalk seemed like a good idea because AWS markets it as "an even easier way for you to quickly deploy and manage applications in the AWS cloud."
However, that doesn't seem to be entirely true if you want to run MySQL with EBS.  From what I gather, I guess everyone uses RDS with Elastic Beanstalk because it scales automatically and presumably has automated snapshot functionality.
So I guess I'm left with these options:
1) Don't use Elastic Beanstalk:  set up an Ubuntu EC2 instance with MySQL running on an EBS volume, as per the Eric Hammond documentation (it sounds like there would be some scalability issues?).
2) Use Elastic Beanstalk:  set up my databases on RDS (no scalability issues).
3) Use Elastic Beanstalk:  but with an ubuntu AMI configured with MySQL running on an EBS volume (is this possible?  will Elastic Beanstalk work with a private AMI?)
4) Use Elastic Beanstalk:  start over and use cyberx86's instructions on adapting the ubuntu instructions to work on CentOS/RHL.
At this point my database and site traffic are quite small.  It would be nice to make it scaleable, but at this point I just want to get it running in a way that lets me deploy new versions using git after I've got my code working on localhost.  Top priority is to get the site up and running and get back to marketing and building features instead of spending time on hosting.  What should I do?
Comment: A few points a) Amazon's Linux is based on CentOS/RHEL - so a guide for CentOS 6 should work quite well (whereas the guide you are following is for Ubuntu 9.10 - and is for EC2, not Elastic Beanstalk specifically). b) (Keeping in mind I don't use Beanstalk - but use EC2 a lot) Your problem seems to be with 'attaching an EBS volume' - the instance should have a root EBS volume (attached at /dev/sda1 => /dev/xvda1), you need to attach a second EBS volume (at /dev/sdh => /dev/xvdh). c) You are mounting the root volume within itself (step 5), which isn't ideal.
Comment: d) Undo the mounts (temporarily) using umount /path or permanently by editing /etc/fstab and removing the entries e) (You are using an EBS-root instance, and not an S3-backed instance, right?) - check under /dev - if you attached an second (in addition to the root) EBS volume it might be there f) MySQL doesn't autoscale (i.e. you can't just stick 2 instances behind an ELB and have it work, it needs to be setup with replication, etc.) Beanstalk usually uses Amazon RDS for this; essentially, if you can get MySQL working with autoscaling, you probably don't need Beanstalk (use EC2 directly)
Comment: And why didn't you post that as an answer? Comments are ephemeral, so no important content should be left there.
Comment: @ErikA: I really think I need to work on my understanding of comments - it seems such a simple concept - don't answer a question with a comment. I think for me, my problem comes down to a bit of uncertainty - I am not a computer person, so there is always this little doubt that my answers are any good. I therefore tend to err on the side of caution and, unless I am certain of the answer and have double checked it, often try to convey the idea as a comment (that, and answers take more effort I find). (I do usually come back to questions I comment on in this way, and will answer them later.)
Comment: Post as an answer. Others will comment or downvote if you are off base. Commrnts should be used for offhand or clarifying questions only, not for answers.
Answer: I don't use Elastic Beanstalk - but the guide you are following is for EC2 (which I can definitely help with). The first difficulty you have is that the guide you are using is for Ubuntu 9.10; Amazon's Linux is based on CentOS/RHEL - so you would have an easier time if you could find a CentOS 6 guide.
The root of your issue seems to stem from 'attaching an EBS volume'. On EC2 you can attach multiple EBS volumes to a single instance. All instances have a root volume - these can be either S3-backed or EBS-backed. By far, the preferred approach is to use an EBS backed root volume (it costs a bit more, but makes up for it in flexibility and durability). An instance with an EBS root volume will almost always have this volume attached as /dev/sda1 - on modern Linux systems, the device actually shows up as /dev/xvda1 (and it is the latter which should be passed to any commands). (Other than trying to format a mounted volume - you were trying to format your root file system with the instance running - i.e. you were trying to erase your operating system, definitely not a good idea, if it is even possible).
In this case, the suggestion is to add a second EBS volume - attach it to your instance (e.g. as /dev/sdh, but use /dev/xvdh for commands), and use that for storing your MySQL data. (Despite not using Elastic Beanstalk) I find it hard to believe that Elastic Beanstalk would not allow you to attach a second volume - as this functionality is fairly central to EC2.
You should be able to get a list of the EBS devices by running cat /proc/partitions (or using fdisk -l).
You will note that in step 5 of what you have done, you are actually mounting the root volume within itself (i.e. /dev/sda1 is already mounted as / and you are mounting /dev/sda1 as /ebsvol) - it is best to avoid doing that.
Also, while /etc/init.d/mysql stop did not work, /etc/init.d/mysqld stop probably would have worked. (Again, you can get a list of the init.d scripts by running ls /etc/init.d - and should be able to use those paths, like you, I usually use the service command though).
The MySQL databases should be in /var/lib/mysql - however, your mountpoints in /etc/fstab are probably incorrect (given the ebsvol within /ebsvol problem). When you cd /var/lib/mysql you should be able to see your databases - if not your mounts haven't worked correctly. (Verify that /var/lib/mysql is mounted on a different device by mountpoint -d /var/lib/mysql and compare the device to cat /proc/partitions).
The basic ideas of the guide you are following are quite valid - it is common practise to put your data and databases on a different EBS volume than your root volume, as it offers numerous advantages (performance, ease of snapshotting, easier to move between instances, etc.), and the basic Linux commands haven't changed - they are just for a Ubuntu.
Undo your mounts with umount /path - just like you normally would, of course, you will need to ensure that the device is not busy (which may not be a problem if you haven't managed to start MySQL). umount is only temporary though - so you will have to edit /etc/fstab and remove any references to the mount points from there also. If you don't have anything of value on the instance, you might be better off starting over (not because it is difficult to unmount a few volumes, but rather because it is always easier to figure out where you went wrong when you start from a known state).
Finally, with regard to MySQL on Elastic Beanstalk: the point of Elastic Beanstalk is supposed to be that it handles provisioning of resources and scaling automatically - it is still based on the core AWS components (e.g. EC2, S3, ELB, etc) but it will do some things for you. Elastic Beanstalk usually uses RDS to handle MySQL databases. RDS is an Amazon managed version of MySQL which simplifies the provisioning and scaling of MySQL instances. Keep in mind that MySQL doesn't lend itself well to autoscaling without a lot of setup. You can't just launch a second MySQL instance and have the load split between your two instances - you need to setup replication, which may not be a simple task).
Essentially, if you are able to setup MySQL in such a way that it runs from your web server instances and can autoscale seamlessly, you'd almost certainly be better off using EC2 directly and not bothering with Elastic Beanstalk. I'd suggest therefore, the most people don't actually setup MySQL on Elastic Beanstalk (what you could do is setup a separate MySQL instance, but if you are using Beanstalk, RDS is probably a simpler approach).

Edit:
Unlike a lot of other services that operate mostly as a black box, Elastic Beanstalk does give you access to the underlying components. That said, if you are going to go through the effort of setting up your EC2 instances manually, you have negated the point of Elastic Beanstalk.
If you are using EC2, there are few approaches to PHP/MySQL:

You can host both your webserver and database on a single instance - when you are starting out, this can be a reasonable approach, however, it doesn't scale horizontally very well (but you can still scale vertically - using larger instances). Hopefully by the time you exceed the capacity of the x-large instances, you will be in a position to setup a more complex setup. That said, it is bad for redundancy - everything is on that single instance, and a failure of any component takes down your whole setup.
You can host your webserver on one instance, and use RDS for your database. Most well designed applications will tax the web server more than the database (and the database load will ideally be read-biased). In such a scenario, you can scale your web server instances relatively easily (e.g. by putting them behind an ELB - with just a bit of effort to ensure that all are serving the same content). RDS is MySQL managed by AWS - it isn't quite fully automatic, but it does go a long way towards autoscaling. Essentially, RDS will provision multiple read-only slaves, and a single write-master, with multiple hot-backups that can take over if you need. The downside is that you are paying for all those instances that are running (and you don't have full control over some of the intricate settings of MySQL).
The final approach would be to use your web server cluster and your own MySQL cluster. Essentially, you can scale your web instances (as above), and then you will setup MySQL instances that will scale separately. You will need to look into MySQL replication (or perhaps use MySQL cluster if you can adapt your application to its data structures).

A few other answers on the same topic:

RDS vs EC2
Deploying applications
Some approaches to EC2 scaling

My perspective is usually that one click solutions aren't the best approach - I like the control that is offered by doing something manually. I find that not only do I usually end up with a more tailored and efficient end result, but I also have a much better understanding of how the system works, which makes figuring out what is wrong much easier. You can always automate your own setups once you have a good understanding of the intricacies of them.
One point to keep in mind about RDS - it is already EBS backed. RDS is MySQL - it isn't something similar, or another relational database. It is a managed instance of MySQL running on EBS backed EC2 instances. AWS will keep the software up to date, and you can do normal EBS snapshots of your data, etc. You just don't have direct access to the underlying software running on the instance.
As for the choice of operating system, I am partial to Amazon's Linux. It is well supported by AWS and uses a minimum of resources - it is fully compatible with CentOS (as a matter of fact, it includes the EPEL repository by default in the latest version). The usual viewpoint is to use whatever Linux distribution you are comfortable with, as the differences are usually minor (CentOS will work just as well as Ubuntu for the instructions you are working from - most commands (except apt-get) are the same on CentOS. Given that my own setup has the databases on a separate EBS volume using Amazon's Linux, I can assure you that it is not difficult to do).
I'd suggest that there are some main considerations:

Comfortability with/willingness to learn Linux systems - if you don't mind setting up your own servers and want to get a better understanding of them, I'd definitely go the EC2 route. You'll end up with a better end result if you do it right and will have more versatility in the long run. I will mention though, that if you are taking this approach, you want to really understand what the commands you are running do - just following a guide will not be enough if you really want to commit to it.
Budget - remember that with AWS everything has a price. The more AWS does for you, the more they charge you. An RDS instance costs about 30% more than an equivalent EC2 instance (and there is not micro instance) and if you want the redundancy they offer, you need to be running multiple RDS instances (and paying for each of those). Elastic Beanstalk will provision instances, load balancers, RDS instances, etc. for you - the costs add up quickly.
Time - if you have no time, want to press a couple of buttons and have something functional, Elastic Beanstalk is probably the best approach for you.

I would advise against using Elastic Beanstalk with MySQL baked into your AMI - it will likely be quite unstable, if it works at all. (Just think about what happens when it adds and removes an instance to you cluster, or when data goes to one instance instead of the other...)
It is great to keep scalability in mind - but don't optimize things too soon, or you will never get anything done. Definitely keep it in mind, but if the cost (time, money, etc.) of making a particular component scalable is not practical at the moment, don't worry too much about it - when the time comes to scale it, you'll figure it out (most popular sites started out that way, afterall).
I'd advise that if your application is designed so that it can take advantage of some caching, it will go a long way.
Typically, on EC2 it is better to scale vertically (to larger instances) than horizontally (to more instances). To begin with however, you want to scale to two instances so that you have some redundancy and minimize your single points of failure. A possible approach, therefore, may be:

Start with a micro instance - have both your database and application on it (you can't get any smaller than this, which makes it a good starting point).

This is of course, quite easy to scale vertically, just keep upgrading your instance until you are using x-large instances. The problem comes down to redundancy - if there is any problem with your instance, your application is offline.

Now, you usually want to separate of your database to another instance (since a) the database will see different load than your application and b) you can't autoscale MySQL in quite the same way as web servers), but micro instances just don't handle load well, so I'd suggest upgrading to a larger instance first, at least a small, and then, perhaps a medium (basically, the idea is that once you need larger instance types, the effect is presumably greater)
Separate your database from your web server. This will allow you to cater to the different needs for databases (e.g. high memory) vs web servers (e.g. higher cpu) and the differences between how you scale each (Recommended reading). At this point you might decide to use RDS instead of running your own MySQL instance.
Now that you have your application running on a dedicated instance, you can scale it and not worry about your database - setup autoscaling so that you have some redundancy. This should automatically add more application nodes as any of them fail or as load exceeds the thresholds you specify.
Add a second database node and configure replication between your nodes (if you opt to use MySQL cluster, or NoSQL solutions, you should be able to setup autoscaling as well). Everything should at this point have redundancy, and even if a node fails, you should still be online.
Upgrade one instance at a time to larger instance sizes as demand merits it.
Comment: @cyber86: thanks again for your help.  I'm going to accept your answer because it not only helped me sort out the meta issue of whether Elastic Beanstalk is right in my circumstances, but it also helped me figure out the process to set up MySQL on Amazon linux.  BTW, I've posted a [question on stackoverflow](http://goo.gl/R5CnJ) about setting up a post-receive hook to pull from bitbucket.  I think it has something to do with permissions, but I'm still too newbie to know why it's not working.  Maybe you've encountered this issue before?
Comment: @cyber86: as per my edit above, I guess I should take a step back and figure out whether Elastic Beanstalk is the right way to go.  I'm not worried about scaling at this point, but it would be nice to have it set up so that it would scale if necessary.  Based on what you've written above, it sounds like it IS possible to adapt the Eric Hammond instructions to work with CentOS/RHEL (I'm assuming it's essentially slightly different commands and configuration on CentOS/RHEL and not a lack of functionality).  In any event, thanks for taking the time to help solve this!
Comment: @edpeciulis: I've added a few points about scaling on EC2 to my answer - I, unfortunately, can't help much with Elastic Beanstalk. You can definitely use Eric's instructions on Amazon's Linux/CentOS/RHEL - almost all the commands from the article (except apt-get) remain unchanged (I use mount binds on my Amazon Linux setup). Good luck with your setup - if you run across problems, just ask another question (and welcome to Server Fault!) (Also - it is nice to see someone put some effort into their question).
Answer: Now that I've become a bit more familiar with Elastic Beanstalk and EC2, I've decided to forego using Elastic Beanstalk because although it's got some cool features, it's too regimented for my liking.  For example, I don't like the fact that I can't change the httpd.conf file (well, you can change it, but those changes disappear when your environment is restarted).  Another reason is that the only way to run Elastic Beanstalk with MySQL (properly, i.e., with auto-scaling and automated backups) is to use RDS.  Even though you can get 3 months of RDS free with a new sign-up, I'm not at the scale where I need its features, so it's not worth it for me to pay ~$76/month for RDS.
Bottom Line:  If you have a reasonable amount of traffic and you need a solution that scales and takes care of itself, Elastic Beanstalk with RDS is a great option.  I like the fact that you can deploy using git.  It's like Heroku for PHP.  The getting started guide should include instructions for setting up MySQL.
What I did:  I opted to use "Synthetic Elastic Beanstalk":  I can recreate its functionality using the various offerings from AWS and have the flexibility to configure it exactly the way I want.  While I'm kicking the tires of AWS, I've set up MySQL on the same EC2 instance as my webapp (not ideal once you need to scale, but perfectly fine while you're learning the ropes of using AWS).
This guide is what I used to set up a LAMP stack on an EC2 instance running an Amazon linux AMI.  I find it easier to import my databases with phpmyadmin.  Because I'm using a micro instance, it uses EBS by default and you can take snapshots to backup your data.  I would recommend setting up the CLI tools for EC2 and running
ec2-modify-instance-attribute --block-device-mapping "/dev/sda1=:false" i-xxxxxxx

where i-xxxxxxxx is the EC2 instance the EBS volume is attached to.  This prevents the EBS volume from being deleted if your instance terminates.  Since that EBS volume is where everything is run from and where my database is stored, I don't want to lose it (while I was playing around with Elastic Beanstalk my EC2 instance was terminated and Elastic Beanstalk instantly started another one with a new EBS volume attached, but I had fortunately changed the DeleteOnTermination setting to "false" for the original EBS volume, so I was able to stop the new instance, detach the new EBS volume, and attach the old EBS volume, thus preserving my MySQL installation and database).
Overall, the whole process of moving a web app to AWS is still quite a pain in the ass.  Now that I've gone through the learning curve, I feel more comfortable with it, but I can't help thinking there should be better documentation for getting started.
Comment: I would make one additional suggestion - separate your data from your root volume. Attach a second EBS volume, and mount bind relevant paths (e.g. /var/www, /var/log, /var/vmail, /var/mysql, etc) to the second EBS volume. In addition to usually offering somewhat better performance, you get smaller deltas on your snapshots, and can move data between instances easier (including scrapping your root volume for any reason).


What does $C^{\text{1,stw}}(0,b)$ mean

Question: I know that this C with the one means continoulsy differentiable functions but what does this stw stand for? Does anyone know this?
Comment: Where did you find it?
Answer: It reminds me of the abbreviation for the german word "stückweise" - in this case, $C^{1,\text{stw}}$ denotes the set of piecewise continuously differentiable functions. But depending on the context, it could mean something totally different.
Comment: @Lipschitz You are welcome. By the way, you can mark answers as the accepted answer by clicking on the check box outline to the left of the answer.
Comment: this is perfect, thank you!

What does $C^{\text{1,stw}}(0,b)$ mean


The polynomial of minimal degree with root $\alpha$ is unique.

Question: So I am working on the following proof:

Problem Statement: Let $\alpha$ be a complex number. Prove that the kernel of the substitution map $\mathbb{Z}[x] \rightarrow \mathbb{C}$ that sends $x \mapsto \alpha$ is a principal ideal, and describe its generator.

To assist me with the proof, I need to show that the polynomial $f(x)$ of minimal degree with root $\alpha$ is unique, so that I can then show that $f(x)$ generates the kernel of the substitution map.
I am trying to do the following:

Let $\varphi: \mathbb{Z}[x] \rightarrow \mathbb{C}$ defined by $x \mapsto \alpha$.
  We will first show that there is only one unique polynomial of minimal degree with root $\alpha$ in $\mathbb{Z}[x]$.
Let's claim that $\exists$ $2$ irreducible polynomials of minimal degree with root $\alpha$ in $\mathbb{Z}[x]$, namely $f(x)$ and $g(x)$.
  Then consider $h(x)=f(x)g(x)$. Then we have
  $$\varphi(h(x))=\varphi(f(x)g(x))=\varphi(f(x))\varphi(g(x))=f(\alpha)g(\alpha)=0\cdot0=0$$
  Since $\mathrm{im}\ \varphi$ is an integral domain, then $fg=0$ implies that $f(x)\equiv 0$ or $g(x)\equiv 0$. Thus, we cannot cannot have more than one polynomial of minimal degree with root $\alpha$ in $\mathbb{Z}[x]$.

I think I am getting a little confused here. My professor told me "The image of a substitution map is an integral domain, but if the ideal is generated by $fg$ them $fg=0$ in the quotient, so the quotient is not an integral domain. Thus, you definitely want the kernel to be generated by an irreducible polynomial."
I think I am just bothered because $f(\alpha)=g(\alpha)=0$, but is my claim that since $\mathrm{im}\ \varphi$ is a domain, that one of $f$ or $g$ are identically $0$ invalid? Because in the image, would they not both be identically $0$?
If someone could verify this or help lead me in the right direction, I would appreciate it!
Comment: if $g$ doesn't divide $f$ and $f$ doesn't divide $g$,  $f(\alpha) =  g(\alpha) =0$ and $deg(f) = deg(g)$ then there exists two integers $b,c$ such that $b f(\alpha) - c g(\alpha) = 0$, $bf-cg$ is not the zero polynomial, and $deg(b f - cg) < deg(f)$, contradicting that $f,g$ are of minimal degree
Comment: So basically, $bf-cg$ would be the minimal degree polynomial... But how is that showing that $bf-cg$ is unique? what if there is some other polynomial with root alpha and same degree as $bf-cg$?
Comment: among the (nonzero) polynomials having $\alpha$ as one of their roots, there is a minimal degree. the question is if there is only one polynomial of that degree, or two or more.
Comment: OP what do you mean by unique minimal polynomial?  $X^2 - 2$ and $2X^2 - 4$ are polynomials in $\mathbb{Z}[X]$ of minimal degree of which $\sqrt{2}$ is a root, and they generate different principal ideals.
Answer: You only have that one of them is zero at $\alpha$, not identically. But remember you have two things you are assuming about $f,g$, that they are zero at $\alpha$, and of minimal degree. So you should try and contradict the minimality in order to show uniqueness.
Comment: Ohh I see. Should I also claim that the two polynomials are monic? Say $\deg f(x) = \deg g(x) = n$, and it they are monic, $\varphi(f(x)-g(x))=\varphi(f(x))-\varphi(g(x))=f(\alpha)-g(\alpha)=0-0=0$. Then $\deg f(x)-g(x) < n$ since the leading term is eliminated. So then there is some irreducible polynomial of degree less than $n$ with root alpha? This causes an issue though because I can continue claiming that there is some polynomial with degree less than $n$ whose root is $\alpha$...
Comment: But the problem was not that the minimal polynomial was of degree $n$, just that there were two distinct minimal polynomials.
Comment: I guess I am getting confused because did I not contradict that there were 2 polynomials with my original proof? I don't see how else I could contradict minimality using the substitution map.
Comment: In an integral domain, the generator of a principal ideal is unique, up to a unit.
Comment: @bernard : that $\mathbb{Z}[x]$ is an integral domain is needed to state that $f$ doesn't divide $g$  and $g$ doesn't divide $f$ ?
Comment: And the kernel is principal ideal... so it must be generated by a minimal polynomial with root $\alpha$? But I feel like I must prove that the kernel is principal ideal...
Answer: First, there may be no "minimal polynomial" that vanishes at $\alpha$ - namely, if $\alpha$ is transcendental. In that case, the kernel is $(0)$, which is indeed a principal ideal, so you are done.
Then suppose $\alpha$ is algebraic. For every polynomial $f$ that vanishes at $
\alpha$, $-f$ ALSO vanishes there, so it doesn't make sense to speak of "the" minimal polynomial (or perhaps it does, but it would not be unique).
Rather, for $\alpha$ algebraic, let $f(x)$ be the non-zero polynomial of minimal degree that vanishes at $\alpha$ AND having the least leading coefficient among ALL polynomials of the same degree and with positive leading coefficient, vanishing at $\alpha$. You may show that this characterization defines a UNIQUE polynomial $f(x)$, but that is not part of the proof. Rather, if $a$ is the leading coefficient of $f$, and $g(x)$ is another polynomial of the same degree, with positive leading coefficient $b$ and that also vanishes at $\alpha$, FIRST show that $a|b$ by the minimal choice of $a$, and then prove that this implies $f|g$ due to the minimal degree. You have to do it in these two separate steps. Then show that $f$ divides all polynomials of higher degree that vanish at $\alpha$. This proves that the kernel is the principal ideal generated by $f(x)$.


Unexpected result on CAST or CONVERT in sql server

Question: SELECT CAST (330/60 AS FLOAT) AS result

or
SELECT CONVERT(FLOAT, 330/60) AS result

Actually I am expecting result as 5.5
But it is returning 5. Why?
And tell me any solution to get result as 5.5
Answer: Use 60.0 to force fixed-point types:
 SELECT CAST (330/60.0 AS FLOAT) AS result
 SELECT CONVERT(FLOAT, 330/60.0) AS result

The calculation of 330/60 is done as integers which will round the result. After that you cast to float. You can use a numeric type right from the beginnging and don't need to cast:
SELECT 330/60.0 AS result

Comment: Because **first** `330/60` is evaluated. **After that** is will be converted to float. And integers can't have floating point results.
Comment: `60.0` is `numeric(3,1)` datatype rather than `float`. `6E1` would be `float`
Comment: **Why** SELECT CONVERT(FLOAT, 330/60) AS result is giving  result **5**
Comment: @MartinSmith: Thanks for the hint. Changed it in the answer.
Answer: Instead of casting the answer to float ,you should try casting the denominator to float and then do the division
 Select 330/CAST(60 as float) as result

or else change the denominator to include decimal part as suggested by juergen d
 SELECT CAST (330/60.0 AS FLOAT) AS result


openvpn without tap module? replacement?

Question: I am trying to instal OpenVPN on a special linux distribution that has no tap kernel module, and all howtos available are outdated.  I was wondering if there is a way to run OpenVPN without this kernel module.
At the moment I can't install OpenVPN due to unresolved dependencies, but I can always force it. I just don't think it's possible, but I do hope somebody knows better ;)
Answer: You really need a TUN module. How else can OpenVPN create the "fake network interface" which accepts the IP-range for your VPN subnet?

on a special linux distribution

What makes it special? Is it some kind of NAS with busybox installed? (because you mention ARM)
My QNAP NAS ships the tun.ko module in /usr/local/modules
This can be easily loaded from a startup script:
( /sbin/lsmod | grep -q tun ) || /sbin/insmod /usr/local/modules/tun.ko

Try to find the tun module at your system in some common system folders, using:
find /lib /usr /opt -name 'tun.ko'

Comment: The distro is jlime on jornada 720.
I've found the file tun.ko in some weird folder deep in /lib/
It seems to load, I'll see if it works.
Answer: Does the kernel in that setup have module support? If so then you may be able to find an appropriate version of the tap module, compile it, and install with modprobe (the same way vmware and similar distribute their driver modules).
Comment: I'll see. I suppose it does have module support. But the machine has an ARM processor. I'll post back when I try it.


Existe algo equivalente ao "FOR UPDATE" do MySQL no SQLite?

Question: Como o próprio título já diz, existe algo equivalente ao FOR UPDATE do MySQL no SQLite?
Exemplo:
SELECT id FROM minha_tabela WHERE id=1 FOR UPDATE

Possuo uma query que utiliza um bloqueio de linha exclusivo dentro de uma transação no MySQL e gostaria de saber se existe uma alternativa que obtivesse o mesmo efeito utilizando o SQLite.
Answer: Não, e não faz sentido ter. O SQLite trabalha com transações de escrita exclusivas, portanto só pode haver uma escrita de cada vez e todo o banco de dados é bloqueado para operações de escrita em cada escrita que queira fazer, e ao mesmo tempo não precisa de nenhuma preocupação com as leituras que ocorrem sem nenhum controle especial, e essa é a beleza do SQLite. Não havendo simultaneidade não tem porque existir essa cláusula que é para lidar com problema de compartilhamento.


$\inf_{n}( \log(a_n))=\log (\inf_{n} (a_n))\;?$

Question: Let $(a_n)_n$ be a positive sequence.

Why
  $$\inf_{n}( \log(a_n))=\log (\inf_{n} (a_n))\;?$$
Answer: Hint: The $\log$ function is continuous and increasing.
Comment: I think that the  fact that log is increasing suffices to get  the  result.
Comment: But it is not. sufficient.
Comment: Could you please give me one other hint ? Thanks a lot. I don't understand where one use the continuity
Comment: For each $n$, $\inf_na_n\leqslant a_n$ and therefore (since $\log$ is increasing) $\log\left(\inf_na_n\right)\leqslant\log a_n$. Since this takes place for each $n$, $\log\left(\inf_na_n\right)\leqslant\inf_n(\log a_n)$. Now, suppose that $\log\left(\inf_na_n\right)<\inf_n(\log a_n)$. Take a subsequence of $(a_n)_{n\in\mathbb N}$ whose limit is $\inf_na_n$. Use the fact that $\log$ is continuous (and, in particular, continuous at $\inf_na_n$) to get a contradiction.