How Can I Best Guess the Encoding when the BOM (Byte Order Mark) is Missing?

Question: My program has to read files that use various encodings. They may be ANSI, UTF-8 or UTF-16 (big or little endian).
When the BOM (Byte Order Mark) is there, I have no problem. I know if the file is UTF-8 or UTF-16 BE or LE.
I wanted to assume when there was no BOM that the file was ANSI. But I have found that the files I am dealing with often are missing their BOM. Therefore no BOM may mean that the file is ANSI, UTF-8, UTF-16 BE or LE.
When the file has no BOM, what would be the best way to scan some of the file and most accurately guess the type of encoding? I'd like to be right close to 100% of the time if the file is ANSI and in the high 90's if it is a UTF format.
I'm looking for a generic algorithmic way to determine this. But I actually use Delphi 2009 which knows Unicode and has a TEncoding class, so something specific to that would be a bonus.
Answer:
ShreevatsaR's answer led me to search on Google for "universal encoding detector delphi" which surprised me in having this post listed in #1 position after being alive for only about 45 minutes! That is fast googlebotting!! And also amazing that Stackoverflow gets into 1st place so quickly.
The 2nd entry in Google was a blog entry by Fred Eaker on Character encoding detection that listed algorithms in various languages.
I found the mention of Delphi on that page, and it led me straight to the Free OpenSource ChsDet Charset Detector at SourceForge written in Delphi and based on Mozilla's i18n component.
Fantastic! Thank you all those who answered (all +1), thank you ShreevatsaR, and thank you again Stackoverflow, for helping me find my answer in less than an hour!
Comment: Thanks for the edit! ChsDet seems to be working!
Answer: Maybe you can shell out to a Python script that uses Chardet: Universal Encoding Detector. It is a reimplementation of the character encoding detection that used by Firefox, and is used by many different applications. Useful links: Mozilla's code, research paper it was based on (ironically, my Firefox fails to correctly detect the encoding of that page), short explanation, detailed explanation.
Comment: All links died. Can you try to restore them?
Comment: Ooooh. That's exactly the type of algorithm I'm looking for. Now if I could figure out how it works, or just find a Delphi equivalent ...
Comment: According to the docs, it's a Python port of Mozilla cpp code. The latter is located at http://mxr.mozilla.org/seamonkey/source/extensions/universalchardet/src/base/ No idea which incarnation is easier to port though!
Comment: (contd.) The CPP version seems to be more amply commented, which might help in porting.
Answer: Here is how notepad does that
There is also the python Universal Encoding Detector which you can check.
Comment: Actually my version is "MS hid the facts" (without quotation marks of course).  Try it.
Comment: The IsTextUnicode is a good first step. Then it says it uses http://www.ietf.org/rfc/rfc2279.txt?number=2279 for the UTF-8 definition, but that doesn't say what to test.
Comment: Actually, WP, it's http://en.wikipedia.org/wiki/Bush_hid_the_facts (some jokes do have to be explained).
Answer: My guess is:

First, check if the file has byte values less than 32 (except for tab/newlines). If it does, it can't be ANSI or UTF-8. Thus - UTF-16. Just have to figure out the endianness. For this you should probably use some table of valid Unicode character codes. If you encounter invalid codes, try the other endianness if that fits. If either fit (or don't), check which one has larger percentage of alphanumeric codes. Also you might try searchung for line breaks and determine endianness from them. Other than that, I have no ideas how to check for endianness.
If the file contains no values less than 32 (apart from said whitespace), it's probably ANSI or UTF-8. Try parsing it as UTF-8 and see if you get any invalid Unicode characters. If you do, it's probably ANSI.
If you expect documents in non-English single-byte or multi-byte non-Unicode encodings, then you're out of luck. Best thing you can do is something like Internet Explorer which makes a histogram of character values and compares it to histograms of known languages. It works pretty often, but sometimes fails too. And you'll have to have a large library of letter histograms for every language.
Comment: Hmmm, I often see bytes with values less than 32 in my text files.  Things like \n, \r and \t.  Rarely some other ones, too.
Comment: ASCII, most ANSI code pages, and UTF-8 understand characters such as carriage return, line feed, horizontal tab, null character, etc., which have byte values less than 32.
Comment: I meant to say ANSI, not ASCII in the question. I've modified the question now. You might want to modify your answer to reflect this.
Answer: ASCII? No modern OS uses ASCII any more. They all use 8 bit codes, at least, meaning it's either UTF-8, ISOLatinX, WinLatinX, MacRoman, Shift-JIS or whatever else is out there.
The only test I know of is to check for invalid UTF-8 chars. If you find any, then you know it can't be UTF-8. Same is probably possible for UTF-16. But when it's no Unicode set, then it'll be hard to tell which Windows code page it might be.
Most editors I know deal with this by letting the user choose a default from the list of all possible encodings.
There is code out there for checking validity of UTF chars.
Comment: Windows still has device drivers. If your kernel code isn't 7 bit clean you'll regret it.
Comment: @Windows programmer: what do you mean kernel code needs to be 7-bit clean? Most (all?) drivers need to deal with Unicode - although sometimes the problem is correctly converting from MBCS to Unicode (do I use OEM or the default codepage?, etc).
Comment: OK, code that handles filenames has to copy and convert character strings in variables (PUNICODE etc.), but the source code still has to be 7-bit clean in order to compile properly at compile time.


Theming Drupal fatal error messages

Question: Whenever Drupal encounters an unrecoverable error, it displays an error message.
Is there a way I can provide a template for this page?
Comment: Can you give an example of a *Drupal* error that you are talking about, or are you talking about a fatal *PHP* error?
Comment: He's talking about exceptions which are thrown and caught. This could be PDO exceptions (database error) or entity wrapper exceptions and a host of other exceptions which are caught and Drupal show an fatal error page.
Comment: Are you missing gramatical php marks like a semi-colon, comma or quotations?
Answer: The closest thing you come by is the maintenance-page.tpl.php, It doesn't cover all cases, but if things go wrong but not that wrong that Drupal can't post a page, it uses maintenance-page.tpl.php to display the error(s).
The error itself is displayed using drupal_set_message
Comment: And for Drupal 8 this would be something like maintenance-page.twig ? Do you know?

Theming Drupal fatal error messages


Como poderia implementar um swipe para essa biblioteca de slide?

Question: Estou utilizando esta biblioteca / Plunker:
Gostaria de fazer funcionar o touch com o efeito:
<code>touch-action: pan-x;
</code>
Como este exemplo, se você habilitar o console, com versão mobile ativo dá pra ver o toque funcionando.
Em tentei fazer usando javascript:
<code> $(document).on("touchstart",'.carousel-item', function(ts) {

               var start_client_x = ts.clientX + parseInt($(this).width());
                   current_client_x = ts.clientX+ parseInt($(this).width());

               if (ts.type == 'touchstart') {
                   var touch = ts.originalEvent.touches[0] || ts.originalEvent.changedTouches[0];
                   ts = touch;
                   //  start_page = ts.pageX;
                    start_client_x = ts.clientX+ parseInt($(this).width());
               }

               $(document).on("touchmove", '.carousel-item', function (e) {

                   if(e.type == 'touchmove') {
                       var touch = e.originalEvent.touches[0] || e.originalEvent.changedTouches[0];
                       e = touch;
                       //current_page = e.pageX;
                       current_client_x = e.clientX+ parseInt($(this).width());
                   }

                   console.log('start->'+start_client_x, 'end->'+current_client_x);
                   // var x_position = e.pageY;
                   //
                   if (start_client_x < current_client_x) {
                       $(this).parent().parent().parent().parent().find('.carousel-next .carousel-btn').trigger('click');
                   } else if (start_client_x > current_client_x) {
                       $(this).parent().parent().parent().parent().find('.carousel-prev .carousel-btn').trigger('click');
                   }

               });
           });
</code>
Mas ficou meio estranho o movimento.
Comment: Tente colocar no container do slider essa outra propriedade `-webkit-overflow-scrolling: touch;` e coloque o prefixo tb nas outra propriedades,  `touch-action` / `-webkit-touch-action`... testa ai pra ver se funciona...
Comment: Infelizmente, nada mudou após colocar isso.
Comment: apresente o seu código na pergunta
Answer: Consegui resolver fazendo assim:
<code>$scope.swipeGallery = function() {
    //PERMITE SWIPE (EVENTO TOUCH DE ARRASTAR) NO CARROSSEL

      $(function() {

            var start_client_x = null, start_client_y = null;
            var current_client_x = null, current_client_y = null;

            $(document).on("touchstart touchend", '.track-wrapper', function(e) {

            var touch = e.originalEvent.touches[0] || e.originalEvent.changedTouches[0];

            if (e.type == 'touchstart') {
                //has_permission_swipe = false;
                start_client_x = null;
                start_client_y = null;
                start_client_x = parseInt(touch.clientX);
                start_client_y = parseInt(touch.clientY);

            }

            if (e.type == 'touchend') {

                e.stopImmediatePropagation();

                //current_page = e.pageX;
                current_client_x = parseInt(touch.clientX);
                current_client_y = parseInt(touch.clientY);

                if (!start_client_x) {
                    return;
                }

                var xDiff = parseInt(parseInt(start_client_x) - parseInt(current_client_x));
                var yDiff = parseInt(parseInt(start_client_y) - parseInt(current_client_y));

                if (Math.abs(xDiff) < Math.abs(yDiff) || Math.abs(xDiff) < 10) {
                    return;
                }

                var meia_tela = parseInt(window.innerWidth / 2);

                if (start_client_x!= null) {

                    start_client_x = null;
                    start_client_y = null;
                    current_client_x = null;
                    current_client_y = null;

                    if (xDiff < 0 || (meia_tela < parseInt(current_client_x) && xDiff < meia_tela)) {
                        /* left swipe */

                        var btn = $(this).parent().parent().find('.carousel-prev .carousel-btn');

                        btn.trigger('click');
                    } else {

                    /* right swipe */
                         var btn = $(this).parent().parent().find('.carousel-next .carousel-btn');
                        btn.trigger('click');
                    }
                }

            }

        });

    });
}
</code>
Comment: vlw, usei no meu plug-in


Java: How to call a function whose name is stored in a string variable

Question: I have a generic string stored in a variable String func = "validate"; and i want to call the function of that name ie, i need to call validate() function.  I mean to say i will pass the variable to some function
public void callFunc(String func){
....
}

so, the above function should call the appropriate function with function name passed as a parameter to the callfunc().
Comment: A common example is loading the appropriate JDBC driver with Class.forName():  http://www.xyzws.com/Javafaq/what-does-classforname-method-do/17
Answer: You can use reflection to do this, e.g.
Method method = getClass().getMethod(func);
// TODO: Validation
method.invoke(this);

That's assuming you want to call the method on this, of course, and that it's an instance method. Look at Class.getMethod and related methods, along with Method itself, for more details. You may want getDeclaredMethod instead, and you may need to make it accessible.
I would see if you can think of a way of avoiding this if possible though - reflection tends to get messy quickly. It's worth taking a step back and considering if this is the best design. If you give us more details of the bigger picture, we may be able to suggest alternatives.
Comment: @ShamsUdDin: Then you use `invoke(this, val1, val2)`. Read the documentation I've linked to.
Comment: what if i want to pass the parameters to the function `func(val1, val2)`
Comment: How do I invoke another method. As in something like this `method.invoke(this).callFunc2()` ? @JonSkeet
Comment: @mypeople: You need to provide more information - e.g. whether you know the method return type at compile-time, and can just cast the result, or whether you want to invoke the *second* method with reflection too.
Comment: I know the return type at compile time
Comment: @mypeople: So cast: `((Foo) method.invoke(this)).callFunc2()`
Comment: Not able to get the callFunc2() in the suggestions even after casting
Comment: @mypeople: Then you need to ask a new question with a [mcve].
Answer: There is another way than using reflections. You can write a pattern matching like the following:
switch (func) {
  case "validate": validate(); break;
  case "apply": apply(); break;
  default: break;
}

But I agree with Jon: try to avoid this. If you use pattern matching you have to apply changes twice. If you add another method for example, you need to add it to the function naming method and to the pattern matching.
Comment: if we use the switch function then our code will not become generic, i mean to say suppose i created a new function called `mypage()` then again i need to add mypage case in switch function.
Comment: That is exactly what I said at the bottom.


Do i need to exclude id, that are like about 70% of records in table from complex index

Question: I hava a simple test table
id iteger pk,
test_id integer fk ....,
someFlag boolean,
other columns....
most of time we need to get a row by test_id and someFlag, so that i made an index
create index test_index(test_id, someFlag) - and everything going good, but 70% of records in this table have same test_id and if i trying to get rows with it - there is no index scan, seq scan only.
So if this test_id do not scan by index should i exlude it from my index like this
create index test_index(test_id, someFlag) where test_id != some value;

it will make the index size much smaller and index scan will be faster
Am i right?
Answer: Quoting from the Postgres docs on Partial Indexes:

One major reason for using a partial index is to avoid indexing common values. Since a query searching for a common value (one that accounts for more than a few percent of all the table rows) will not use the index anyway, there is no point in keeping those rows in the index at all. This reduces the size of the index, which will speed up those queries that do use the index.


jquery instead of javascript

Question:

$(document).ready(function() {
        var table = document.getElementById("tab");
        table.style.borderCollapse = "collapse";
        var rows = table.rows;

        for(i = 0; i < rows.length; i++) {
         if(i % 2 == 0) {
          rows[i].style.backgroundColor = "red";
         }
        }
       });

      <title>JJJJS</title>

        1
        2
        3

        4
        5
        6

        7
        8
        9

It is working, but how can I achieve similar result using only jQuery? I was trying to use something like $("#tab"); but it isn't working for me. Where I make mistake? I mean I want to color i % 2 == 0 rows.
@edit:
for example it not working

Comment: "but it isn't working for me" What exactly is not working?
Comment: `#tab { border-collapse: collapse } #tab tr:nth-child(odd) { background-color: red }`
Comment: [jQuery Learning Center](https://learn.jquery.com)
Comment: (It got it backward, but it's the same answer: Try to do it, and then if you get stuck, post a question about what you're stuck on.)
Answer: Try this:

$(document).ready(function() {
var i = 0;
  $('#tab tr').each(function() {
     if(!(i++%2)) { // if(i % 2 == 0 and increment i as well
       $(this).css('background','red');
     }
  });
});

      <title>JJJJS</title>

        1
        2
        3

        4
        5
        6

        7
        8
        9

jquery instead of javascript


Why is the determinant of a proper rotation matrix 1?

Question: For any rotation matrix, R, I know that $RR^T = I$ and thus $det(R)=±1$. And I understand that proper rotations preserve orientation and that is why a proper matrix, R, is one such that  $det(R)= 1$ However, I am unsure how to actually prove this for a rotation matrix of general finite dimension.
Thank you!
Comment: Well, how do you *define* orientation and proper rotation? Start from there
Comment: I think you need to explain what is meant by "a rotation matrix of general finite dimension" first. Some people (me included) refer to matrices in $SO(n;\mathbb R)$ as "rotation matrices" (note: this is a nonstandard usage; *many* people reject it when $n>3$), but in this case, $\det R=1$ *by definition*.
Answer: If you are talking about 2D or 3D rotations, the reason is simple: every proper rotation $R$ about an axis for some angle is the square of the proper rotation $Q$ about the same axis for the half angle; therefore $\det R=(\det Q)^2$ cannot be negative.
Answer: If it preserves orientation, so you begin from identity and get again identity after a full rotation. This means that the proper rotation must contain identity matrix for some special values. The determinant of identity matrix is $+1$. You can check that some sort of transformations like reflection about one axis has determinant $-1$ as it changes orientation.
Answer: For a 2D/3D rotation matrix one of the eigenvalues must be $1$ the other $2$ are complex conjugates of unit modulus $e^{\pm i \theta}$. Since the determinant is the product of the eigenvalues then $\det(R)=+1$
(The $2 \times 2$ $2$D rotation matrix is a shorthand for the $3\times3$ matrix rotating about and preserving the z axis)
Answer: The determinant of a 2D rotation is clearly one:
$$ det\begin{pmatrix} cos(\theta) & -\sin(\theta) \\
sin(\theta) & cos(\theta) \end{pmatrix} = cos^2(\theta) + sin^2(\theta) = 1$$
A general rotation in N dimensions can be obtained by composing (multiplying) $\begin{pmatrix} N \\ 2 \end{pmatrix}$ (N takes 2) 2-dimensional rotations, because we have a perpendicular plane on which it is possible to rotate for each pair of axes. (For example in 3D we have $\begin{pmatrix} 3 \\ 2 \end{pmatrix} = 3$ perpendicular planes on which to rotate).
Given that a 2D rotation in N dimensions (with identity for all the perpendicular axes) still has determinant 1 (in fact for example):
$$ det\begin{pmatrix} cos(\theta) & -\sin(\theta) & 0\\
sin(\theta) & cos(\theta) & 0 \\
0 & 0 & 1 \end{pmatrix} = cos^2(\theta) + sin^2(\theta) = 1$$
and:
$$ det (A B ) = det (A) det(B) $$
We can see that a rotation in N dimensions still has determinant 1.


Node token in rules (Events: When an order is first paid in full)

Question: In comments there is Product reference-Inline entity form-Single value, so when a product is posted in comments, it can be bought by somebody.
Is it possible to have a node token in Conditions or Actions in rules when in Events 'When an order is first paid in full' is used? For example, it would be nice to have a node token in Conditions-'Entity has field' and in Actions-'Set a data value'.
Comment: Not sure if I fully understand your question, but maybe you should try something like using a Rules Action like "Fetch entity by property"?
Answer: You can have a node token for that action, but you must load one manually using Fetch entity by id.
Comment: Hello, I tried several times to get this token but it doesn't work. May I miss anything? Entity by id mustn't be a particular one, it must also be provided by using a token.


Create database fields to an existing table when installing a module

Question: I'm new to drupal and I need some help.
I created a new module with name user_extra. Which create necessery table when Installing the module.
But I also need to insert some database field in 'users_field_data' table ('user_extra_id' and 'user_office')
In baseFieldDefinitions I wrote all necessery database table fields for user_extra table.
 public static function baseFieldDefinitions(EntityTypeInterface $entity_type) {
    // Standard field, used as a unique if primary index.
    $fields['id'] = BaseFieldDefinition::create('integer')
        ->setLabel(t('Unit ID'))
        ->setDescription(t('The unit ID.'))
        ->setReadOnly(TRUE)
        ->setSetting('unsigned', TRUE);

    $fields['uuid'] = BaseFieldDefinition::create('uuid')
        ->setLabel(t('UUID'))
        ->setDescription(t('The unit UUID.'))
        ->setReadOnly(TRUE);
        .
        .
        .
    $fields['user_info'] = BaseFieldDefinition::create('string')
        ->setLabel(t('User Info'))
        ->setDescription(t("The User Info"))
        ->setSetting('max_length', 255)
        ->setDisplayOptions('form', array(
          'type' => 'string_textfield',
          'weight' => 10,
        ))
        ->setDisplayConfigurable('form', TRUE);
     return $fields;
     }

When I install the module it inserts all values that I define in baseFieldDefinitions.
But I also need to create 'user_extra_id' and 'user_office' in 'users_field_data' table with setDisplayOptions and setDisplayConfigurable options.
This is the commenting that I made in entity file
/**
 * @ContentEntityType(
 *   id = "user_extra",
 *   label = @Translation("user extra"),
 *   handlers = {
 *     "storage" = "Drupal\user_extra\UserextraStorage",
 *     "storage_schema" = "Drupal\user_extra\UserextraStorageSchema",
 *     "access" = "Drupal\user_extra\UserextraAccessControlHandler",
 *     "list_builder" = "Drupal\user_extra\UserextraListBuilder",
 *     "view_builder" = "Drupal\user_extra\UserextraViewBuilder",
 *     "views_data" = "Drupal\user_extra\UserextraViewsData",
 *     "form" = {
 *       "default" = "Drupal\user_extra\Form\UserextraForm",
 *       "add" = "Drupal\user_extra\Form\UserextraAddForm",
 *       "edit" = "Drupal\user_extra\Form\UserextraEditForm",
 *       "delete" = "Drupal\user_extra\Form\UserextraDeleteForm",
 *     }
 *   },
 *   admin_permission = "administer user_extra",
 *   base_table = "user_extra",
 *   entity_keys = {
 *     "id" = "id",
 *     "label" = "name",
 *     "uuid" = "uuid",
 *   },
 *   links = {
 *     "canonical" = "/admin/user_extra/{user_extra}",
 *     "add-form" = "/admin/user_extra/add",
 *     "delete-form" = "/admin/user_extra/{user_extra}/delete",
 *     "overview-form" = "/admin/user_extra/${user_extra}/overview",
 *     "edit-form" = "/admin/user_extra/{user_extra}",
 *     "collection" = "/admin/user_extra",
 *     "children" = "/admin/user_extra/{user_extra}/children",
 *   },
 * )
 */

Comment: See http://drupal.stackexchange.com/questions/144796/extend-drupal-8-user-profiles/144839#144839 for a question that should help help to answer some of your questions.
Answer: There are a couple of inconsistencies with the question and code so I am not sure whether or not this can be accepted as an answer to the strict question "How do I create database fields to an existing table when installing a module".

"user_extra" is defined as a new content entity. What this means is that it is a fundamental new thing that is not related to a Drupal User entity nor the User entity's storage. This might be a similar use case to Profile2.
Modifying base field definition is probably a bit too heavy/complex for adding in the "user_office" and "user_extra_id" data that your site needs. The "Drupal Way" would be to add a "Bundle Field" to the entity.

This could be done as a site builder in the user interface (See Configuration > People), programmatically or providing YAML in config/install (Berdir). See field.storage.user.user_picture.yml.
Adding many fields to the User entity may have a performance impact, which could be solved by using a module such as Profile2 to only load that information on certain pages.
Comment: Adding default config works perfectly for user, the only thing you need to take care of is configuring the widget and formatter so that it shows up in the UI. That needs to be done in hook_installl(), through the API.


Views slideshow numbered pager

Question: Im using Drupal 7, and I am trying to make a slide show with a numbered pager like the one shown in this page. I added PHP code in Views, thinking I could simple put PHP code, but the problem is that it is not producing the expected result.
I used this code, which is supposed to print "1 2 3 4", and that instead prints "1 1 1 1."
$i = 0;
$i++;
echo $i;

In this code I tried using one of the variables, and I think I am on the right track with this one, but it prints "10 9 8 7," while I want it prints "1 2 3 4."
$i = 0;
$i = $row->field_image;
echo $i;

I know that this is probably something simple, but I can't figure out how to save my life; any help is greatly appreciated.
Comment: I've been trying to do the same thing. Where do you put that code (default argument?) and i'm assuming that 'mycode_iterator_count' stands for something else. Any help is appreciated.
Comment: you put it in global php null
Answer: Much easier way:

Add "Global: View result counter" Field
Exclude it from Display
In Slideshow setting: choose that field as the Pager Type
Answer: The first code you wrote doesn't work because, all times it is invoked

it initializes $i to 0
it increments $i by 1
it prints the value of $1

Because the first step, the result is always 1.
To get a result that is different in any cases, the code should be similar to the following.
$i = variable_get('mycode_iterator_count', 0) + 1;
echo $i;
variable_set('mycode_iterator_count', $i);

The code is not optimized, and probably there is a better way to write the code; although, the code works, as it will print a different number every time it is invoked.
As the code is passed to eval(), using a static variable would not probably serve to the purpose because the static variable will probably be reset to 0 every time the code is executed through eval(). Probably the code would work even if it would use a global variable.
global $i;

if (empty($i)) {
  $i = 1;
}

echo $i++;

Answer: Embedding PHP inside the view is a bad idea. PHP Views should be avoided. Instead I have come up with the following which works with default views:

Click "add" on the fields box.
Add a "Global: Math expression" field to your fields.
In the "Expression" field enter [expression] + 1
Click "Exclude from display".
Set your pager field to be the expression.

Views slideshow numbered pager


Como criar um vetor a partir de outro, porém seus elementos devem ser maiores que x?

Question: Gostaria que v2 assumisse apenas os valores que fossem maiores que x. Ele até chega perto, porém cria um vetor com elementos lógicos. Gostaria que fossem os elementos de v1.
v1 <-c (1500, 1600, 1700, 1800)

x <- 1600

v2 <- v1 > x

Comment: `v1[v1 > x]` ou `subset(x = v1, v1 > x)`
Answer: Sem apelar para uma função pronta, você pode simplesmente utilizar um condicional dentro de um for loop como no exemplo abaixo:
v1=c(1500,1600,1700,1800)
v2=c()
for (i in v1) {
  if (i>1600) {
   v2=c(v2,i)
  }
}
print(v2)

É ineficiente e leva muita sintaxe, mas você entende melhor o que o computador está fazendo. Soluções pré-prontas incluem o uso de funções. Um exemplo é a função Filter:
v1=c(1500,1600,1700,1800)

v2=Filter(function(x) x>1600, v1)

print(v2)

Ou, em uma solução ainda mais simples sugerida no comentário por outro usuário:
v2=v1[v1 > x]
print(v2)

Comment: No `Filter` é `v1` e naõ `v`.
Comment: obrigado @RuiBarradas . Corrigido


How to arrive at a specific formulation of the relative median deviation? Related to integration and statistics.

Question: So my title is not very specific but here is the question in more detail.
I am an economist currently working with this book: Frank Cowell - Measuring Inequality
On page 25 a formulation of the relative mean deviation is given as follows:
$$
M = 2 \left[ F\left(\bar{y}\right) - \Phi(\bar{y}) \right]
$$
$F$ is the CDF, $\Phi$ is the proportion of total income received by persons who have an income less than or equal to $y$ ( per the book's definition: $\Phi=\frac{1}{\bar{y}} \int_0^y zdF(z)$), and $\bar{y}$ is the mean.
All this is also defined on page 152 in the appendix.
The appendix also gives a definition of $M$:
$$
M = \int \left| \frac{y}{\bar{y}} -1\right|dF
$$
The book says that the former formulation can be derived from the latter, but I have no idea how to begin with this. How do I perform the integration here and get to the first formulation?
Any help would be greatly appreciated.
Comment: Also why does it say median in your title but the question is about mean?
Comment: I think ive done it. I will post a solution soon.
Comment: The methodological problem here is that the first relation contains definite integrals while the second is given as an indefinite integral. What are the (effective) integration limits for the second relation? Since $y$ is income, are they $(0,\infty)$?
Comment: Yes they are $(0, \infty)$
Answer: I think you want your variable to be positive, so replace all the $-\infty$ in my proof by 0.
Comment: (+1) for the _literally_ back-of-the envelope calculation -which is also correct. The crucial thing here is that the way $M$ is defined, its value is decomposed symmetrically around $\bar y$, irrespective of whether $F$ is a symmetric distribution or not.
Comment: Ok, then I think that you have got it.
Comment: If we assume $F$ is differentiable (which is, for standard distributions, then $dF(y) = f(y) dy$ where $f$ is the pdf of the distribution. Maybe you are more use to this notation? $y$ is the 'dummy' variable being integrated over so the $y$ bit inside the integral is the thing being integrated over.
Comment: @AlecosPapadopoulos thanks and oh nice answer to the random walk identity. If did asks something, it was never going to be easy, so kudos to you.
Comment: Thanks. But these are the really enjoyable ones.
Comment: @option_select replace all the $-\infty$ by 0. It does not matter what the lower limit of your random variable is. It can be -10, 7/2 or $e^\pi$. I used $-\infty$ because a general random variable is defined on $(-\infty,\infty)$ and certainly that is how the integral in your text book should be interpreted, because he did not write down the domain of integration
Comment: Just one final thing that is probably fairly noobish, but for the integration it is irrelevant whether the variable y is in the integral but also contained in $F(y)$. Is that generally the case?
Comment: @option_select please say say that again, I don't quite follow.
Comment: Yes, I am more used to the other notation, but both result in the mean. I think I follow.


Any recommendations for books on CFA and SEM, R?

Question: Context:
-Psychology student interested in learning more about CFA and SEM with the use of R.
Recommendations appreciated.
Answer: Below are two books I have referenced in the past. I particularly like the 2015 book as it also goes into other latent variable modeling techniques (e.g., Item Response Theory) in an accessible way. It may also be worth using the lavaan website (also below) as a learning tool, as it has many good examples.
Finch, W. H., & French, B. F. (2015). Latent variable modeling with R. Routledge.
Gana, K., & Broc, G. (2019). Structural equation modeling with lavaan. John Wiley & Sons.
https://www.lavaan.ugent.be/


Problema com Criação e localização de modulo pelo AngularJS

Question: Observem o código;
<!DOCTYPE html ng-app="helloWword">

    <title>Ola Mundo</title>

    {{message}}

Como vocês podem perceber estou criando o modeulo aqui nessa linha;
angular.module("helloWword", []);

E aqui estou fazendo com que o modulo seja localizado;
    angular.module("helloWword").controller("helloWwordCtrl", function ($scope) {
        $scope.message = "ESTA APARECENDO NO SITE";
    });
</script>

e pela linha de código era para parecer ESTA APARECENDO NO SITE em tela do navegador, sendo que não é isso que acontece, ele simplesmente aparece {{message}}
O que será que fiz de errado?
Comment: angular 1.x ou 2.x?
Comment: Já experimentas-te com o `ng-app="helloWword"` no ``? funciona para mim... http://jsfiddle.net/s8wp9pob/
Answer: Seu erro está nas duas primeiras linhas:
<!DOCTYPE html ng-app="helloWword">

Você precisa declarar a app no inicio do html e não na declaração do tipo de arquivo. Embora uma melhor prática seja declarar o ngApp  no body como o Sergio♦ já comentou. Para funcionar o código deveria ser algo como:
<!DOCTYPE html>

  <title>Hello World</title>

    {{message}}

Comment: Gostaria de tira uma dúvida, tem como ter thymeleaf e AngularJS no mesmo projeto? eu acho que não tem como porque é necessário colocar as tag´s de thymeleaf para funcionar e no mesmo lugar que colocar as tag´s do thymeleaf é o mesmo lugar onde coloca a tag do AngularJS, se colocar a tag´s do thymeleaf o AngularJS não funciona.


Extrema of $(\tan{x})^{\sin{2x}}$ on $(0,\frac{\pi}{2})$ interval

Question: Function $(\tan{x})^{\sin{2x}}$ has its minimun for $x=u\in(0,\frac{\pi}{2})$ and maximum for $x=v\in(0,\frac{\pi}{2})$.
Find $u+v$
Having calculated the derivative: $$\frac{d}{dx}(\tan{x})^{\sin{2x}} = (\tan{x})^{\sin{2x}}\cdot(2\cos{2x\cdot\ln{\tan{x}}+2})$$  I find it quite difficult to find roots of it (as if there was some other special way/technique to approach this problem)
Can You give me any hint? I will appreciate everything.
Answer: Since $u,\,v$ are roots of $1+\cos 2x\ln\tan x$, which is invariant under $x\mapsto \pi/2-x$, $u+v=\pi/2$.
Comment: I think you mean $\pi$, good observation
Comment: @KingTut No, I was right.
Comment: Yes I made error
Answer: Note that $\tan x>0$ for $x\in (0,\pi/2)$. Let $y=\tan^{\sin 2x} x$ then taking logarithms yields
$$\ln y=\sin 2x\ln \tan x\Rightarrow \frac{y'}{y}=2\cos 2x\ln\tan x+\sin 2x\frac{(\tan x)'}{\tan x}$$
Overall using definition of $y$ and $(\tan x)'=1/\cos^2 x$ one obtains
$$y'=\tan^{\sin 2x} (2\cos 2x\ln\tan x+2)$$
Therefore $y'=0$ implies
$$2\cos 2x\ln\tan x+2=0\Leftrightarrow \cos 2x\ln\tan x=-1$$
since $\tan^{\sin 2x}>0$ for all $x\in(0,\pi/2)$. Note that $g(x):=\cos 2x\ln\tan x$ is strictly increasing on the interval $(0,\pi/4)$ since
$$g'(x)=-2\sin 2x\ln\tan x+\frac{\cos 2x}{\cos x\sin x}>0$$
moreover $\lim_{x\to 0^+}g(x)=-\infty$ and $g(\pi/4)=0$ therefore $g(x)$ intersects the horizontal line $y=-1$ only once. On the other side $g(x)$ is strictly decreasing for $x\in (\pi/4,\pi/2)$ since $g'(x)<0$ there. Likewise $g(\pi/4)=0$ and $\lim_{x\to\pi/2^-}g(x)=-\infty$ (since $\cos 2x<0$ for $x\in(\pi/4,\pi/2)$) Hence $g(x)$ intersects the line $y=-1$ only once more on the interval $(\pi/4,\pi/2)$. On the other hand we have
$$\cos 2x\ln\tan x=\cos(2(\frac{\pi}{2}-x))\ln\tan(\frac{\pi}{2}-x)$$
this implies that $x\in(0,\pi/4)$ solves $g(x)=-1$ if and only if $(\pi/2-x)$ solves the same equation. Therefore the two roots $x_1$ and $x_2$ are related by $x_2=\pi/2-x_1$. Since $x_1$ and $x_2$ are unique the sum of the roots of $g(x)=-1$ is then equal to $$x_1+x_2=\frac{\pi}{2}$$
Answer: Since logarithm and exponential are increasing functions, inverse to one another, you can study instead
$$
f(x)=\sin 2x \log\tan x=\sin 2x(\log\sin x-\log\cos x)
$$
Its derivative is
\begin{align}
f'(x)
&=2\cos2x\log\tan x+\sin 2x\left(\frac{\cos x}{\sin x}+\frac{\sin x}{\cos x}\right)\\[6px]
&=2\cos2x\log\tan x + 2 \\[6px]
&=2(1+\cos 2x\log\tan x)
\end{align}
which agrees with your computation (apart from my preference for $\log$ over $\ln$).
Now consider
$$
\cos2x=2\cos^2x-1=\frac{2}{1+\tan^2x}-1=\frac{1-\tan^2x}{1+\tan^2x}
$$
Since $\tan x>0$ in your setting, you can set $t=\tan^2x$ and study instead
$$
g(t)=2(1+t)+(1-t)\log t \qquad (t>0)
$$
and find its zeros. We have
$$
g'(t)=1-\log t+\frac{1-t}{t}=1-\log t+\frac{1}{t}
$$
and
$$
g''(t)=-\frac{1}{t}-\frac{1}{t^2}<0
$$
This means $g$ is concave. Moreover
$$
\lim_{t\to0}g(t)=-\infty
\qquad
\lim_{t\to\infty}g(t)=-\infty
$$
so we conclude that $g$ has a single point of maximum $t_0$, where $\log t_0=1+\frac{1}{t_0}$; now
$$
g(t_0)=2+2t_0+(1-t_0)\left(1+\frac{1}{t_0}\right)=\frac{(1+t_0)^2}{t_0}>0
$$
We conclude that $g$ vanishes twice, say at $t_1$ and $t_2$, being negative over $(0,t_1)$, positive over $(t_1,t_2)$ and negative again over $(t_1,\infty)$.
The same behavior will have $f'$: negative over $(0,\arctan\sqrt{t_1})$, positive over $(\arctan\sqrt{t_1},\arctan\sqrt{t_2})$ and negative over $(\arctan\sqrt{t_2},\pi/2)$.

Now that you know the existence and uniqueness of $u=\arctan\sqrt{t_1}$ and $v=\arctan\sqrt{t_2}$, it should be easy to exploit the symmetry and find their sum.


Impressão de relatórios no Firefox utilizando Telerik Reporting

Question: Estou com um problema, que pelo visto está incomodando outros desenvolvedores que utilizam os componentes da Telerik....
http://www.telerik.com/forums/print-button-in-firefox-starts-pdf-download
E os desenvolvedores Telerik...
http://www.telerik.com/support/kb/reporting/details/print-in-firefox
O fato é que no Firefox ele não imprime ao clicar no botão de impressão, ele pede para fazer o download ou abrir o PDF do relatório.
Alguém sabe se é possível resolver este problema? Se sim, como resolvo?
Comment: É a primeira vez que ouço falar de Telerik, mas se os desenvolvedores ainda não tem resolução oficial, acho que vai ser complicado... Lá no fórum, o admin publicou uma gambiarra, ela não atende? Tem um link também pra [esta discussão no bugzilla](https://bugzilla.mozilla.org/show_bug.cgi?id=840439), viu já?
Comment: Sim, vi a gambiarra e vi a discussão no bugzilla, a gambiarra não funciona...
Comment: Achou alguma solução? Poste como resposta para ajudar outras pessoas.
Comment: Não encontrei nenhuma solução ainda, se você conhecer alguma, poste por favor. Atualmente o problema persiste, apenas foi solicitado que o usuário utilize o Google Chrome.
Answer: O telerik reporting não funciona em algumas versões do Firefox por usar pdf.js. Veja o ticket https://bugzilla.mozilla.org/show_bug.cgi?id=843342
Como solução de contorno mande exportar para pdf.
Obs.: acho que a nova versão do reporting e do firefox já estão ok pois antes ele não fazia nada, agora faz download do pdf ou abre o pdf no firefox, depende da configuração do firefox.
Comment: Testei com a última versão do Telerik Reporting e com a última versão do Mozila, mas ainda ele pede para salvar em PDF. Isso até que não seria um grande problema, porém os meus usuários não aceitam a ideia de ter que salvar para depois imprimir, deste modo, continuo solicitando a utilização do Google Chrome. Obrigada.
Comment: Mas é essa a solução. No melhor caso oriente os clientes a abrir o relatório usando o plugin do adobe reader configurando o firefox em Opções > Aplicativos e configurando o firefox para abrir os documentos pdf diretamente pelo firefox. Infelizmente é isso mesmo.


SQL syntax error were am i goign wrong?

Question: Hello guys and girls im trying to a sql update but think i forgot a ' or a "
im getting this error messege
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''Brock'='1'WHERE username = 'admin'' at line 1
The fault lies with in this bit of code if i take the code out the page loads witht he rest of the scripts on it. But need it two do the update.
$blah = mysql_query("UPDATE users SET '".$_SESSION['gymleader']."'='1'WHERE username = '".$_SESSION['username']."'")
or die(mysql_error());

Were am i going wrong ?
Comment: You have a column named Brock?.... love your hard-coding! It'll cause you loads of problems in the future. Column names aren't strings to be quoted using ' If they do need quoting because they're reserved words, then you use backticks `
Comment: the error message points you directly to your error. See Robin's answer for more information and read carefully what the computer says and what you type :)
Comment: Wow. I think it would help to see the table structure.
Answer: You miss a space between the '1' and the WHERE if I am not mistaken. And you should use backticks (`) when you want to escape a column name
So your code becomes:
$blah = mysql_query("UPDATE users SET `".$_SESSION['gymleader']."`='1' WHERE username = '".$_SESSION['username']."'")

Note the ` instead of the ' around the column name (right after the SET).
Further possible improvements:

In case the column is of type INT, you can replace the '1' by 1 (without the ')
You should never directly use the $_SESSION,$_POST,$_GET or other values which can be altered by users in your queries. Do a Google search on SQL injection for more information
Comment: Have you also taken care of the comment of Mark Baker (the use backticks around your column names, and only when needed) ?
Comment: No i don't know what he is on about please explain ?
Comment: I updated my answer and included some of the comments made by others in it. Hopefully you understand it now
Answer: UPDATE user SET field = '1' WHERE ...

instead of
UPDATE user SET 'field' = '1' WHERE ...

and if your field is of type int, you might use
UPDATE user SET field = 1 WHERE

If you want to escape your fieldname, use
`field`

in backticks `
Comment: I want to change were you have "field " into a session virable  $_SESSION['gymleader']  im getting ''Brock' = 1 WHERE username = 'admin''      if you notice there are 2  '' in front of the word brock maybe that is the problem ?
Comment: that would be mysql_query("UPDATE users SET ".$_SESSION['gymleader']." = '1' WHE...
Answer: Besides the fact that this looks like a bad idea to code like this, assuming you have a column named Brock then you should use this types of quotes instead:
$blah = mysql_query("UPDATE users SET `".$_SESSION['gymleader']."`='1' WHERE username = '".$_SESSION['username']."'")
or die(mysql_error());

Notice I replaced your ' with `
Comment: @nute, Stackoverflow uses backquotes to mark up code - this is also achieved by indenting, which then leaves the backquotes unaffected. I have amended your answer accordingly.
Comment: Thanks for getting to the point and not taking the mick like others about the `

SQL syntax error were am i goign wrong?


vb.net open word document return null

Question: I use the following code to convert doc to pdf file, but when opening doc it returns nothing:
Dim wordApplication As New Microsoft.Office.Interop.Word.Application
Dim wordDocument As Microsoft.Office.Interop.Word.Document = Nothing
Dim outputFilename As String = IIf(fileout = "", System.IO.Path.ChangeExtension(filename, "pdf"), fileout)
wordDocument = wordApplication.Documents.Open(filename)
wordDocument.ExportAsFixedFormat(outputFilename, Microsoft.Office.Interop.Word.WdExportFormat.wdExportFormatPDF, False, Microsoft.Office.Interop.Word.WdExportOptimizeFor.wdExportOptimizeForOnScreen, Microsoft.Office.Interop.Word.WdExportRange.wdExportAllDocument, 0, 0, Microsoft.Office.Interop.Word.WdExportItem.wdExportDocumentContent, True, True, Microsoft.Office.Interop.Word.WdExportCreateBookmarks.wdExportCreateNoBookmarks, True, True, False)

Answer: There are many not defined variables in your code:
fileout,filename,outputFilename.
this is a working code
Dim wordApplication As New Microsoft.Office.Interop.Word.Application
Dim wordDocument As Microsoft.Office.Interop.Word.Document = Nothing
Dim filename As String = "yourworddocument.docx"
Dim fileout As String = "yourworddocument.pdf"
wordDocument = wordApplication.Documents.Open(filename)
wordDocument.ExportAsFixedFormat(fileout, Microsoft.Office.Interop.Word.WdExportFormat.wdExportFormatPDF, False, Microsoft.Office.Interop.Word.WdExportOptimizeFor.wdExportOptimizeForOnScreen, Microsoft.Office.Interop.Word.WdExportRange.wdExportAllDocument, 0, 0, Microsoft.Office.Interop.Word.WdExportItem.wdExportDocumentContent, True, True, Microsoft.Office.Interop.Word.WdExportCreateBookmarks.wdExportCreateNoBookmarks, True, True, False)

vb.net open word document return null


Linux software RAID: Is a disk faulty?

Question: I have a setup with three identical harddrives, recognized as sdb, sdd and sde. I have one RAID0 partition (md0) and two RAID5 partitions (md1 and md2) across these three disks. All my RAID partitions appear to be working normally, and have done so since I created them. I have seen messages on the console about md[12] being "active with 2 out of 3 devices", which to me sounds like a problem.
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [linear] [multipath] [raid1] [raid10]
md2 : active raid0 sdb3[0] sdd3[1] sde3[2]
      24574464 blocks super 1.2 512k chunks

md1 : active raid5 sdd2[1] sde2[3]
      5823403008 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]

md0 : active raid5 sdd1[1] sde1[3]
      20462592 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]

unused devices: <none>

I'm not experienced with mdadm, but this to me seems like arrays md[12] are missing the sdb disk. However, md2 does not seems to be missing anything. So, has the sdb disk failed or is this just some configuration issue? Any more diagnostics I need to do to figure that out?
EDIT:
# mdadm --examine /dev/sdb2
/dev/sdb2:
         Magic : a92b4efc
       Version : 1.2
   Feature Map : 0x0
    Array UUID : 94d56562:90a999e8:601741c0:55d8c83f
          Name : jostein1:1  (local to host jostein1)
 Creation Time : Sat Aug 18 13:00:00 2012
    Raid Level : raid5
  Raid Devices : 3
Avail Dev Size : 5823404032 (2776.82 GiB 2981.58 GB)
    Array Size : 5823403008 (5553.63 GiB 5963.16 GB)
 Used Dev Size : 5823403008 (2776.82 GiB 2981.58 GB)
   Data Offset : 262144 sectors
  Super Offset : 8 sectors
  Unused Space : before=262064 sectors, after=1024 sectors
         State : active
   Device UUID : cee60351:c3a525ce:a449b326:6cb5970d

   Update Time : Tue May 24 21:43:20 2016
      Checksum : 4afdc54a - correct
        Events : 7400

        Layout : left-symmetric
    Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)

# mdadm --examine /dev/sde2
/dev/sde2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 94d56562:90a999e8:601741c0:55d8c83f
           Name : jostein1:1  (local to host jostein1)
  Creation Time : Sat Aug 18 13:00:00 2012
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5823404032 (2776.82 GiB 2981.58 GB)
     Array Size : 5823403008 (5553.63 GiB 5963.16 GB)
  Used Dev Size : 5823403008 (2776.82 GiB 2981.58 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=1024 sectors
          State : clean
    Device UUID : 9c5abb6d:8f1eecbd:4b0f5459:c0424d26

    Update Time : Tue Oct 11 21:17:10 2016
       Checksum : a3992056 - correct
         Events : 896128

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : .AA ('A' == active, '.' == missing, 'R' == replacing)

So --examine on sdb shows it is active, while the same command on sdd and sde show it as missing.
# mdadm --detail --verbose /dev/md1
/dev/md1:
        Version : 1.2
  Creation Time : Sat Aug 18 13:00:00 2012
     Raid Level : raid5
     Array Size : 5823403008 (5553.63 GiB 5963.16 GB)
  Used Dev Size : 2911701504 (2776.82 GiB 2981.58 GB)
   Raid Devices : 3
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Tue Oct 11 22:03:50 2016
          State : clean, degraded
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : jostein1:1  (local to host jostein1)
           UUID : 94d56562:90a999e8:601741c0:55d8c83f
         Events : 897492

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       50        1      active sync   /dev/sdd2
       3       8       66        2      active sync   /dev/sde2

EDIT2:
The event count for the device no longer part of the array is very different from the others:
# mdadm --examine /dev/sd[bde]1 | egrep 'Event|/dev/sd'
/dev/sdb1:
         Events : 603
/dev/sdd1:
         Events : 374272
/dev/sde1:
         Events : 374272

Smartmontools for the disk that is not part of the array:
# smartctl -d ata -a /dev/sdb
smartctl 6.4 2014-10-07 r4002 [x86_64-linux-4.2.0-36-generic] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (AF, SATA 6Gb/s)
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ2185619
LU WWN Device Id: 5 0014ee 25c58f89e
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Wed Oct 12 18:54:30 2016 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (51480) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 494) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   147   144   021    Pre-fail  Always       -       9641
  4 Start_Stop_Count        0x0032   099   099   000    Old_age   Always       -       1398
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   090   090   000    Old_age   Always       -       7788
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       1145
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       45
193 Load_Cycle_Count        0x0032   097   097   000    Old_age   Always       -       309782
194 Temperature_Celsius     0x0022   124   103   000    Old_age   Always       -       28
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Comment: check the logfiles
Comment: There are no errors in dmesg. Any other logs I should check?
Comment: `mdadm --examine /dev/sdb2`
Comment: Look at the SMART data for the failed drive?  smartctl. See if it is reporting as failed?
Comment: @Zoredache smartctl shows the drive is fine, as far as I can tell, see output as comment to the question. The output is very similar to other disks of the same type that work just fine.
Answer: Your mdstat file says it all.
[3/2] [_UU] means that while there are 3 defined physical volumes, only 2 are in use at the moment. Similarly the _UU says the same.
For grater details on the raid devices (before going to the physical ones) you'd run (as root)
mdadm --detail --verbose /dev/md0
mdadm --detail --verbose /dev/md1
mdadm --detail --verbose /dev/md2

On my system (using raid6) I have simulated a failure and this is an example output:
dev/md0:
        Version : 1.2
  Creation Time : Thu Sep 29 09:51:41 2016
     Raid Level : raid6
     Array Size : 16764928 (15.99 GiB 17.17 GB)
  Used Dev Size : 8382464 (7.99 GiB 8.58 GB)
   Raid Devices : 4
  Total Devices : 5
    Persistence : Superblock is persistent

    Update Time : Thu Oct 11 13:06:50 2016
          State : clean   <<== CLEAN!
 Active Devices : 4
Working Devices : 4
 Failed Devices : 1
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : ubuntu:0  (local to host ubuntu)
           UUID : 3837ba75:eaecb6be:8ceb4539:e5d69538
         Events : 43

    Number   Major   Minor   RaidDevice State
       4       8       65        0      active sync   /dev/sde1  <<== NEW ENTRY
       1       8       17        1      active sync   /dev/sdb1
       2       8       33        2      active sync   /dev/sdc1
       3       8       49        3      active sync   /dev/sdd1

       0       8        1        -      faulty   /dev/sda1  <<== SW-REPLACED

Comment: So I have "state: clean, degraded" in the middle and "removed" for the listing of disk number 0 (which I assume should have been sdb). Can I recover from "removed" somehow? The full output has been included in the question.
Comment: @josteinb You'd add spare devices so the RAID can be automatically reconstruced. This is my current setup, as I showed in the answer.
Comment: @josteinb By the way, I prefer to 1) create a RAID device over entire disks, 2) then partition (and 3) then LVM eveything). You first did partitioning, then the RAID. My starting point is: does that make any sense to you? When a drive is gone, then all partitions are (supposed to be) gone as well.
Comment: I see your point about raid first, partitioning second. It is a long time since I created this setup, but I would think the reason why I partitioned first is that I have one raid0 and two raid5 partitions.
Answer: md1 and md2 are raid5 arrays, degraded because their respective partitions on /dev/sdb are failed or marked falty. Run mdadm --examine on the array itself for more details (madam --examine /dev/md1).
If all is well with /dev/sdb, re-add the partitions to the arrays. Get the correct partition numbers from your /etc/mdadm.conf or the output of --examine on the array.
mdadm --re-add /dev/sdb[?] /dev/md1
Comment: I tried this, --re-add does not work. mdadm just responded that the device could not be re-added. It may be related to the event counts shown in EDIT2 of the question.
Answer: Yes, /dev/sdb1 and /dev/sdb2 was kicked from /dev/md0 and /dev/md1 respectively. You can doing grep on your syslogs (/var/log/messages* on the RHEL/CentOS/etc based distros, /var/log/syslog* on the Debian/Ubuntu based distros) to find what has caused this if these are still kept.
To fix this now, first thing I suggest you to issue a SMART test on the /dev/sdb. This can be done with smartctl -t long /dev/sdb (this is a non-destructive test) and then you can check both it's progress and results with smartctl -a /dev/sdb.
If everything looks to be fine with the disk afterwards, you can re-add partitions to the RAID arrays which might cause rebuilds (and most likely will), which is still non-destructive:
mdadm /dev/md0 --add /dev/sdb1
mdadm /dev/md1 --add /dev/sdb2

You can watch rebuild progress by issuing watch cat /proc/mdstat command (which will print /proc/mdstat every 2 seconds on screen).
If you didn't had write-intent bitmap before, I strongly suggest to add it after rebuild:
mdadm -G /dev/mdX -b internal

Replace X with your array number. Proceed for every array you have active. It doesn't takes much space, and is non-destructive too. But it helps with data consistency and rebuild speed sometimes.

Linux software RAID: Is a disk faulty?


Evaluation of $\int_{0}^{\frac{\pi}{4}} \frac{\log(\log(\tan(\frac{\pi}{4} + x))) \cdot \log(\tan(\frac{\pi}{4} + x)))}{\tan(2x)} \,dx$

Question:
$$\int_{0}^{\frac{\pi}{4}} \frac{\log(\log(\tan(\frac{\pi}{4} + x))) \cdot \log(\tan(\frac{\pi}{4} + x)))}{\tan(2x)} \,dx$$

$$\int_{0}^{\frac{\pi}{4}} \frac{\log(\log(\tan(\frac{\pi}{4} + x))) \cdot \log(\tan(\frac{\pi}{4} + x))}{\tan(2x)} \,dx = -\int_{0}^{\frac{\pi}{4}} \log(-\log(\tan x)) \cdot \log(\tan x) \cdot \tan(2x) \,dx$$
$$= -2 \int_{0}^{\frac{\pi}{4}} \frac{\log(-\log(\tan x)) \cdot \log(\tan x) \cdot \tan x}{1-\tan^2 x} \,dx \Bigg|_{\tan x=e^{-t}}$$
$$= 2 \int_{0}^{\infty} \frac{t \cdot \log(t \cdot e^{-2t})}{1 - e^{-4t}} \,dt = 2 \sum_{n=0}^{\infty} \int_{0}^{\infty} t \cdot \log(t \cdot e^{-(4n+2)t}) \,dt$$
$$= 2 \sum_{n=0}^{\infty} L \cdot (t \cdot \log(t)) \cdot (4n + 2)$$
Comment: What is the question?
Comment: @KurtG. How to go further or a better way to approach?
Comment: What is that $L\,?$ Not that I am *that* interested. If I were you I would check my formulas numerically. If they pass I hand in that solution and do something more interesting.
Answer: $$\begin{align}I=&\int_{0}^{\frac{\pi}{4}}\frac{\ln\left(\ln\left(\tan\left(x+\frac{\pi}{4}\right)\right)\right)\cdot\ln\left(\tan\left(x+\frac{\pi}{4}\right)\right)}{\tan\left(2x\right)}dx\\
=&-\int_{\frac{\pi}{4}}^{\frac{\pi}{2}}\ln\left(\ln\left(\tan\left(x\right)\right)\right)\cdot\ln\left(\tan\left(x\right)\right)\tan\left(2x\right)dx\\
=&\int_{1}^{+\infty}\ln\left(\ln\left(t\right)\right)\ln\left(t\right)\cdot\frac{2t}{t^{4}-1}dt\\
=&\int_{0}^{\infty}\ln\left(s\right)s\frac{2e^{2s}}{e^{4s}-1}ds\end{align}$$
Using the Feymann's trick:
$$f(z):=\int_{0}^{\infty}x^{z}\frac{2e^{2s}}{e^{4s}-1}ds$$
$$f(z)=(2^{-z} - 2^{-2z - 1}) \zeta(1 + z) \Gamma(1 + z)$$
$$I=f'(1)$$
$$I=\frac{\pi^2}{48}\left(3 + \ln(2) + 3 \ln(\pi)-36 \ln(A)\right)\approx -0.375831$$
Where:

$\zeta(z)$ is the zeta Riemann function
$\Gamma(z)$ is the Gamma function
$A$ is the Glaisher-Kinkelin Constant


How do I get the file URI from an EntityMetadataWrapper object?

Question: Getting hold of file objects related to nodes through a field is easy using an EntityMetadataWrapper:
$wrapper = entity_metadata_wrapper('node', $node);
$images = $wrapper->field_images;

So that's great. However, I want to use some of the properties of the standard file object, and they're not available. When I look at the properties available on $images[0]->file using the above code, I see these:

fid
name
mime
size
url
timestamp
owner

Most notably, uri is missing, and in its place there's a url property, which is the full external URL to the image file. I want to manipulate these images via their URI (create thumbs, etc), so I'm left doing this:
$fid = $images[0]->file->fid->value();
$uri = file_load($fid)->uri;

Which seems to negate the whole point of using an EntityMetadataWrapper in the first place.
What am I missing? Where is the URI on the wrapped file object? If it's not available, for the love of all things good and pure, why not?!
I'm aware I could deconstruct the URL back to a URI, but that also feels very unnecessary.
Comment: Are you using file_entity?
Comment: Did a quick test and `uri` is there with file_entity.
Comment: @MPD Ahhhh, great call, no I'm not...maybe I shouldn't be expecting to see what I'd consider 'properties' if the object isn't acting like an entity. URL seems a strange choice instead of URI for the Entity module guys to have chosen, though. If the answer is "you're going to need to implement `hook_entity_property_info_alter()`" then that's fine, just wanted to know
Comment: I'm actually not sure if that will make a difference, but having that module enabled does end up in some subtle differences here and there.  I have found File integration with EMW a bit lacking in other places, but haven't been bothered enough by it to submit a patch.
Comment: @MPD I think it will - just found out the 'offender' is `entity_metadata_system_entity_property_info()`, where a bunch of new properties get tacked onto `$info['file']['properties']`. I'm willing to bet if file_entity was in there that array would already have the entity properties themselves in there. I'll try to write an answer up later or tomorrow if no-one's already done it
Answer: Big thanks to MPD for pointing me in the right direction. In short, without the File Entity module, files aren't entities, so they don't naturally have the 'properties' I was expecting.
The Entity module provides some on their behalf, but URI isn't one of them. Fortunately it's very simple to make it available using hook_entity_property_info_alter() and a custom property callback:
function MYMODULE_entity_property_info_alter(&$info) {
  $info['file']['properties']['uri'] = array(
    'label' => t("URI"),
    'description' => t("The URI of the file."),
    'getter callback' => 'MYMODULE_entity_get_file_properties',
    'schema field' => 'uri',
  );
}

function MYMODULE_entity_get_file_properties($file, array $options, $name) {
  switch ($name) {
    case 'uri':
      return $file->uri;
      break;
  }
}

After that, $images[0]->file->uri->value() is available as expected.
Answer: Below works without the hack:
 $node_wrapper->field_media[0]->value()['uri'];

Comment: It's good to point out that this is supported from PHP 5.4 on.
Answer: This is what works for me to get the full path of a single image...
// wrap it
$wrapper = entity_metadata_wrapper('node', node_load(16));
// get the value
$image = $wrapper->field_image->value();
// check it
if ($image) {
   // get the full path
   $path = file_create_url($image['uri']);
}
print_r($path);

Alternatively...
  $image = $wrapper->field_image->value();
  $image_path = $image ? file_create_url($image['uri']) : '';


How to redefine default theme_image() function?

Question: By default drupal use theme_image() for render images. How to set for render custum function, ex. theme_cus_image()?
Answer: If your purpose is overriding theme_image(), and make Drupal (or third-party modules) use your theme function when calling theme('image'), then you need to implement hook_theme_registry_alter().
function mymodule_theme_registry_alter(&$theme_registry) {
  if (isset($theme_registry['image'])) {
    $theme_registry['image']['function'] = 'theme_cus_image';
  }
}

Implementing that code, when a module uses code similar to the following one, it will call your function, and not theme_image().
$form['enable'] = array(
  '#markup' => theme('image', array('path' => 'misc/watchdog-error.png', 'alt' => $status_short, 'title' => $status_short)),
);

If you need to override theme_image(), and use your code when the image class is a specific one, and theme_image() in the other cases, then you should use code similar to the following one.
function mymodule_theme_registry_alter(&$theme_registry) {
  if (isset($theme_registry['image'])) {
    $theme_registry['image']['function'] = 'theme_cus_image';
  }
}

function theme_cus_image($variables) {
  if (!empty($variables['attributes']['class']) && $variables['attributes']['class'] == 'the class you are looking for') {
    // Add your code.
  }
  else {
    return theme_image($variables);
  }
}

The problem with that code is that it is assuming no other module overridden the same theme function. If another module with a lower weight (or for which its hook_theme_registry_alter() is executed first) overrides theme_image(), the new theme function would not be called from the code I shown, as it explicitly calls theme_image().
It is also not possible to call the original theme function with theme('image') as that would cause an infinite loop: theme('image') would call theme_cus_image(), which would call theme('image'), which would call theme_cus_image(), which would call theme('image'), which would call […].
Keep in mind that, with Drupal 7, preprocess functions are invoked also for theme functions that don't use template files. If you want to replace theme_image() just to alter one of the variables it gets, then you can use code similar to the following one, which unsets the width, and the weight of the image.
function mymodule_preprocess_image(&$variables) {
  foreach (array('width', 'height') as $key) {
    unset($variables[$key]);
    unset($variables['attributes'][$key];
  }
}

Similarly, if you would want to alter the class assigned to the image, you could use the following code.
function mymodule_preprocess_image(&$variables) {
  if (!empty($variables['attributes']['class']) && $variables['attributes']['class'] == 'the class you want to alter') {
    $variables['attributes']['class'] = 'your class';
  }
}

If your purpose is defining a new theme function that is used from your module, then you simply need to implement hook_theme().
function mymodule_theme() {
  return array(
    'cus_image' => array(
      'variables' => array('path' => NULL, 'width' => NULL, 'height' => NULL, 'alt' => '', 'title' => NULL, 'attributes' => array()),
    ),
  );
}

In this case, theme_cus_image() will be used from code that calls theme('cus_image'). The code that calls theme('image') will still be using theme_image().
Comment: I need to use my theme_cus_image only for images with some class. It will be able?
Comment: I have updated my answer to include that information. I also expanded the answer in the case you just need to define a new theme function that is used only from your module, and not a theme function that replaces `theme_image()` when used from Drupal, or other modules apart yours.
Comment: You should never call a theme function directly. That will basically cancel are the functionality that comes with the theme layer. Source: [Calling theme functions](http://themery.com/book/export/html/83) and [theme hook](http://api.drupal.org/api/drupal/includes%21theme.inc/function/theme/6)
Comment: @Max That is true, but not for a function that is overriding a theme function. Consider what would happen if the `theme_cus_image()` function I am referring in my answer would call `theme('image')`: `theme('image')` would call the "image" theme function, which in this case is `theme_cus_image()`, which would call `theme('image')`, which would call the "image" theme function, which in this case is `theme_cus_image()`, which would call `theme('image')`, which would call....
Comment: We're talking about different things. If you want to override a Drupal's default theme functions, you should do that in the `template.php` file. In this case `theme_image` is the standard implementation but using `theme('image')` will make sure that the overridden version is used. But if you want a totally different theme function and still keep the default one then you should just define a new function using `hook_theme`. But you should still use `theme('cus_image')`.
Comment: `hook_theme()` is not used for overriding another theme function; that is the purpose `hook_theme_registry_alter()`. Since the OP seems to want to overriding the default function when called from other modules (or Drupal), that is what I am suggesting.


Iterate through string char by char

Question: I try to iterate through a string char by char. I tried something like this:
void print(const string& infix)
{
char &exp = infix.c_str();
while(&exp!='\0')
{
         cout<< &exp++ << endl;
    }
}

So this function call print("hello"); should return:
h
e
l
l
o

I try using my code, but it doesn't work at all. btw the parameter is a reference not a pointer. Thank you
Comment: The parameter might be a reference, yes, but so is `exp`. By the way, `std::string` has overloaded `operator[]`, and it has `begin()` and `end()` functions, as well as working with the free version of said function pair for iterating.
Comment: possible duplicate of [How can I iterate through a string and also know the index (current position)?](http://stackoverflow.com/questions/1315041/how-can-i-iterate-through-a-string-and-also-know-the-index-current-position)
Comment: possible duplicate of [For every character in string](http://stackoverflow.com/questions/9438209/for-every-character-in-string)
Answer: Your code needs a pointer, not a reference, but if using a C++11 compiler, all you need is:
void print(const std::string& infix)
{
    for(auto c : infix)
        std::cout << c << std::endl;
}

Comment: Best answer. Very clean
Answer: for(unsigned int i = 0; i<infix.length(); i++) {
    char c = infix[i]; //this is your character
}

That's how I've done it. Not sure if that's too "idiomatic".
Comment: @user1988385, For printing, even `std::cout << infix;` works.
Answer: If you're using std::string, there really isn't a reason to do this. You can use iterators:
for (auto i = inflix.begin(); i != inflix.end(); ++i) std::cout << *i << '\n';

As for your original code you should have been using char* instead of char and you didn't need the reference.
Comment: But, maybe he doesn't only want to copy it? He's simply asked how to iterate through the string character by character.
Comment: Depending on what you're doing with it, there's probably something to replace the `copy` call, though, that works just about the same.
Comment: The funny thing about the output is that you can and should just `std::cout << infix;`. However, needing to do anything else, an algorithm or iterator loop works. Also, if using `auto` already, I'd use a range-based for loop over an iterator loop as well :p
Answer: std::string::c_str() returns const char*, you can't use char& to hold it. Also exp is pointer already, you don't need reference:
Better use iterator though:
void print(const string& infix)
{
  for (auto c = infix.begin(); c!=infix.end(); ++c)
  {
    std::cout << *c << "\n";
  }
  std::cout << std::endl;
}

To fix your original code, try:
void print(const string& infix)
{
  const char *exp = infix.c_str();
  while(*exp!='\0')
  {
    cout << *exp << endl;
    exp++;
   }
 }

Comment: Is there any reason we would use char* over just a for loop (in my answer)? I've never been sure if that's how other people have done it.
Comment: @DhaivatPandya, Not really, no. Iterating a `std::string` by using a `const char *` seems like a pointless endeavour to me.

Iterate through string char by char


Finding the limit of recursive sequence

Question: Assuming that the solution of $e^{-x}=x$ is $c\in (0,1)$
And give the following sequence
$$a_{n+1}=e^{-a_n}$$
$$a_1=1$$
How can i prove that the sequence converge and that the limit is  $$\lim_{n \to \infty}a_n=c$$
I tried to define a new sequence $$b_n=a_n-c$$ and to prove it using the ratio test but without success
Answer: $f(x) = \mathrm{e}^{-x}$ is a contraction in the relevant range $(0, 1]$, thus $x_{n + 1} = f(x_n)$ converges if started inside that range.
Comment: Can you elaborate or provide some reference? I undertand contraction is a kind of coordinate transform, how does it work here?
Comment: @YuriyS Check [this document](http://www.math.uchicago.edu/~may/VIGRE/VIGRE2011/REUPapers/Peeler.pdf). The relevant theorem is 3.6. You can also search for Banach Fixed Point Theorem.
Answer: Write $e^y=\exp(y)$ - it's easier to write recursive sequence:
$$a_n=\exp(-\exp(-\exp(-...\exp(-1)))))$$
Then you can write:
$$x=\lim_{n \rightarrow \infty} a_n=\exp(-x)$$
Since just one step of recursion will not change the limit of infinite sequence.
The rest is clear.
As for proving the limit exists - every term in the sequence is positive, and every term is bounded.
$$0 \leq a_n \leq 1$$
The sequence is actually not monotonic (as can bee seen by numerical experiment $a_3=0.500, a_4=0.606, a_5=0.545, a_6=0.580$ etc), so the proof of convergence is tricky
Comment: You assume that the limit exist,but how do you prove it exist?
Comment: See the edit I made. We start with 1 and you can see that the sequence is bounded. That's a start, not the whole proof of course
Answer: Exponential function is continuous,then
$$
\lim_{n\to\infty}a_n=\lim_{n\to\infty}a_{n+1}=\lim_{n\to\infty}e^{-a_n}=e^{-\lim_{n\to\infty}a_n}.
$$
So the limit of this sequence is the solution of $x=e^{-x}$ which is $c$.
To see the existence, first note by induction that $1\geq a_n\geq 0$. Now we claim that subsequence with odd terms decreasing and subsequence with even terms increasing. For $k=1$, $
a_{2k-1}\geq a_{2k+1}
$ is immediate and to see $a_{2k}\leq a_{2k+2}$ note that $1\geq a_n$ and $e$ is monotone. Thus, $
a_2=e^{-1}\leq a_4=e^{-a_3}.
$
Suppose now for some $k$, $a_{2k-1}\geq a_{2k+1}$ and $a_{2k}\leq a_{2k+2}$. Then
$$
e^{-a_{2k}}\geq e^{-a_{2k+2}}\implies a_{2k+1}\geq a_{2k+3},\\
e^{-a_{2k+1}}\leq e^{-a_{2k+3}}\implies a_{2k+2}\leq a_{2k+4}.
$$
This tells us that both $\lim_{k\to\infty}a_{2_k}=l_0$ and $\lim_{k\to\infty}a_{2k-1}=l_1$ exist by monotone convergence theorem. Now we claim that every odd term is bigger than every even term. Trivially for $k=1$, this claim holds. Suppose now that it is true for some $k$, $a_{2k-1}\geq a_{2k}$. Then
$$
a_{2k+1}=e^{-a_{2k}}\geq e^{-a_{2k-1}}=a_{2k}\implies e^{-a_2k}=a_{2k+1}\geq e^{a_{2k+1}}=a_{2k+2}
$$
which shows that the claim is true for $k+1$ and induction is completed. So we show that $a_{2k}\leq a_{2k+2}\leq a_{2k+1}\leq a_{2k-1}$. Note also that $l_0$ is supremum and $l_1$ is infimum for corresponding sequences. Then $l_1\geq l_0$.
By induction one can show $a_{2k+1}=a_{2k}^{1/e}\geq a_{2k+1}^{1/e}=a_{2k+2}$. Taking limit on both sides  gives us $l_0^{1/e}\geq l_1^{1/e}$ and $l_0\geq l_1$. Thus $l_0=l_1=l$ and $a_k\to l$.
Comment: _If_ the limit exists, it must be $c$. Showing the existence of the limit is the less easy task.

Finding the limit of recursive sequence


When Run As at Server,Eclipse - Tomcat - java.lang.NoClassDefFoundError: javax/servlet/ServletConfig

Question: I'm fighting the whole day with this:
I have DynamicWebProject. It worked fine. But at some point I desided to clean extra jar wich didn't used in project. After my cleaning, I've got a bunch of errors. So I rollback all my changes (I mean I retuned all jars back). But Also I played with Build Path of project, of Eclipse, of Debug config. Unfortunately I didn't remember this changes (with Path stuff). And when I run as at server, tomcat shows me this:
`
04.12.2012 16:37:43 org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NoClassDefFoundError: javax/servlet/ServletConfig
    at org.apache.axis2.deployment.WarBasedAxisConfigurator.<init>(WarBasedAxisConfigurator.java:103)
    at org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:584)
    at org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:454)
    at org.apache.axis2.webapp.AxisAdminServlet.init(AxisAdminServlet.java:60)
    at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
    at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
    at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
    at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
04.12.2012 16:37:43 org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /library threw load() exception
java.lang.NoClassDefFoundError: javax/servlet/ServletConfig
    at org.apache.axis2.deployment.WarBasedAxisConfigurator.<init>(WarBasedAxisConfigurator.java:103)
    at org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:584)
    at org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:454)
    at org.apache.axis2.webapp.AxisAdminServlet.init(AxisAdminServlet.java:60)
    at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1266)
    at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1185)
    at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1080)
    at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5027)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1559)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1549)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
04.12.2012 16:37:43 org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
04.12.2012 16:37:43 org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8009"]
04.12.2012 16:37:43 org.apache.catalina.startup.Catalina start
INFO: Server startup in 3556 ms

`
I'm real tied with this endless fight. What I'm doing wrong? What Eclipse and Tomcat tries to tell me?

Done.
So. How I restore functionality.
First of all, my project was backup in SVN repository.
Next I create new Workspace in Eclipse. new workspace in different folder. And Checkout project from SVN.
After that, at step - run at server, I caught errors, and added one by one missed libraries(jars). And whoa-la, it works!
So how I thought, problem resides in configuration of eclipce and project, also in Run/Debug Configuration. How to find errors in configuration, I don't know :( but I'll try at old workspace.
Comment: can you please post your servlet config file?
Comment: I don't realy use any myself servlets. I guess this servlet for axis2. I didn't change anything in it.
Comment: Do you check if all required jars are present in $CALINICA_HOME/lib ?
Comment: at this path all needed jars are present. I guess my problem lie within Eclipse configuration, project conf, or debug/run configuration..
Answer: You are missing servlet-api.jar in your class path. Check it
UPDATED
1) Right click on project
2) Got to java build path
3) Click on add library in right hand side.
4) Choose server run time.
5) Click next and select your tom cat server.
6) Click on finish.

Then try again
Comment: Yep, in my searches i've found that this mistake could be with server-api.jar But where to write it? I'm using Eclipse. in DynWebProj, JavaResources->Libraries->ApacheTomcatv7.0 - servlet-api.jar - is present. When I'm trying to write this jar in "Debug Configurations.." I get more mistakes.
Comment: Nope. I've done this. But there is the same condition :( But maybe this gives some add.info  `INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [718] milliseconds.
2012-12-04 17:29:10,067 [WARN ][host-startStop-1][.transport.http.AxisAdminServlet] - Web application uses org.apache.axis2.transport.http.AxisAdminServlet; please update web.xml to use org.apache.axis2.webapp.AxisAdminServlet instead
04.12.2012 17:29:10 org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.NoClassDefFoundError: javax/servlet/ServletConfig`
Comment: Check your WEB-INF/lib folder have this jar if not add it manually and re try whether this helps to you.
Comment: checked. axis2-transport-http-1.6.2.jar in WEB-INF/lib. Also i added it to debug/run configuration. and the error stack has changed to this `    INFO: Starting Servlet Engine: Apache Tomcat/7.0.29
    04.12.2012 17:43:33 org.apache.catalina.core.ApplicationContext log
    INFO: Marking servlet AxisAdminServlet as unavailable
    04.12.2012 17:43:33 org.apache.catalina.core.StandardContext loadOnStartup
    SEVERE: Servlet /library threw load() exception
java.lang.NoClassDefFoundError: javax/servlet/http/HttpServlet`
Answer: Go to the Java Build Path, go to Libraries, click on Add Library, choose Server Runtime, click Next, choose your webserver and click Finish. Clean and deploy to see if it works.
Comment: Yep, i've done it like in the prevoius answer. And also clean project, clean tomcat work directory. the problem not gone. Maybe i doubled some jar?
Comment: You can use `Ctrl` + `Shift` + `t` and type javax.servlet.ServletConfig to see if you have added any duplicate jars. Just keep this runtime library and remove any other library/jar which provided this class and try once.
Comment: I like this hotkey combination. But it shows (javax.servlet.ServletConfig) only once. So I don't know if it doubled some where.
Comment: In this case, there is no chance for a duplicate jar being added. Could you check if the `axis` jars are there of different versions?


Como exibir comentários no console do RStudio?

Question: Quando escrevo um comentário (precedido pelo símbolo "#") em um script do RStudio, e aperto o comando 'Control+Enter', o meu comentário não é exibido na tela do console. Alguém saberia me informar como posso fazer os comentários do script aparecerem no console do RStudio?
Comment: Oi Arthur, podes comentar em baixo algumas duvidas em relação à tua pergunta?
Comment: Olá, Arthur, É para funcionar da maneira que você está fazendo. Qual versão do R e RStudio você está utilizando? Caso não esteja com as versões mais recentes, atualize e tente novamente. Outra opção é selecionar o comentário que você quer exibir no console e rodar. Espero te ajudado (:
Answer: Após a versão 1.0.36 será necessário alterar uma configuração, para isso siga os passos.

Tools ( Ferramentas )
Global Options... ( Opções globais )
Code ( Código )

e desmarque a opção Execute all lines in a statement
Answer: Uma outra opção pode ser a de atrelar um comentário a um objeto qualquer e emití-lo no Console.
Exemplo com um vetor (x) qualquer:
x<-1:10

comment(x)<-'My vector'

comment(x)

[1] "My vector"


What is the Acc parameter in roster module mean?

Question: Many functions in the mod_roster.erl file accept an "Acc" parameter. What does this parameter store? It is always empty in my function calls.
e.g.
<code>  get_user_roster(Acc, US) ->
  ....
</code>
Comment: [Events and Hooks][1]

Acc = same type as the return type of the function

https://support.process-one.net/doc/display/MESSENGER/Events+and+hooks
Answer: Basing on implementation I would say that is just syntactic sugar over propagating final roster list.
I never used it with Acc =/= []. As I see, non of ejabberd's module does otherwise (always Acc = [], so nothing is appended to roster retrieved from database).
I can imagine that we can call this function with some custom/client data and make implementation of custom module more compact.
Comment: That's exactly what I want to use it for. Passing custom data to the get roster method.


AIC BIC Mallows Cp Cross Validation Model Selection

Question: If you have several linear models, say model1, model2 and model3, how would you cross-validate it to pick the best model?
(In R)
I'm wondering this because my AIC and BIC for each model are not helping me determine a good model. Here are the results:
<code>Model - Size (including response) - Mallows Cp - AIC - BIC
Intercept only - 1 - 2860.15 - 2101.61 - 2205.77
1 - 5 - 245.51 - 1482.14 - 1502.97
2 - 6 - 231.10 - 1472.88 - 1497.87
3 - 7 - 179.76 - 1436.29 - 1465.45
4 - 8 - 161.05 - 1422.43 - 1455.75
5 - 9 - 85.77 - 1360.06 - 1397.55
6 - 10 - 79.67 - 1354.79 - 1396.44
7 - 17 - 27.00 - 1304.23 - 1375.04
All Variables - 25 - 37.92 - 1314.94 - 1419.07
</code>
Note - assume the models are nested.
Comment: why isn't you're AIC helping? AIC and BIC appear even to coincide here.
Comment: The lowest AIC/BIC is obtained on the 17 variable model - so this is the best? Shouldn't I do some calculations to check if the 'additional AIC or BIC' is worth an extra variable? Also - would you use LASSO to get several models, subsets regression, or neither? I used Subsets to create 6 models but now I'm wondering if its better to use LASSO? The question also is - is this method even correct for justifying which model is the best 'compromise'?
Answer: The difference in AIC between two specified models is the estimated expected difference in Kullback–Leibler divergence from each to the true model, & therefore a useful model-selection criterion. The difference in AIC between the model with the lowest AIC out of several & the one with the second-lowest is something else, & therefore has to be taken with a pinch of salt as a model-selection criterion. If 'several' becomes 'many' it's not much use at all. As @gung says here, "if you run a study several times and fit the same model, the AIC will bounce around just like everything else".
What you need to cross-validate is the fit of the final model, repeating the entire model selection procedure within each cross-validation fold. You mention in a comment that you used "Subsets" to create six models, which suggests some selection going on there. Even if you use a measure of fit calibrated by cross-validation instead of AIC to pick the best model, you still need to cross-validate that procedure (see @Dikran's answer here, or @Bogdanovist's here).
Comment: Interesting. If I use a cross-validated LASSO function, then change the 'fraction' so it produces several models, then select the best one on AIC/BIC change then is it okay as the models have been cross-validated? This is what is confusing me quite a bit...
Comment: I'm not sure how to say what I've already said differently. See @cbeleites' answer [here](http://stats.stackexchange.com/questions/65128/nested-cross-validation-for-model-selection). Basing model selection on cross-validation is itself a selection procedure & can be cross-validated - the necessity for cross-validation  increases with the number of possible models you're selecting from.
Comment: Is Kullback–Leibler divergence = exp((AIC min - AIC i)/2)? I found this on wiki but it doesn't say what 'number range' we look for in a model to select it. I have created 10 cross validated models using LASSO just for different 'fractions'. The link you provided explains I need to cross validate it again but I'm unsure how to implement this in R (assume all models are in linear model form).
Comment: For Kullback–Leibler divergence I need a distrubiton per linear model right? How do i get this. I found ~AIC = exp((AIC min - AIC i)/2) on wiki, to see if a model has more info then the minimum AIC, but it doesn't say what 'number range' we look for in a model to select it. I have created 10 cross validated models using LASSO just for different 'fractions'. The link you provided explains I need to cross validate it again but I'm unsure how to implement this in R (assume all models are in linear model form). Infact reading the link again it seems I cannot pick a model?
Comment: Ignore the comment above about number range. It says if the exp(AICmin-AICi/2) > 10 we reject model i. In this case I reject all my models except the OLS. However, this is bad, as I know several terms are collinear. Something here is not making sense :(


Distribuindo pontos uniformemente em um círculo

Question: Comecei a fazer um código para distribuir aleatoriamente pontos uniformemente em um círculo, porém, ao gerar um ponto localizado com um ângulo teta aleatório entre 0 e 2π e com uma distância do centro do círculo aleatória de 0 ao raio R, os pontos ficam amontoados mais próximos do centro.
Fiz com o seguinte código:
Pontos = R * rand(1,N) .* exp(j * 2 * pi * rand(1,N));

onde rand(1,N) é um vetor que se relaciona a N valores aleatórios de 0 a 1.
No momento em que eu tiro a raiz quadrada da primeira operação rand(1,N), ao plotar os gráficos, os pontos ficam uniformemente distribuídos:
Pontos = R * sqrt(rand(1,N)) .* exp(j * 2 * pi * rand(1,N));

Como isso pode ser explicado?
Comment: Seu problema me parece mais matemático que qualquer outra coisa.
Eu colocaria as tags *matemática* e *random* e colocaria uma imagem do seu gráfico com as duas distribuicões. Numa olhada nos resultados, eu percebo a diferenca e vejo a razão, afinal `sqrt(0.2)~0.45`, mas não consegui lembrar de um método eficiente de qualificar *randomness*.
Comment: Gente, a edição que eu fiz não foi correção do texto, foi apenas a adição que o Guto sugeriu, das tags, não entendi porque alguém colocou em correção o texto como se eu tivesse modificado e dificultado o entendimento se tá como tava antes, kkkkk
Comment: Eu consgui perceber porque que no primeiro caso os pontos ficam mais concentrados no centro. Não sei entretanto explicar o porquê de ao tirar a raíz quadrada eles se espalharem melhor, mas eu acho que eles continuam não uniforme
Comment: Não percebi o porquê você multiplica pelo exponencial de um número no fator de `j`, assim como não vi você colocando o valor do ângulo teta;.
Comment: Eu sei que faz tempo já a pergunta mas como fazia tempo que eu não abria o SOF, resolvi explicar. Uma forma de escrever um ângulo é utilizando uma exponencial com expoente i2πX, sendo i o valor do conjunto dos imaginários tal que sqrt(i)=-1. No fim, basta a gente tirar o módulo do eixo imaginário e calculamos a tangente que temos magicamente um ângulo, é bem mais simples do que parece, e j é como o matlab representa o valor imaginário.
Answer: Conforme apontado anteriormente, de fato sua questão é de matemática.
A origem do problema
O problema é que você está tomando uma distribuição uniforme de raios, i.e., a probabilidade de um ponto estar a uma distância qualquer r do centro não depende de r. A maneira intuitiva de ver que isso não produz uma distribuição uniforme na área do círculo é a seguinte: se a probabilidade de um ponto estar a uma distância r do centro é a mesma pra qualquer r, você espera encontrar em média o mesmo número de pontos a uma distância r do centro, para todo r.
Ademais, como a probabilidade independe do ângulo, os pontos a uma distância r devem se distribuir igualmente sobre uma circunferência de raio r. Acontece que para r grande as circunferências vão ficando maiores, e como você está distribuindo o mesmo número de pontos sobre elas, é de se esperar que os pontos fiquem mais espaçados uns dos outros!
Para que a distribuição fique uniforme em termos de área, você precisa colocar mais pontos sobre as circunferências maiores, ou em outras palavras, a probabilidade de se colocar um certo ponto a uma distância r deve crescer com r. Mas crescer como? Uma vez que o tamanho das circunferências cresce linearmente com o raio (=2πr), é de se esperar que essa probabilidade cresça também linearmente com r. Para ver isso de forma mais rigorosa precisamos de mais matemática.
Distribuição de raios
Ocorre que os elementos de área em coordenadas esféricas dependem do raio, ficando maiores para r maiores:
dA = r dr dθ,

e para que a distribuição seja de fato uniforme, você quer que a probabilidade de haver um ponto em um dado elemento de área seja constante para qualquer elemento. Essa probabilidade é, pela definição de densidade de probabilidade, P(r,θ) dA, ou seja, P(r,θ) r dr dθ.
Integrando isso para r entre 0 e R e θ entre 0 e 2π você tem a probabilidade total do ponto cair em algum lugar no círculo, que deve ser 1. Não é difícil ver que isso fixa
P(r,θ) = 1 / (πR²),

o que é óbvio: numa distribuição homogênea sobre uma determinada área total, a probabilidade associada a um elemento de área deve ser a razão entre a área do elemento e a área total. Veja que é exatamente isso que dá o produto P(r,θ) dA:
P(r,θ) dA = r dr dθ / (πR²).

Variável inomogênea
Pois bem, então agora você pode interpretar esse resultado de forma um pouco diferente. Em vez das variáveis r e θ, mudamos para duas variáveis aleatórias x (que varia entre 0 e R) e y (que varia entre 0 e 2π) que possuem agora uma densidade de probabilidade
P(x,y) = x / (πR²)

de modo que a integral disso pra x indo de 0 a R e y indo de 0 a 2π seja 1:
∫∫ P(x,y) dx dy = 1.

Veja que agora sua densidade de probabilidade P(x,y) de fato não depende de y mas é proporcional a x! Esse é o resultado que menciono no início: na verdade você está procurando uma variável aleatória x entre 0 e R mas cuja distribuição não é uniforme, sendo na verdade proporcional a x.
Para entender a raiz quadrada em rand(1,N) ainda precisamos avançar um pouco mais.
Uma pequena tecnicalidade: as variáveis x e y são independentes, pois podemos reescrever P(x,y) como um produto
P(x,y) = f(x)g(y).

É fácil ver que a escolha adequada é
g(y) = 1/2π

de modo que ∫g(y) dy = 1 para a integral entre 0 e 2π, e
f(x) = 2x/R²

tal que ∫f(x) dx = 1 para x entre 0 e R. Observe que f(x)g(y) de fato recupera P(x,y).
Como g(y) não depende de y, vemos que o ângulo é realmente distribuído homogeneamente (o que explica porque você pode usar 2*pi*rand(1,N) dentro da exponencial).
O desafio então agora é buscar uma variável x distribuída de acordo com a densidade de probabilidade f(x) acima.
Método da transformação
Um método para se fazer isso a partir de variáveis distribuídas uniformemente é o chamado método da transformação.
Para variáveis x distribuídas de 0 a R de acordo com uma distribuição f(x), a fração de números que caem entre 0 e x, que chamaremos de F(x), é dada por
F(x) = ∫f(x') dx',

com a integral realizada entre 0 e x.
Acontece que para números uniformemente distribuídos entre 0 e 1, a fração de números que caem entre 0 e G (para um dado G entre 0 e 1) é exatamente G. Veja: a fração de números entre 0 e 1/2 é 1/2, entre 0 e 1/3 é 1/3, e por aí vai.
Então a fração de números entre 0 e um dado F(x) para números distribuídos homogeneamente entre 0 e 1 é F(x), que é exatamente a fração de números entre 0 e x na distribuição f(x) definida entre 0 e R.
O que queremos portanto é mapear o intervalo entre 0 e F(x) de números r distribuídos homogeneamente no intervalo de 0 a x da nova distribuição. E isso é bem fácil de fazer: os extremos dos intervalos devem corresponder! Assim o número x da distribuição não-uniforme deve ser obtido quando o número
r = F(x)

for obtido na distribuição uniforme. Conclusão: dados os números r distribuídos uniformemente, basta inverter essa relação - encontrando a função inversa de F(x) - para obter o x. Vamos aplicar essa ideia à nossa f(x) acima.
Temos f(x) = 2x/R², de modo que, lembrando que a integral é realizada entre 0 e x,
F(x) = ∫f(x') dx' = ∫2x'/R² dx' = x²/R².

Inverter essa relação é fácil:
r = F(x) => r = x²/R² => x = R √r

o que (finalmente!) mostra que uma variável x distribuída entre 0 e R com densidade de probabilidade f(x) = 2x/R² pode ser obtida simplesmente multiplicando-se R pela raiz quadrada de uma variável r homogeneamente distribuída entre 0 e 1. Isso é exatamente o que você está fazendo com
R * sqrt(rand(1,N))

na sua segunda linha de código. Essa sua solução 'empírica' portanto realmente fornece exatamente números homogeneamente distribuídos sobre um círculo de raio R.