Migrating Trac 0.12 from Debian wheezy to Trac 1.0.2 on jessie including all plugins

Question: I try to migrate a trac installation from a debian wheezy server to another debian jessie.
If I copy all files to the new server I get the message, that I have to upgrade with trac-admin /var/trac/blimus upgrade, which seems to work, but it seems not to update all plugins.
for example I had a plugin installed in the old trac, that lets you login on a webpage instead of the htaccess popup.
On the new server I now get the error
Error: Not Found
No handler matched request to /login

How do I get the web-login back?
Is there a way to analyze the old trac folder, which plugins were installed, so I can install the missing ones in the new installation?
I checked the new trac.ini file and added the missing options, this is my trac.ini
These are my versions:
# trac-admin --version
Warning: Detected setuptools version 5.5.1.
Welcome to trac-admin 1.0.2

Comment: The `Warning: Detected setuptools version 5.5.1.` is due to using setuptools in the range 5.4 to 5.6, which had performance problems. There is a [workaround](https://trac.edgewall.org/wiki/TracInstall#MandatoryDependencies), or just use a version outside that range. You can just use the latest setuptools, which is currently 42.
Comment: Trac 1.0.2 is rather old. [Debian Buster](https://packages.debian.org/buster/trac) and [Debian Stretch](https://packages.debian.org/stretch/trac) have a 1.2.x version of Trac.
Comment: But unfortunately I have to install it on a Debian Jessie server
Answer: If the plugin were installed in the Environment plugins directory as eggs, they will be carried over to your new server when you copied the the environment. However, you almost certainly need to upgrade to newer versions, so I suggest deleting the eggs from the Environment plugins directory. The plugins might also have been installed in Python's site-packages. Or they could have been installed using Debian's package manager, I'm not sure if plugins are available there, you might want to search using apt. See Trac Plugin documentation.
Which version of Trac are you running now? I assume it's 1.2.3.
You trac.ini shows that you have 3 plugins installed, and you no longer need one of them in Trac 1.2 (NeverNotifyUpdaterPlugin). You should install these two:
 * AccountManagerPlugin
 * XmlRpcPlugin
I don't know if Debian packages those for distribution. If not, you can install them with pip.
Comment: Those plugins exist, I installed with `apt install trac trac-accountmanager trac-xmlrpc` but still the same problem: `No handler matched request to /login`
Comment: I had those eggs: `Genshi-0.5.1-py2.5-linux-x86_64.egg-tmp` and `TracAccountManager-0.2.1dev_r4679-py2.5.egg-tmp` I deleted those folders, do I have to rebuild the eggs somehow?
Comment: `apache2ctl graceful` was enough to reload trac
Comment: What happens, if you don't delete the tmp files in the egg folder?
Comment: `service apache2 reload` is another good way. Looks like the `.egg-tmp` files are related to the egg extraction (`PYTHON_EGG_CACHE`), so they should be harmless, whether you delete them or not.
Comment: In fact, those `.egg-tmp` files should only be found under `$INSTALLPATH/.python-eggs`, and if they are found in another location such as `$INSTALLPATH/plugins`, that may be due to a previous configuration where the `PYTHON_EGG_CACHE` pointed to `$INSTALLPATH/plugins`.
Comment: I see you are running 1.0.2, so you'll need to install [NeverNotifyUpdater](https://trac-hacks.org/wiki/NeverNotifyUpdaterPlugin) if you want that functionality.
Answer: This instruction will work on Debian jessie (Trac 1.0.2), as well as Ubuntu 14.04 trusty (Trac 1.0.1)
Prerequisites
On your old Debian wheezy server:
# cd ~/
# tar -cvzf trac-example.tgz /var/trac/example
# cp /var/trac/example/conf/trac.ini trac.ini-example
# update your trac.ini-example to match new settings

Installation on the new server
#!/usr/bin/env bash

####################### config ##############################
INSTALLPATH=/var/trac/example
DEPLOYPATH=/var/www/trac-example
# where you store your tgz backup and the new trac.ini file
BACKUP_PATH=~/
#############################################################

# install packages without user interaction:
export DEBIAN_FRONTEND=noninteractive

apt-get -y install unzip apache2 trac trac-accountmanager trac-xmlrpc libapache2-mod-python libapache2-mod-python-doc libapache2-mod-wsgi
a2dismod python
a2enmod rewrite

# add trac user for apache WSGIDaemonProcess:
adduser --shell /bin/sh --no-create-home --disabled-password trac
mkdir -p /home/trac/.keep-for-mod_wsgi

mkdir -p /var/trac

# extract the tar-gz file of the trac installation from the old server
tar -C /var/trac/ -xvzf $BACKUP_PATH/trac-example.tgz

 # copy your new config here:
cp $INSTALLPATH/conf/trac.ini $INSTALLPATH/conf/trac.ini-backup
cp $BACKUP_PATH/trac.ini-example $INSTALLPATH/conf/trac.ini

# remove old plugins and eggs
rm -rf $INSTALLPATH/plugins/nevernotifyupdaterplugin-0.0.* $INSTALLPATH/eggs/*

# update Trac
cd $INSTALLPATH/
trac-admin $INSTALLPATH upgrade
trac-admin $INSTALLPATH wiki upgrade
trac-admin $INSTALLPATH deploy $DEPLOYPATH/
chmod ugo+x $DEPLOYPATH/cgi-bin/ $DEPLOYPATH/htdocs/

# downgrade genshi from 7.3 to 6.0 due to error when adding an attachment:
easy_install -U Genshi==0.6
# upgrade setuptools
easy_install -U setuptools==1.4.2

# install neverNotifier Plugin
# (This is still needed on trac 1.0.x, on 1.2 ist is obsolete)
cd /tmp
wget "https://trac-hacks.org/browser/nevernotifyupdaterplugin/1.0?r│
ev=17630&format=zip"
unzip 1.0\?r*
cd 1.0/
python setup.py bdist_egg
cp dist/nevernotifyupdaterplugin-1.0-py2.7.egg $INSTALLPATH/plugins/

# setup hosts file
VHOST=$(cat <<EOF
<VirtualHost *:80>
  Alias /trac/chrome/common $DEPLOYPATH/htdocs/site/common
  Alias /trac/chrome/site $DEPLOYPATH/htdocs/site
  <Directory "$DEPLOYPATH/htdocs">
    Require all granted
  </Directory>
  <Location "/trac">
    SetEnv TRAC_ENV "$INSTALLPATH"
    SetEnv PYTHON_EGG_CACHE "$INSTALLPATH/.python-eggs"
    SetEnv TRAC_ENV_INDEX_TEMPLATE $INSTALLPATH/templates
  </Location>
  ##trac mit mod_wsgi
  WSGIDaemonProcess trac user=trac group=trac threads=25
  WSGIScriptAlias /trac $DEPLOYPATH/cgi-bin/trac.wsgi
  <Directory $DEPLOYPATH/apache>
    WSGIApplicationGroup %{GLOBAL}
    Require all granted
  </Directory>
  <Directory $DEPLOYPATH/cgi-bin>
    Require all granted
  </Directory>
  <Directory $DEPLOYPATH/htdocs/common>
    Require all granted
  </Directory>
  <Directory $DEPLOYPATH/htdocs/site>
    Require all granted
  </Directory>
</VirtualHost>
EOF
)

# apache config
echo "${VHOST}" > /etc/apache2/sites-available/trac-example.conf
a2ensite trac-example

On a fresh installed systen, disable the default apache config
rm /etc/apache2/sites-enabled/000-default.conf

then restart apache
service apache2 restart

Comment: I would definitely run the `deploy` command, as you could be using old versions of the static resources if you were previously running an older version of Trac. If you encounter a recursion error then you will need to relocate the assets to a `www` directory. I would also suggest updating your Apache config to use `Require ...`, but that might be little less important.
Comment: The recursion problem exists because `$INSTALLPATH/htdocs` is actually the environment `htdocs` dir, used for site customizations, and should not be used as the target for extraction of static resources, which is the result you have in your environment. This is a common problem due to the way the docs were written in older versions of Trac.
Comment: 3. Give [TracUpgrade](https://trac.edgewall.org/wiki/TracUpgrade) a read through.
Comment: 5. Create a `virtualenv` to avoid polluting the global `site-packages`, which `apt` will install packages to, with manually (`easy_install`/`pip`) installed packages.
Comment: 6. Use `pip` rather than `easy_install`. It's the modern package manager for Python and includes an uninstall command. It will install packages in the wheel format, which avoids potential egg-extraction problems. I'd also recommend upgrading `setuptools` to the latest, and I think the error adding attachments was resolved in a Genshi 0.7.x version.
Comment: 8. You don't need ` libapache2-mod-python libapache2-mod-python-doc` since your Apache configuration uses only `libapache2-mod-wsgi`.
Comment: 9. [trac-users mailing list](https://trac.edgewall.org/wiki/MailingList#Trac-users) is another good resources if you have more questions, though you may just get a response from me there as well ;)
Comment: thaks, @RjOllos, I used `deploy` now (you can revise some of your comments now)


Map enumareted properties to array by Automapper

Question: Given is an class with indexed properties
    public class Foo
    {
        public int Bar1 { get; set; } = 17;
        public int Bar2 { get; set; } = 42;
        public int Bar3 { get; set; } = 99;
        ...
                   Bar<n>
    }

result is
List of int contain 17,42,99 ...
How to configure mapper that I can consume Automapper like this
List<int> bars = mapper.Map<List<int>>(foo);

Comment: First of all I don't think that counts as "indexed properties", but only something for human readability. However with a combination of a custom value resolver and reflection you should be able to get this done with automapper.
Comment: Start [here](http://docs.automapper.org/en/latest/Dynamic-and-ExpandoObject-Mapping.html).
Answer: You can achieve this with Reflection
List<int> bars = new List<int>(foo.GetType()
                                  .GetProperties()
                                  .Where(x => x.PropertyType == typeof(int))
                                  .Select(x => (int)x.GetValue(foo)));

Or if you also need the property names to sort or filter them in some way
 Dictionary<string, int> bars = foo.GetType()
                                   .GetProperties()
                                   .Where(x => x.PropertyType == typeof(int))
                                   .ToDictionary(x => x.Name,  x => (int)x.GetValue(foo));

Comment: This misses the automapper configuration that was asked for. Also this just puts all properties of type `int` in a list instead of using the "indexed" name convention OP mentioned, though that might not be needed since it might be implied that the class only contains those indexed properties
Answer: I agree with Innat3 on this but his answer misses few points. Here:
        var bars = foo.GetType()
            .GetProperties(BindingFlags.Public | BindingFlags.Instance)
            .Where(x => x.PropertyType == typeof(int))
            .Where(x => x.Name.StartsWith("Bar"))
            .Select(x => (int)x.GetValue(foo))
            .ToList();

And of course it doesn't involves Automapper as well as your indexed properties. I didn't rely on them cuz I don't know how exactly you implemented those.
Comment: OrderBy is missing :)


Apache - how to serve pages with users other than www-data

Question: I have a webserver that uses apache.   When I do a ls -l on /var/www/project1/public_html and /var/www/project2/public_html, I see that they are owned by projectuser1 and projectuser2 respectively.
On some of other servers I've looked at, both /var/www/project1/public_html and /var/www/project2/public_html are owned by only www-data.  How would I go about changing these ownerships to projectuser1 and projectuser2 such that these new users can login to their areas and manage their own websites?
I created a user projectuser1 then did a chown -R projectuser1 /var/www/project1, but any time projectuser1 adds a new file to the directory, Apache gives me a Permission Error.  If do a chown -R www-data /var/www/project1, then everything works again.
Ultimately, I want apache to serve the /var/www/project1 directory with projectuser1 owning it.
Answer: There a two simple answers for your question:

You could add the www-data group to each projectuser with usermod -aG www-data projectuser% and then ensure that group permissions are at least g+rw with chown -R g+rw /path/to/dir.
There is a module that runs each child process as a different user called mpm-itk. This allows you to run each vhost under a separate userid.
Answer: Having the served files owned by the server uid (www-data) is often a bad idea. The server should be able to server files owned by other users. You just must make sure the permission bits on the files allow that.
So, instead of:
chown -R www-data /var/www/project1

which changes owner of the files. And you don't want the server to be owner of those files,
unless the server is supposed to write them.
You should rather do:
chmod -R a+rX /var/www/project1

which will set read permission to all files and execute permission to all directories under /var/www/project1. This will make the files read-accessible by anyone. Make sure it is ok. If other users are not supposed to be able to read these files, then this answer is not for you.
You could also use a group for the HTTP server and the users who are allowed to upload files for it. This will make things a bit more complicated.
Another problem are the permission of the files uploaded by your users. The thing that matters is 'umask' – setting stating which permission bits should be disabled for newly created files. How to change that depends on how the files are uploaded (FTP, ssh session, scp, anything else). The files are probably uploaded with umask 0077 or 0007 now, which means any permission only for the owner of the files (the one who uploads them) and maybe his gid.


Acesso negado ao atualizar PIP

Question: Estou executando o seguinte commando no prompt Windows
pip install --upgrade pip

E estou recebendo um erro:

PermissionError: [WinError 5] Acesso negado:
  'd:\leonardo\python\django-treinaweb\env\scripts\pip.exe'

Lembrando que estou dentro de uma virtualenv.
Answer: Tenta executar o cmd como administrador.
Vai a inicio/accesorios/Command Prompt e da clique direito/ executar como administrador.
depois tenta o mesmo comando:
pip install --upgrade pip

Comment: Deu certo. Mas acho que é porque eu estava dentro da virtualenv. As vezes tava apontando pra outro lugar, acha que pode ser isso?
Answer: pip install --upgrade pip --user

Vai dar certo.

Acesso negado ao atualizar PIP


How can I collapse notification in Flutter? using Local Push Notification plugin and CloudFunctions

Question: I'm using cloud function to send notifications by using device tokens.
    const payload: admin.messaging.MessagingPayload = {
        notification: {
            title: `updatedBy: ${updatedBy}`,
            body: `nova messagem ${increment}`,
        },

    }
    return fcm.sendToDevice(userToken, payload, options).then((value) => {
        console.log(value.results);
    }).catch((error) => {
        console.log(error);
    })

But, all notification got pilled, I wanna so much to administrate these notification that stay stuck in system tray, is there anyway I can moderate notifications with such "complexity" using these plugins? (Local push Notification, FCM, cloudfunctions) without using native code (kotlin, android)? because I don't know how the android structure work...
I can use cancelAll (of Local push notification)
FlutterLocalNotificationsPlugin.cancelAll()

from Local Push Notification: https://pub.dev/documentation/flutter_local_notifications/latest/
But is too inflexible
I can also use cancel notification using an specific "ID", but I don't know how to get that notification "id"
FlutterLocalNotificationsPlugin.cancel(notificationId)

so, How can solve this issue?
In that way I can wisely remove whatever notification I want,
But basically, I just wanna collapse notification that is the same type, like on Whatsapp, when a person send you more than 1 message, and in the system tray, there's only one notification saying that "he" sent more than 1 message (and not a heavy pile of notification).
I just looked at https://github.com/chetdeva/collapsing-notifications/tree/a5e8867e24f53ff8caca761c7faa13242c607dba
This github code is for the article https://medium.com/fueled-engineering/collapsing-fcm-notification-like-a-pro-102a4946b350
would be perfect if it would not too difficult to implement in a flutter project, I also looked at every post here, in stackoverflow, seens that no one yet ever answered this question
[EDIT]: the reason I can't use
FlutterLocalNotificationsPlugin.cancelAll()

It's because I'm using fcm method to listen for notifications, I tried to put that method in some of fcm listener but not succeed.
_firebaseMessaging.configure(
      onMessage: (Map<String, dynamic> message) async {
        print('on message $message');
      },
      onResume: (Map<String, dynamic> message) async {
        print('on resume $message');

      },
      onLaunch: (Map<String, dynamic> message) async {
        print('on launch $message');
      },
    );

Comment: Have you found a way in which this can be done?
Answer: Use the "collapse key" mentioned in the docs: https://firebase.google.com/docs/cloud-messaging/concept-options#ttl


Finding the Jordan / Prejordan Basis.

Question: I have been given the following matrix:
$\begin{bmatrix}5 & 4 & 3 \\-1 & 0 & -3 \\ 1 & -2 & 1  \end{bmatrix}$
And I want to find the Jordan Basis. I have found the Characteristic polynomial which  is $(x+2)(x-4)^2$, the same as the mimum polynomial and I thus know the Jordan Canonical form.
So far what I have tried:
I first considered $\lambda = -2$ and I found the nullity of this eigen space here to be 1, with an eigen vector $(1, -1, -1)^T$, although I am not sure if this is correct.
Now with $\lambda = 4$, after reduction $T- 4I$ we get:
\begin{bmatrix}1 & 4 & 3 \\-1 & -4 & -3 \\ 1 & -2 &-3  \end{bmatrix}
I found an eigen vector $(-1, 1, -1)^T$ but I'm a bit stuck on what to do next? Do I need to find $(T-4I)^2?$ Doing this I found $(0,-1,1)^T$. How does this eigen vector relate to the ones I have already found.
Also how do I go about creating a prejordan basis? I know that once I have one, I can just apply $T-\lambda I$ on the 'lower dimension' basis vectors to get a Jordan basis, but I'm struggling to find this one. Do I just 'guess' any linearly independent vector from my eigen vectors?
Thanks
Answer: So there is the generalized eigenspace $\ker (T-4)^2$ in your case you can check that this is all vectors $\begin{pmatrix}a\\b\\-b\end{pmatrix}$. What you want is a vector in this eigenspace but not in $\ker(T-4)$ The latter you have calculated as being generated by the vector given in the question. So a vector in $\ker(T-4)^2-\ker(T-4)$ is $v=\begin{pmatrix}1\\0\\0\end{pmatrix}$.
This is one element of your basis. The next element of your basis is $(T-4)v$ (which with ny choice of $v$ is just the negative of the vector you gave.) These two vectors $v, (T-4)v$ are then a basis for the gereralized eigenspace of $T-4$.

Finding the Jordan / Prejordan Basis.


"If-Match" or "x-goog-if-generation-match" equivalent for PUT requests with AWS/S3

Question: Is there an equivalent to the "If-Match" (Azure) or "x-goog-if-generation-match" (Google Storage) equivalent for PUT requests sent to Amazon S3 blob storage?
Both these headers do the same thing - they allow you to specify that a PUT request to overwrite an object should only succeed if you are overwriting a specific previous version of the object. Details:
https://learn.microsoft.com/en-us/azure/storage/common/storage-concurrency?toc=/azure/storage/blobs/toc.json#optimistic-concurrency-for-blobs-and-containers
https://cloud.google.com/storage/docs/xml-api/reference-headers#xgoogifgenerationmatch
Thanks!
Answer: Unfortunately, S3 doesn't support conditional PUT. You need to cooperate with other AWS services, such as DynamoDB, to achieve such behaviour.


Renaming the label of the Title textfield

Question: Adding new fields allows you to specify the label of the field but the Title field does not. In fact, it just sits there on top acting all high and mighty. I just want to change the label of Title to something like Your name but retain the machine name. Does anyone have an idea how to do it?
Answer: Go to Content type, you will have a list of content types of your site. Under Operations click on edit  for the content type you want to change the title.  Under Submission form  settings you can change the "Title field label"
Answer: Use themename_preprocess_field in template.php file in theme folder.
function themename_preprocess_field(&$variables, $hook) {

    if($element['#field_name'] == 'field_xyz') {
      $variables['label'] = "your label name";
    }

}


Porque as views ficaram lentas após migrar os dados via script no Firebird 2.5?

Question: Estou trabalhando em uma base de dados padronizada. Recentemente refatorei a estrutura dela (A) renomeando todas as PK e FK existentes, porque tinham os nomes que são gerados automaticamente.
Ao migrar os dados de uma outra base (B) para esta refatorada (A), agora estou tendo uma lentidão muito grande na leitura de algumas views, mesmo elas possuindo a mesma estrutura.

Ao analisar a performance, reparei que na base nova (A) estão sendo lidos muito mais registros que na anterior, para exatamente a mesma consulta. Porém não consegui entender porque isso acontece.
Abaixo o print da análise de performance.
Comment: https://www.upscene.com/documentation/dbw3/index.html?fb_db_databasestatistics.htm   Estatiscas atualizadas ?
Comment: Obrigado pela ajuda Motta, eu já tinha tentado mas isso não tinha resolvido.
Answer: Consegui resolver o problema, foi só recompilar a view. Bom, não entendi porque, mas felizmente o problema está resolvido!


Open Excel Workbook with vba code - error to notify VSTO

Question: I am trying to open an Excel file from my C# code.
<code>workBook = workBooks.Open(fileName,               //filename
                          0,                      //updatelinks
                          true,                   //readonly
                          5,                      //format
                          "",                     //password
                          "",                     //WriteResPassword
                          true,                   //IgnoreReadOnlyRecommended
                          XlPlatform.xlWindows,   //Origin
                          "\t",                   //Delimiter
                          false,                  //Editable
                          false,                  //Notify
                          0,                      //Converter
                          true,                   //AddToMru
                          1,                      //Local
                          0);                     //CorruptLoad
</code>
I see that the code hangs at workBooks.Open if the excel has some error within it.
I have kept the display property to true and that is when I noticed that there is an error in the excel. On further investigation I got to know that the error is caused due to some vba code in the excel which is failing. My requirement is how do I get my C# code to be notified about this, so that it can ignore the error within the file or ignore the file itself?
Comment: For the uninitiated, maybe some `// comments` inline to describe what each of the 400 arguments are for? Not your fault of course, but ugh, that's some signature...
Comment: In .Net4 with named parameter you can pass parameter in better way, interop is really ugly:(, anyway, doesn't this throw any exception? in bad circumstances?
Comment: @Saeed, Thanks..Interop is indeed very ugly. I am struggling a lot with it. It does not throw any exception. Just hangs!!! Do you have any idea how to do this in 4.0. I believe it would internally use the Interop. But would be more than happy if it can manage it in a clean manner!!!
Comment: Nishat see [named parameter and interop in .net4](http://msdn.microsoft.com/en-us/library/dd264733.aspx) or [named parameter](http://msdn.microsoft.com/en-us/library/dd264739.aspx) do not need comments, Also for your problem, In fact I don't have an good idea, but some crazy way is open it in another thread and if takes more time finish it :))
Comment: Try tinkering with the Application.AutomationSecurity property.  You'll want the 'ForceDisable' setting.
Answer: What do you try to achive? Maybe just using the Microsoft Open XML SDK would be enough?
You can read, create and edit Excel files with it.
Comment: I would like to fetch all the named ranges in the excel file. Is it possible with Open XML?
Answer: Another alternative to using the intreop library is http://code.google.com/p/excellibrary/  which  doesn't require you to have excel or the excel.dll installed and registered to your machine or server that you are putting it on.


Add All link on view pager

Question: Is there a way to add a  Show All Link on my view pager.
I have a Block View.
I would like to be able to add an All Link button when the number of Items reach 5. I do not need the pager "Next, last 1,2,3" only the All Link button.
Thanks a lot
Comment: see my edit(s) to my answer.
Answer: Yes see the Pager Options for your Block or Views Page:

EDIT:
Allow me to clarify. This gets the - All - text option into the configuration of the View. If you want custom business logic that says if there is 5 or more results I want to show/hide the Next and Previous and include the All options I recommend you look at hook_views_pre_view() or other Views hooks to implement hiding/removing options as needed. For ease I recommend hiding/showing the options using CSS or Javascript.
Again, this option builds the option into the page -- but doesn't really enforce your business logic.
EDIT 2:
The module Pagerer might get you a little further, but probably not all of your custom logic is possible out of the box with that module. It has an API to fine-grain it's custom pager views.
Comment: Can you provide me a sample code for Edit One. ?
Answer: If you want to do this via coding then this can be helpful to you. I've used this in my own projects.

Add a new file with ".views.inc" extension. Preferred module_name.views.inc.
Theme the block form and add a variable to the views output via HOOK views_pre_render.
Then with the HOOK_pre_build function you can modify the pager items per page.

function MYMODULE_views_pre_render(&$view) {
  if ($view->name = 'your_view') {
    $view->attachment_after['show_all'] = l('Title', 'show-all', array('query' => array('view_name' => $view->name)));
  }
}

function MYMODULE_views_pre_build(&$view) {
  if ($view->name == 'your_view' && (isset($_GET['view_name']) && $_GET['view_name'] == $view->name)) {
    if (arg(0) == 'show-all') {
      $view->pager['items_per_page'] = 200; //The number you want to have.
    }
  }
}
Comment: I try MYMODULE_views_pre_build and it's never firing up. To be sure that is not called, I create a txt file when this method is called
Comment: have you cleared the cache??
Comment: Yes I do. I'm using Omega 4. Is it possible that it could cause thé problem to not firing up thé prebuild hook
Comment: This has nothing to do with the theme you are using. Check about the function name and spelling. If your module is called "foo_mod" then the hook will be function foo_mod_views_pre_build(&$view) {}
Please check and let me know.
Answer: I think the Views Flexible Pager module should solve this for you: https://www.drupal.org/project/views_flexible_pager

Add All link on view pager


String comparison for button events

Question: I have a String named updatedDisplay that is set to empty in the constructor.
The buttons[] are JButtons and alarmCode is a String field.
I want the user to press four buttons (and they should be concatenated and stored in the updatedDisplay field).
The checkCode() method is executed to try match updatedDisplay against alarmCode. Trouble is, they never match. I think it may be something to do with a "space" when I originally declare my updatedDisplay as follows:
private String updatedDisplay = " ";
The updatedDisplay field doesn't seem to be storing the e.getActionCommand() value.
//add actionListeners to each button (except the "clear" button) to display value on screen
for (int i = 0; i< (buttons.length -1); i++) {
  buttons[i].addActionListener(new ActionListener() {
         public void actionPerformed(ActionEvent e)
          {
          //store the name of the button in a local variable
          String command = e.getActionCommand();

          System.out.println("You clicked " + command);

          updatedDisplay = updatedDisplay + command;
          //updatedDisplay = command;
          System.out.println (updatedDisplay);

          screen.setText(updatedDisplay);
}
     });}

I have an armButton that, when pressed, should trigger the checkCode() method. The method checks if updatedDisplay and alarmCode are equal:
//add actionListener to the arm button
armButton.addActionListener(new ActionListener() {
        public void actionPerformed(ActionEvent e)
        {
         checkCode();
        }
    });

checkCode():
public void checkCode() {
//check if user entered the correct code
if (updatedDisplay == alarmCode)
{
  updatedDisplay =  "System Armed!";
  screen.setText(updatedDisplay);
}
else
{
  updatedDisplay  = "Incorrect Code, Try again!";
  screen.setText(updatedDisplay);
}
}

Even when I output the button presses to the terminal window they look right - but as I said, I suspect a "space" is being entered at the start.
Any ideas?
Answer: Solution
Try:
if( updatedDisplay.equals( alarmCode ) { // ...

Comparison
To understand this, read:
http://leepoint.net/notes-java/data/expressions/22compareobjects.html
Summary
Since updatedDate and alarmCode are object references, you must ask the objects to compare their values. You can think of them as pointers whose values are locations in memory that contain strings. Rather than comparing the value of the pointers (references), you want to compare the text that starts at that memory location.
Comment: @user: Don't forget to accept correct answers, as it encourages people respond to your questions.
Comment: Perfect. Worked straight away! I actually should have known to use that. Thanks a million.

String comparison for button events


Fixed Regressor Conspiracy and Connection to Exchangeability

Question: In simple regression model regressors are treated as fixed rather than stochastic. Whoever picks the experimental values for the regressors, decides in which frequency to include each value. This can be equally weighted (i.e 10 samples of 20mg medicine, 10 samples of 30mg medicine, 10 samples of 0mg-placebo) or in some other way.
Once the model is built and coefficients are determined, when it comes to use the model, it will be illogical to run the model with values of different frequency. It will be even more questionable to use values that did not exist in the experiment in the first place.
Moreover assuming that $Xs$ are fixed entails the following distributions:
$Y_i \sim N(\beta x_i,\sigma)$
Now if we consider the random sequence
$Y_1,Y_2,...,Y_n$
It appears to me that it is not an exchangeable sequence without having a random $X$ to index them. Each $Y_i$ has a different mean. Hence the inference is not converging to something meaningful. $\hat\beta$ that is found is implicitly linked to the frequency of the values picked by the designer. Changing the frequencies changes the $\hat\beta$. The difference with respect to the random predictor case is that in the random case we acknowledge that there is a distribution, we don't get to mess with it during the estimation, and it will be maintained into the future for the sake of making predictions and having a comfort in terms of average prediction error. In the fixed case nothing can be said about the frequency of the future usage.
Another interesting aspect is that variability of $\hat\beta$ with respect to the design frequencies sneaks into cross-validation procedures. How can one safely split the design matrix into folds if we cannot assume convergence to the same $\hat\beta$ in the limit for each fold? We simply cannot sample from the design matrix if we consider it fixed. Same argument applies to various resampling procedures.
Moreover the fixed-X model doesn't actually contain a placeholder for future $x$ values, i.e. $Y_i \sim f_i(y_i)$. We are just lucky that fixed-X and random-X inference coincides under correct specification, so that we can abuse notation as if we have a placeholder.
For this reason I conclude, perhaps ignorantly, either there is no such thing as a fixed regressor (hence it is a scientific conspiracy), or there is a different way of looking at this to rationalize it.
EDIT
https://economics.mit.edu/files/11856
On section 3, there is a discussion of the situation. It seems the problem emerges under misspecification. But this is almost always true when we have any data set at hand. Nobody knows the truth.

A point that has not received attention in the literature is that
  under general misspecification, the random versus fixed regressor
  distinction has implications for inference that do not vanish with the
  sample size.

..........

One way to frame the question is in terms of different repeated
  sampling perspectives one can take. We can consider the distribution
  of the least squares estimator over repeated samples where we redraw
  the pairs Xi and Yi (the random regressor case), or we can consider
  the distribution over repeated samples where we keep the values of Xi
  fixed and only redraw the Yi (the fixed regressor case). Under general
  misspecification both the mean and variance of these two distributions
  will differ.

I would appreciate if somebody could make a connection to the exchangeability argument for an explanation, if there is a merit to it. How is it even working on correctly specified fixed-X situation, the basic proof is in every text book for $E[\hat\beta] = \beta$, but did we just get lucky?
Comment: Can you show what you mean by changing coefficients when changing frequencies? I mean show it in an experimental setup, not in a mental experiment. Maybe you have the lab results that demonstrate it. It just sounds bizarre to me. I've done experimental physics long ago, and this claim doesn't correlate with my experience at all.
Comment: I still cannot comprehend what you might mean by "long run error rate"--and understanding this seems to be essential to understanding the question.  Calling it "a formula to accumulate" really doesn't tell us anything.  Could you please edit the question to explain this key concept?
Comment: Please revise it then!
Comment: What's "long run error rate" mean here exactly?
Comment: @CagdasOzgenc, regarding another question of yours that had a bounty which ended a few days ago (as of today, 10 March 2015). I cannot find that question anymore. You said you found out an answer but did not post it. I wonder what it was...
Comment: Well you wouldn't expect performance to be the same on a very different set of predictor values. But you can calculate/simulate what it should be. E.g. the coefficient of determination obtained in a designed experiment is often greater than that obtained when the model fitted is applied to a sample from a naturally occurring population.
Comment: Look into the concept of *partial exchangeability*. Also, the following paper might interest you: http://stat.wharton.upenn.edu/~buja/PAPERS/Notes-on-Wrong-Models-and-Random-X.pdf  (I found it very interesting)
Answer:
A regression model gives predictions of the response conditional on predictor values; so there's no problem in applying a model fitted to one set of predictor values fixed by design to another set of predictor values, even if the latter are randomly sampled from a population. With an experimental design matrix $X$, the expectation & variance of the predicted response $\hat y$ for a (new) predictor vector $x$ are given by
$$\operatorname{E}{\hat y \,|\, x} = x^\mathrm{T}\beta$$
$$\operatorname{Var}{\hat y\,|\,x}=\sigma^2\left(1+x^\mathrm{T}(X^\mathrm{T}X)^{-1}x\right)$$
where $\beta$ is the coefficient vector & $\sigma^2$ is the error variance—so the particular predictor values used for the fit don't affect the expectation of predictions, but do affect the variation in their precision throughout predictor space. Note that any aggregate fit metrics, say root mean square error of predictions, don't carry over from the experiment to the new sample.
The above discussion assumes the model is right: in practice there will be extra-statistical considerations when applying it. You need to think about e.g. variation of effects that weren't investigated in the original experiment, the reliability of extrapolation into new regions of predictor space, selection bias in the population, & whether experimental manipulation is comparable to a natural cause. An engineer might model resistivity as a linear function of temperature from experimental data & be confident in applying the model to a particular collection of resistors used in a circuit board. The medical researcher in your example might assert that the medicine reduces blood cholesterol level, & confidently predict the results of further experiments; but would be unlikely to claim that, in a random sample from, say, all hospital admissions, those patients taking the medicine would have lower cholesterol levels than those who weren't.
Comment: Note that we are not modeling the process that generates the design matrix, merely the relationship between the entries in the design matrix and the target variable.  That relationship should not depend on how the design matrix was generated.  If it does, there's something missing from the model, namely, the relationship between how the design matrix was generated and the target variable.  Analogously, consider $y = 3x + \epsilon$; the "$3$" remains the same whether or not I generate $x$ from a binomial$(5,0.2)$ distribution or from a Gaussian$(0,1)$ distribution.
Comment: Don't want to play burden tennis, but can you give a hint as to why you think it's not? Or perhaps an example of what could go wrong?
Comment: @jbowman: Wish I'd put that so clearly.
Comment: Exploration doesn't have to be into the new regions. If model's repeated usage for values in the design matrix has a different frequency (but same values), then this is still not the correct model. Design matrix will have to be reweighted to accomodate. Basically this model is not good for anything.


What is the L1 norm of the difference of two products?

Question: Assume that $f_1$ and $f_2$ are two bounded functions from $\mathbb R^2$ to $\mathbb R$ such that $||f_1-f_2||_{L^1}\leq \varepsilon$ for some $\varepsilon>0$.
Is it possible to bound in a convenient way the following $L^1$ norm  $$||f_1^n-f_2^n||_{L^1}$$ where $n\in\mathbb N$
Answer: Say $|f_1| \le M$ and $|f_2| \le M$.  By the mean value theorem,
$$
|f_1^n - f_2^n| \le |f_1-f_2|nM^{n-1}
$$
so integrate to get
$$
\|f_1^n-f_2^n\|_{L^1} \le nM^{n-1}\|f_1-f_2\|_{L^1} \le nM^{n-1}\epsilon .
$$


Geometric Interpretation of members of $\mathrm{O}(2)\setminus\mathrm{SO}(2)$

Question: I recently came across a question which asked to prove the defining properties of the orthogonal matrices (members of $\mathrm{O}(2)$), then to subsequently determine that they can be written in the form:
$$\mathbf{R}(\varphi)=\begin{pmatrix}\cos(\varphi) & \mp \sin(\varphi) \\ \sin(\varphi) & \pm \cos(\varphi)\end{pmatrix}$$
It then asks us which of these matrices correspond to two-dimensional rotations (i.e. members of the group $\mathrm{SO}(2)$). However, my question is as follows:

What is the geometrical interpretation of matrices $M\in\mathrm{O}(2)\setminus \mathrm{SO}(2)$?

My gut feeling is that these matrices somehow invert the points that they rotate, because $\det(M)=-1$, however, I am not sure if this is true and would like clarification.
Answer: The matrices in $O(2) \setminus SO(2)$ are exactly those that can be written as $FR$, where $R$ is a rotation, and $F$ is the diagonal matrix with $-1, 1$ as diagonal entries, i.e., a flip around the y-axis. So every matrix in your class is just a rotation followed by an orientation-reversal, or "flip", or whatever you like.
(Proof of my claim: suppose $M$ is in $O(2)\setminus SO(2)$. Then $FM$ is in $SO(2)$ because it has determinant $+1$. Hence it's a rotation $R$. Since $FM = R$, and $FF = I$, we have $M = FFM = FR$. )
Answer: They are reflections around a line.


sum each value in a list of tuples

Question: I have a list of tuples similar to this:
l = [(1, 2), (3, 4), (5, 6), (7, 8), (9, 0)]

I want to create a simple one-liner that will give me the following result:
r = (25, 20) or r = [25, 20] # don't care if tuple or list.

Which would be like doing the following:
r = [0, 0]
for t in l:
  r[0]+=t[0]
  r[1]+=t[1]

I am sure it is something very simple, but I can't think of it.
Note: I looked at similar questions already:
How do I sum the first value in a set of lists within a tuple?
How do I sum the first value in each tuple in a list of tuples in Python?
Comment: possible duplicate of [Python element-wise tuple operations like sum](http://stackoverflow.com/questions/497885/python-element-wise-tuple-operations-like-sum)
Comment: @CiroSantilli: it is not a duplicate. The question that you've linked works with *two* tuples. This question about a *list* of tuples. The transposition is the essential part of the solution. Though the answers work almost verbatim in both cases. Still `a,b` tuples and `a_list_of_tuples` are different (the diffence might be exposed in what solution is the most efficient).
Answer: Use zip() and sum():
In [1]: l = [(1, 2), (3, 4), (5, 6), (7, 8), (9, 0)]

In [2]: [sum(x) for x in zip(*l)]
Out[2]: [25, 20]

or:
In [4]: map(sum, zip(*l))
Out[4]: [25, 20]

timeit results:
In [16]: l = [(1, 2), (3, 4), (5, 6), (7, 8), (9, 0)]*1000

In [17]: %timeit [sum(x) for x in zip(*l)]
1000 loops, best of 3: 1.46 ms per loop

In [18]: %timeit [sum(x) for x in izip(*l)]       #prefer itertools.izip
1000 loops, best of 3: 1.28 ms per loop

In [19]: %timeit map(sum, zip(*l))
100 loops, best of 3: 1.48 ms per loop

In [20]: %timeit map(sum, izip(*l))                #prefer itertools.izip
1000 loops, best of 3: 1.29 ms per loop

Comment: @nick_w for `zip()` when used with * acts like unzipping a list. http://docs.python.org/2/library/functions.html#zip
Comment: `map` sometimes outperforms `list comprehension` when used with built-in functions.
Comment: Out of interest, what does the asterisk mean when you go `zip(*l)`?
Comment: O_O can't believe how simple that was, how could I have forgot about `zip`? thanks. Which is more efficient? the `map` or `list-comprehension` method?
Comment: This can be made slightly faster by using 'map' instread of a loop. As follows:

map(sum,zip(*res))
Comment: @jhrf Perhaps you should look at the answer one more time.
Comment: @Aशwiniचhaudhary OPPS! Sorry for not reading the answer completely. I am surprised, however, that map fails to beat an inline for loop even using izip (even if the difference is slight). My understanding was that inline for-loops were slower than map. Am I wrong about this? Pardon my silly comment!
Comment: Just want to mention that the timing results is for Python 2, since in Python 3 `zip` is an iterator this the `itertools.izip` doesn't exist.
Answer: Without using zip
sum(e[0] for e in l), sum(e[1] for e in l)

Answer: I want to add something to the given answer:
If I have an array of dict e.g.
l = [{'quantity': 10, 'price': 5},{'quantity': 6, 'price': 15},{'quantity': 2, 'price': 3},{'quantity': 100, 'price': 2}]

and i want to obtain two (or more) sums of calculated quantity over the values e.g. sum of quantities and of price*quantity
I can do:
(total_quantity, total_price) = (
sum(x) for x in zip(*((item['quantity'],
                       item['price'] * item['quantity'])
                      for item in l)))

Instead of:
total_quantity = 0
total_price = 0
for item in l:
     total_quantity += item['quantity']
     total_price += item['price'] * item['quantity']

Maybe the first solution is less readable, but is more "pythonesque" :)
Comment: What does that add to Ashwini's answer? The question is not about a list of dicts
Comment: you're right, but it adds something about using sum and zip for complex operations. I was looking for something like that, I've found this answer and I used it for my own problem, which was about dicts. Thus I thought it would be nice to share my solution with somebody else who could have my same issue.

sum each value in a list of tuples


Too many return values - golang

Question: Eu tenho o seguinte arquivo de conexão com o banco:
package banco

import (
    "database/sql"

    _ "github.com/go-sql-driver/mysql" // Driver de conexão com o MySQL
)

//Abre a conexão com o banco de dados
func Conectar(*sql.DB, error) {
    stringConexao := "#####:#####@/######?charset=utf8&parseTime=True&loc=Local"

    db, erro := sql.Open("mysql", stringConexao)
    if erro != nil {
        return nil, erro
    }

    if erro = db.Ping(); erro != nil {
        return nil, erro
    }

    return db, nil
}

Porém em todos os meus returns, eu obtenho o seguinte erro:
too many return values
have (*sql.DB, nil)
want ()

Eu já chequei alguns tópicos da comunidade, mas sigo sem sucesso com esses erros.
Answer: Conectar não retorna nenhum valor e você esta retornando dois.
func Conectar(*sql.DB, error) { <--
        stringConexao := "#####:#####@/######?charset=utf8&parseTime=True&loc=Local"

        db, erro := sql.Open("mysql", stringConexao)
        if erro != nil {
            return nil, erro <--
        }

        if erro = db.Ping(); erro != nil {
            return nil, erro <--
        }

        return db, nil <--

}

O correto seria algo como Conectar() (*sql.DB, error)
De uma olhada de como se retorna mais de um valor https://gobyexample.com/multiple-return-values
Comment: Ah sim, entendi... no meu caso seria Conectar() (*sql.DB, error)... muito obrigado, deixei passar isso mesmo.

Too many return values - golang


Is the ♭VII chord still a dominant chord in the natural minor scale?

Question: If I have the progression Cm > Ab > Bb > Cm from the C natural minor scale, does the Bb chord still work as a dominant chord as it would in Eb major? I know that in the minor scale, technically the Gm chord would be the v chord or perhaps G7 if we are talking about the harmonic minor scale but I was thinking that this progression does sound like a cadence even though there is no V chord in it.
Comment: Could you transform the "is X a ..." question into a "does X do ..." or "can X do ..." question? For example, "can the V major chord be used to perform a dominant function in a minor key".  As you've already seen, "is" questions are ambiguous and generate unnecessary debate and misunderstanding, because people tend to have different opinions as to what "is" implies. Some people take it as meaning, "can X be labelled as Y in all imaginable circumstances while retaining all the same mental associations".
Answer: You seem a bit confused with your scales. In the C natural minor scale the Bb would work as the dominant, but to Eb, if it was followed by Eb, but not to Cm. Your progression is Cm > Ab > Bb > Cm. In the Eb major scale that would be vi IV V vi, whereas in the C natural minor scale this would be i bVI bVII i.
If you want the dominant for the C natural minor scale, that would be Gm. But in practice you'll see that people more often use G, as a borrowed chord from the C major scale (or the melodic/harmonic minor). The reason is that G has the leading tone (B), which gives the chord the "dominant function".
So, the progression you're looking for, with G as dominant would be Cm > G > Cm or Cm > Gm > Cm if you don't want to borrow the G major chord.
Comment: @armani there are many chords that can be nicely resolved to a tonic,and can *act* as a dominant chord of some kind, but the dominant chord per se would be V
Comment: I don't see that I am confused by the scales. The i bVI bIIV i progression is exactly what I was talking about so not sure why you said I was confused about the scales.  Also,  I thought the G comes from the raised notes in the harmonic minor scale. It is the first time someone told me that the V7 chord in the minor scale is borrowed from the major scale.
Comment: @armani you said that Bb would act as a dominant to Cm; it doesn't. Also, G can be borrowed both from the harmonic, the melodic minor and the major scale
Comment: @Grace yes. Unless you choose to use B for the aforementioned reasons
Comment: Well, when I play the progression it does kind of resolve to the i chord (to my ears anyway)so I was asking if it did. I know it is not the correct v chord of the scale.
Comment: @armani - Shev is exactly correct. There is only one letter name for a dominant chord in a key. It's the V chord in that key. In your scenario, the Bb can and will *act* like a dominant, but it's wrong to *call* it the dominant. There's the difference.
Comment: It is quite common to use the chord **V** as the dominant to the chord **i**. Assuming you are in a natural minor scale, would it best to use chord **V** with the raised 7th, or use the chord **v** which is of the natural minor scale? @Shevliaskovic
Comment: @Grace the dominant uses flat 7th; that means (in C minor scale) G B D F
Comment: @Shevliaskovic flat 7th? Are you referring to the natural minor?
Comment: @Grace yes. In C *natural minor scale*, the G chord would have an F
Comment: @Shevliaskovic then the dominant would be G Bb D F, making it a Gm7 chord. Is that right?
Answer: I think you are saying that bVII sounds like a dominant because bVII I feels like conclusion, even though you know that the dominant is V (or v).
I think the concept you are referring to is a cadence, which is exactly that: a chord progression that feels like conclusion. Turns out the (maybe) most common cadence is the Perfect Cadence (V I), which involves the dominant, and maybe this is causing you some confusion.
There are countless types of cadences, some feel more like conclusion than others, depending on the situation. Keep in mind that "dominant" is just a name for the fifth degree of the scale, it isn't necessarily in a cadence, nor a cadence must necessarily have it!
Comment: Yes you are right, I now understand the terminology better. bVII > i does sound like a cadence to my ears and I should have used this term in my question. I will edit it.
Answer: There are various types of cadences that sound resolved.  Classical music tends to focus on perfect cadences (V>I) and, to a lesser extent, plagal cadences (IV>I) in terms of resolved cadences.  But there are other cadences that sound resolved too, and bVII can have a relatively strong "pull" to the I.  Not as much as a V7 does, but it's still very much there, and it's an interesting colour
There is a cadence sometimes called the "super mario cadence" or the "lady madonna" cadence which uses this pull upwards by a tone twice, bVI > bVII > I.  So the bVI pulls to the bVII, then the bVII to the I.  And it doesn't even have to be in key; you're talking about Ab Bb Cm but in many cases (mario and the beatles for example) you hear this in a major context i.e. Ab major > Bb major > C major
There are a series of chord progressions that do this.  The most famous is the "secondary dominant" (5 of 5 to 1) but you also see 4 of 4 to 1 and, in your case, 7 of 7 to 1.
e.g. (all examples here work with C major too)
5: Cm D7 G7 Cm
4: Cm Bb F Cm
7: Cm Ab Bb Cm
b2: Cm D Db Cm   (linked example in Dm and it's Dm > F > E7 > Emaj7 > Dm)
The confusion comes that in Jazz terms we like to talk (often quite loosely) about chords with a "dominant function" aka pulling to the 1, and how flexible you can stretch that terminology depends on who you're talking to, in what situation.  Often it's restricted more for chords that actually function as a substitution of a literal 5>1 (diminished chords, tritone substitutions etc.), but sometimes people do use it just to mean "any chord that pulls to a resolved 1", and in that sense you could say bVII can have a "dominant function" in a "pulls to the tonic" sense, but it's probably best to avoid that terminology in most cases.


Proof of logical statement using induction

Question: I've been struggling for a whole day with this proof in " discrete mathematics and its applications " by Rosen. I even found a solution online on sladder but wasn't conviced ( 90% the proof on sladder is correct and i just didn't get it ) . However i just came up with a proof but it's nothing like the one online so i wanted to share with more experienced people to check if the proof is indeed correct.(ps: the question is after induction is explained so we're supposed to use induction )
question:

My proof:
Answer: There is no need for induction (and fortunately not even a big urge to make all those "$\ldots$" rigorous).
In order to show
$$ \color{red}{[(p_1\to p_2)\land\cdots\land (p_{n-1}\to p_n)]}\to[\color{green}{(p_1\land\cdots \land p_{n-1})}\to\color{blue}{p_n}]$$
We assume the red part and show the gren/blue part, which again means that we assume the green part and show the blue part.
So assume "red". As it is of the form "something and $p_{n-1}\to p_n$", we have
$$ p_{n-1}\to p_n.$$
Next form assuming "green", which is of the form "something and $p_{n-1}$, we have
$$ p_{n-1}.$$
Now by modus ponens,
$$p_n, $$
as desired.

For  a similar statment that can be proved by induction, try
$$ \color{red}{[(p_1\to p_2)\land\cdots\land (p_{n-1}\to p_n)]}\to[\color{green}{p_1}\to\color{blue}{p_n}]$$


Trouble parsing blocks of text with python: incomplete parsing of text file

Question: I'm a chemist and very new to programming. I try to write programs to make my life easier when handling data. After scouring StackOverflow all day, I was finally able to write a short python script that parses a text file containing blocks of similar data separated by a blank line. My code works well, but it doesn't parse the last block. I'm not sure why. I tried searching for an answer but couldn't find one that helped.
In a typical text file, there are 361 blocks of data each containing information to construct a molecule in 3-D space with a different torsion angle for a set of four atoms. Here is an example of a text file I have tried parsing which only includes the first two blocks.
!Coordinate: -51.45857  Energy: *****
6 0.006074 0.000915 0.000760
6 0.003070 -0.004811 1.496641
6 1.065644 -0.015789 2.367841
6 2.500078 -0.010542 1.993114
6 3.043633 -0.885454 1.109936
6 2.319723 -2.061360 0.571949
6 1.651211 -3.009615 1.308815
16 0.964940 -4.223294 0.280714
6 1.598121 -3.476004 -1.156548
6 2.300403 -2.353600 -0.830192
1 2.774538 -1.713316 -1.566133
6 1.370973 -4.039010 -2.492108
6 2.306097 -3.847669 -3.514857
6 2.051238 -4.378854 -4.772466
7 0.959825 -5.084236 -5.080872
6 0.075629 -5.271691 -4.098835
6 0.226680 -4.776825 -2.808825
1 -0.547454 -4.952070 -2.067650
1 -0.811208 -5.846075 -4.358490
1 2.771093 -4.237936 -5.576037
1 3.231185 -3.312215 -3.327250
6 1.484740 -3.110171 2.791981
1 2.271126 -2.537323 3.291578
1 0.521994 -2.699519 3.116631
1 1.545489 -4.149268 3.130100
6 4.425208 -0.728995 0.567929
6 5.293981 -1.825349 0.536092
6 6.575924 -1.699782 0.012540
6 7.002467 -0.480078 -0.506308
6 6.138453 0.611969 -0.498798
6 4.860426 0.488085 0.033453
1 4.189564 1.341843 0.040929
1 6.459401 1.563510 -0.912065
1 8.000697 -0.382509 -0.922563
1 7.242127 -2.557541 0.005802
1 4.957135 -2.781274 0.928240
6 3.298894 1.044689 2.682189
6 2.806965 2.352662 2.756428
6 3.525634 3.346796 3.410575
6 4.740700 3.044040 4.018965
6 5.230033 1.741208 3.969123
6 4.514468 0.749369 3.308424
1 4.901734 -0.264238 3.270300
1 6.171693 1.494548 4.450468
1 5.300110 3.817950 4.536063
1 3.131670 4.358007 3.451132
1 1.851909 2.586965 2.294231
6 0.644628 0.032167 3.735978
6 -0.708788 0.041750 3.903716
16 -1.501825 0.018225 2.355367
6 -1.460523 0.074589 5.163238
6 -0.916630 -0.463354 6.334489
6 -1.645855 -0.393694 7.514376
7 -2.861426 0.150339 7.612820
6 -3.380262 0.652483 6.490232
6 -2.733763 0.643955 5.260195
1 -3.211536 1.093615 4.394957
1 -4.369681 1.095963 6.579511
1 -1.232419 -0.806908 8.432018
1 0.055022 -0.946356 6.323493
1 1.348290 0.078304 4.560069
1 -0.126732 -1.007882 -0.406234
1 -0.790297 0.637669 -0.396423
1 0.964526 0.378020 -0.366958

!Coordinate: -52.45859  Energy: *****
6 0.016006 0.016117 -0.001167
6 0.008091 0.004202 1.494640
6 1.068924 -0.017801 2.367520
6 2.503392 -0.009246 1.992562
6 3.048080 -0.887580 1.113704
6 2.322345 -2.062968 0.576734
6 1.653555 -3.010561 1.314091
16 0.963790 -4.222595 0.286393
6 1.595670 -3.475347 -1.151441
6 2.300257 -2.354228 -0.825550
1 2.774156 -1.714212 -1.561877
6 1.365619 -4.037046 -2.487061
6 2.299829 -3.846714 -3.510831
6 2.042363 -4.376373 -4.768547
7 0.949180 -5.079357 -5.076134
6 0.065835 -5.265841 -4.093142
6 0.219443 -4.772314 -2.802916
1 -0.554143 -4.946542 -2.060928
1 -0.822495 -5.838195 -4.352173
1 2.761473 -4.236208 -5.572914
1 3.226175 -3.313192 -3.323941
6 1.489754 -3.111703 2.797517
1 2.276398 -2.538063 3.295797
1 0.527124 -2.702199 3.123917
1 1.552391 -4.150812 3.135284
6 4.429609 -0.733119 0.571119
6 5.297405 -1.830292 0.541209
6 6.579288 -1.706863 0.016976
6 7.006698 -0.488561 -0.504432
6 6.143617 0.604255 -0.498839
6 4.865654 0.482526 0.034036
1 4.195506 1.336862 0.039937
1 6.465258 1.554673 -0.914138
1 8.004858 -0.392683 -0.921241
1 7.244728 -2.565225 0.011647
1 4.959792 -2.785154 0.935276
6 3.299443 1.049518 2.679214
6 2.802410 2.355625 2.752465
6 3.517994 3.353520 3.404255
6 4.735140 3.056448 4.011255
6 5.229631 1.755519 3.962371
6 4.517166 0.759897 3.304042
1 4.908495 -0.252167 3.266846
1 6.172959 1.513307 4.442708
1 5.292160 3.833294 4.526530
1 3.119965 4.363158 3.444095
1 1.845729 2.585480 2.291417
6 0.646126 0.024358 3.735114
6 -0.707598 0.038379 3.900610
16 -1.498235 0.027194 2.350780
6 -1.461222 0.067452 5.159060
6 -0.920246 -0.476558 6.328874
6 -1.650941 -0.410067 7.508021
7 -2.865451 0.136232 7.607012
6 -3.381624 0.644001 6.485729
6 -2.733382 0.639158 5.256592
1 -3.209059 1.093310 4.392548
1 -4.370246 1.089176 6.575392
1 -1.239629 -0.827839 8.424551
1 0.050370 -0.961638 6.317360
1 1.348629 0.063341 4.560569
1 -0.118307 -0.990869 -0.412258
1 -0.776594 0.657152 -0.398933
1 0.977453 0.391011 -0.363519

Each block contains the following information:

Header line containing the torsion angle.
Each line after the header line contains 4 columns: atomic number, x, y, z

I need to do the following to each block:

Extract the torsion angle. Delete line after extracting the torsion angle.
Change each atomic number to the corresponding element.
Write a separate *.xyz file which has the element instead of the atomic number and number of atoms at the top.

Here is a sample of my code:
import os
import re

#I just paste the file path for now. And change \ to \\
filepath = os.path.normpath("file.xyz")

#Dictionary for atomic number and element
replacements = {'1': 'H', '6': 'C', '7': 'N', '16':'S'}

#Open read and write files
originalFile = open(filepath, 'r')
writeEditedFile = open('output_all(edited).txt', 'w')
readEditedFile = open('output_all(edited).txt', 'r')

#Replace atomic numbers with element symbol
for lines in originalFile:
    writeEditedFile.write(re.sub('(^\d+)', lambda m: replacements[m.group()], lines))

#Extract torsion angle and append to array
with open('output_all(edited).txt', 'r') as wEF:
    torsionAngles = []
    for line in wEF:
        if '!' in line:
            for number in line.split():
                try:
                    torsionAngles.append(str(float(number)))
                except ValueError:
                    pass

#Write each line into a new file until a blank line
#The file is closed and a new one is opened
#This should continue until the last block
with readEditedFile as rEF:
    record = 0
    separateFile = open('Step_' + str(record+1) + '_TorsionAngle_' + torsionAngles[record] + '.xyz', 'w')
    separateFile.write('64 \n \n')
    for lines in rEF:
        if lines == "\n":
            record += 1
            separateFile.close()
            separateFile = open('Step_'+ str(record+1) + '_TorsionAngle_' + torsionAngles[record] + '.xyz', 'w')
            separateFile.write('64 \n \n')
        else:
            if '!' in lines:
                lines = ''
            else:
                separateFile.write(lines)

Sorry for the sloppy code! Here is an example of the first two files it outputs:
Filename: Step_1_TorsionAngle_-51.45857.xyz
64

C 0.006074 0.000915 0.000760
C 0.003070 -0.004811 1.496641
C 1.065644 -0.015789 2.367841
C 2.500078 -0.010542 1.993114
C 3.043633 -0.885454 1.109936
C 2.319723 -2.061360 0.571949
C 1.651211 -3.009615 1.308815
S 0.964940 -4.223294 0.280714
C 1.598121 -3.476004 -1.156548
C 2.300403 -2.353600 -0.830192
H 2.774538 -1.713316 -1.566133
C 1.370973 -4.039010 -2.492108
C 2.306097 -3.847669 -3.514857
C 2.051238 -4.378854 -4.772466
N 0.959825 -5.084236 -5.080872
C 0.075629 -5.271691 -4.098835
C 0.226680 -4.776825 -2.808825
H -0.547454 -4.952070 -2.067650
H -0.811208 -5.846075 -4.358490
H 2.771093 -4.237936 -5.576037
H 3.231185 -3.312215 -3.327250
C 1.484740 -3.110171 2.791981
H 2.271126 -2.537323 3.291578
H 0.521994 -2.699519 3.116631
H 1.545489 -4.149268 3.130100
C 4.425208 -0.728995 0.567929
C 5.293981 -1.825349 0.536092
C 6.575924 -1.699782 0.012540
C 7.002467 -0.480078 -0.506308
C 6.138453 0.611969 -0.498798
C 4.860426 0.488085 0.033453
H 4.189564 1.341843 0.040929
H 6.459401 1.563510 -0.912065
H 8.000697 -0.382509 -0.922563
H 7.242127 -2.557541 0.005802
H 4.957135 -2.781274 0.928240
C 3.298894 1.044689 2.682189
C 2.806965 2.352662 2.756428
C 3.525634 3.346796 3.410575
C 4.740700 3.044040 4.018965
C 5.230033 1.741208 3.969123
C 4.514468 0.749369 3.308424
H 4.901734 -0.264238 3.270300
H 6.171693 1.494548 4.450468
H 5.300110 3.817950 4.536063
H 3.131670 4.358007 3.451132
H 1.851909 2.586965 2.294231
C 0.644628 0.032167 3.735978
C -0.708788 0.041750 3.903716
S -1.501825 0.018225 2.355367
C -1.460523 0.074589 5.163238
C -0.916630 -0.463354 6.334489
C -1.645855 -0.393694 7.514376
N -2.861426 0.150339 7.612820
C -3.380262 0.652483 6.490232
C -2.733763 0.643955 5.260195
H -3.211536 1.093615 4.394957
H -4.369681 1.095963 6.579511
H -1.232419 -0.806908 8.432018
H 0.055022 -0.946356 6.323493
H 1.348290 0.078304 4.560069
H -0.126732 -1.007882 -0.406234
H -0.790297 0.637669 -0.396423
H 0.964526 0.378020 -0.366958

Filename: Step_2_TorsionAngle_-52.45859.xyz
64

C 0.016006 0.016117 -0.001167
C 0.008091 0.004202 1.494640
C 1.068924 -0.017801 2.367520
C 2.503392 -0.009246 1.992562
C 3.048080 -0.887580 1.113704
C 2.322345 -2.062968 0.576734
C 1.653555 -3.010561 1.314091
S 0.963790 -4.222595 0.286393
C 1.595670 -3.475347 -1.151441
C 2.300257 -2.354228 -0.825550
H 2.774156 -1.714212 -1.561877
C 1.365619 -4.037046 -2.487061
C 2.299829 -3.846714 -3.510831
C 2.042363 -4.376373 -4.768547
N 0.949180 -5.079357 -5.076134
C 0.065835 -5.265841 -4.093142
C 0.219443 -4.772314 -2.802916
H -0.554143 -4.946542 -2.060928
H -0.822495 -5.838195 -4.352173
H 2.761473 -4.236208 -5.572914
H 3.226175 -3.313192 -3.323941
C 1.489754 -3.111703 2.797517
H 2.276398 -2.538063 3.295797
H 0.527124 -2.702199 3.123917
H 1.552391 -4.150812 3.135284
C 4.429609 -0.733119 0.571119
C 5.297405 -1.830292 0.541209
C 6.579288 -1.706863 0.016976
C 7.006698 -0.488561 -0.504432
C 6.143617 0.604255 -0.498839
C 4.865654 0.482526 0.034036
H 4.195506 1.336862 0.039937
H 6.465258 1.554673 -0.914138
H 8.004858 -0.392683 -0.921241
H 7.244728 -2.565225 0.011647
H 4.959792 -2.785154 0.935276
C 3.299443 1.049518 2.679214
C 2.802410 2.355625 2.752465
C 3.517994 3.353520 3.404255
C 4.735140 3.056448 4.011255
C 5.229631 1.755519 3.962371
C 4.517166 0.759897 3.304042
H 4.908495 -0.252167 3.266846
H 6.172959 1.513307 4.442708
H 5.292160 3.833294 4.526530
H 3.119965 4.363158 3.444095
H 1.845729 2.585480 2.291417
C 0.646126 0.024358 3.735114
C -0.707598 0.038379 3.900610
S -1.498235 0.027194 2.350780
C -1.461222 0.067452 5.159060
C -0.920246 -0.476558 6.328874
C -1.650941 -0.410067 7.508021
N -2.865451 0.136232 7.607012
C -3.381624 0.644001 6.485729
C -2.733382 0.639158 5.256592
H -3.209059 1.093310 4.392548
H -4.370246 1.089176 6.575392
H -1.239629 -0.827839 8.424551
H 0.050370 -0.961638 6.317360
H 1.348629 0.063341 4.560569
H -0.118307 -0.990869 -0.412258
H -0.776594 0.657152 -0.398933
H 0.977453 0.391011 -0.363519

The simple code does what I want it to do for every block except the last one!  Any suggestions or tips would be greatly appreciated! Thanks for reading my post!
Answer: This script will read the sample input data (as written in question) from file.txt and writes two files Step_1_TorsionAngle_-51.45857.xyz and Step_2_TorsionAngle_-52.45859.xyz:
import re

replacements = {'1': 'H', '6': 'C', '7': 'N', '16':'S'}

with open('file.txt', 'r') as f_in:
    data = f_in.read()

torsion_angles = re.findall(r'!Coordinate:\s+(.*?)\s+Energy', data)
blocks = [b.splitlines() for b in re.findall(r'^(\d.*?)(?=\s*!|\Z)', data, flags=re.DOTALL|re.M)]

for step, (angle, block) in enumerate(zip(torsion_angles, blocks), 1):
    with open('Step_{}_TorsionAngle_{}.xyz'.format(step, angle), 'w') as f_out:
        f_out.write(str(len(block)) + '\n\n')
        lines = [' '.join([replacements[s[0]], *s[1:]]) for s in [v.split() for v in block]]
        f_out.write('\n'.join(lines))

The content of files are like this:
64

C 0.006074 0.000915 0.000760
C 0.003070 -0.004811 1.496641
C 1.065644 -0.015789 2.367841
C 2.500078 -0.010542 1.993114

...etc.

Comment: Thank you so much! This does everything I needed! It does give me an error though which I assume is because it's trying to parse the non-existent 362 block.

lines = [' '.join([replacements[s[0]], *s[1:]]) for s in [v.split() for v in block]]
IndexError: list index out of range

Is there any way to suppress this error? I'm not sure if this will be a problem when constructing a GUI out of this. Regardless, I get every file now!  I can live with the error. Thank you so much!


call a method of class instance located inside a python list

Question: I have following classes in python:

QueryElement as a root class
ElemMatch and GT which inherit from the root.

I have a list in ElemMatch class which is supposed to have instances of QueryElement.
My problem is in invoking a method called compute from the instances inside the list, in ElemMatch class(compute method). The type of object inside the list is not identified by Python, and I do not know how to assign a type to the list. I do not have such a problem in Java since I could 'cast' to a type I like, but here I do not know how to solve it.
I appreciate if you could help.
class QueryElement(object):
    __metaclass__ = abc.ABCMeta

    @abc.abstractmethod
    def addQueryElement(self, queryElement):
        raise NotImplementedError( "Should have implemented this" )

    @abc.abstractmethod
    def compute(self):
        raise NotImplementedError( "Should have implemented this" )

class ElemMatch(QueryElement):

    def __init__(self):
        self._queryElements = []

    def addQueryElement(self, queryElement):
        self._queryElements.append(queryElement)

    def compute(self):
        elemMatch = {}
        if len (self._queryElements) > 0:
            elemMatch['e'] = self._queryElements[0].compute()

        return elemMatch

class GT(QueryElement):

    def __init__(self):
        print 'someThing'

    def addQueryElement(self, queryElement):
        return None

    def compute(self):
        print 'compute GT!'

class PALLAS(object):
    def foo(self):
        gt = GT()
        elemMatch = ElemMatch()

        elemMatch.addQueryElement(gt)
        elemMatch.compute()

p = PALLAS()
p.foo()

Comment: Is there an error being thrown when you use this code?
Comment: Are you trying to change the type of the object, or call a `compute` function belonging to a different class from the object? Or is your list already made up of `QueryElement` objects and you just want to call `compute`?
Comment: @TheSoundDefense : I am not trying to change the type of object. I just make an empty list(in init) and create two methods to populate it and call compute method.
Comment: @TheSoundDefense: I think python does not identify the type since populating the list is another class (PALLAS).
Comment: @IanAuld: It just says that compute is not attribute of queryElements :(
Comment: @user3376020 Python does not need to know the type before attempting to call `compute`. It will just check to see if the object has a `compute` method, and call it if it does. It's not like Java where the type has to be specifically known ahead of time. See the current answer.
Comment: If you're getting an exception, can you include the full stack trace? The code you've shown seems just works when I try it, so if it is not working for you you'll need to elaborate a bit more on what is going wrong.
Answer: In Python, the objects, not the names referring to them, are typed. If the object has a compute method, you can call it, regardless of what the type of the object is.
A quick example:
class A(object):
    def foo(self):
        print "I'm an A"

class B(object):
    def foo(self):
        print "I'm a B"

lst = [A(), A(), B(), A(), B()]
for l in lst:
    l.foo()

Each element of lst is either an instance of A or of B. Since both have a method named foo, you don't have to know the type of the object referenced by l each time through the loop; the lookup of foo will find the correct method.
This is commonly referred to as duck typing; if l looks like a duck and acts like a duck (i.e., if it has a method foo), then it is a duck (i.e., then we can call the method foo).
Comment: Thanks for the explanation. It is very useful. In my case, the objects which eventually will be added to the list, are all in the same type, with compute() method. What are possible reasons that python does not identify it?
Comment: If I understand your question correctly, it's a design decision. Python is often said to have names, not variables. The names can refer to anything: one moment, `x` might refer to an integer, the next, a function, and after that a class. The downside is that you can't catch errors like "`x` doesn't have an attribute `foo`" at compile time, because you won't know for sure what `x` refers to until you actually try to access the `foo` attribute of whatever `x` refers to at the moment.
Comment: Put another way, many of the things you may be used to determining at compile time aren't actually done until run time in Python.


3D forms of a two variables separated function

Question: I am looking for all the possible forms in 3D space of the function defined as $$ \Psi(x,t) = \psi(x) e^{-it}$$
There is this funny constraint: $$|\Psi (x,t)|^2 = \psi^{\ast}(x)\psi(x) e^{it} e^{-it}$$
$$|\Psi (x,t)|^2 = |\psi (x)|^2$$
For every $t$, so $|\Psi|$ doesn't depend on t.
So far, I've found one possible form, a function looking like a spring like in this image (but with the x-axis inside of the spring):

As time goes by (yes, I am of the ones that have a very hard time to see time as an extra axis) the spring rotates exactly as a screw does
Do you guys know another form of this function in 3D?
EDIT: After the reply below, I am looking now forward to vizualize what the $\Psi$ function, where $\psi$ has the form $A(x)e^{it}$, would look like.
I am wondering if it looks like the function that is in the two following images... or not:

Here the $A(x)$ would be a sort of gaussian?
Answer: $\psi(x) = A(x) e^{i \phi(x) }$
Therefore,
$ \Psi(x) = A(x) e^{i(\phi(x) - t )} = A(x) \cos(\phi(x) - t) + i A(x) \sin(\phi(x) - t) $
Let $y = A(x) \cos(\phi(x) - t) $ , $ z = A(x) \sin(\phi(x) - t) $
To generate the screw in the picture, set $A(x) = A $ a constant, and $ \phi(x) = x $
Then
$(x,y, z) = ( x, A \cos(x - t), A \sin(x - t))$
For a fixed $t$ this is a spiral (helix) along the $x$ axis.  As $t$ advances,the spiral rotates.
Comment: Yes.  It does.  And $A(x)$ looks gaussian $ A(x) = a \exp(- b x^2) $
Comment: So the $A(x)$ is now giving me a hard time to conceptualize it. Could you tell if the two new images I have appended to my OP correspond to this $A(x) e^{it}$ function please?


Apache Flink streaming word count aggregate over lines

Question: I've bumped into a class of issues which are non-existent in batch processing, but seem non-trivial for streaming case. Let's consider the classic word count example:
lines
  .flatMap(_.split("\\W+"))
  .map(word => (word, 1))
  .keyBy(0)
  .sum(1)

This will print the result for every word in the stream, e.g.:
input: "foo bar baz foo"
output: (foo, 1) (bar, 1) (baz, 1) (foo, 2)

What I'd like to do instead is to process each line as a whole and only then print the result, i.e. use a window over each line:
input: "foo bar baz foo"
output: (foo, 2) (bar, 1) (baz, 1)

Obviously, neither time-based nor count-based windows are applicable here. What is the correct way to tackle the problem?
Comment: You need to merge `flatMap` and `map` into a single operator, and also do some pre aggregation (ie counting how often each word is in a sentence). Thus, input foo bar baz foo" would be transformed into "(foo, 2) (bar, 1) (baz, 1)" *before* the keyBy
Comment: I need to parallelise operations on individual words before grouping them back.
Comment: I do not understand what you mean by "window over each line". I think adding more input lines to your example and showing the desired output would help here.
Comment: @FabianHueske I mean that I want to count words per line, i.e. wait until all words in a line are processed before emitting any output. This is just an example, the general problem is: split each input record into different parts (aka flatMap), process these parts in parallel and then aggregate them (group by each initial record) for further processing.
Comment: I.e. logically it looks like this: `lines.map(line => line.split("\\W+").map(processWord).reduce(aggregateWords))`, which is what Mathias suggested, but the processing function is quite heavy, so I'd like to parallelise it somehow: `lines.flatMap((_.id, _.split("\\W+"))).map(processWord).groupAllRecords(_.id).reduce(aggregateWords))`
Comment: You could also add a (random) line id to the word you emit in flatMap and use "lineId + word" as key.
Comment: How do I know when I consumed all records with a given lineId and should do some aggregation? That's the problem.
Answer: Processing both words and lines in parallel is not possible even in batch mode because nested groupBy (or keyBy) is not supported in Flink. If however, you want the streaming version of the following batch word-count:
lines
  .flatMap(line => (lineId,word,1))
  .groupBy(0)
  .reduceGroup {aggregateWords}

where aggregateWords iterates over words for that specific key and counts them, then you can implement it in the following way: for each line you emit the words along with a special record at the end, then you use a GlobalWindow with a custom trigger that fires once the special record is received.
A streaming version of the previous batch job could look like:
public static void main(String[] args) throws Exception {

    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

    env.fromElements("foo bar baz foo", "yes no no yes", "hi hello hi hello")

            .flatMap(new FlatMapFunction<String, Tuple3<Double, String, Integer>>() {
                @Override
                public void flatMap(String s, Collector<Tuple3<Double, String, Integer>> collector) throws Exception {
                    String[] words = s.split("\\W+");
                    Double lineId = Math.random();
                    for (String w : words) {
                        collector.collect(Tuple3.of(lineId, w, 1));
                    }
                    collector.collect(Tuple3.of(lineId, "\n", 1));
                }
            })
            .keyBy(0)
            .window(GlobalWindows.create())
            .trigger(new Trigger<Tuple3<Double, String, Integer>, GlobalWindow>() {
                @Override
                public TriggerResult onElement(Tuple3<Double, String, Integer> element, long l, GlobalWindow globalWindow, TriggerContext triggerContext) throws Exception {
                    if (element.f1.equals("\n")) {
                        return TriggerResult.FIRE;
                    }
                    return TriggerResult.CONTINUE;
                }

                @Override
                public TriggerResult onProcessingTime(long l, GlobalWindow globalWindow, TriggerContext triggerContext) throws Exception {
                    return TriggerResult.CONTINUE;
                }

                @Override
                public TriggerResult onEventTime(long l, GlobalWindow globalWindow, TriggerContext triggerContext) throws Exception {
                    return TriggerResult.CONTINUE;
                }
            })
            .fold(new HashMap<>(), new FoldFunction<Tuple3<Double, String, Integer>, HashMap<String, Integer>>() {
                @Override
                public HashMap<String, Integer> fold(HashMap<String, Integer> hashMap, Tuple3<Double, String, Integer> tuple3) throws Exception {
                    if (!tuple3.f1.equals("\n")) {
                        hashMap.put(tuple3.f1, hashMap.getOrDefault(tuple3.f1, 0) + 1);
                    }
                    return hashMap;
                }
            }).print();

    env.execute("Test");

}

Output:
{bar=1, foo=2, baz=1}
{no=2, yes=2}
{hi=2, hello=2}

Comment: Omg, that looks complicated! In batch mode this problem doesn't exist because you don't need to trigger anything - you just collect all existing records for each key in `groupBy`, while in streaming mode you need to know when you collected enough and need to trigger.


BeanIO fixed length file writer Scala- Avoid one field from writing

Question: I am trying to write a csv file from scala case class using fixed length file BeanIO library.
sample code
case class employee(id:String,name:String,dob:String)

<record name="emp" class="employee">
 <field name="id" position="0" length="5" getter="#1" setter="id"/>
 <field name="name" position="5" length="5" getter="#2" setter="name"/>
 <field name="dob" position="10" length="5" getter="#3" setter="dob"/>
</record>

But I want to avoid dob from writing in the csv file.
If I removed that line from xml, it will throw error
Could anyone suggest any way to do that other than removing it from case class or make field length as "zero".
Comment: I don't know scala and how it reads XML files, are you sure about the colon ':' that is used on the attributes of the xml mapping file? Should they not be all '=' instead (I changed it in my answer)?
Comment: yes, unintentionally put ':' in my question . Actually its '='. I edited my question. Thanks
Answer: How about treating the dob field as a constant?

If a bean property does not map to a field in the stream, a constant
property value can still be set using a property element. Like a
field, all properties must specify a name attribute, which by default,
is used to get and set the property value from the bean object.
Properties also require a value attribute for setting the textual
representation of the property value. The value text is type converted
using the same rules and attributes (type, typeHandler and format)
used for field type conversion described above. Collection type
properties are not supported.

Try this:
<record name="emp" class="employee">
 <field name="id" position="0" length="5" getter="#1" setter="id"/>
 <field name="name" position="5" length="5" getter="#2" setter="name"/>
 <property name="dob" position="10" length="5" getter="#3" setter="dob" value="" />
</record>

The value for dob would then not be dependent on the actual value it is set to or not in your class. This way you control the output. You can also experiment with changing the name of the property to something that is not present in the class.


An algorithm for pavement usage calculation

Question: Given an area of specific size I need to find out how many pavement stones to use to completely pave the area. Suppose that I have an empty floor of 100 metre squares and stones with 20x10 cm and 30x10 cm sizes. I must pave the area with minimum usage of stones of both sizes. Anyone knows of an algorithm that calculates this?
(Sorry if my English is bad)
C# is preferred.
Comment: +1 for not seeking the easy route, but the one that leads to learning :-)
Comment: Write out how you would solve three different scenarios, put the solution into an algorithm and then see if you can generalize it.
Comment: There is a difference between an algorithm and an implementation.  An algorithm describes how to solve a problem.  An implementation is code that implements the algorithm.  Are you looking for an algorithm or some C# code that does the homework for you?
Comment: Both will do but an algorithm will be better.
Answer: This is a category of problem known as a Packing Problem.
This DevX article provides background and outlines approaches without directly solving the homework for you (it also provides code, but you'll have to think a bit to apply it to your situation).
Answer: You can solve this in your head (assuming the area to fill is rectangular). If you have to fill an area of N squares, and your tiles are 2x1 and 3x1, you never need more than two 2x1 tiles. This gives the total number of tiles you need to be N/3 (rounded up) except for the case N=1 which is impossible.
Proof:
Suppose your area is A x B and that not both of A and B are 1. Assume without loss of generality that A != 1.
You can tile a rectangular area A x 3 with 3x1 tiles easily. Repeat this pattern to fill up the are as much as possible.
If there are no rows left, you're done (you've tiled the entire area with 3x1 tiles)
If there's one row left, fill with 3x1 tiles until you have either 0, 1 or 2 spaces left. 0 => you're done, 1 => replace the last 3x1 with two 2x1, 2 => fill the last square with a 2x1.
If there's two rows left, do a similar construction. You'll be left with either 0 columns (you're done), 1 column (fill it with a 2x1), or 2 columns (fill it with two 2x1).


ggplot not showing inside ipython notebook output area, rather popping up

Question: I'm using IPython notebooks to save my results and perhaps to share code including graphics. I am using ggplot right now. But I cannot get ggplot to plot inside the notebook output area. It always gives me a pop-up window that shows the plot. I don't know how to save it along with the notebook easily. Is there something I need to configure to make that happen? "%matplotlib inline" I saw in a ggplot tutorial that below code should do it. What am I missing?
My code:
<code>plot = ggplot(my_dataframe, aes("x")) + geom_histogram()
print plot
</code>
Answer: I got my answer elsewhere. It worked like a charm!
%pylab inline


C# Async Sockets (*Async, SocketAsyncEventArgs)

Question: I've been working with the .NET 3.5 asynchronous socket API and found that ll of the *Async methods return a bool, which is false if the operation completed synchronously.
I'm a little confused as to how/why this could happen. It's not always an error condition, right? The MSDN examples immediately call the event handler manually in this case. Is that a good practice to follow?
Additionally, when I call Socket.SendAsync for instance, and attach a handler to SocketAsyncEventArgs.Completed, am I guaranteed that "Completed" is fired (or SendAsync returns false) only after all of my data has been sent? Or is it possible that it will call my handler somewhere in the middle?
Answer: It may return synchronously if the action requested can be preformed instantly, for instance, if the TCP Buffer already contains the data required to complete the ReceiveAsync method, or a client is already waiting to be accepted ect.
The examples the handler because the AsyncMethod will not. I would be happy to use invoke the handler directly in this case (if there is only one subscribing). However there is a practice which suggests that you shouldn't, rather that the handler should not do anything except call another method (posibly with more meaningful params) which handles the logic - and that you should invoke this method rather than the actual event/handler.
The event will not be raised before the operation completes, Unless of course you do it manually when the async method returns true (is not complete).
From MSDN:

Returns true if the I/O operation is pending. The SocketAsyncEventArgs.Completed event on the e parameter will be raised upon completion of the operation.
Returns false if the I/O operation completed synchronously. In this case, The SocketAsyncEventArgs.Completed event on the e parameter will not be raised and the e object passed as a parameter may be examined immediately after the method call returns to retrieve the result of the operation.
Comment: When you say "the operation completes" (for SendAsync), does "complete" mean that the entire buffer of data (that you specify) has been sent? Inuitively, semantically, I am thinking "of course", but I just want to be absolutely positive.
Comment: Yeap, for that instance of SocketAsyncEventArgs. Perhaps the Socket.BeginXYZ / Socket.EndXYZ methods would be nicer (no need to worry about completing synchronously - although that info is passed to the callback aswell)


ORA-03134: Connections to this server version are no longer supported

Question: Environment:Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.6.1586.0
Backend: Oracle 9i
Tools : Visual Studio 2012, ODAC 11.2 Version
The application works fine when I debug through Visual studio 2012 and all CRUD operation works perfectly.
But when I deploy it to the IIS server it said "ORA-03134: Connections to this server version are no longer supported as in below image"

IIS configuration(Application Pool) is as follows :
Comment: The terminal patchset for 9iR2 is _over 10 years old_. Time to upgrade already.
Comment: Then you need to use tools that support that ancient version.
Comment: Here is [an Oracle forum thread which has a similar sounding problem](https://community.oracle.com/thread/2603090?start=0&tstart=0). The solution there was to change the configuration of the GAC. Maybe it could help you?
Comment: The client is not willing to upgrade the backend side.
Comment: So I am using ODAC 11.2 Version that supports 9i but the problem is, it does not work in IIS but works perfectly in Visual study debuging mode
Answer: You can use a gateway DB version 11.2 to access the old database 220.127.116.11.
Create a dblink from the gateway DB version 11.2 for the old database 126.96.36.199.
C:\oracle\instantclient_12_2>.\sqlplus.exe  scott/tiger@oracle11

SQL*Plus: Release 18.104.22.168.0 Production on Mon Feb 18 07:31:38 2019

Copyright (c) 1982, 2017, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Release 188.8.131.52.0 - 64bit Production

SQL> select * from dept@oracle9;

    DEPTNO DNAME          LOC
---------- -------------- -------------
        12 RESEARCH       SARATOGA
        10 ACCOUNTING     CLEVELAND
        11 ART            SALEM
        13 FINANCE        BOSTON
        21 SALES          PHILA.
        22 SALES          ROCHESTER
        42 INT'L          SAN FRAN

7 rows selected.

Oracle client 18.104.22.168 can connect to Oracle RDBMS 22.214.171.124.
C:\oracle\instantclient_11_2>.\sqlplus.exe  scott/tiger@oracle9

SQL*Plus: Release 188.8.131.52.0 Production on Mon Feb 18 07:37:22 2019

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle9i Enterprise Edition Release 18.104.22.168.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 126.96.36.199.0 - Production

SQL>

Oracle client 22.214.171.124 can not connect to Oracle RDBMS 184.108.40.206.
C:\oracle\instantclient_12_2>.\sqlplus.exe  scott/tiger@oracle9

SQL*Plus: Release 18.104.22.168.0 Production on Mon Feb 18 07:36:45 2019

Copyright (c) 1982, 2017, Oracle.  All rights reserved.

ERROR:
ORA-03134: Connections to this server version are no longer supported.


How can we restrict module developers advertising in Magento SE?

Question: I came across few situations in recent days some module developers digging in old questions or the questions which are related to their developed module and using comments and answers to advertise their modules I flagged some of them which were done on the answers of mine but how to stop them doing this?How to keep MSE away from this kind of things?
Answer: First take a look at the user profile. If you see that all of the answers (or almost all of them) are self promoting, flag one of their answers for moderator's attention and fill in a message saying "This user only self promotes". You can even put the link to the user profile.
Most probably their accounts will be suspended temporarily or deleted.
Comment: You cannot link spam user accounts through some feature. But when flagging something you have the "Other" option. You should fill there the link to the user account.
Comment: I just thought I can only flag the comments I'm not aware of linking spam users account thanks for this info surely from now I follow to do so.


Singular value decomposition proof

Question: I need help in the following question. I'm not sure how to even begin to answer this. What is a possible proof for the following question?
If $A$ is an $m \times n$ matrix and $b$ is an $m$-vector, prove that the solution $x$ of minimum Euclidean norm to the least-squares problem $A\cong B$ is given by
$$x=\sum_{\sigma_i \neq 0} \frac{u_i^Tb}{\sigma_i}v_i$$
where the $\sigma_i$, $u_i$, and $v_i$ are the singular values and corresponding singular vectors of A.
Comment: I reported this as unclear what you're asking, but at second look it simply appears to be a malicious edit of a valid question. That is to say, a pseudo-deletion.
Comment: It would be nice to have the objective function. I have no idea what $\rm A \cong B$ means.
Comment: The least squares solution you post is derived here: http://math.stackexchange.com/questions/772039/how-does-the-svd-solve-the-least-squares-problem/2173715#2173715
Comment: Possible duplicate of [How does the SVD solve the least squares problem?](http://math.stackexchange.com/questions/772039/how-does-the-svd-solve-the-least-squares-problem)
Comment: Steve, care to explain why you undo edits by others? It's not like you own this post! Did you not read [the fine print](http://math.stackexchange.com/help/licensing)?
Answer: Preliminaries
Start with a tighter specification on the target matrix:
$$
  \mathbf{A} \in \mathbb{C}^{m\times n}_{\rho}
$$
where the rank $\rho  < \min (m,n)$. The matrix is rank defective and both null spaces are nontrivial.
We are given the data vector $b\notin\color{red}{\mathcal{N}\left( \mathbf{A}^{*} \right)}$ to insure a least squares solution exists. Defining the vector of residual errors
$$
r(x) = \mathbf{A} x - b,
$$
the desire is to find the solution vector $x$ which minimizes the total error $r^{2}$. The solution set is given by the minimizers
$$
  x_{LS} =
\left\{
  x\in\mathbb{C}^{n} \colon
\lVert
  \mathbf{A} x - b
\rVert_{2}^{2}
\text{ is minimized}
\right\}
$$
For each element in this set, $r^{2}\left(x_{LS}\right)$ achieves minimum value. We are searching for the element in this set which has minimum length.
In Laub's book (p. 66), he shows the solution is in general an affine space:

represented as the dashed red line in the figure below.

The singular value decomposition is
$$
\begin{align}
  \mathbf{A} &=
  \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\
%
 &=
% U
  \left[ \begin{array}{cc}
     \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}}
  \end{array} \right]
% Sigma
  \left[ \begin{array}{cc}
     \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\
     \mathbf{0} & \mathbf{0}
  \end{array} \right]
% V
  \left[ \begin{array}{c}
     \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\
     \color{red}{\mathbf{V}_{\mathcal{N}}}^{*}
  \end{array} \right]  \\
%
\end{align}
$$
The Moore-Penrose pseudoinverse is constructed from the SVD:
$$
\begin{align}
  \mathbf{A}^{\dagger} &=
  \mathbf{V} \, \Sigma^{\dagger} \, \mathbf{U}^{*} \\
%
 &=
% U
  \left[ \begin{array}{cc}
     \color{blue}{\mathbf{V}_{\mathcal{R}}} &
     \color{red} {\mathbf{V}_{\mathcal{N}}}
  \end{array} \right]
% Sigma
  \left[ \begin{array}{cc}
     \mathbf{S}^{-1} & \mathbf{0} \\
     \mathbf{0} & \mathbf{0}
  \end{array} \right]
% V
  \left[ \begin{array}{c}
     \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\
     \color{red} {\mathbf{U}_{\mathcal{N}}}^{*}
  \end{array} \right]  \\
%
\end{align}
$$
The object of the proof is to show that the pseudoinverse solution has minimum length.
Trick
Cast the total error in terms of the SVD and perform a unitary transformation
$$
  r^{2}(x) =
\lVert
  \mathbf{A} x - b\rVert_{2}^{2}
=
\lVert
  \mathbf{U} \, \Sigma \, \mathbf{V}^{*} x - b
\rVert_{2}^{2}
=
\lVert
  \Sigma \, \mathbf{V}^{*} x - \mathbf{U}^{*} b
\rVert_{2}^{2}
$$
to separate the $\color{blue}{range}$ and $\color{red}{null}$ spaces.
$$
\begin{align}
  r^{2}(x) =
\lVert
  \Sigma \, \mathbf{V}^{*} x - \mathbf{U}^{*} b
\rVert_{2}^{2}
&=
\Bigg\lVert
\left[
\begin{array}{cc}
 \mathbf{S} & \mathbf{0} \\
 \mathbf{0} & \mathbf{0} \\
\end{array}
\right]
  \left[ \begin{array}{c}
     \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\
     \color{red} {\mathbf{V}_{\mathcal{N}}}^{*}
  \end{array} \right]
x
-
  \left[ \begin{array}{c}
     \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} \\
     \color{red} {\mathbf{U}_{\mathcal{N}}}^{*}
  \end{array} \right]
b
\Bigg\rVert_{2}^{2} \\
%
&=
\big\lVert
\mathbf{S} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} x -
  \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} b
-
\color{red} {\mathbf{U}_{\mathcal{N}}}^{*} b
\big\rVert_{2}^{2} \\
%
&=
\big\lVert
\mathbf{S} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} x -
  \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} b
\big\rVert_{2}^{2} + \big\lVert
\color{red} {\mathbf{U}_{\mathcal{N}}}^{*} b
\big\rVert_{2}^{2} \\
%
\end{align}
$$
How to minimize the total error now? We control the range space component, and we force that contribution to $0$:
$$
r^{2}(x) =
\underbrace{\big\lVert
\mathbf{S} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} x -
  \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} b
\big\rVert_{2}^{2}}_{\text{controlled}} +
\underbrace{\big\lVert
\color{red} {\mathbf{U}_{\mathcal{N}}}^{*} b
\big\rVert_{2}^{2}}_{\text{uncontrolled}} \\
$$
That is, force
$$
 \mathbf{S} \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} x -
  \color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} b = 0
$$
by setting
$$
  \color{blue}{x_{LS}} = \color{blue}{\mathbf{V}_{\mathcal{R}}}\mathbf{S}^{-1}\color{blue}{\mathbf{U}_{\mathcal{R}}}^{*} b = \color{blue}{\mathbf{A}^{\dagger} b}.
$$
The least value for the sums of the squares of residuals is
$$
  r^{2}\left( x_{LS} \right) = \big\lVert
\color{red} {\mathbf{U}_{\mathcal{N}}}^{*} b
\big\rVert_{2}^{2}
$$
The derivation deliberately avoided the requirement that $\mathbf{A}$ be of full column rank. When the null space $\color{red}{\mathcal{N}\left( \mathbf{A}^{*} \right)}$ is nontrivial, the least squares solution includes a projection onto that null space. The least squares solution set is
$$
  x_{LS} =
\color{blue}{\mathbf{A}^{\dagger}b} +
\color{red} {\left( \mathbf{I}_{n} - \mathbf{A}^{\dagger} \mathbf{A} \right)y},
\quad y\in\mathbb{C}^{n}
$$
Refer back to the blue vector and red dashed line in the previous figure.
What are the lengths of the least squares solution vectors?
$$
\lVert
  x_{LS}
\rVert_{2}^{2}
=
\lVert
\color{blue}{\mathbf{A}^{\dagger}b}
\rVert_{2}^{2}
+
\lVert
\color{red} {\left( \mathbf{I}_{n} - \mathbf{A}^{\dagger} \mathbf{A} \right)y}\rVert_{2}^{2}
$$
What the least squares solution of minimum length?
It is the vector which is entirely in $\color{blue}{\mathcal{N}\left( \mathbf{A}^{*} \right)}$. It is the pseudoinverse solution.
$$
\lVert
  \color{blue}{x_{LS}}
\rVert_{2}^{2}
=
\lVert
\color{blue}{\mathbf{A}^{\dagger}b}
\rVert_{2}^{2}
$$
Comment: Why is it possible to separate controlled and uncontrolled parts in the 2-norm?
Comment: @errorist: Excuse the sloppy vernacular. The theorem of Pythagorus is used to separate range spaces (blue) from null spaces (red). The range space portion contains the $x$ term which we vary in order to find the best result.  We can adjust $x$ to eliminate the range space contribution to the error. Colloquially, we say "we can control the range space contribution" because of the freedom to choose $x$. There are no such variable parameters in the null space term. The residual error term, $r^{2}(x)$ is a function of $x$ which highlights the role of $x$ in reducing the total error.

Singular value decomposition proof


Show PDF/print icons inside a template file

Question: I am using a template file to render a content type. So I print variables as with the following code.
print $node->body

I have installed the Print module, and I have selected the option to display print icons for this content type to, but the icons are not getting printed.
How can the icons be printed inside a template file?
Answer: As the README.txt says

Administration > Structure > Content types (admin/structure/types)
  For each content type it is possible to enable or disable the
  PF link via the "Show printer-friendly version link"
  checkbox. (default: enabled) It is also possible to
  enable or disable the PF link in individual comments via the
  "Show printer-friendly version link in individual comments"
  checkbox. (default: disabled)

API
print_insert_link(), print_mail_insert_link(), print_pdf_insert_link()

The *_insert_link functions are available to content developers that
  prefer to place the printer-friendly link in a custom location. It is
  advisable to disable the regular Printer-friendly link so that it is
  not shown in both locations.

Take a look at THEMEABLE FUNCTIONS
It is a best Practice to read modules README.txt.