Can I have Composer or something else download a Git repo as a dependency of my contrib module?

Question: I'm working on a module and there is a sub module for scanning barcodes that requires this, quagga.
Currently you have to manually download the library and put it into the libraries dir. If I add this as a dependency to the module you cannot install it without doing this first.
Can I have composer or something download a git repo to the libraries when installing the submodule?
Edit:
libraries.yml
form:
  version: VERSION
  css:
    layout:
      css/commerce_pos_barcode_scanning.css: {}
  js:
    js/commerce_pos_barcode_scanning.js: {}
  dependencies:
    - core/jquery
    - core/drupal
    - commerce_pos/quagga
quagga:
  remote: https://github.com/serratus/quaggaJS.git
  version: 1.0.0
  license:
    name: MIT
    url: https://github.com/serratus/quaggaJS/blob/master/LICENSE
    gpl-compatible: yes
  js:
    /libraries/webrtc/adapter-latest.js: {}
    /libraries/quagga/quagga.min.js: {}

install.yml
function commerce_pos_barcode_scanning_requirements($phase) {
  $requirements = [];
  if ($phase == 'install' || $phase == 'runtime') {
    $path = 'libraries/quagga/quagga.min.js';
    if (!file_exists($path)) {
      $requirements['commerce_pos_barcode_scanning'] = [
        'title' => t('POS Barcode Scanning'),
        'description' => t('To scan barcodes, you need to install the quagga JavaScript library. How To'),
        'severity' => REQUIREMENT_WARNING,
      ];
    }
    else {
      $requirements['commerce_pos_barcode_scanning'] = [
        'title' => t('POS Barcode Scanning'),
        'severity' => REQUIREMENT_OK,
        'value' => t('quagga JavaScript library has been configured.'),
      ];
    }
  }

  return $requirements;
}

Comment: It's a JS lib hosted on GitHub OK. So you can either go for a [*.libraries.yml](https://www.drupal.org/node/2947530) pulling that in automatically or use [hook_requirements](https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Extension%21module.api.php/function/hook_requirements/8.2.x) to declare this dep explicitly though this will not prevent your library from being installable if the requirement isn't met. It just creates a warning or error message on the status page.
Comment: But maybe you can it get it to work somehow with a [composer.json](https://www.drupal.org/docs/8/creating-custom-modules/add-a-composerjson-file) in your module. Though I'm not sure about how this works for (JS) libs not packaged for Composer usage.
Comment: @leymannx Thank you for the response. The submodule has a libraries and install already. These do not download the library, just warn that we need it. I'll add them to the Q.
Comment: This is outside the scope of composer, see https://getcomposer.org/doc/faqs/why-can%27t-composer-load-repositories-recursively.md
Comment: Just as a side note: Quagga is available on [asset-packagist.org](https://asset-packagist.org/package/search?query=quagga&platform=bower%2Cnpm) (but this still is a foreign repo that has to be added to the root project)
Answer: You can edit file composer.js
find "web/libraries/{$name}": ["type:drupal-library"], change to
"web/libraries/{$name}": [
                "type:drupal-library",
                "type:bower-asset",
                "type:npm-asset",
                "vendor:npm-asset",
                "vendor:bower-asset",

            ],

add repositories
{
            "type": "package",
            "package": {
                "name": "serratus/quaggaJS",
                "version": "0.12.1",
                "type": "drupal-library",
                "source": {
                    "url": "https://github.com/serratus/quaggaJS.git",
                    "type": "git",
                    "reference": "master"
                },
                "dist": {
                    "url": "https://github.com/serratus/quaggaJS/zipball/master/serratus-quaggaJS-v0.12.1-14-g862df88.zip",
                    "type": "zip"
                },
                "require": {
                    "composer/installers": "^1.0.20"
                }
            }
        }

and run composer require "serratus/quaggaJS"
Comment: OP is asking how to manage dependencies for a (contrib) module.
Comment: ley is right here I believe. I understand how composer works for a project, but not for a module honestly.
Comment: While not the answer to the question, this IMHO is the way better solution than leymannx's anwer. Manually create the composer entry. Don't mix composer-managed and self-downloaded shared libs. What happens when another module wants the same lib with conflicting version requirements?
Comment: @Hudri – This version argument doesn't count when you define a proper library with a proper version. The question is just how this library could be downloaded automatically.
Comment: I was hopefully going to make it so that this library would auto install when enabling this sub-module, correct. This would go in the project's composer rather than the modules correct? Not what I want sadly
Answer: To have this be done automatically people will simply configure their project's composer.json themselves. So I'd recommend to just explain one more time how this is supposed to be done via Composer on your project's page on drupal.org or in the documentation to your module.

https://stackoverflow.com/q/43121292/2199525
How to use Composer to install third party libraries
How to require a JS library in a composer.json file

Another inadvisable approach would be to provided a Drush command to download the library. Like Semantic UI API does it. Browse their code on GitLab. The command has not been ported to D8 yet. But I think you'll get the idea. Probably worth looking at their 8.x-1.x branch as well. And see how they solved it in D8.
Below code is from 7.x-1.x. SEMANTIC_UI_DOWNLOAD_URL leads to a Zip file on GitHub.

  $args = func_get_args();
  if ($args[0]) {
    $path = $args[0];
  }
  else {
    //we have dependencies on libraries module so no need to check for that
    //TODO: any way to get path for libraries directory?
    //Just in case if it is site specific? e.g. sites/domain.com/libraries ?
    $path = drush_get_context('DRUSH_DRUPAL_ROOT') . '/sites/all/libraries/semantic_ui';
  }
  // Create the path if it does not exist yet. Added substr check for preventing
  // any wrong attempts or hacks !
  if (substr($path, -11) == 'semantic_ui' && !is_dir($path)) {
    drush_mkdir($path);
  }
  if (is_dir($path . '/dist')) {
    drush_log(dt('Semantic Ui already present at @path. No download required.', array('@path' => $path)), 'ok');
  }
  elseif (drush_op('chdir', $path) &&
      drush_shell_exec('wget ' . SEMANTIC_UI_DOWNLOAD_URL . '  -O semantic_ui.zip') &&
      drush_shell_exec('unzip semantic_ui.zip') &&
      drush_shell_exec('rm -rf semantic_ui.zip') &&
      drush_shell_exec('mv Semantic-UI-master/dist/* .') &&
      drush_shell_exec('rm -rf Semantic-UI-master')) {
    drush_log(dt('The latest Semantic Ui library has been downloaded to @path', array('@path' => $path)), 'success');
  }
  else {
    drush_log(dt('Drush was unable to download the Semantic Ui library to @path', array('@path' => $path)), 'error');
  }

Comment: IMHO a bad idea. What happens when another module wants the same lib with conflicting version requirements? This approach kind of contradicts using composer at all.
Comment: I don't know any solution, and [composer doesn't even want to this](https://getcomposer.org/doc/faqs/why-can%27t-composer-load-repositories-recursively.md)
Comment: Yes, this is a solution and it does answer the OP's question. I just think it is a problematic advice to bypass a composer limitation this way, that's why I've given a -1. You don't need to download it manually, but you have to manually add the foreign repo to the root project.
Comment: @Hudri – Of course you also register it as a proper library. Do you know any other way to auto-download a third-party lib not packaged for Composer from a contrib module?
Comment: Or don't put the lib in a shared, composer-managed folder, then this is fine too.
Comment: @Hudri – Got it. OP should simply explain in the docs how this is supposed to be done via the project's `composer.json`. Totally agree. Updated answer.


Understanding a Java syntax error

Question: I am rather new to java and I do not understand an error I am getting. Eclipse is not recommending to me how to fix it.
I am getting the following error with my below code:
"Syntax error on token "getChannel", Identifier expected after this token"
import java.io.FileOutputStream;
import java.net.URL;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;

public class main {
    URL website = new URL("http://ts.vtggames.net/extras.zip");
    ReadableByteChannel rbc = Channels.newChannel(website.openStream());
    FileOutputStream fos = new FileOutputStream("extras.zip");
    fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
}

Comment: You have to put statements in a block of some sort, like a method.
Comment: See: http://stackoverflow.com/questions/18795467/system-out-println-statement-outside-any-method-in-java, http://stackoverflow.com/questions/12062481/why-cant-i-do-assignment-outside-a-method
Answer: This is a statement:
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);

It has to go in a block: basically, it has to be in some syntactic construct inside the class wrapped in {}.
The other lines are variable declarations. These can go outside blocks; this is why the first syntax error is on the fos.getChannel() line.
You can put all of the lines in a main method, like this:
public class main {
  public static void main(String[] args) {  // <-- Add this line
    URL website = new URL("http://ts.vtggames.net/extras.zip");
    ReadableByteChannel rbc = Channels.newChannel(website.openStream());
    FileOutputStream fos = new FileOutputStream("extras.zip");
    fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
  }  // <-- and this line
}

Answer: As @AndyTurner mentioned, this isn't proper Java code

Classes in Java should begin with an uppercase letter
You should put your main code in a method
A public static void main(String[] args) should be provided to run the class standalone
Comment: "Classes in Java need to begin with an uppercase letter" No they don't. This is mere convention.
Comment: "All executable code needs to be in a method" No, it can be in an initializer block, which is not a method.
Comment: @AndyTurner I realized that as soon I posted it :).
Comment: "A public static void main(String[] args) method must be provided as a runner" Not to compile the class.
Comment: @AndyTurner you're right, should've phrased it better

Understanding a Java syntax error


Apache security

Question: I've been reading up on server/apache security and I've run into a contradiction. I read that you shouldn't serve any files that aren't inside Apache's scope, but I've been told that you should keep files outside Apache's scope.
My current setup is:
Apache's scope: public_html/front. In the front folder I keep html, css, and javascript files. I also have a folder public_html/back, outside Apache's scope, which houses my PHP, database, and website backend files. All website directories and files have an owner/group of www-data. I'm running this all on Free BSD.
Should I have website files outside of Apache? Any other server security tips are appreciated.
Answer: If you want the files served by Apache, they should be accessible by Apache.
However, you should not let Apache have write access to any files which it is not required.
If you have an upload directory, you can make Apache able to write in that directory, but it must not be able to write in your PHP files, css and such.
Comment: How do you define Apache's scope?   If a script is vulnerable to some attack, it can possibly read a lot of files on the system. And if the files are writable by www-data it will be possible write to the files via that.
Comment: My frontend directory contains only functions to render graphics/html and the index page.  No credentials or any other security related information is stored there.

My backend directory contains db connection information, scripts which process requests which will be sent to the frontend for rendering.

A basic requests goes like this:
index.php contains a phpController method which calls the appropriate backend class.
The backend method makes a db connection, gathers the appropriate information, stores it into an array, then calls the frontend method to render the html, css, and js.
Comment: I've been told that it's ok to store files outside of apache's scope, but another source said that you should not store anything outside of apache.  My question is which method is more secure?
Comment: Everything works fine with the current setup, I'm not sure why there might be a security risk if my php security files are outside Apache's scope.
Comment: Inside of Apache's config file, where you set DocumentRoot, I have it set to public_html/front, which contains html, css, and js files.  But I keep all the important php, db and everything inside public_html/back.
Comment: And how to web users access dynamically generated web pages (PHP) then?
Comment: I have a page which sends a request to the backend files, then after the data is gathered, I have it call a frontend file which renders the html.
Comment: How does it send the request to the backend files? Please, stop being so secretive about how you did it and give so little information. If you want help you can use for anything, you need to lay out your cards.. Describe in the question exactly how your setup works.
Answer: Your setup is perfectly acceptable and I'd say it's even recommended.
Your DocumentRoot should ideally point only at files that need to be served to end users.  Scripts and other assets that are used by your client-facing scripts can sit in a different directory (e.g. public_html/back as you have it).  For example, your configuration files, database connectivity classes etc would sit in the non-public folders.
If you put everything under DocumentRoot, then you would have to secure those files so they don't get served to end users (for example, by blocking their download using .htaccess or using PHP to kill the request when those files are loaded directly).
One note I would have is that the name "public_html" suggests public facing files.  In our company we have the DocumentRoot point to public_html (or equivalent) and the "back" files under a totally different directory as to avoid confusion.

Apache security


how can i add to cart a downloadable product with two different downloadable packages from product listing page

Question: i have set two different packages to be downloaded for a single product (buy developer and buy standard).
it can be added to cart page from detail page, but wat i want is to add it to cart from listing page too.
Mean when add to cart on product listing page is clicked, it should by default add to cart the standard package of a product.
Rite now when i click on add to cart from listing page, it takes me to detail page with a msg to select one of the package.
Any Help ????
Answer: If you already have a controller and you are able to send your onclick to that controller and are able to send the product id to it simply do:
<code>
$prod_id = $this->getRequest()->getParam('product_id');
$product = Mage::getModel('catalog/product')->load($prod_id);
$param = array( 'product' => $product->getId(),
                                                'options' => '',
                                                'qty'     => 1
                                              );
$cart = Mage::getModel('checkout/cart')->init();
$cart->addProduct($product, new Varien_Object($param));
Mage::getSingleton('checkout/session')->setCartWasUpdated(true);
$cart->save();
</code>
Answer: If however you want to simply change the reference on the add button in the template you can check template: /var/www/.../app/design/frontend/default/default/template/catalog/product/list.phtml


Piano fingering: B flat dominant 7th arpeggio

Question: I am having trouble playing this arpeggio:
B♭ - D - F - A♭ - B♭ - D...
For the right hand I have tried this fingering:
B♭ - D - F - A♭ - B♭ - D
2  - 1 - 2 - 3  - 4  - 1
But on the way down, I am extremely slow switching between 1 (D) and 4 (B♭). It's actually easier for me to switch on two white keys, like this:
B♭ - A♭ - F - D - B♭ - A♭
3  - 2  - 1 - 4 - 3  - 2
But that doesn't seem right. What is the generally recommended way to play this?
Comment: Using your fingering, how are you playing Bb on finger 3 going down?
Answer: Technically, there is no 'correct' fingering for any arpeggio - there is fingering which is generally accepted as the easiest or most comfortable way to play an arpeggio.
However, I would advise against starting on 4, and would suggest starting on 2 as you would in, say, an Eb major scale. Then, I recommend going to 1, which would allow you to:
B♭ - D - F - A♭ - B♭ (- D - F - Ab - Bb)
2  - 1 - 2 - 3  - 4  (- 1 - 2 - 3  - 4 )
As I said, there is no single 'correct' way of doing it, but for me that is the easiest. It's up to you as to how you do it - what ever you find most comfortable. There is no point in contorting your hand to play an arpeggio in a way that one pianist likes, if you know an easier one.
Comment: You're right, when starting on B♭ and going up with the right hand it would make more sense to start on 2. I have fixed this in the question
Answer: This is one of the more difficult arpeggios for most people, because of getting the thumb under the fourth finger for a major third.
Try playing F and D together with 2-1.  Note your hand position.  Now, while holding down D with the thumb, play Bb with 4.  You should notice that your wrist has to move pretty far laterally.
Now, see how far you can turn your wrist to the left (I mean turn laterally with the palm roughly parallel to the keys, not rotating the wrist--don't do that) on the first two notes.  Then play the Bb again with the thumb still on the D.  This will show you how to minimize the lateral movement of the wrist, which represents an economization of movement.
Another way to think of this is to see how closely you can position your wrist, while playing the F and D, in the position needed to reach over the thumb to play Bb.  The closer to that position you can be when playing the F, the less you have to move your wrist when you get to the Bb.
Answer: Forget the first note for a moment. The D,F,Ab and Bb fall nicely under thumb, index, middle and ring fingers.You then have to move the right hand along, left to right, parallel to the way the music sheet is on the piano.At the same time, the thumb will be moving under the hand, in order to play the next D, and so on.Now, the first Bb. It really doesn't matter whether it's played with the ring, as it will be as you ascend, or the index or middle. I prefer middle to start, as it gives a little more space underneath for the thumb to hit the D.
On the way down. you start with top Bb using ring, and use up all the digits, and do the same lateral movement, right to left, though, as you did before. Again, the lowest, last Bb can be played with whatever you like.If you were going down another octave, it'd need to be played using ring, but as the last note, it doesn't make any difference. It's a scale - not as if you're going anywhere after it.
Your third idea is not so great. Generally scales come down backwards from the way they're played going up.Try not to flick your elbow up, or raise your wrist too much, more of a smooth transition of thumb under, whilst moving your hand/wrist/forearm steadily from side to side, going either up or down.


Higienizar endereço de email

Question: Assumindo o seguinte endereço de email:
$email = "zuúl@ so.pt";

Tentei proceder à higienização do mesmo fazendo uso de:
filter_var($email, FILTER_SANITIZE_EMAIL); // firstname.lastname@example.com

Tentei também:
preg_replace('/[[:punct:]]/', '', $email); // zuúl sopt

A ideia é que pelo fato do utilizador digitar um acento ou um espaço em branco por engano não seja forçado a voltar "atrás" para retificar o endereço pois este tipo de cenários pode ser controlado pela aplicação tirando o trabalho ao utilizador.
Pergunta
Como higienizar o endereço de email zuúl@ so.pt para que fique email@example.com ?
Answer: Para higienizar endereços de email você pode utilizar as funções preg_replace e iconv:
$email = preg_replace('/[^a-z0-9+_.@-]/i', '', iconv('UTF-8', 'ASCII//TRANSLIT//IGNORE', $email));

É importante que você assegure que os caracteres de entrada estejam na codificação esperada, seja em tempo de execução com setlocale ou através de variáveis de ambiente (ex.: Apache):
Comment: Parece-me que ele também esta a falar de caracteres especiais, e não apenas de espaços
Comment: @CesarMiguel falha de atenção minha... já corrigi a resposta :)
Comment: Uma entrada de `#zuúl@ so.pt` resulta em: `firstname.lastname@example.com` utilizando `setlocale(LC_ALL,'pt_PT.utf8')`. Impecável.
Answer: Com recurso à função iconv (Inglês) que nos permite fazer a conversão de uma string para um charset especifico:
iconv('UTF-8', 'ASCII//TRANSLIT', "zuúl@ so.pt"); // zu?l@ so.pt

Agora, é preciso indicar qual o locale (Inglês) onde a lingua deve ser recolhida para que os caracteres acentuados não resultem em ?:
setlocale(LC_ALL,'pt_PT.utf8');
iconv('UTF-8', 'ASCII//TRANSLIT', "zuúl@ so.pt"); // zuul@ so.pt

Por último, aplica-se isto à primeira resposta do @Willy Stadnick para remover também os espaços:
setlocale(LC_ALL,'pt_PT.utf8');
preg_replace('/\s/', '', iconv('UTF-8', 'ASCII//TRANSLIT', "zuúl@ so.pt"));  // firstname.lastname@example.com

Nota: A solução nesta resposta lida apenas com espaços e acentuação. Caracteres especiais não são removidos.

Higienizar endereço de email


Android Studio not showing source set when adding new resource file

Question: When I try to add a new resource file (ie. File>New>Android Resource File) "Source set: " is missing. Does anyone have any ideas why that could be?
Comment: Can you please explain your problem more? what type of resource file you are adding? and where are you trying to add it, under which directory? what is the error message exactly? can you share some screenshot if possible?
Comment: Unfortunately, I don't have enough rep to include a screen shot. It is the same for any resource file for any directory. I added some instructions to my question. I hope that makes it clearer.
Answer: I also had the same problem. I couldn't solve it. But you can still manually add a source set for different flavors.
In ur project view, right click on app >> New >> Folder >> Res Folder.
Then choose a specific flavor in "Target Source Set".
You can see the details here in the link https://developer.android.com/studio/build/build-variants.html.
Android Studio will create a source set for you for that flavor.
Try to add whatever resource file you want manually there.
Cheers!
Answer: I think it is by design -
When you add a new resource file by right clicking on the res folder, you cannot select a source set; but if you add a new resource folder first, you will be able to select a source set.
After that, just right click on the new source set and add new resource files.
Answer: Looks like a bug. File >  New > XML > Values (or Layout) Resource File does create a new resource file in the current build variant.
Answer: if you want Source set to appear you have to add the new resource file in res folder not any other one like drawable or layout
Answer: You can creat it from

new >> folder >> Resfolder

Then change the build flavor from the buildvariance tap at the left and add the files you want by new at the flavor folder.
Answer: As Moe Han answered you could refer to https://developer.android.com/studio/build/build-variants.html.
One thing you need to concern is that you need to choose one of the Build Variants before creating a source set directory you want to make. If not, Android studio couldn't recognize that source directory I think(no icon changes).


Aggregate raster in R with NA values

Question: I have a 1km resolution raster in R with widespread NA values throughout, but at irregular locations (i.e. the cells with data are not contiguous and have NA values scattered throughout). I am trying to aggregate this raster (using aggregate() command in the {raster} package) at, say, 5km resolution (factor=5) with a user-defined function for averaging circular angles (included below). As of now, I can't figure how to get aggregate() (or my function, if that's the problem) to provide a result value unless the raster has a contiguous 5x5 cell area. In other words, where the aggregate window hits a 5x5 cell area with only, say, 5 cells with data values (20 NA cells), I still want it to return an average value for those 2 cells. I've tried modifying the na.action options of both the function and the aggregate() command, with no luck. I'm not super experienced with functions, so the problem may be in there.
Sorry for no working example, but not sure how to generate a similar example raster layer within R.
Here's my circular average function:
library(circular)
avg.ang <- function(x,...){
  mean.circular(circular(x, units="degrees", rotation="clock", zero=pi/2, modulo="2pi"))
}

And here's the aggregate code I'm using (where 'angle' is a 1km raster with NA values scattered throughout):
library(raster)
angle5k <- aggregate(angle, fact=5, fun=avg.ang, expand=T)

But this returns a raster layer with a value ONLY at aggregated locations where every cell of the 5x5 window contains a value.
Comment: You need (at least) to set `na.rm=TRUE` in your call to `mean.circular()`.
Comment: Yep, that did the trick, with one more small if/else modification to deal with an error that occurs when all cells=NA. Will post the modified function below. Thanks!
Answer: Thanks Josh for the guidance. Here's the modified function that produces what I'm looking for:
avg.ang <- function(x, ...){
  if (sum(is.na(x))==length(x)) {
      NA
  } else {
      round(mean.circular(circular(x, units="degrees", rotation="clock",
                                   zero=pi/2, modulo="2pi"), na.rm=TRUE))
  }
}

The na.rm=TRUE is the key. The if/else statement is to deal with occurrences where all cells=NA (otherwise breaks with an error). If anyone has a more elegant way to deal with the if/else, I'm all ears.
Comment: Great. In that second line, you could alternatively test that condition with `all(is.na(x))`
Comment: this is nice. what kind of variable is to be manipulated with this function?
Comment: @Paulo I have circular angle measurements (0-360°, hence units option) on compass bearings (hence zero, rotation, and modulo options) that I need to average over wider areas that just the 1km resolution of the raster. The application is for climate change migration estimates.

Aggregate raster in R with NA values


Try for multiple statements in Python

Question: I need some help/advice on the best approach to do this. I have a block of code in Python, which is basically more or less the same line (a function/method applied to different variables). Now, sometimes, depending on other things, some of those variables may not exist, causing the lines that have any of them to throw an exception.
My goal is to run the block of code allowing the lines with the existing variables to run and the others to be skipped. I have a million workarounds for this, but i wanna see if there is a "clean" way, like, using a try/exception for the "entire" block.
My idea is to do something like this, where a1, a2, a3, etc., are the variables that could or could not exist:
try:
    functionx(a1)
    functionx(a2)
    functionx(a3)
    ...
except:

And somehow, to get that try/except wrap to skip the ones that don't exist, but execute the ones that exist.
Any idea?
Edit: The point is to see if, somehow, i can make the try to "continue" after it gets an exception inside its code block instead of breaking and going to the except.
Comment: What do you mean "variables may not exist"? Are they `None`?
Comment: Not defined. Doesn't really matter, assume the line will throw an exception. My point is to know if i can, somehow, make the try call to "continue" with the rest of the block when it throws an exception instead of breaking.
Answer: Based on this answer, you can call your function only after checking the variable exists like so:
functionx(a1) if 'a1' in locals() else None
functionx(a2) if 'a2' in locals() else None
functionx(a3) if 'a3' in locals() else None

Comment: Well, gave it a couple thoughts and i'll just stick to this solution, thanks!
Comment: Yep, that's one of the workarounds.. i've also just initialized the variable empty, but i wanna see if there is a cleaner way to do it (like with a single try call).
Answer: Assuming you meant some vars might be None: only run your function on vars that aren't None:
for non_null_var in [v for v in [a1, a2, a3, ...] if v is not None]:
  functionx(non_null_var)

map & filter version:
map(functionx, filter(lambda x: x is not None, [a1, a2, a3, ...]))

If you really mean they aren't defined at all, then, modifying Martin's answer:
for non_null_var in [v for v in ['a1', 'a2', 'a3', '...'] if v is in locals() and locals()[v] is not None]:
      functionx(locals()[v])

Comment: Yes, that's another workaround i've used, but please read the description again (i added some edit to point on that). I'm looking for a solution following the layout i wrote (a way to make the try-catch to "continue" after it finds an exception instead of just breaking to tho except)
Answer: Maybe something like *args or **kwargs will do the trick.
def apply_on(*args):
    for arg in args:
    try:
        functionx(arg)
    except:
        pass

You could then call like:
apply_on(1, 2, 3, 4)

Using kwargs if the name of the "variable" is somehow important:
def apply_on(**kargs):
    for name, arg in args.items():
        try:
            functionx(arg)
        except:
            pass

Usually it is not clean to assume that variables exist or not. Pick some other data structure in this case, like list or dictionary, depending on your use case.

Try for multiple statements in Python


Generating elements of a Galois Field using an irreducible polynomial

Question: I am practicing some cryptography problems and I am having problems with one involving Galois Fields and irreducible polynomials. Here is the problem:

Using the irreducible polynomial $f(x) = x^5 + x^4 + x^3 + x^2 + 1$ to
a) generate the elements of the field $GF(2^5)$
b) based on the results of a, calculate the following in $GF(2^5)$:
1b. $(x^4 - x + 1)^{-1}$
2b. $(x^3 - x + 1) * (x^4 + x^2 + 1)$
3b. $(x^4 - x^3 + 1) / (x^2 + x + 1)$

I am not exactly sure how to even start this problem. If anyone can please help me, I would appreciate it very much! This is what I have:
$x^5 = x^4 + x^3 + x^2 + 1 \Rightarrow {F}={1111}$
$0 = 0 0 0 0 0$
$1 = 0 0 0 0 1$
$g = 0 0 0 1 0$
$g^2 = 0 0 1 0 0$
$g^3 = 0 1 0 0 0$
$g^4 = 1 0 0 0 0$
$g^5 = g^4 + g^3 + g^2 + 1 \rightarrow 1 1 1 0 1 $
$g^6 = 0 0 1 1 1$
$g^7 = 0 0 0 1 0$
$g^8 = 1 0 0 0 0$
...
I'm not exactly sure if I am even doing this correctly. Please help! Thanks!
Answer: It seems like you want your five-bit string $b_4b_3b_2b_1b_0$ to correspond to the (equivalence class of the) polynomial $b_4x^4+b_3x^3+b_2x^2+b_1x+b_0$. Therefore $0$ is indeed 00000, but $1$ is 00001 and $g$ is 00010. You have $g^2$ through $g^4$ right, but $g^5$ is 11101, not 1111.
Comment: The field has 32 elements, so if you want to list each power of $g$ as a polynomial, you'll have to go way beyond $g^7$. But there are ways to answer part b without listing all the powers of $g$.
Comment: Ok, so I guess I was sort of on the right track. Another question of mine is, up until what number do i go up to? g^7?


Two domains accessing same folder

Question: I've just taken a new role in a school and am still familiarizing myself with their network, how ever I have recently been given a task and I'm having a little trouble finding out the fundementals of it.
I have an admin network/domain 10.49.x.x and a classroom network/domian 192.168.1.x both connect to a Proxy server 10.49.202.231/192.168.1.51.
Each domain has it's own shared folders as you'd expect, files and software installs etc,  how ever there is a folder "staff" on the classroom network that all the teachers on the classroom network can access. The users on the admin network would like to access this same folder.
How do I go about making this happen?
Answer: can you apply "trust" on these active directory servers?
trust relationship between two AD.DS can solve your problem
Comment: @liamQuinn can you apply it?

Two domains accessing same folder


Proof by Contradiction?

Question: How does one construct the proof by contradiction? I know Direct Proof and Proof by Contrapositive really well but just can't understand proof by contradiction.
Comment: Welcome to MathSE. The question unfortunately needs a *lot* more context before anyone can attempt to answer it.
Answer: Statement: If $p$ is true, then $q$ is true.
Direct Proof: Assume $p$ is true, try to proof $q$ is true.
Contrapositive: Assume $q$ is false, try to proof $p$ is false.
Contradiction: Assume $p$ is true and $q$ is false, try to proof the assumption will lead to result makes no sense.
Answer: Brief summary.  Assume provisionally that the statement you want to prove true is actually false.  Work out some consequences until you arrive at something you know is false.  As long as your deductions are all accurate, the only way you could have obtained a false conclusion is if the provisional assumption you made was incorrect.  That is, the statement you want is not false, it is true.

Possible format (not the only way of setting out this kind of proof) -
Theorem.  The statement $S$ is true.
Proof.  Assume that $S$ is false.  Then
...$\langle$various deductions$\rangle$...
and therefore $\langle$some conclusion$\rangle$.
But this is false.  $\langle$For example, you might have proved that $0=1$.$\rangle$
Therefore the statement $S$ is true.

Example.  We observe by calculation that to $8$ significant figures,
$$8+31\sqrt{15}\quad\hbox{and}\quad20\sqrt{41}$$
are both equal to $128.06248$.  Are these numbers really equal?
Answer.  No, they are not equal.
Proof.  Suppose the contrary, that is,
$$8+31\sqrt{15}=20\sqrt{41}\ .$$
Squaring both sides,
$$14479+496\sqrt{15}=16400\ ,$$
so
$$496\sqrt{15}=1921\ .$$
Squaring both sides again,
$$3690240=3690241\ .$$
But this is false.  Hence our original supposition was false; that is,
$$8+31\sqrt{15}\ne20\sqrt{41}\ .$$

What next?  There are many examples of proof by contradiction online, please read a few and then try to do some yourself.  Make sure that at some stage you read the two classics of proof by contradiction: $\sqrt2$ is irrational, and there are infinitely many primes.
Good luck!

Proof by Contradiction?


Pre-registration of data that is already collected

Question: I collected survey data in late 2018 through mid 2019. With the pandemic and other projects, I haven't had time to do anything but enter the data. I have not yet analyzed the data, or even looked at basic descriptives or correlations. I do have hypotheses and theory that were generated in 2018.
Is it too late to pre-register the study? When I collected the data, pre-registration was not a mainstream practice. Regrettably, I didn't pre-register then.
Comment: What would be the purpose of pre-registering in your situation?
Answer: Pre-registration of the data analysis depends on the level of contact you have had with the data. Given you haven't analyzed the data, this is still essentially a primary data analysis. However, the collection of data before pre-registration and the length in time between collection/analysis makes it difficult for this project to be pre-registered in that way. This is because you are supposed to pre-register how you will sample, stopping rules, etc. in advance. Obviously that is not the case here since the data was already collected.
If you are really worried about scrutiny about the timeline of your project, the most obvious solution to me is to pre-register this as a secondary data analysis instead, where the pre-registration templates on places like OSF will ask what your level of contact is with the data anyway after the data has been collected.
Comment: As far as I am aware, the only more flexible pre-registration that accounts for this is the [open-ended registration](https://docs.google.com/document/d/1_hdYTyOj90WOJq1dI7UTpVmqwCGHenFc5DBjmAdpXGg/edit#heading=h.fwbi14d4b65g), which is essentially just a semi-blank doc you fill in yourself. Perhaps OP can combine elements of the SDA format (supplying dates of data collection, etc.) and filling other vital information as needed. My only concern is that the primary data versions explicitly ask you to register things in advance that OP has already done.
Comment: hmm, I'm not sure if we could call that a secondary data analysis, as it's the original study for which the data has been collected in the first place. I'd tend to call that a retrospective registration, and I'm not sure if OSF accepts this kind of submission. But in any case, the obvious problem is how to avoid to involuntarily mislead readers about the timeline, so no matter the solution (registration as reprospective or secondary data analysis), the situation should be made explicit and abundantly clear.


How to shift labels in Python using Apache Spark

Question: I'm trying to shift labels of a LabeledPoint so that they start from 0. Right now parsedDataInit.take(1) looks like this before the code at the bottom:
[LabeledPoint(2001.0,[0.884123733793,0.610454259079,0.600498416968,0.474669212493,0.247232680947,0.357306088914,0.344136412234,0.339641227335,0.600858840135,0.425704689024,0.60491501652,0.419193351817])]

The labels are the years, for example, 2001.0 is the max year and 1922.0 is the min year. The difference is 89.0 and what I am trying to do is shift the labels so that they start from 0, making the new minYear 0 and the new maxYear 89 while still maintaining the same features. I'm new to apache spark and would appreciate any help. Thanks in advance! Here is my code:
parsedData = parsedDataInit.map(lambda lp: lp.label) # This is where I'm having trouble
print type(parsedData.take(1)[0])
print '\n{0}'.format(parsedData.take(1))

Comment: "_Questions asking for homework help must include a summary of the work you've done so far to solve the problem, and a description of the difficulty you are having solving it._"  [What topics can I ask about here?](http://stackoverflow.com/help/on-topic)
Comment: This is from the Edx Scalable machine learning MOOC Lab homework!! There is piazza for asking questions about your homework.
Comment: @eliasah, thanks for your comment. I just today stuck on another apache spark course. And after reading your comment  I understood now, that this course (I can't named it here) just copy paste from edx)))
Answer: You just subtract the minimum from each year.
def normalize(data):
  labels = data.map(lambda lp: lp.label)
  minLabel = labels.min()
  return data.map(
    lambda lp: LabeledPoint(lp.label - minLabel, lp.features))

Answer: based on @Daniel response a one line solution to the problem (without creating any new functions):
parsedData = parsedDataInit.map(lambda lp:LabeledPoint(lp.label - minYear, lp.features))


Como chamar método Linq na View?

Question: Bom dia pessoal.
a minha dúvida é a seguinte, tenho o seguinte método no meu model:
public IQueryable GetConsulta()
{
    var q = (from c in Datacontext.Tabela1
                join b in Datacontext.Tabela2 on c.CodClase equals b.CodClase
                join d in Datacontext.Tabela3 on b.CodImobilizado equals d.CodImobilizado
                where c.CodClase == b.CodClase && b.CodImobilizado == d.CodImobilizado
                select new
                {
                    taxa = c.Taxa,
                    ano = DateTime.Parse(b.Data).Year,
                    data = d.Data,
                    valorAquisicao = b.Valor,

                });

    return q;
}

Na View está assim:
void getConsulta(){
    var a = model.GetConsulta();
    foreach(var linha in a){
      //faça algo
    }
}

O problema é que não consigo acessar as variáveis criadas no método do model no foreach da View.
Alguém poderia me ajudar.
Comment: Você está retornando isso para a View?
Comment: sim Marconi. eu preciso acessar o resultado do select feito no model, dentro do ciclo foreach que aparece no método getconsulta que se encontra na viewer
Comment: E a resposta abaixo, acho que te ajuda. :)
Answer: O problema está no tipo de retorno que você está utilizando, nesta abordagem, você não especifica qual o retorno do IQueryable, consequentemente na view, você não vai conseguir visualizar os dados desejados.
Uma sugestão seria você tipar seu retorno com uma classe de modelo:
class Modelo {
    public string taxa { get; set; }
    public int ano { get; set; }
    public DateTime data { get; set; }
    public decimal valorAquisicao { get; set; }
}

public IQueryable<Modelo> GetConsulta()
{
    var q = (from c in Datacontext.Tabela1
                join b in Datacontext.Tabela2 on c.CodClase equals b.CodClase
                join d in Datacontext.Tabela3 on b.CodImobilizado equals d.CodImobilizado
                where c.CodClase == b.CodClase && b.CodImobilizado == d.CodImobilizado
                select new Modelo()
                {
                    taxa = c.Taxa,
                    ano = DateTime.Parse(b.Data).Year,
                    data = d.Data,
                    valorAquisicao = b.Valor,

                });

    return q;
}

Na View ficaria assim:
void getConsulta(){
    var a = model.GetConsulta();
    foreach(Modelo linha in a){
      //faça algo
    }
}

Comment: Pode ser isso mesmo, bem lembrado. :)
Comment: Provavelmente seja esta a situação.Neste caso a minha variavel q teria que ser do tipo modelo certo?
Comment: Sim, vou editar a resposta
Comment: Julio muitissimo obrigado. a sua resposta foi muito util Obrigado Mesmo. Nota: Sou novo neste forum. aonde posso marcar como resposta válida?
Comment: Abaixo do marcador de votos tem um V, basta clicar nele para marcar a resposta como válida.

Como chamar método Linq na View?


Why is $AUC=0.5$ and a 45-degree line for a ROC curve considered baseline performance?

Question: $AUC=0.5$ and an ROC curve of a 45-degree line often are considered the baseline performance of a model, one that gets absolutely nothing from the features.
If we predict the same (prior) probability of class membership every time, how do we wind up with a 45-degree line for the ROC curve? Either we get perfect sensitivity and zero specificity, or we get zero sensitivity and perfect specificity. The 45-degree line implies that a sensitivity of $0.4$ is possible when the specificity is $0.6$.
How does that happen?
And why, with the ROC curve being so funky, do we say that $AUC=0.5$ is the baseline performance? Is this related to how AUC is related to the Wilcoxon test (Harrell's concordance index) and not just the geometry of the ROC curve?
Comment: My understanding of this is that the baseline used for ROC/AUC is a random ranking of the outcomes, rather than equal predictions.
Comment: I assume this is a duplicate, though I couldn't find one. If this is not a duplicate, consider me surprised that Cross Validated took 12 years to get such a question.
Comment: The AUC is the probability that a randomly selected positive is ranked higher than a randomly selected negative. So an AUC of 0.5 is the performance of a classifier that does not rank positives higher than negatives, nor the opposite. https://stats.stackexchange.com/questions/190216/why-is-roc-auc-equivalent-to-the-probability-that-two-randomly-selected-samples and https://stats.stackexchange.com/questions/180638/how-to-derive-the-probabilistic-interpretation-of-the-auc fill in the particulars.
Comment: @Sycorax I didn't read those in detail (yet), but I figured it was something like that for the AUC/c-index. However, how do we justify plotting that diagonal line for the ROC curve, such as `plot(pROC::roc(c(0,1,0,1,0,1), c(1,1,1,1,1,1)/2))` in `R`?
Comment: Given that the probability of a positive is ranked higher than a negative is 0.5, then at each choice of threshold, the true positive rate must equal the false positive rate.
Comment: @Sycorax But when all of the predictions are equal (regardless of class), how do we wind up with a continuum?
Comment: An AUC of 0.5 does not **require** that all predictions are equal regardless of class. AUC measures the probability that a randomly selected positive is ranked higher than a randomly chosen negative, so an AUC of 0.5 means there's a 50% chance that a randomly selected positive is ranked higher than a randomly chosen negative.
Answer: The AUC is the probability that a randomly selected positive is ranked higher than a randomly selected negative. So an AUC of 0.5 is the performance of a classifier that does not rank positives higher than negatives, nor the opposite. Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked? and How to derive the probabilistic interpretation of the AUC? fill in the particulars.
Therefore, given that the probability of a positive is ranked higher than a negative is 0.5, then at each choice of threshold, the true positive rate must equal the false positive rate. In other words, given some classification threshold, the model designates as positives some portion of the samples. Among those positives, half of them are true positives, and half of them are false positives.
Comment: If all of the predictions are equal, then the probability of ranking a positive case above a negative case is zero.
Comment: All predictions being equal does not have any bearing on AUC = 0.5, because AUC can equal 0.5 even if the predictions are not all equal. This is the second time you've made reference to this, but it's still not relevant. As an example: `pROC::roc(c(rep(0, N/2), rep(1, N/2)), runif(N))` will have AUC 0.5 in expectation, and the ROC curve will appear to be a diagonal line (but any particular sample will exhibit some small deviation, even for large `N`). Each prediction is unique (almost surely), yet the AUC is 0.5 (in expectation).
Answer: When it comes to AUC, baseline performance means the performance of a model that has the same distribution of predictions for both classes.
From this, the diagonal line and $AUC=0.5$ follow naturally from the usual definitions.
If you want to make an argument related to a model that outputs the same value every time (something related to my thoughts on the baseline performance in $R^2$-style statistics that I discuss here, here, here, and here), then you can view it this way, which is how software, such as the pROC::roc function in R, seems to work when given distributions with just one value.
Picking a threshold above that single value being predicted every time results in $TPR=TNR=1$, and picking a threshold below that single value results in $TPR=TNR=0$. Now connect those two points with a straight line (one of many options, but the simplest option and how I would expect software to interpolate) to get your ROC curve. From that diagonal line, $AUC=0.5$ follows immediately.
Answer: The following minimal example might help.
First, we build the most frequent class classifier from small toy data with the prevalent positive class (`True') having 66.6% probability. We then apply it to an even smaller test dataset, predicting the same probability of 66.6% for each instance.
y_train=[True,True,False,True,True,False]
# get the probability of the positive class
# we will use this as a prediction for all test instances
pos_probability=sum(y_train)/len(y_train)
y_test=[False,True,True]
y_pred_prob = [pos_probability for i in range(0,len(y_test))]

Next, we get the values of false positive rate and true positive rate used as an input for the ROC curve.
from sklearn.metrics import roc_curve
fpr, tpr, thresholds = roc_curve(y_test, y_pred_prob, pos_label=True)

Finally, we plot the curve:
from sklearn.metrics import RocCurveDisplay
roc_display = RocCurveDisplay(fpr=fpr, tpr=tpr).plot()

How do we wind up with a 45-degree line for the ROC curve?
This follows from the input data for the ROC plot.
The output of fpr is [0., 1.]
The output of tpr is [0., 1.]
The ROC plot is created by connecting only these two points ([0,0] and [1,1]) with a straight line.
Why don't we get more points?
The reason for this is that the value of the false positive rate and of the true positive rate (almost) do not react to changes in the threshold, which is used to create the input for the ROC curve. In our case, the output of thresholds is [1.66666667, 0.66666667]
The first threshold is arbitrarily set in scikit-learn above 1.0 to ensure that no instances get predicted as positive. The second threshold corresponds in our case to the value of `pos_probability'.
Recall that $tpr=\frac{tp}{positives}$ and  $fpr=\frac{fp}{negatives}$ for a given threshold value.
Since these relative values are used for axes, it does not matter how many positives or negatives (absolute) are in the dataset.


What are some of the tips for musicians starting out as teachers?

Question: What are some of the pitfalls and tips for starting out as a teacher? What do you need to start your own successful music teaching business and what are some of the things you should avoid?
Comment: Can we consider making this question a wiki?
Comment: A lot of this is going to be location-specific.  What country are you in?
Comment: @LaurencePayne - from the profile, S.Africa. But there are a lot of factors that will be similar for most places. Not sure how 'music teaching *business*' fits into teaching in schools.
Answer: For starters, merely being a good or great musician is nowhere near enough to be a successful teacher. Neither is having qualifications and certificates. Those downvoting - please tell your valid reasons!

Having knowledge is one thing. Having the propensity to pass that knowledge on to others is far more important for teaching. So, no.1 - being able to communicate and empathise.
O.k. So you passed loads of exams. That usually means in the first place that you're good at studying and passing exams. It doesn't make you a good teacher - necessarily. There may be parents, or potential students, who pose the question about qualifications, but I've only ever been asked about that once, and the fact that I'd passed no exams on the instrument concerned was in the end of no concern.
Having a suitable room is pretty important. For a while I visited students, but trying to teach in the kitchen or hallway hardly worked. Plus as much time was spent travelling as teaching, and resources were rarely at hand. So, good room, good equipment -  dependant on the instrument concerned. For guitar, a spare amp is basic, music stands, guitar stand, footstool etc. essential.
Somewhere for waiting, for parents, next pupil, etc.
Resources - something on which to play music, books, paper, manuscript paper, laptop, tuner, own instrument, etc. By own instrument, I mean just that. I prefer teaching guitar with my own on my lap; keyboard with another to explain things without shifting the student; two sets of drums facing each other.
In some countries, having been certified as safe (for wont of a better phrase!) as in a Police check (in UK it was called C.R.B) is essential. Difficult for a parent, in some cultures, to let their offspring stay with what would initially be a complete stranger for lessons. I would insist on parent being there for the first few, even though I had been 'certified'.
Being reliable - ready on time, prepared, not cancelling at the last minute.
Having material suitable for each individual. Being flexible enough to change what you prepared if it's not going as well as you hoped. Being able to transcribe anything the student brings, there and then, to work on, using that material - love it or hate it - as a vehicle. After all, that's probably more motivating than what you'd like him to practise! But not allowing him to call all the shots. You're the teacher, after all.
Being aware of publications, trends, being able to answer questions immediately - or - being certain to find out answers ready for next lesson! Obviously here, experience helps - and as a beginner teacher, one wouldn't have that, maybe.

Most of this comes from experience with private teaching, but a lot will be appropriate for schools and colleges.
That'll do for now, except to say that behind every great performer, be it athlete, player, whatever, there's often a coach who is not as good as they are - just that that coach knows how to make the performer improve. Teachers are often like that coach. So, back to the beginning - you don't need to be a brilliant player to teach!
Comment: I would add that it is extremely important to come up with a consistent and efficient method of collecting payment from your students (or more likely their parents).
Answer: A few things you should consider when starting out.
Decide what kind of teacher you want to be
A professor of music at a college once told someone close to me that he always finds it amazing when he sees a teacher, teaching young children. He says that even though he did go to Julliard, it was something he was never taught.
Just as this professor decided to focus on adults studying music at a Collegiate level so you must also decide on what kind of teacher you want to be.
Do you want to focus on primary school children, do you want to focus on teenagers, would you take on adults? All question that you must ask yourself
Where are you going to teach?
Do you want to teach at a school, or do you want to teach in a private practice? Some people do well at a school while others prefer to be their own boss.
In certain places teaching from home has a stigma associated with it. In Britain, for instance, the RCM will not give out the details of graduates to parents who inquire for lessons if the graduate works from home.
Do you have the space to work from home? In Europe it is hard to have the home where you can teach from, most people live in a flat and that is not really suited to teach from.
In North America people tend to own homes that allow for this more. If you do have space there is still more to consider. If you are teaching from home you need an open space from which the parents can see you working with their children.
 There is a certain amount of natural apprehension when parents let new people in their children's lives, you need the setup where the parents can sit in on the lessons and see what you are like with them.
If you work from home you must make it clear to the children that some parts of the house are out of bounds. You cannot have children entering your bedroom.
If you live with someone then this someone must take reasonable steps to divorce him/herself from your teachings. You must also ask yourself if this person represents him or herself in a manner that aids my teaching or not?
Working for a school can also be worthwhile, having a steady paycheck and benefits like sick-leave and a pension can be great. You do give out a lot of control over your lessons when working for a school.
Many music teachers who work at schools, especially those in full-time positions, have their roster of pupils fill in for them. Some school parents apply to the school for a position and the school decides whether they have a place for the pupil. There are schools where this happens in a much more authoritarian way that what would strike some as good. It is also often not easy to cancel lessons as there is often a stringent process for this to happen.
Do you have the mentality of a teacher?
Some people are natural born teachers, some are not. So have the social graces for teaching, some do not. Teaching is both a selfless and a thankless job.
Regardless of how good you are, you are never going to be rich teaching. You are not going to be the person who plays to an audience and gets rapturous applause and the love of the masses. You are going to be the person who will work with young people, teaching them for a certain amount of time and then when they are done using you they will leave and you will never see them again.
This is not to say that teaching cannot be richly rewarding, you must just be aware of the non-tangible rewards it provides. Teaching is not a profession for people who have the locus of their self-worth foreign to themselves.
Comment: How did I guess this would be self-answered..! And why highlight young people? I've taught many adults, 70+ yr olds!
Answer: This is a very broad question, and entire books are available on the subject.  This answer will focus on considerations of starting a music teaching business, and I'll be re-iterating some excellent points from the other answers. Some of my points may be specific to the United States.
There are a number of opportunities for teaching music. Some of these tips are for someone who is not working as an employee of an organization, such as a school, but is working independently as their own employer.
GETTING STARTED
Setting yourself up as an independent music teacher is essentially starting your own business.  You may teach from your own location, contract with a music school or through a music store, and/or travel to client's locations.
License and Certification
Depending on your location you will need to look up local licensing and certification requirements. In the States, a Business License is usually required to sell services to people. In some cases things like Certification and posting of Bond are required to operate a teaching business.
As mentioned, a safety certification is extremely useful, or being able to provide some sort of background check. It is also useful to have First Aid and CPR training.
You will also be responsible for reporting and paying any taxes that are required in your area.  Reporting and paying taxes may be different when you are set up as a business than personal employment income.
Location
If you are teaching from your own location, you should check local zoning rules to see if your type of business is permissible in your location. Your location should be clean and well lit, and have easy access for the client. Your clients shouldn't have to climb some rickety back stairs and travel through your kitchen to get to your music studio. It should also be clean: no pet hair on chairs, no laundry being folded, no equipment other than the music gear you need for teaching. Did I mention that it should be clean?
Contracting with a music studio or store can be a way to have a location without setting up one of your own. Teachers may pay rent on the studio by space or by student, or the studio may handle the customer and pay you directly for the lessons you teach. There are many different configurations.
If your are going to be traveling to  the client's location, travel time and the additional expense of your transportation needs to considered in your pricing.
Customers
As a business you need to attract and retain customers. You should do Market Research in your area to find out what competition you have and what the going rate is for lessons of the type you are providing.
Music lessons are often a referral business, so you will want to reach out to local music stores, local school music directors and anyone else in related music business in your area and introduce yourself and the services you are offering.
Make sure you look professional when you are contacting people and potential clients.
Contracts
It is a good idea to have a Contract prepared for your lesson services. The contract should detail all the expectations between the client and you, including but not limited to Scheduled Time, Duration of lesson, Payment requirements, Policies on absences and make-up lessons, Contract cancellation, late fees &c.
You should also collect on your contract all the customer's contact information, including where you can mail your invoices, e-mail addresses, emergency contact numbers, and any special considerations for the student.
You should maintain a list or database of your customers, including past customers and customer history, and keep records of all payment transactions.
Billing and Accepting Payments
You will want some way of Invoicing your clients, and collecting payment. There are many options to how you may set this up and it is a personal choice for how much you want to set up. Accepting credit cards has become easy today with many services available for individuals and small businesses.
BUILDING THE BUSINESS
Advertising and Marketing
Besides the very many books and web pages available on the subject, there are also often non-profit and government resources available to help new small-businesses with starting out.
Gaining and Keeping Students
Make sure you know enough to actually teach your instrument. I often have musicians come to me looking for a teaching job who have played for a number of years, but can't describe what Key Signatures are.  If you don't have a degree in music or haven't passed various certifications, check your knowledge. If you are taking on students you may have them for a long time and you want to be sure that you can provide all the knowledge to put them on a path to success.
Student retention is one of your primary goals. The teaching business may start slowly, with only a few students to start with, and limited income. Students can stay with a teacher for years, and you build up more and more students as you add them into your schedule.
Clear communication with the client and acting Professional is very important.
Consistency is also very important, especially with scheduling. Students (or their parents) like to have the schedule be regular and predictable. If you are constantly changing the schedule, re-arranging the lessons or cancelling last minute you will loose your students.  I have seen this happen with a number of good teachers. Eventually the client gets tired of the constant changes and re-scheduling, and drops the teacher.
Referrals are often one of the main avenues to gaining more students. Having a referral discount for your current customers, family discounts for additional family members, free lesson vouchers that students can give to their friends, and marketing material on hand such as business cards or brochures can all help bring in new business.
Get involved with your community music events, get to know your community leaders and business people,  and always be on the lookout for the opportunity to mention your lesson business.


Is there a "correct" position for the right hand when playing guitar?

Question: Normally when I am playing I support my right hand by putting my pinky finger below the E string, but when I try to play past a certain speed I have difficulty keeping up. I have found a couple sites where they advise supporting the hand by putting the wrist on the bridge, but when I try it I tend to mute the A and low E strings. It's also more tiring because I am not used to that position, and my pick hits the pickups. Another thing I tried was to leave my hand unsupported, and I actually found it easier than supporting it on the bridge.
Is there a "proper" technique for this that would help me in the long run even if it is difficult to get used to? It would also be nice to know the advantages/disavantages of each of these listed methods, or whether it is totally irrelevant and you can get to a good level using any of them.
Comment: You may find some helpful information in this related question: [Flatpicking: how can I make my floating-pick-hand less bouncy?](http://music.stackexchange.com/questions/7801/flatpicking-how-can-i-make-my-floating-pick-hand-less-bouncy) Note that the answers there recommend anchoring for fast, accurate picking, so the fact that you're anchoring isn't a problem in itself, but there may be a better way to do it.
Answer: Normally, you'll get all the support you need when you simply rest your wrist/forearm on the upper body of the guitar. With your full hand completely free to move, you'll find that it is much, much easier to play smoothly. This is how I was taught to play both classical and rock style guitar.
Answer: If your pick is catching your pups, you have too much showing.For faster play there needs to be only a little sticking out, so it shouldn't catch.
You will use all sorts of hand/arm positions to play. Some will need a 'floating' arm, where the contact is inside elbow on guitar body. Sometimes the palm will rest  just behind the saddles, as in palm-muting, but slightly further along thr strings, away from the pups.
If you find one position is strange, angle the guitar differently, so your arm/wrist is comfortable. This will morph into a good shape for you in time. Partly because everyone's arms are slightly different from each other, and wrist flexibility also differs, a personal rather than textbook position works better for each individual.I.e some find that resting pinky on the top e works well. Obviously not while trying to play that string though !
Answer: About 99% of where your hand should go is determined by two things:
 1. the style of music and sound you want
 2. what feels comfortable to you
When I finger-pick an acoustic guitar, I find myself holding my pinky below the e-string on the pick guard (if applicable).
If I'm wanting a pulsing (I don't know the word, think punk-rock rhythm) strummed rhythm, I usually take the outside edge of my hand (the side that's opposite from the thumb) and rest my hand centimeters from the strings (not all bridges allow this comfortably). Then, for a slightly muted or faster-decay-time sounds, I lower my hand so it grazes the strings as the strings vibrate.
For basically everything else on acoustic or electric, the "floating hand" works best for me. I basically use my elbow to push the guitar towards me a little and just strum. I'm pretty sure I aim my pick so it's just to the top of the bottom pickup. I usually don't think about it so hard to give more detail than that.
If you're hitting the strings with your hand, then you could try moving your hand or tilting the guitar. Maybe even lower or raise the guitar. The last thing you could get checked out (by a professional) is an incorrectly aligned neck - I've seen on two guitars where a player's kept hitting the strings due to the neck being bowed inward too much. You'll find what works and it'll be second nature when you start playing to hold it whichever way you find is best.
Overall, I'd say personal preference and comfort are more important than any one "correct" way of holding your hand. Hendrix, SRV, Eric Clapton, and Slash all played completely different ways - not just styles, but they also held their hands differently, held the strings with different intensities, as well as numerous other things.
Answer: Others have provided good answers but I think  this also depends on the shape of the guitar and what you're playing.
Eg for strumming an acoustic guitar, you probably want a bit of movement available so resting your forearm (just below the elbow) on the uppder body of the guitar will work - or possibly just have it free with no support - so you can get nice even sweeps across the strings.
Assuming you're talking about intricate/fast playing on an electric guitar, you may get different results on say a Les Paul as opposed to a Stratocaster because the bridges are different.
On a Les Paul you'll find resting your palm on the bridge at all will mute a string somewhere. On a Strat you can often get away with resting it a bit on the bridge because the height adjuster grub screws may stop your pam hitting the string immediately.
Also the shape of the body on a strat means you can move your arm in a little closer to the strings and kind of rest your arm along the top of the body. Not so on a Les Paul or a Telecaster - there's no chamfer in the upper body so your arm is forced outwards a little more.
To an extet, what you're aiming fgor is subjective in that it's "whatever works for you" so the actual answer to your queston is "No, there isn't a 'correct' position" but what works for me is to rest my wrist on the body of the guitar just next to the bridge, holding my hand quite still, and allow my fingers to move about as necessary. I use a Strat (can you tell..) so the bridge isn't too raised above the body, and this feels comfortable.
I also have a Telecaster and a Les Paul and use the sometimes. One of the reasons I return to the Strat is because it feels strange when first using the Les Paul after a while, having to hold my hand out a little further because the bridge is higher from the body on the Les Paul, and the body of the Tele pushed my arm out whcih feels strange at first.


Masked JFormattedTextField prevent deleting slash char

Question: I'm creating a frame with a JFormattedTextField masked with MaskFormatter:
public static void main(String[] args) {
    DateFormat df = new SimpleDateFormat("dd-mm-yyyy");
    JFrame frame = new JFrame("");
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    JPanel panel = new JPanel();
    frame.setLayout(new BorderLayout());
    JFormattedTextField tf = new JFormattedTextField(df);
    tf.setColumns(20);
    panel.add(tf);
    try {
        MaskFormatter dateMask = new MaskFormatter("##-##-####");
        dateMask.install(tf);
    } catch (ParseException ex) {
        Logger.getLogger(MaskFormatterTest.class.getName()).log(Level.SEVERE, null, ex);
    }

    frame.add(new JButton(), BorderLayout.LINE_START);

    frame.add(panel, BorderLayout.LINE_END);
    frame.pack();
    frame.setVisible(true);
}

OK. It's alright, but when i delete (with keyboard key) the text i write in the field too delete slashes. Are there any mode to prevent it? I want delete the text but the slashes not, like:
Writing date:
"12-12-1212"

After delete:
"  -  -    "

Answer: How do you delete the text? Are you using setText("")? You should not do that. Use setValue(null) instead.
If that is not your problem, please elaborate on what you are doing, and what is happening instead.
edit: Try out this code and see if the problem persists.
public class Tester {
  public static void main(String[] args) {
    DateFormat df = new SimpleDateFormat("dd/mm/yyyy");
    JFrame frame = new JFrame("");
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    JPanel panel = new JPanel();
    frame.setLayout(new BorderLayout());
    JFormattedTextField tf = new JFormattedTextField(df);
    tf.setFocusLostBehavior(JFormattedTextField.COMMIT);
    tf.setColumns(20);
    panel.add(tf);
    try {
      MaskFormatter dateMask = new MaskFormatter("##-##-####");
      dateMask.install(tf);
    } catch (ParseException ex) {
    }

    frame.add(new JButton(), BorderLayout.LINE_START);

    frame.add(panel, BorderLayout.LINE_END);
    frame.pack();
    frame.setVisible(true);
  }
}´

edit2:
try adding
    tf.setFocusLostBehavior(JFormattedTextField.COMMIT);
see the JavDoc for further reading.
Comment: Thanks! it's right! I throught that setFocusLostBehavior only works when the focus is changed, not when delete the text. Thanks! Accepted ;-)
Comment: This does work in your example, if I try it out. The `-` character stays in place. Usually the JFormattedTextField behaves this way, if you give it an invalid text, it kind of breaks this way. Can you post all of your code?
Comment: What is the button supposed to do? Are you pressing it? If I do that, it breaks the field. Though currently not sure why.
Comment: For now nothing, is only a test, in the future only prints the jtextfield in screen. When i press it without press "tab" the field deletes at all, when i press "tab" the correct date persists and when i delete with the keyboard it deletes all the chars in the field (slashes included)


How to fix disable validation in Dynamo-db against null / empty string?

Question: I am trying to push the data to Dynamodb from node JS code.
I am getting the issue like
DynamoDB.DocumentClient should support empty string properties
does anyone knows how to disable validation in DynamoDB ?
Comment: The validation cannot be disabled.
Comment: @VishnuRanganathan did you find solution?
Comment: @AmodGokhale : yes , i have added the solution in section.
Answer: By adding this we will be able to insert empty values to Dynamodb.
var dynamoDocClient = new AWS.DynamoDB.DocumentClient({ region : 'us-west-2', convertEmptyValues : true });

Comment: convertEmptyValues will convert empty strings to NULL, then save it. Be careful that that is indeed what you want.
Comment: How to you achieve this in Java instead of Javascript?

Thanks!
Answer: 2020 May 18 Update:
DynamoDB now supports empty string values for non-key attributes: https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-dynamodb-now-supports-empty-values-for-non-key-string-and-binary-attributes-in-dynamodb-tables/
In order to have consistent behavior, make sure that the client does not have { convertEmptyValues: true }, or else these attributes will be saved as NULL as opposed to an empty string.


Why is post used in a search route?

Question: Hey fellow stackerenos!
Consider this snippet of code I found in a tutorial on custom routes:
In routes.rb,
<code>resources :users do
  resources :tweets
  post :search, on: :collection
end
</code>
Why is the action "post" and not "get"?  As I understand it, "post" is used to insert into a table.  But this creates a custom route to search all the tweets.
Answer: Form submissions are generally sent as POST requests (whether or not this results in inserting into a table is immaterial). Presumably this route is invoked by a user submitting a "search" form.

Why is post used in a search route?


rest-framework @detail_route without pk

Question: In rest framework exists some way to use session user instead pk and self.get_object() in a @detail_route view?
I want to use request.user and don't send pk in the url.
Or maybe use another decorator instead @detail_route...
Comment: Use @list_route instead. Detail route needs a pk by definition ( as per rest )
Comment: Can you expain a bit more about your use case maybe with some code.
Answer: It's hard to tell what you're trying to archive in the first place. Maybe it's easier to use a separate view instead of the viewset action this case? For example here is a view snippet that lets currently logged in user manage his account:
class CurrentUser(generics.RetrieveUpdateDestroyAPIView):
    serializer_class = UserSerializer

    def get_object(self):
        return self.request.user

    def perform_destroy(self, instance):
        instance.is_active = False
        instance.save()

Add this to your urlpatterns without pk in the pattern:
    url(r'^users/me/$', CurrentUser.as_view()),

Comment: @LuisSieira yes, any url you want. Just no capture group since it isn't needed. It doesn't *delete* it just sets is_active to False as recommended by Django. Obviously, you can remove or change this logic as you like, this is just an example
Comment: @LuisSieira Ah! Got it
Comment: So where it says "me" it might be whatever you want to write there?, seems legit !

I'd not allow users to delete their own account, though
Comment: @LuisSieira What was wrong with my answer? This answer is also using `self.request.user`
Comment: @KishorPawar It doesn't look semantically correct to retrieve the information of a single user, using a class representing "a collection of model instances", which is the reason why the OP was trying to use detail_route and not list_route, and why I opened the bounty.
Answer: In my project, I am using both plain Django and Django-rest-framework and I am using request.user as below.
class ProfileRunCounts(generics.ListCreateAPIView):

    def get(self, request, *args, **kwargs):
        id = self.request.user

update after comments.

HttpRequest.user
  From the AuthenticationMiddleware: An instance of AUTH_USER_MODEL representing the currently logged-in user. If the user isn't currently logged in, user will be set to an instance of AnonymousUser.
REST framework's Request class extends the standard HttpRequest, adding support for REST framework's flexible request parsing and request authentication.
APIView classes are different from regular View classes in the following ways:
Requests passed to the handler methods will be REST framework's Request instances, not Django's HttpRequest instances.

With above extract from the documentation I believe user object is always available in request.
Sometimes it is in self.request.user and sometimes it is in request.user.
Comment: These are used for read-write endpoints to represent a collection of model instances, so still the same hacky feeling. http://www.django-rest-framework.org/api-guide/generic-views/#listcreateapiview
Comment: It would be more appropriate to use an api_view

rest-framework @detail_route without pk


Zabbix and logrt.count doesn't get data

Question: I'm running Debian and Zabbix 3.4.
I've created a custom item with the following syntax:
logrt.count["/var/log/^req_log.*.txt$","TIMEOUT"]

I'm trying to detect any log file in the format req_log2018-06-26.txt with the word TIMEOUT and return the number of occurenses.
But no data is gathered with this item, even though there are some files in the directoty which does contain the word.
Is there a way to manually run and test item on server with Zabbix agent running?
Answer: From Zabbix version 3.4+, Perl compatible regular expressions (PCRE) are implemented in the Zabbix frontend/backend. Your regexp doesn't look right:
$ touch req_log2018-06-26.txt
$ ls -lah | grep -P '^req_log.*.txt$'
<empty result>
$ ls -lah | grep -P 'req_log.*.txt'
-rw-r--r--.  1 root root     0 Jul 12 20:37 req_log2018-06-26.txt

Try:
logrt.count["/var/log/req_log.*.txt","TIMEOUT"]

Zabbix and logrt.count doesn't get data


Recording stream locally with VLCKit on MacOS - looking for example

Question: In my MacOS app (Swift) I'm receiving an RTSP stream, which I want to record to a local file. I found a few examples using the .startStreaming() and .stopStreaming() functions from the VLCStreamSession object, but these are marked as "deprecated" since VLCKit 3.0 version.
I'm looking for an up-to-date example or hints how to do that. Thanks for your help
Answer: Meanwhile I was told that recording is not an official part of the VLCKit lib anymore. The old version was deprecated, but the VLC team plan to integrate this with a complete new solution mid term.


Pytorch GPU implementation is much slower than Numpy

Question: can someone explain to me why this simple operation takes so much time using pytorch with GPU? I understand that maybe the equation is not so hard for CPU (numpy code) but still, 0.6899s seems to be horrible when I used pytorch for much harder problems with more samples and it took 4ms. Is there a way to make that function much more efficient? Thank you
import numpy as np
import torch
import time

# Define the functions to be tested
ndof = 2
x_dim = 20
n_samples = 1000

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
def NP_cost(x):
    x_t_minus1 = np.concatenate((x[:, :ndof], x[:, :(x_dim - ndof)]), axis=1)
    x_t_minus2 = np.concatenate((x_t_minus1[:, :ndof], x_t_minus1[:, :(x_dim - ndof)]), axis=1)
    k_order = x + x_t_minus2 - 2 * x_t_minus1
    return np.sum(k_order ** 2, axis=1)

def GPU_cost(x):
    x_tensor = torch.tensor(x, dtype=torch.float32).to(device)
    x_t_minus1 = torch.cat([x_tensor[:,:ndof], x_tensor[:, :(x_dim - ndof)]], dim=1)
    x_t_minus2 = torch.cat([x_t_minus1[:,:ndof], x_t_minus1[:,:(x_dim - ndof)]], dim=1)
    k_order = x_tensor + x_t_minus2 - 2*x_t_minus1
    return torch.sum(k_order**2, dim=1).cpu().numpy()

# Generate a random input array
x = np.random.rand(n_samples, x_dim)

# Test the functions and measure execution time
start_time = time.time()
print(NP_cost(x)[0])
print(f"NP_cost execution time: {time.time() - start_time:.4f} seconds")    # 0.0003 seconds

start_time = time.time()
print(GPU_cost(x)[0])
print(f"GPU_cost execution time: {time.time() - start_time:.4f} seconds")   # 0.6899 seconds

Comment: You should move the 'print()' calls out of code region you are trying to time.
Comment: Measure computation time only, not copying data from cpu-to-gpu and gpu-to-cpu
Answer: I'm no expert at this, but I tried this code, and it's probably just a bunch of copying back and forth and not much compute for a gpu.
Tried it with n_samples = 10000000, where the gpu did it faster.
4.644260007225965
NP_cost execution time: 3.8376 seconds
4.6442604
GPU_cost execution time: 2.0928 seconds
Gpu execution needs your tensors to be copied to the device, make the computation, and copy it back. Concatenation is really just another few copies.
The only real compute you have is a few additions, a subtraction, square, and a sum, which is not much compared to how many data elements you have. I would like to hear someone else's opinion though with more experience.
Answer: Thanks to @balazon I have found out that creating the tensor takes over 600ms, which is most of the time. After putting the torch.cuda.synchronize() command at the beginning of the code, creating the tensor was fast. I cannot explain it...


Wix bootstrapper won't detect installed .NET framework

Question: I have a Wix installer and bootstrapper application to install my application and a service. As a prerequisite I need the .NET 2.0 SP2 Framework installed and tried to detect that using the following codesnippet:
    <Chain>
      <PackageGroupRef Id='Netfx2Package' />
      <MsiPackage SourceFile="..\Wix.CHL7.Dispatcher.Service.AZHF\bin\Debug\Wix.CHL7.Dispatcher.Service.AZHF.msi" Id="Wix.CHL7.Dispatcher.Service.AZHF_PackageId" Cache="yes" Visible="no">
        <MsiProperty Name="INSTALLDIR" Value="[INSTALLPATH]" />
        <MsiProperty Name="WixAppFolder" Value="[INSTALLSCOPE]"/>

      </MsiPackage>
    </Chain>
  </Bundle>

  <Fragment>
    <WixVariable Id="WixMbaPrereqPackageId" Value="Netfx2Package" />
    <WixVariable Id="WixMbaPrereqLicenseUrl" Value="NetfxLicense.rtf" />

    <PackageGroup Id="Netfx2Package">
      <ExePackage Id="Netfx2Exe"
                  Cache="no"
                  Compressed="no"
                  PerMachine="yes"
                  Permanent="yes"
                  Vital="yes"
                  SourceFile="C:\Installation\Wix.CHL7.Dispatcher.Service.Bootstrapper\lib\NetFx20SP2_x86.exe"
                  DownloadUrl="http://download.microsoft.com/download/c/6/e/c6e88215-0178-4c6c-b5f3-158ff77b1f38/NetFx20SP2_x86.exe"
                  DetectCondition="NETFRAMEWORK20"
                  InstallCondition="FALSE"/>
    </PackageGroup>
  </Fragment>

When I run the installer on a machine which has .NET 2.0 SP2 installed (checked that in the registry) my installer wants to download and install the framework.
I assume my DetectCondition is incorrect but can't seem to find a correct solution to my problem.
So how do I detect if .NET 2.0 SP2 is installed in my bootstrapper?
Thanks in advance!
W
Answer: Make sure you are referencing the NetFxExtension properly.
Source: WixNetfxExtension

Using WixNetfxExtension Properties To use the WixNetfxExtension
  properties in an MSI, use the following steps:
Add PropertyRef elements for items listed above that you want to use
  in your MSI. Add the -ext  command line
  parameter when calling light.exe to include the WixNetfxExtension in
  the MSI linking process.
For example:

<PropertyRef Id="NETFRAMEWORK20" />

Also, you probably want to use the property NETFRAMEWORK20_SP_LEVEL to get the appropriate service pack level if you require SP2. When troubleshooting I'd also check the install log to see if it indicates what the property values are set to.


Why can I combine a bitwise operator with a math operator, like &+, in Java?

Question: I was writing some code earlier today, and forgot I was writing Java instead of Swift and accidentally typed &+. Surprisingly, I didn't get any errors, and the code compiled and ran fine.
I played around a little bit, and found that &+, &-, |+, |-, ^+, and ^- all compile and run without errors. It looks like they all act like the first bitwise operator when acting on numbers:
1 &+ 2 == 1 & 2
3 |+ 4 == 3 | 4
7 ^+ 9 == 7 ^ 9

On the other hand, it seems like any other combination of a bitwise operator and a math operator, like +&, &/, and ^*, all give compile-time errors.
Why do &+, &-, |+, |-, ^+, and ^- compile and run without errors? Do they have some purpose that I'm not seeing, or is it just some compilation oddity?
Comment: You're just formatting it wrong. `1 &+ 2` is actually `1 & +2`. You can prefix any number with a `+` or `-`.
Answer: The + stands for the Unary Plus Operator + resulting in a positive number.
The - means the Unary Minus Operator - resulting in a negative number.

1 &+ 2 is 1 & +2
3 |- 4 is 3 | -4
7 ^+ 9 is 7 ^ +9

Notice that * and / operators cannot be used in the same way (1 &/ 2).
Comment: To name it correctly, the `+` sign is an [Unary Plus Operator](https://docs.oracle.com/javase/specs/jls/se12/html/jls-15.html#jls-15.15.3), and the `-` is an [Unary Minus Operator](https://docs.oracle.com/javase/specs/jls/se12/html/jls-15.html#jls-15.15.4).
Comment: Just also to point out, for interest's sake: there is no such thing as a negative integer literal. It's the unary minus operator and a non-negative integer literal. You can see this because you can't write `-2147483648`, even though that is ostensibly equal to `Integer.MIN_VALUE`, because it is interpreted as `-` then `2147483648`, which is too large for an int literal.


SQL query the Drupal way, complex SELECT, conditional aggregate

Question: I'm starting to invest time in learning custom Drupal entities like the ones explained here and I want to make sure that I am going to be able to do something in the future.
I am working with Drupal 7 with MySQL and have the entities API. Being tied to MySQL is okay.
Is it possible to perform a query like the one listed below in Drupal? If not, can I access a MySQL database view?
 select
        `listProducts`.`productId` AS `productId`,

        group_concat(if((`listProducts`.`className` = 'manufacturer'),
                `listProducts`.`optionValue`,
                '')
            separator '') AS `manufacturer`,

        group_concat(if((`listProducts`.`className` = 'model'),
                `listProducts`.`optionValue`,
                '')
            separator '') AS `model`,

        group_concat(if((`listProducts`.`className` = 'sub model'),
                `listProducts`.`optionValue`,
                '')
            separator '') AS `sub model`,

         group_concat(if((`listProducts`.`className` = 'year'),
                `listProducts`.`optionValue`,
                '')
            separator '') AS `year`
    from
        `hqr`.`listProducts`
    group by `listProducts`.`productId`

It's sort of an intense select statement. The data I'm working with is in the form
productId  className      optionValue
---------------------------------------
1          manufacturer   abcCompany
1          model          aModel
2          manufacturer   xyzCompany
2          year           2005

and should come out like
productId  manufacturer  model  sub model  year
-------------------------------------------------
1          abcCompany    aModel    -        -
2          xyzCompany      -       -       2005

Is doing this possible using the Drupal tools? If so, knowing how would be a big bonus too. Somewhere in my travels I saw a Drupal patch to enable group_concat in Drupal views (if that changes anything).
Answer: Drupal does not provide database transposition, unfortunately. It's database access system is pretty good at filtering, joining and reordering the data, but not at restructuring them.
There are many patches and hacks for specific task, but none was general enough and needed enough to be accepted into core or views so far.
Your safest bet is to define a view, and then access it just like any other table. Drupal sees no difference as long as you don't write (and if you want to play, you could make MySQL to accept writes, too).
Answer: Yes you can easily reproduce that query using Drupal's DB API. In fact you run pretty much any valid MySQL query through that API; the only type I've had problems with in the past is LOAD DATA INFILE.
The key is the SelectQuery::addExpression() method, which allows you to inject arbitrary expressions into the query. For example:
$query = db_select('listProducts')->groupBy('listProducts.productId');

foreach (array('manufacturer', 'model', 'sub model', 'year') as $field_name) {
  $sql = "group_concat(if((listProducts.className = '$field_name'), listProducts.optionValue, '') separator '')";
  $query->addExpression($sql, $field_name);
}

$results = $query->execute();

That produces:
SELECT
  group_concat(if((listProducts.className = 'manufacturer'), listProducts.optionValue, '') separator '') AS manufacturer,
  group_concat(if((listProducts.className = 'model'), listProducts.optionValue, '') separator '') AS model,
  group_concat(if((listProducts.className = 'sub model'), listProducts.optionValue, '') separator '') AS submodel,
  group_concat(if((listProducts.className = 'year'), listProducts.optionValue, '') separator '') AS year
FROM {listProducts} listProducts
GROUP BY listProducts.productId