Poetry Update Fails: Unable to Find Installation Candidates for pyqt5-qt5 (5.15.11)

Question: My Environment:

Operating System: [Windows 22H2]
Python Version: [3.11]
Poetry Version: [(version 1.7.1)]

Attempted Solutions:

Clearing Poetry's cache
Directly specified different versions of pyqt5-qt5 in pyproject.toml.
Checking network settings and proxy configurations

Issue Description:
I am encountering an issue when trying to update my Python project dependencies using Poetry. Specifically, Poetry is failing to find installation candidates for pyqt5-qt5 version 5.15.11. Here are the steps I have tried which did not resolve the issue:

Cleared Poetry's cache
Confirmed that my network connection is stable
Tried specifying different versions of pyqt5-qt5

Here is the output when I run poetry update:
(jsonrpc-py3.11) PS C:\PythonProject\pythonProject\jsonRPC> poetry update
Updating dependencies
Resolving dependencies... (4.5s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing pyqt5 (5.15.1): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  Traceback (most recent call last):
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\sipbuild\api.py", line 46, in build_wheel
      project = AbstractProject.bootstrap('wheel',
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\sipbuild\abstract_project.py", line 87, in bootstrap
      project.setup(pyproject, tool, tool_description)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\sipbuild\project.py", line 586, in setup
      self.apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpl4qbme9n\PyQt5-5.15.1\project.py", line 64, in apply_user_defaults
      super().apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\pyqtbuild\project.py", line 70, in apply_user_defaults
      super().apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\sipbuild\project.py", line 237, in apply_user_defaults
      self.builder.apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpcxilq025\.venv\Lib\site-packages\pyqtbuild\builder.py", line 69, in apply_user_defaults
      raise PyProjectOptionException('qmake',
  sipbuild.pyproject.PyProjectOptionException

  at ~\AppData\Roaming\pypoetry\venv\Lib\site-packages\poetry\installation\chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pyqt5 (5.15.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyqt5 (==5.15.1)"'.

Instead,it is normal when I install pyqt5 by pip:
(jsonrpc-py3.11) PS C:\PythonProject\pythonProject\jsonRPC> pip install pyqt5
Collecting pyqt5
  Using cached PyQt5-5.15.10-cp37-abi3-win_amd64.whl.metadata (2.2 kB)
Requirement already satisfied: PyQt5-sip<13,>=12.13 in c:\users\nbdhc\appdata\local\pypoetry\cache\virtualenvs\jsonrpc-1g8b7ov1-py3.11\lib\site-packages (from pyqt5) (12.13.0)
Collecting PyQt5-Qt5>=5.15.2 (from pyqt5)
  Using cached PyQt5_Qt5-5.15.2-py3-none-win_amd64.whl (50.1 MB)
Using cached PyQt5-5.15.10-cp37-abi3-win_amd64.whl (6.8 MB)
Installing collected packages: PyQt5-Qt5, pyqt5
Successfully installed PyQt5-Qt5-5.15.2 pyqt5-5.15.10

after my installation by pip, I fix my pyproject.toml's [tool.poetry.dependencies] to
[tool.poetry.dependencies]
python = "^3.11"
loguru = "^0.7.2"
sip = "^6.7.12"
pyqt5-sip = "^12.13.0"
pyqt5 = "=5.15.10"
pyqt5-qt  = "5.15.2"

the poetry's error are unchanged.
Additionally, running pip wheel --no-cache-dir --use-pep517 "pyqt5 (==5.15.1)" also results in errors:
$ pip wheel --no-cache-dir --use-pep517 "pyqt5 (==5.15.1)"

I have got error,too.
(jsonrpc-py3.11) PS C:\PythonProject\pythonProject\jsonRPC> pip wheel --no-cache-dir --use-pep517 "pyqt5 (==5.15.2)"
Collecting pyqt5==5.15.2
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))': /packages/28/6c/640e3f5c734c296a7193079a86842a789edb7988dca39eab44579088a1d1/PyQt5-5.15.2.tar.gz
  Downloading PyQt5-5.15.2.tar.gz (3.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 4.4 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      Traceback (most recent call last):
        File "C:\Users\nbdhc\AppData\Local\pypoetry\Cache\virtualenvs\jsonrpc-1g8b7oV1-py3.11\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\nbdhc\AppData\Local\pypoetry\Cache\virtualenvs\jsonrpc-1g8b7oV1-py3.11\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\nbdhc\AppData\Local\pypoetry\Cache\virtualenvs\jsonrpc-1g8b7oV1-py3.11\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\sipbuild\api.py", line 46, in build_wheel
          project = AbstractProject.bootstrap('wheel',
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\sipbuild\abstract_project.py", line 87, in bootstrap
          project.setup(pyproject, tool, tool_description)
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\sipbuild\project.py", line 586, in setup
          self.apply_user_defaults(tool)
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-wheel-gtrn2jcu\pyqt5_d0e386869adb4ed9bf122a2a8aa3e1c7\project.py", line 63, in apply_user_defaults
          super().apply_user_defaults(tool)
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\pyqtbuild\project.py", line 70, in apply_user_defaults
          super().apply_user_defaults(tool)
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\sipbuild\project.py", line 237, in apply_user_defaults
          self.builder.apply_user_defaults(tool)
        File "C:\Users\nbdhc\AppData\Local\Temp\pip-build-env-r_utcu4y\overlay\Lib\site-packages\pyqtbuild\builder.py", line 69, in apply_user_defaults
          raise PyProjectOptionException('qmake',
      sipbuild.pyproject.PyProjectOptionException
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Certainly,I tried use
$poetry cache clear --all pypl

but it doesn't work.
After installing with pip, I updated pyproject.toml as follows, but the issue persists:
pyqt5 = "=5.15.10"

$poetry update
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/66/f5/0a684b2b33b1f1dceaf894d4e8a105f523bb08003f81c89a91e509c5f854/PyQt5_Qt5-5.15.11-py3-none-macosx_10_13_x86_64.whl  99%
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/66/f5/0a684b2b33b1f1dceaf894d4e8a105f523bb08003f81c89a91e509c5f854/PyQt5_Qt5-5.15.11-py3-none-macosx_10_13_x86_64.whl  99% Resolving dependencies... (43.5s)

Package operations: 0 installs, 1 update, 0 removals

  • Updating pyqt5-qt5 (5.15.2 -> 5.15.11): Failed

  RuntimeError

  Unable to find installation candidates for pyqt5-qt5 (5.15.11)

  at ~\AppData\Roaming\pypoetry\venv\Lib\site-packages\poetry\installation\chooser.py:73 in choose_for
       69│
       70│             links.append(link)
       71│
       72│         if not links:
    →  73│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       74│
       75│         # Get the best link
       76│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       77│

Cannot install pyqt5-qt5.

pyqt5 = "=5.15.2"

$poetry update
Updating dependencies
Resolving dependencies... (9.3s)

Package operations: 0 installs, 1 update, 0 removals

  • Downgrading pyqt5 (5.15.10 -> 5.15.2): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  Traceback (most recent call last):
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Roaming\pypoetry\venv\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\sipbuild\api.py", line 46, in build_wheel
      project = AbstractProject.bootstrap('wheel',
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\sipbuild\abstract_project.py", line 87, in bootstrap
      project.setup(pyproject, tool, tool_description)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\sipbuild\project.py", line 586, in setup
      self.apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpdh6v9hcw\PyQt5-5.15.2\project.py", line 63, in apply_user_defaults
      super().apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\pyqtbuild\project.py", line 70, in apply_user_defaults
      super().apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\sipbuild\project.py", line 237, in apply_user_defaults
      self.builder.apply_user_defaults(tool)
    File "C:\Users\nbdhc\AppData\Local\Temp\tmpau4hqchz\.venv\Lib\site-packages\pyqtbuild\builder.py", line 69, in apply_user_defaults
      raise PyProjectOptionException('qmake',
  sipbuild.pyproject.PyProjectOptionException

  at ~\AppData\Roaming\pypoetry\venv\Lib\site-packages\poetry\installation\chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pyqt5 (5.15.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyqt5 (==5.15.2)"'.

Summary of the Issue:
Currently, I am able to successfully install pyqt5 using pip, but I encounter issues when attempting to configure the same via poetry. This leads me to ponder whether the root cause of this problem lies in my specific configuration, the version of Python I am using, or some unknown aspects of poetry settings that I might not be aware of. I am looking for insights or solutions to effectively resolve this discrepancy between pip and poetry.
Additional Context:

pip install pyqt5 works without any issues.
poetry add pyqt5 or poetry update leads to errors, specifically in finding installation candidates or in building the wheel for pyqt5.
My environment uses Python 3.11, and I have tried clearing poetry's cache and directly specifying the version in pyproject.toml.
The issue persists regardless of the version of pyqt5 specified in pyproject.toml.

I have checked relevant documentation and other similar questions but haven't found a solution.
Has anyone encountered a similar issue, or does anyone have any suggestions on how to resolve it?
Thank you!
What I Tried:
Using pip: I successfully installed pyqt5 using pip without encountering any issues.
Using poetry:
I attempted to add pyqt5 using poetry add pyqt5, which resulted in errors related to finding installation candidates.
Running poetry update also led to similar issues, specifically errors in building the wheel for pyqt5.
I tried specifying different versions of pyqt5 in pyproject.toml, but the issue persisted.
Other Attempts:
Cleared Poetry's cache using poetry cache clear --all pypi.
Checked and confirmed network settings and proxy configurations.
What I Was Expecting:
I was expecting poetry to handle the installation of pyqt5 as smoothly as pip does. Specifically, I anticipated that poetry add or poetry update would successfully resolve and install the pyqt5 dependency without encountering errors, especially since the installation via pip was successful and straightforward.
Answer: I solved installing the module using pip through poetry, i.e:
poetry run pip install pyqt5-qt5


Is it possible to be integrate Alexa Skills Kit with iOS mobile app to trigger a event?

Question: For example, say I want to launch camera within my ios app to take a photo, Can I utilize ASK and iphone microphone to understand users speech command("launch camera") to launch camera and trigger a function within ios?
Comment: You're better off using something like [pocketsphinx](https://github.com/cmusphinx/pocketsphinx) for basic command and control speech recognition. What you mentioned is not really ideal with Alexa since you have to make a lot of roundtrips.
Answer: Short answer: No.
Long answer: Yes, but not really a good us case.
The Alexa Skills Kit is used to create an endpoint (skill) that the Alexa service can invoke to do something. The Alexa Skills Kit does not cover the acquisition and recognition of the speech. That all happens within the Alexa service.
The Alexa Voice Kit is a newly released kit that lets you do the other end. You can use it to integrate anything with a microphone into the Alexa service. Using this kit you can stream raw sound recorded from your microphone up to the service, and it will do the recognition and invocation of services.
So, in theory, you could use the Alexa Voice Kit and enable your iphone as a device that could send sound and invoke Alexa skills. You could use the Alexa Skill Kit to enable a web service that can be invoked by Alexa. You could do some sort of push/pull notification between your web service and your phone to trigger whatever you want. Then you would be able to invoke your Alexa skill with your iPhone, and have it do something on your iPhone.
Probably not where you really want to go. The better way to implement that use case is to use whatever speech recognition and trigger options iOS gives you and not via Alexa.


Enviar posts de sites Wordpress parceiros para meu site

Question: Boa noite.
Preciso desenvolver um plugin para Wordpress que funciona pegando o post publicado no site onde o plugin está instalado (que é de um site parceiro que fornece conteúdo) e enviando os campos do post (título, imagem de destaque, autor, conteúdo, tags...) para outro site Wordpress, em outro provedor/banco de dados, que será usado para alimentar um aplicativo para celular. Tem como fazer isso? Já existe algum plugin ou função que faça algo parecido ou tenho que começar do zero? Desenvolver um plugin para fazer isso é a melhor solução, ou existem outras?
Obrigada,
Mey
Answer: Existe uma tabela no banco de dados que guarda os posts (não lembro o nome da tabela, mas acho que é algumacoisaPosts). Basta passar esses dados pro outro banco...
Comment: E é simples fazer isso? Desculpa a ignorância, mas é a primeira vez que vou fazer isso, e estou aprendendo php agora.
Comment: você usa phpMyAdmin?
Comment: Sim, uso phpMyAdmin
Comment: vê se você acha essa tabela posts, depois abre ela e clica na aba "exportar", ai ele vai fazer download do arquivo .sql com todos dados da sua tabela.
Comment: Entendi, mas no meu caso, eu não tenho acesso direto ao banco de dados onde os posts foram criados, pois são de outros sites parceiros que vão fornecer o conteúdo para minha aplicação. Por isso eu precisava de um plugin que esses parceiros instalariam e, cada vez que eles publicasse um novo post, este iria para o meu banco de dados.
A minha dúvida é qual seria a melhor forma de fazer isso, se é criando um plugin, ou widget. Ou se já existe algum plugin que já faça isso. E qual o método/função que tenho que usar pra fazer isso.
Comment: Não sei se existe plugins, aconselho dar uma pesquisada..
se não achar, existe a possibilidade de eles fazerem uma página que fornece as informações dos posts (ex: posts.php) e você acessar essa página, pegar as informações e gravar no seu banco..


php post as side by side

Question: in my project dynamically creating list of 's as heading category and its images.like

Heading 1

Image 1

Image 2

Image 3

Image 4

Image 5

Heading 2

Image 1

Image 2

Image 3

Heading 3

Image 1

Each heading is a category and image names are images under each category.
Now its coming one by one and i want to display it as side by side as shown in image.

Like in image one heading and 2 images below and next li side to heading.if li under one category finises next heading comes to new column.
How can i create a table like in image using php. each heading and and its images.
Comment: use `table` element you get like this
Comment: yes but how can i contol li as new when list under one category completed
Comment: why should you use `li` ?
Comment: the images are dynamic or static ? whatever use grid like table no bother about 'li` also no need
Comment: images and category names all are dynamic..i want to show in a slider so want to display as li's
Comment: `slider` ??? or table format ?? i am confusing
Comment: how can i implement using table
Answer: You can use Table then the html:

        Heading 1
        Image 3
        Heading 2
        Image 3
        Heading 3

        Image 1
        Image 4
        Image 1
        Image 4
        Image 1

        Image 2
        Image 5
        Image 2

This is a simple trick: example
<?php
    $images = array('Image 1', 'Image 2', 'Image 3', 'Image 4', 'Image 5');
    $fixRow = 3;  //total fixed row that you want

    $L1 = 5;/*List-1 item*/ $L1C = ceil(($L1+1)/$fixRow);// 2 row for 1st list
    $L2 = 3;/*List-2 item*/ $L2C = ceil(($L2+1)/$fixRow);//2 row for 2nd list
    $L3 = 1;/*List-3 item*/ $L3C = ceil(($L3+1)/$fixRow);//1 row for 3rd list

?>

        <?php
            //list-1 column=2
            $k=0;
            for ($i=0; $i<$L1C ; $i++){
                if($i==0){
                    echo "Heading 1	";
                }else{
                    echo "".$images[$k-1]."	";
                }
                $k = $k+$fixRow;
            }

            //list-2 column=2
            $k=0;
            for ($i=0; $i<$L2C ; $i++){
                if($i==0){
                    echo "Heading 2	";
                }else{
                    echo "".$images[$k-1]."	";
                }
                $k = $k+$fixRow;
            }

            //list-3 column=1
            $k=0;
            for ($i=0; $i<$L3C ; $i++){
                if($i==0){
                    echo "Heading 3	";
                }else{
                    echo "".$images[$k-1]."	";
                }
                $k = $k+$fixRow;
            }
        ?>

    <?php for ($j=0; $j <2 ; $j++){
        echo "
";
            $k=$j;
            for ($i=0; $i<$L1C ; $i++) {
                if($i==0){
                    if($j<$L1){
                        echo "".$images[$j]."	";
                    }else{
                        echo "	";
                    }
                }else{
                    if($k<$L1){
                        echo "".$images[$k]."	";
                    }else{
                        echo "	";
                    }
                }
                $k = $k+$fixRow;
            }

            $k=$j;
            for ($i=0; $i<$L2C ; $i++) {
                if($i==0){
                    if($j<$L2){
                        echo "".$images[$j]."	";
                    }else{
                        echo "	";
                    }
                }else{
                    if($k<$L2){
                        echo "".$images[$k]."	";
                    }else{
                        echo "	";
                    }
                }

                $k = $k+$fixRow;
            }

            $k=$j;
            for ($i=0; $i<$L3C ; $i++) {
                if($i==0){
                    if($j<$L3){
                        echo "".$images[$j]."	";
                    }else{
                        echo "	";
                    }
                }else{
                    if($k<$L3){
                        echo "".$images[$k]."	";
                    }else{
                        echo "	";
                    }
                }
                $k = $k+$fixRow;
            }
        echo "
";
    } ?>

You can modify by using common function
Comment: how can i restrict only 3 rows
Comment: http://stackoverflow.com/questions/9821103/dynamic-create-rows-and-colum-with-the-help-of-php-and-html

php post as side by side


why is lmer splitting up my factor into all its levels in the results?

Question: I am running a linear mixed model for a single response variable on a predictor root.type that has  4 levels; when I run the model, I just want the information on the entire factor, but it keeps splitting it up into the levels. Any ideas?
Ca.auto <- lmer(Ca ~ root.type + (1|pot), data)
summary(Ca.auto)

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: Ca ~ root.type + (1 | pot)
   Data: autotroph

REML criterion at convergence: -17.5

Scaled residuals:
    Min      1Q  Median      3Q     Max
-2.2204 -0.4872 -0.1147  0.4371  3.6250

Random effects:
 Groups   Name        Variance Std.Dev.
 pot      (Intercept) 0.00000  0.0000
 Residual             0.02884  0.1698
Number of obs: 42, groups:  pot, 12

Fixed effects:
                               Estimate Std. Error       df t value
(Intercept)                     1.62108    0.05120 38.00000  31.660
root.typeunparasitized host    -0.99282    0.07241 38.00000 -13.711
root.typeattached hemiparasite -0.57593    0.07420 38.00000  -7.762
root.typeparasitized host      -0.97373    0.07420 38.00000 -13.123
                               Pr(>|t|)
(Intercept)                     < 2e-16 ***
root.typeunparasitized host    2.72e-16 ***
root.typeattached hemiparasite 2.35e-09 ***
root.typeparasitized host      1.09e-15 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr) rt.typnh rt.typth
rt.typnprsh -0.707
rt.typttchh -0.690  0.488
rt.typprsth -0.690  0.488    0.476
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular

I was expecting just the one fixed effect (root.type). I updated the packages, but nothing changed.
Answer: This is fairly standard behaviour for R modeling packages (i.e., the summary() function reports parameter-level information). Some options for getting the term-level results are: car::Anova(), afex::mixed(); since you're using lmerTest rather than lme4, anova() and drop1() will also work.
For the particular example you show above,

since you have 38 df, the finite-size correction is pretty small (e.g. 2.5% upper tail is at qt(0.975, 38)==2.02 rather than 1.96)
your termwise result for root will likely be reported as < 2e-16, since the p-values for the individual contrasts are all already quite small ...


Passing Control's Value to Modal Popup

Question: Just would like know how to pass textbox value to a modal popup after clicking a button using ModalPopUpExtender in ASP.NET, I've tried these codes but seems that  I have no luck :(

    <asp:ScriptManager ID="ScriptManager1" runat="server">
    </asp:ScriptManager>
    <asp:TextBox ID="TextBox1" runat="server"></asp:TextBox>
    <asp:Button ID="Button1" runat="server" Text="Button" OnClick='showModalPopup(); return false;' />

    <cc1:ModalPopupExtender ID="ModalPopupExtender1" runat="server" TargetControlID="Button1"
        PopupControlID="Panel1" CancelControlID="btnCancel" OkControlID="btnOkay" BackgroundCssClass="ModalPopupBG">
    </cc1:ModalPopupExtender>
    <asp:Panel ID="Panel1" Style="display: none" runat="server">

                Header

                <asp:Label ID="Label1" runat="server"></asp:Label>

                <input id="btnOkay" type="button" value="Done" />
                <input id="btnCancel" type="button" value="Cancel" />

    </asp:Panel>

javascript
 function showModalPopup() {
        //show the ModalPopupExtender
        var value;
        value = document.getElementById("TextBox1").value;
        $get("<%=Label1.ClientID %>").value = value;
        $find("<%=ModalPopupExtender1.ClientID %>").show();

    }

I wonder what I miss out :(, Thanks and I hope someone could help me :)
Answer: use
value = document.getElementById('<%=TextBox1.ClientID %>').value;
instead of
value = document.getElementById("TextBox1").value;

Passing Control's Value to Modal Popup


Show that $\int_a^b f(u) \int_a^u f(v) dv du=\int_a^b f(u) \int_u^b f(v) dv du$

Question: Let $f:[a,b]\rightarrow \mathbb{R}$ be integrable on $[a,b]$. Show that ($\forall u\in[a,b]$)
$$\int_a^b f(u)\int_a^u f(v) dv\; du=\int_a^b f(u)\int_u^b f(v) dv\; du$$
What i've done so far: I suppose $\int f$ has $F$ as an antiderivative (which is not guaranted by $f$ being just integrable, but i can't think other way). So,
$$\int_a^b f(u)\int_a^u f(v) dv\; du=\int_a^b F(u)f(u)\;du-F(a)\int_a^b f(u) du=\int_a^b F(u)f(u)\;du+F(a)^2-F(a)F(b)$$
$$\int_a^b f(u)\int_u^b f(v) dv\; du=F(b)\int_a^b f(u)\;du-\int_a^b F(u)f(u) du=
F(b)^2-F(a)F(b)-\int_a^b F(u)f(u)\;du$$
In terms of areas, for let´s say $f(x)=2x$ with $[-2,3]$, and $F(x)=x^2$ what we're integrating from $-2$ to $3$ is: $$(F(u)-F(a))f(u)=
2(u^2-(-2)^2)u=2(u^2-4)u$$ and
$$(F(b)-F(u))f(u)=2(9-u^2)u$$
If i graph these from $-2$ to $3$, they don't seem related, but the area under both curves is the same.
Can someone give me an approach. ¿Is this a known result? Thanks
Comment: Assuming that response is to me, you should go back to the beginning, don't suppose $f$ has an antiderivative, and use Fubini's theorem.  (As an additional hint, it may help if you swap $u$ and $v$ on one side of the equation)
Comment: hint $\int Ff=\int FF'=[\frac 12F^2]$
Comment: Can you use Fubini's theorem?
Comment: Yes, i saw that one in class
Comment: This looks incorrect to me. It's supposed to be an exercise in changing the order of integration, but it's just wrong.
Answer: For $ u\in\left[a,b\right] $ define a function $ f_{u}:\left[a,b\right]\rightarrow\mathbb{R} $ as following : $ f_{u}\left(v\right)=\left\lbrace\begin{aligned}f\left(u\right)f\left(v\right),\ \ \ \ &\textrm{If }a\leq v\leq u\\ 0\ \ \ \ \ \ \ \ ,\ \  \ \ &\textrm{If }u\leq v\leq b\end{aligned}\right. \cdot $
Fubini's theorem allows us to write the following : $$ \int_{a}^{b}{\int_{a}^{b}{f_{u}\left(v\right)\mathrm{d}v}\,\mathrm{d}u}=\int_{a}^{b}{\int_{a}^{b}{f_{u}\left(v\right)\mathrm{d}u}\,\mathrm{d}v} $$
Since the left side can be written as : $ \int\limits_{a}^{b}{\int\limits_{a}^{b}{f_{u}\left(v\right)\mathrm{d}v}\,\mathrm{d}u}=\int\limits_{a}^{b}{\int\limits_{a}^{u}{f\left(u\right)f\left(v\right)\mathrm{d}v}\,\mathrm{d}u}=\int\limits_{a}^{b}{f\left(u\right)\int\limits_{a}^{u}{f\left(v\right)\mathrm{d}v}\,\mathrm{d}u} \cdot $
And the right side can be written as : $ \int\limits_{a}^{b}{\int\limits_{a}^{b}{f_{u}\left(v\right)\mathrm{d}u}\,\mathrm{d}v}=\int\limits_{a}^{b}{\int\limits_{v}^{b}{f\left(u\right)f\left(v\right)\mathrm{d}u}\,\mathrm{d}v}=\int\limits_{a}^{b}{f\left(v\right)\int\limits_{v}^{b}{f\left(u\right)\mathrm{d}u}\,\mathrm{d}v} \cdot $
We have : $$ \int_{a}^{b}{f\left(u\right)\int_{a}^{u}{f\left(v\right)\mathrm{d}v}\,\mathrm{d}u}=\int_{a}^{b}{f\left(v\right)\int_{v}^{b}{f\left(u\right)\mathrm{d}u}\,\mathrm{d}v} $$
We can substitute $ \left(u,v\right) $ to $ \left(v,u\right) $ in the right side to get : $$ \int_{a}^{b}{f\left(u\right)\int_{a}^{u}{f\left(v\right)\mathrm{d}v}\,\mathrm{d}u}=\int_{a}^{b}{f\left(u\right)\int_{u}^{b}{f\left(v\right)\mathrm{d}v}\,\mathrm{d}u} $$
Answer: You are on the right track (with the assumption of antiderivative existing, as stated in the comments).
$$\int_a^b f(u)\int_a^u f(v) dv \;du=  \int_a^b f(u)(F(u)-F(a)) du = \int_a^b f(u)F(u) du - F(a)(F(b)-F(a))$$
$$\int_a^b f(u)\int_u^b f(v) dv \;du = \int_a^b f(u)(F(b)-F(u)) du = -\int_a^b f(u)F(u) du +F(b)(F(b)-F(a))$$
All that's left to show is:
$$ \int_a^b f(u)F(u) du = 1/2\left( F(b)^2 - F(a)^2\right) $$
Remember that $f=F^\prime$.
Answer: The left hand integral is of the form
$$
I=\int_{u=a}^{u=b}\int_{v=a}^{v=u}g(u,v)dvdu
$$
It has a region of integration given by $a\leq u \leq b$ and $a \leq v \leq u$. This region of integration can be equivalently written as $a \leq v \leq b$ and $v \leq u \leq b$ so a direct application of Fubini's theorem gives
$$
I=\int_{v=a}^{v=b}\int_{u=v}^{u=b}g(u,v)dudv
$$
If the integrand satisfies $g(u,v)=g(v,u)$ for all $u,v$ in the domain of integration then we can swap variables by $u\to v$ and $v\to u$ to obtain
$$
I=\int_{u=a}^{u=b}\int_{v=u}^{v=b}g(v,u)dvdu=\int_{u=a}^{u=b}\int_{v=u}^{v=b}g(u,v)dvdu
$$
The given integrand $g(u,v)=f(u)f(v)$ is one example of such an invariant integrand. Some more would be $g(u,v)=u^2+v^2$ and $g(u,v)=\sin uv+e^{u+v}+u^4+v^4$, and so on. So, the result is more general than for integrands of the form $g(u,v)=f(u)f(v)$.


Loading large CSV into Postgres

Question: I'm trying to load a CSV of about 100M records (around 8GB on disk) into Postgres via the copy command: copy mytable from 'path/to/myfile.csv' with CSV; I have been monitoring the progress by checking the reported table size in pgAdmin and comparing it with the CSV size. I know that's going to be a loose comparison at best, and I'd love to hear if there's a better way to monitor progress.
Here's the issue: this load has been going on for quite a long time (too long, I think) and as I keep checking the table size, the loading seems to be decelerating. That is, it takes much longer now to load in a new 100MB of data than it did earlier on in the load. Why?
Is there any tuning, configuration, or alternate approach I can take for a faster load other than breaking up my CSV into many smaller files?

Update: schema/data specifics
One representative data row:
1234567890,FOOBARF,2010-01-15 03:07:05,0.924700,0.925000

Complete schema definition:
CREATE TABLE mytable
(
  id integer NOT NULL,
  rname character varying(7) NOT NULL,
  ts timestamp without time zone NOT NULL,
  stat1 numeric NOT NULL,
  stat2 numeric NOT NULL,
  CONSTRAINT pk_id PRIMARY KEY (id)
)
WITH (
  OIDS=FALSE
);
ALTER TABLE mytable OWNER TO postgres;

Answer: You might have better luck if you can disable indexes.  But this is not a good answer as you did not provide enough information about the table.
Please post table defn, constraints, indexes!!!, and triggers.
Also, are you [make] sure the CSV data is correct and matches your table?
Comment: Removing the PK did the trick. I guess the issue is that the PK needed to ensure uniqueness, and therefore as the list grew it had more and more old records to check to ensure each new one was unique. The result: every subsequent records takes a bit longer to insert than the previous one.
Comment: See edits on post. There are no indexes (yet; I was planning on adding after I got the data in).
Comment: I've run tests:  (first: you do clearly have an index on id)
I created a 10M row file with your data incrementing only the ID.
On very nice server it took 1:40s to load.
Then I dropped you index and it took 55s to load.

Loading large CSV into Postgres


theme_table override not working

Question: I am trying to override / hook theme_table from theme.inc.
My theme name is stat.
I have added following function to my theme template.php.
function stat_table($variables) {
  //table logic here
  return $output;
}

I have dev env configured with xdebug and while function is 'loaded'/processed when displaying page with table, it is never called.
Original theme_table is called instead.
I already tried clearing cache via drupal_theme_rebuild();
Using Drupal 7.
Is there anything I am doing wrong?
How can I debug why is the function not called?
Answer: use hook_theme_registry_alter.
Example:
function MYMODULE_theme_registry_alter(&$theme_registry) {
  $theme_registry['table']['function'] = '_MYMODULE_my_table_theme';
}
function _MYMODULE_my_table_theme($variables) {
...
...
}
Comment: I do not have my own module, I am using theme.
This seems to only work for modules.

theme_table override not working


Achieving numpy like fast interpolation in Fortran

Question: I have a numerical routine that I need to run to solve a certain equation, which contains a few nested four loops. I initially wrote this routine into Python, using numba.jit to achieve an acceptable performance. For large system sizes however, this method becomes quite slow, so I have been rewriting the routine into Fortran hoping to achieve a speed-up. However I have found that my Fortran version is much slower than the first version in Python, by a factor of 2-3.
I believe the bottleneck is a linear interpolation function that is called at each innermost loop. In the Python implementation I use numpy.interp, which seems to be pretty fast when combined with numba.jit. In Fortran I wrote my own interpolation function, which reads,
  complex*16 function interp(x_dat, y_dat, N_dat, x)
    implicit none
    integer, intent(in) :: N_dat
    real*8, dimension(N_dat), intent(in) :: x_dat
    complex*16, dimension(N_dat), intent(in) :: y_dat
    real*8, intent(in) :: x

    complex*16 :: y, y1, y2
    integer :: i, i1, i2, im

    if(x <= x_dat(1)) then
      y = y_dat(1)
    else if(x >= x_dat(N_dat)) then
      y = y_dat(N_dat)
    else
      im = MINLOC(DABS(x_dat - x), DIM=1)
      if(x_dat(im) >=x ) then
        i1 = im
        i2 = im - 1
      else
        i1 = im + 1
        i2 = im
      end if

      y1 = y_dat(i1)
      y2 = y_dat(i2)

      y = y1 + (x-x_dat(i1))*(y2 - y1)/(x_dat(i2) - x_dat(i1))
    end if
    interp = y
    return
  end function interp

Note that I need to interpolate complex data. If my diagnostics are correct this function is much slower than numpy.interp, which, since the interpolation has to be called in each loop, greatly reduces the speed of the whole program.
Does anyone know if there is a way to achieve Numpy like interpolation speeds in Fortran? Or if my interpolation function as shown above here is somehow horribly inefficient? I don't have much experience coding Fortran yet.
Thanks!
Comment: 1) Have you profiled this so you know which line(s) are taking the most time? 2) How big is n_dat? 3) Is x_dat sorted? 4) Can you please, provide a complete program, without that it is much more difficult to say anything 5) Note `DABS` should not have been used since 1977, use `ABS` 6) Note complex*16 is not standard Fortran, and has never been standard Fortran  - learn about kinds https://stackoverflow.com/questions/838310/fortran-90-kind-parameter to do it properly
Answer: At a guess (and see @IanBush's comments if you want to enable us to do better than guessing), it's the line
im = MINLOC(DABS(x_dat - x), DIM=1)

which is taking all your time, as this line is O(N) in the size of x_dat, where everything else is O(1).
If x_dat is linearly spaced then you can replace this line with
im = 1 + nint((N_dat-1)*(x-x_dat(1))/(x_dat(N_dat)-x_dat(1)))

or better yet, skip im entirely and calculate i1 and i2 as
i1 = 1 + floor((N_dat-1)*(x-x_dat(1))/(x_dat(N_dat)-x_dat(1)))
i2 = i1 + 1

If x_dat is not linearly spaced, but has other useful properties, then you want to use these properties to calculate im if possible.
Comment: My data is indeed linearly spaced, silly that I didn't think of this immediately. Thanks a bunch!


Which groups have non-trivial cyclic quotient groups?

Question: Under what conditions can it be determined whether or not any group G has some subgroup N such that the quotient group G/N is a cyclic group? In other words, how can it be determined whether or not a group has a non-trivial cyclic quotient group?
I know that for any group G, if there is a group N of order g/p where g is the order of G and p is the smallest prime factor of the order of that group, then the group N will be normal and G/N will be a prime order cyclic group. Does the group N always exist for a finite group? Under what conditions does it exist?
What about for infinite groups? In what cases can we say they have a subgroup N such that G/N is a cyclic group?
My motivation in asking this question is that for any cellular automaton C on a group G, C can simulate a simpler cellular automaton if every element in each coset of G is the same. For example, if G is the group of a two dimensional grid, which corresponds to the free abelian group on two generators, making every element in each column have the same state causes the columns, which are cosets of G, to act as a one dimensional cellular automaton. The group of this "factor" cellular automaton is N, the free abelian group on one generator.
I am curious to know under what conditions any group could simulate a 1-dimensional cellular automaton, hence, under what conditions it has a cyclic quotient group.
Comment: ...and there are infinite non-abelian simple groups. For instance, the alternating group of the positive integers (permutations with finite support and signature 1).
Comment: There are finite simple groups which are not cyclic like $A_5$. You need them to be Abelian for your claim to work.
Comment: This question would benefit from added context, as it is slightly odd and confused. (You seem to think maximal subgroups are normal?) For example, simply telling us where you got the problem from or explaining what you tried would be a big step forward! (For further feedback/help with asking questions, you can ask [here](https://chat.stackexchange.com/rooms/43593/constructive-feedback).)
Comment: While every subgroup of prime index is maximal, not every maximal subgroup is of prime index; and not every maximal subgroup is normal. There are finite groups that have no nontrivial cyclic quotient.
Comment: **If** there is a subgroup whose index is the smallest prime that divides the order of the group, then that subgroup is normal. However, not all maximal subgroups have orders like that, and not every finite group has a subgroup whose index is the smallest prime that divides the order of the group: $A_5$ has order $60$, but no subgroup of order $30$ (index $2$, the smallest prime that divides $60$). And $A_5$ has a maximal subgroup of order $6$ (index $10$).
Comment: It is not true that for a finite group, the quotient of the whole group by **a** maximal subgroup is a prime cyclic group. I'm assuming you mean a maximal *normal* subgroup, as otherwise 'the quotient' doesn't even make sense.
Comment: I'm sorry. That was sloppy language. By maximal group I mean the subgroup with the largest order in the group. I got this from the Wikipedia article which says, "Furthermore, if p is the smallest prime number dividing the order of a finite group, G, then if G/H has order p, H must be a normal subgroup of G." https://en.wikipedia.org/wiki/Quotient_group
Comment: @Electro-blob Can you edit this argument into the question (and maybe remove your "Edit:...")? It think then the question can be reopened. I know you now understand the errors, but Arturo's answer pointing them out is quite nice and might help other people in the future.
Comment: @User1729 I tried to improve it.
Answer: Both the question and your answer contain several misconceptions/errors.

Your question seems to assert that every finite group has a maximal normal subgroup of prime index. While every subgroup of prime index is necessarily maximal (this follows from the multiplicativity of the index, if $H\lt K \lt G$ then $[G:H]=[G:K][K:H]$), it is not true that every such group is necessarily normal, and it is not true that every maximal subgroup has prime index. For example, $A_5$ has a maximal subgroup of index $5$ (the subgroup fixing $5$, say), but it is not normal; and it has a maximal subgroup of order $6$ and index $10$, generated by $(123)$ and $(12)(45)$.

The statement "if you divide the order of a finite group by the smallest prime possible, that any subgroup will be normal and of prime cyclic order" is hopelessly garbled. On the basis of your comment on the post, you tried to say that if the index of a subgroup is the smallest prime that divides the order of the group then that subgroup will be normal and the quotient cyclic of prime order. That is true, but not every finite group has such a subgroup (see above: $A_5$ has no subgroup of index $2$).

The statement in the answer that "any group not composed of cyclic simple groups will not have any cyclic simple quotient" is false if you mean "a group in which not every composition factor is a cyclic simple group". It is false for $\mathbb{Z}$, for example; and for finite groups, it is false for $S_5$ (which has a quotient isomorphic to $C_2$) and more generally, any group of the form $A\times B$ where $A$ is finite abelian and $B$ is nonabelian simple. If you mean "for which every composition factor is not cyclic simple group", then that is true for finite groups but your phrasing is extremely confusing.

To your questions:
Theorem. A finite group $G$ has a quotient that is nontrivial cyclic if and only if the commutator subgroup is a proper subgroup, $G\neq [G,G]$.
Proof. A nontrivial finite abelian group has nontrivial cyclic quotients, hence nontrivial cyclic quotients of prime order. If $G=[G,G]$, then the only abelian quotient of $G$ is trivial so $G$ cannot have nontrivial cyclic quotients. If $G\neq[G,G]$, then $G/[G,G]$ is finite abelian and hence has itself a quotient which is cyclic of prime order, and therefore so does $G$ by the isomorphism theorems. $\Box$
For infinite groups the question is harder. There are groups (even abelian groups) with no subgroups of finite index. For example, the Prüfer $p$-group $\mathbb{Z}_{p^{\infty}}$ is infinite, but all proper subgroups are finite. And the additive group of rationals $\mathbb{Q}$ does not have any maximal subgroups, and in particular cannot have a subgroup of finite index. The quotients of $\mathbb{Z}_{p^{\infty}}$ are either trivial or isomorphic to $\mathbb{Z}_{p^{\infty}}$ itself, which is quasicyclic but not cyclic. No nontrivial quotient of $\mathbb{Q}$ can be nontrivial and cyclic (even infinite cyclic) because $\mathbb{Q}$ is divisible, hence so is every quotient, and no nontrivial cyclic group is divisible. Both of these groups are abelian, so the commutator test certainly does not work in general.
It does work for finitely generated groups:
Theorem. Let $G$ be a finitely generated group. Then $G$ has a quotient that is nontrivial cyclic if and only if $[G,G]\neq G$.
Proof. It follows as above once you verify that a finitely generated abelian group has such quotients, and note that a quotient of a finitely generated group is finitely generated. $\Box$
Comment: On point 3 I did mean the latter, but it definitely was confusing. I didn't really ask the question well but you answered it well anyways.


MapBox for iOS, custom style

Question: I created new map style with Mapbox Studio, then I uploaded it to my account (so I see it in styles tab), then I added StyleID to my iOS MGLMapView
but when I run app it throws error
<code>[ERROR] {Map}[Setup]: loading style failed: HTTP status code 404
</code>
If I don't use my style, map is working correctly.
Comment: Was anyone able to figure out this issue?
Comment: This is frustrating. I was under the impression you could style your own map, upload it to your account, and then pull it into an app.
Answer: Try this in viewDidload:
<code>mapView.styleURL = NSURL(string: "asset://styles/dark-v7.json")
</code>
Update 1:
For this moment it is not possible to use custom map style with Mapbox GL for iOS.
'First steps with Mapbox GL for iOS':
Changing the map style
At the moment, Mapbox GL comes bundled with a handful of map styles. That'll all change in a few months, but for now let's change the map style to Mapbox Dark...
Comment: Seems, you need to wait :
"At the moment, Mapbox GL comes bundled with a handful of map styles. That'll all change in a few months, but for now let's change the map style to Mapbox Dark"

[source](https://www.mapbox.com/guides/first-steps-gl-ios/)
Comment: I see; would you recommend using the previous obj-C versions of Mapbox, I ask because the custom style is somewhat a deal breaker and the option to wait a few months would not fly well with the client
Comment: Sorry @kdgwill,  but I don't have an experience with old MapBox SDK for iOS. I  switched a few weeks ago from MapKit to MapBox due designer new point of view in our app, but fortunately, the "dark-v7" that what he wants. Try to contact with MapBox team about custom styles availability for iOS.
Comment: "Very soon, you'll be able to easily create your own custom GL styles and have them hosted by Mapbox. _In the meantime, it's necessary to edit styles by hand, following the GL Style Spec. Our GL styles can be shared between native apps and Mapbox GL JS web apps._ See the bundled styles for examples of how to get started and of powerful dynamic styling techniques." Anyone have any idea how to do this? I've adjusted the CSS in Mapbox Studio but how can it be done so in app?

MapBox for iOS, custom style


How to access already provided rest api in magento

Question: I am a completely new in Magento. I can say that i am not much familiar with file structure and flow also. My current task is rest web-service.
These steps i have followed.

I have created role and user in system > webservice. and also created key and secret key.
Than i am hitting this url
<code>www.my-magento-store.com/api/rest/products
</code>

But still its showing me 404 Page not found. Any one can tell me step by step process with folder structure, how can i use already provided rest api service and how can i create new service. Treat me as fresher.
Please excuse me for bad English. Thank You.
Answer: <code>$proxy = new SoapClient('http://magentohost/api/soap/?wsdl');
$sessionId = $proxy->login('apiUser', 'apiKey');
$allCategories = $proxy->call($sessionId, 'category.tree'); // Get all categories.
</code>
Run this from your root of magento you will get the result.


How to stop auto sliding of react-bootstrap carousel

Question: By installing "npm i react-bootstrap". I added Below code in html. I have used interval="false" but it was not working. In bootstrap it working but not working in react "react-bootstrap". Please help me out
function ControlledCarousel() {
   const [index, setIndex] = useState(0);

  const handleSelect = (selectedIndex, e) => {
    setIndex(selectedIndex);
  };*strong text*

  return (
    <Carousel activeIndex={index} onSelect={handleSelect}>
      <Carousel.Item>

        <Carousel.Caption>

First slide label

Nulla vitae elit libero, a pharetra augue mollis interdum.

        </Carousel.Caption>
      </Carousel.Item>
      <Carousel.Item>

        <Carousel.Caption>

Second slide label

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

        </Carousel.Caption>
      </Carousel.Item>
      <Carousel.Item>

        <Carousel.Caption>

Third slide label

            Praesent commodo cursus magna, vel scelerisque nisl consectetur.

        </Carousel.Caption>
      </Carousel.Item>
    </Carousel>
  );
}

Answer: Using React-Bootstrap version 1 I had to set interval={null} as with 0 it was sliding non-stop.
Answer: You need to pass interval={0} to Carousel.
Answer: You need to pass interval as 0. This can be done via interval prop, by setting as below. The default value is '5000'
<Carousel activeIndex={index} onSelect={handleSelect} interval={0}>


Codeigniter, uploading multiple files with same name

Question: I have an upload form that allows me to add as many files as needed. However when I start trying to upload the files I get an error.
Controller
$this->load->library('upload');
$error = "";
$file  = "";
$this->total_count_of_files = count($_FILES['user_certificates']['name']);
print_r($_FILES['user_certificates']['name']);

for($i=0; $i<$this->total_count_of_files; $i++)
{
    $_FILES['user_certificates']['name'] = $_FILES['user_certificates']['name'][$i];
    $_FILES['user_certificates']['type'] = $_FILES['user_certificates']['type'][$i];
    $_FILES['user_certificates']['tmp_name'] = $_FILES['user_certificates']['tmp_name'][$i];
    $_FILES['user_certificates']['error'] = $_FILES['user_certificates']['error'][$i];
    $_FILES['user_certificates']['size'] = $_FILES['user_certificates']['size'][$i];

    $config['encrypt_name'] = TRUE;
    $config['file_name'] = $_FILES['user_certificates']['name'];
    $config['upload_path'] = './certificate_files/';
    $config['allowed_types'] = 'txt|pdf|doc|docx';
    $config['max_size'] = 0;

    $this->upload->initialize($config);

    if (!$this->upload->do_upload())
    {
        $status = 'error';
        $msg = $this->upload->display_errors();
    }
    else
    {
        $data = $this->upload->data();
        $file = $data['raw_name'] . $data['file_ext'];
    }
    if($file)
    {
        $status = "success";
        $msg = "Image successfully uploaded";
    }
    else
    {
        $status = "error";
        $msg = "Something went wrong when saving the image, please try again.";
    }
}
echo json_encode(array('status' => $status, 'msg' => $msg));
exit();

The print_r($_FILES['user_certificates']['name'])) shows me the files I have added:
Array ( [0] => license.txt [1] => license.txt )

I am totally stuck on how to get the upload to work. Any ideas?
Cheers in advance!
EDIT
If I change this:
if (!$this->upload->do_upload())

to this:
if (!$this->upload->do_upload('user_certificates'))

it works, but only for one file, it doesn't seem to loop round again for some reason
Comment: What's d **error** you are getting???
Comment: The file is not getting uploaded to any folder, and the 'echo json_encode(array('status' => $status, 'msg' => $msg));' shows '{"status":"error","msg":"Something went wrong when saving the file, please try again."}'. So the error is originating from after the config is initialized. Any idea?
Comment: It seems your file is getting uploaded but from where are you setting value for `$data['raw_name']`???
Answer: Your loops seems to be incorrect.
...
$_FILES['user_certificates']['name'] = $_FILES['user_certificates']['name'][$i];
...

these lines overwrite the original $_FILES array, so after completion of first loop, it will not find anything else in the loop because it got overwritten.
Instead you may first store the $_FILES array to a local variable and loop through it to upload one by one.
Comment: As I suggested, you can store the $_FILES in a temp variable before starting the loop, like $temp = $_FILES, and loop through $temp, then inside loop you initialize your $_FILES variable using the temp variable. Try it out. If it doesn't work, let me know.
Comment: Any idea how I would go about fixing that?
Comment: Ah excellent, that worked! Thank you very much, such a little change makes a huge difference!


Is it possible to migrate users with passwords from centos 6 to centos 7?

Question: Is it possible to migrate users with passwords from centos 6 to centos 7? If yes - should it just work by copying passwd/group and shadow entries to a new system?
Answer: Yes. Be careful to only copy your own users, not any system users.


how to remove the warnings in Jtidy in java

Question: I am using Jtidy parser in java.
URL url = new URL("www.yahoo.com");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
InputStream in = conn.getInputStream();
doc = new Tidy().parseDOM(in, null);

when I run this, "doc = new Tidy().parseDOM(in, null);"
  I am getting some warnings as follows:
Tidy (vers 4th August 2000) Parsing "InputStream"
line 140 column 5 - Warning:
 lacks "summary" attribute

InputStream: Doctype given is "-//W3C//DTD XHTML 1.0 Strict//EN"
InputStream: Document content looks like HTML 4.01 Transitional

1 warnings/errors were found!

These warnings are getting displayed automatically on console. But I don't want these
warnings to be displayed on my console after running
doc = new Tidy().parseDOM(in, null);

Please help me,how to do this,how to remove these warnings from console.
Answer: Looking at the Documentation I found a few methods which may do what you want.
There is setShowErrors, setQuiet and setErrout. You may want to try the following:
Tidy tidy = new Tidy();
tidy.setShowErrors(0);
tidy.setQuiet(true);
tidy.setErrout(null);
doc = tidy.parseDOM(in, null);

One of them may be enough already, these were all the options I found. Note that this will simply hide the messages, not do anything about them. There is also setForceOutput to get the output, even if errors were generated.
Answer: If you want to redirect the JTidy warnings to (say) a log4j logger, read this blog entry.
If you simply want them to go away (along with other console output), then use System.setOut() and/or System.setErr() to send the output to a file ... or a black hole.
For JTidy release 8 (or later), the Tidy.setMessageListener(TidyMessageListener) method deals with the messages more gracefully.

Alternatively, you could send a bug report to email@example.com. :-)
Answer: Writer out = new NullWriter();
PrintWriter dummyOut = new PrintWriter(out);
tidy.setErrout(dummyOut);

Answer: Looking at the documentation I found another method that seems a bit nicer to me in this particular case: setShowWarnings(boolean). This method will hide the warnings, but errors will still be thrown.
For more info look here:
http://www.docjar.com/docs/api/org/w3c/tidy/Tidy.html#setShowWarnings(boolean)
Answer: I think this is the nicest solution, based on the answer of Joost:
Tidy tidy = new Tidy();
tidy.setShowErrors(0);
tidy.setShowWarnings(false);
tidy.setQuiet(true);

All three are necessary.


What good is an F-test and is using it as a test for significance of independent variable worth it?

Question: An F-test may lead to the conclusion that a predictor is significant by an F-test low p-value. This is because while an independent variable like smoke (below) is statistically unimportant (high p-value), the F-test has a low p-value suggesting its inclusion in the model. Why not remove smoke all together?
Case in point: Smoking is not significant
Call:
lm(formula = Height ~ Wr.Hnd + Sex + Smoke, data = survey)

Residuals:
     Min       1Q   Median       3Q      Max
-17.4869  -4.7617   0.7604   4.3691  22.1237

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) 137.4056     6.5444  20.996  < 2e-16 ***
Wr.Hnd        1.6042     0.3301   4.860 2.36e-06 ***
SexMale       9.3979     1.2452   7.547 1.51e-12 ***
SmokeNever   -0.0442     2.3135  -0.019    **0.985**
SmokeOccas    1.5267     2.8694   0.532    **0.595**
SmokeRegul    0.9211     2.9290   0.314    **0.753**
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 7.023 on 201 degrees of freedom
  (30 observations deleted due to missingness)
Multiple R-squared:  0.5085,    Adjusted R-squared:  0.4962
F-statistic: 41.59 on 5 and 201 DF,  p-value: < 2.2e-16

Case in point: F-test shows that combine with sex and writing handspan, smoke's inclusion in the model is acceptable.
survmult2 <- lm(Height~Wr.Hnd+Sex+Smoke, data = survey)
R2 <- summary(survmult2)$r.squared
R2

n <- nrow(survey)-30
n

p <- length(coef(survmult2))-1
p

n-p-1

Fstat <- (R2*(n-p-1))/((1-R2)*p)
Fstat
1-pf(Fstat,df1=p,df2=n-p-1)

**[1] 0**  # p-value is way less than <0.05, practically zero

Comment: Could you please explain the title and the question in the first line?  I cannot make sense of them.  And could you explain what a "variance equality" test is?
Comment: I'm with whuber -- I can't follow the premises and the steps in your title and start of your body text and I still can't follow it in your comment just now. Can you *explain* the situation/reasoning/steps in more detail, please. Imagine you have a smart but ignorant person (someone who has less stats than you) that you're trying to get to understand the problem, so you have to explain in simple but clear terms what you're getting at.
Comment: Looking at survmult2, where two of the predictors yielded small p-values (first gray region above), so then too the omnibus F-test suggests strong evidence against null hypothesis. Although the smoking frequency variable itself doesn't appear to contribute anything statistically important (first gray region above), the F-test for that model still suggest survmult2 should be preffered over a no-predictor model, because both handspan and sex are important.Should F-test be used as a test of model acceptance? I've read it has other components: variance equality & independent variable significance.
Comment: Data is random; there is no guarantee that every true null hypothesis will fail to be rejected, nor that every false null hypothesis will be rejected.  That doesn't mean that the tests are useless, just that they aren't oracles.
Comment: Variance (variability) equality--null hypothesis: https://www.youtube.com/watch?v=g9pGHRs-cxc
Comment: "Could you please explain the title and the question in the first line?"

If the F-test were to be use to double-check the significance of an independent variable, which already is verifiable with summary() or its use in variance (variability) equality, it decompensates by potentially wrongfully claiming significance or no signifiance for an independent variable. This would seem to throw off its functionality. I'm not understanding why its being used given the wrong assumption it may give, since it makes it that much worse.
Answer: The model F-test is an omnibus test that tests the hypothesis that all slopes are jointly equal to zero (or, equivalently, whether the population R2 is zero). If any slope is different from zero in the population, then the null hypothesis is false, and, hopefully, you'll get a significant F-statistic to confirm that. Because it's an omnibus test, it doesn't tell you which slope is different from zero. You have to examine the Wald t-tests for each slope to determine whether each is significantly different from zero. So, the omnibus model F-test and the Wald t-tests for each slope tell you different things. They are not redundant. In your data example, there is evidence that the slopes for SexMale and Wr.Hnd are different from zero based on the Wald tests. This is consistent with the model F-test which tests whether any slopes are different from zero.
You may ask why you should use the F-test when you can just look at the Wald tests for the slopes and see if any are significant. When you do this, you implicitly perform many comparisons, and the probability of a type I error increases if the slopes are actually zero in the population. The model F-test doesn't suffer from this; it only performs a single comparison, so its type I error rate is nominal (i.e., .05). The recommended course of action is to first examine whether the F-test is significant or not; if not, don't bother looking at the coefficients; if so, look at the coefficient tests to see which ones are significant. This is in general how all omnibus testing proceeds: start with the broadest test, and only probe further if that test indicates significance.
Note that the F-test you manually computed is the same one given by summary() at the very bottom, so you didn't need to do that (which I think added confusion). Also I think there is a problem with the clarity of your English that makes it a bit hard to figure out what you're asking and why your explanations are not really helping.


Problem on combinatorics solving by using invariance principle

Question: 2015 ones are written in a row..It is allowed to delete any two written numbers a and b and is replaced by a+b/4 the process is continued as long as one number remains or the number is less than .0001.Prove that the last number is greater than .0001
Comment: I don't quite understand; can you give an example of what a valid operation on the $2015$ ones is? Can $a$ and $b$ only be $1$, or can $a = 11$ and $b = 111$, for example?
Comment: @Ross: I wonder if *as long as* should have been *until*, and the operation $(a+b)/4$.
Comment: '2015 ones', i.e. $1111...1$ 2015 times?
Comment: At the end you say "or the number is less than $.0001$".  Does that mean any number less than $0.0001$ is removed, or you stop if any number is less than $0.0001?$.  We already stopped if we were down to one number.
Comment: A trivial answer, even if we can group $1$s into a single number, is that every number on the list is at least $1$.  The operation $a+\frac b4$ results in a number greater than $a$, so greater than $1$.  You will never get below $1$, let alone $0.0001$.  Why does this fail?
Comment: It is one of the question in lativian mathematics site
Comment: Hint given Invariant - - the inverse of all recorded numbers is always less than or equal to 2015
Comment: Stop if number is less than .0001
Answer: Suppose that at some point the sum of the reciprocals is $S$. When you replace $a$ and $b$ with $\frac14(a+b)$, the sum of the reciprocals becomes
$$\begin{align*}
S-\frac1a-\frac1b+\frac4{a+b}&=S-\left(\frac{a+b}{ab}-\frac4{a+b}\right)\\
&=S-\frac{(a+b)^2-4ab}{ab(a+b)}\\
&=S-\frac{(a-b)^2}{ab(a+b)}\\
&\le S\;.
\end{align*}$$
Initially $S=2015$, so throughout the procedure we must have $S\le 2015$, and it follows that the last remaining number is at least $\frac1{2015}>0.0001$.


Search method always null

Question: I'm having issues with a method I've written to search a class called Item. No matter what I search, it is returning null. I believe I'm having issues with variable scope:
public Item search(String itemSearch) {
     Item search = null;
     for(Item i : items){
        if (i.getName() == itemSearch){
        search = i;
        }
     }
     return search;
  }

The getName method returns the name attribute of the item. No matter what the Item search is always null, I'm guessing this is due to variable scope and it is not assigning in the for each loop? Why is this method always null?
Thank you
Comment: Note that after the matching `Item` is found, you should `break;` to exit the loop.
Comment: Or just get rid of the `search` variable altogether and directly return if you find something...
Answer: You can't use the == to compare the content of two strings in java. You need to use the .equals() method
Using the == will only compare the adress of the two strings, while equals will compare their values.
Answer: You are comparing strings using ==. You should instead use equals() method. E.G
i.getName().equals(itemSearch)

Also instead of looping the entire loop use return i in the if statement, instead of assigning i to search and then returning search.

Search method always null


Foil, greatest common factor

Question: I had this on a test and couldn't figure it out... It was written like this:
$12p^2-75$
The answer has to be $(p+\_ ) (p-\_)$
There must be a positive and negative number when combined (addition or subtraction) to equal $12$ but when multiplied to equal $-75$.
Answer: You've greatly garbled the routine for factoring a trinomial $ap^2+bp+c$. The answer does not "have to be" $(p+\dots)(p-\dots)$. In fact it won't be if $a\neq 1$.
If you're trying to factor $ap^2+bp+c$, then you want to find two numbers which when added give you $b$ and when multiplied give you $ac$, and if you are successful you can find a factorization of the trinomial.

In your case, we don't need that process anyway, because there is a simpler path. You should start by factoring 3 out of both (always, always try to find common factors first!):
$12p^2-75=3(4p^2-25)$
Now the idea is to notice that the quantity in parentheses is a difference of squares. The difference of squares always factors this way $X^2-Y^2=(X-Y)(X+Y)$ no matter what $X$ and $Y$ are.
Can you see what $X$ and $Y$ are for $(4p^2-25)$? After you know, you can write $3(X-Y)(X+Y)$ and you will be done.

Foil, greatest common factor


getting scripts to run exclusively

Question: I'm trying to get a bash script to run exclusively -- if another instance of the script is already running, then wait for the other instance to finish before starting the new one.   I found some references to flock which sounds like it should do what I need, but it doesn't seem to be working the way I expect.   I have the following script:
#!/bin/bash

inst=$1
lock=/nobackup/julvr/locks/_tst.lk
exec 200>$lock
flock -x -w30 200 || { echo "$inst: failed flock" && exit 1; }
echo "$inst:got lock"
for i in {1..2}; do
        echo "$inst: $i"
        sleep  1
done
echo "$inst:done script";

And then I run
> flocktest.sh test1 & flocktest.sh test2
[1] 25213
test1:got lock
test1: 1
test2:got lock
test2: 1
test1: 2
test2: 2
test1:done script
test2:done script
[1]+  Done                    flocktest.sh test1

It seems both instances of flocktest are running in parallel...   When does flock release its lock?  How to make it keep the lock until the script is complete?
(An aside, if I do flock -x -w 20 200, then it complains flock: 20: fcntl: Bad file descriptor..., which seems odd, as the man page seems to imply I can add a --w timeout parameter before the lockfile...)
Comment: Works for me. Which OS are you running this on, and what fs is your lock file on?
Comment: The file is on a Linux box (2.6.32-696.18.7.el6.x86_64), and the file is on a mounted directory.
Comment: Ok, got it.   Apperently someone remapped the PATH, and it was grabbing a really old version of flock, which doesn't work the way I'd expect...
Answer: flock seems to me very complicated.
Perhaps you can try this way.
cat script_unique.sh
while test -n "$run_sh"
  do
    sleep 2
  done
  export run_sh="run_sh"
  sleep 2
  echo "$run_sh"
  sleep 4
  echo "$0 $1"
  run_sh=""

Comment: Thanks, but flock is indeed cleaner.   My issue was that I was using an older version of flock, with a different interface than what the man page described.  I'm closing this issue.
Answer: Ok, I found my bug -- I was using a really old version of flock which had a different interface than the one described in the man page.   I updated to a new version of flock, and it worked.

getting scripts to run exclusively


Make  background-color equal array value, but each group of 3 have same value

Question: I have
for (var i = 0; itr < $("td").length; i++) {
    $("td").eq(i).css("background-color", Colors[i]);
}

Which works but it colors every individual td a new color and the row is 3 columns so I need every  in a row (3) have the same color. I cannot color the table row itself, so I need a work around by modifying the above code.
Comment: You want to color every row with a diferent color?
Comment: have you tried colgroup to color down a column?
Comment: I really rather a JS option rather than HTML
Answer: Do you mean this?
var Colors = ["red", "black", "purple", "blue", "gray", "green"];
for (var i = 0; i < $("tr").length; i++) {
    $("tr:eq(" + i + ")").children().css("background-color", Colors[i]);
}

Example here: http://jsfiddle.net/WhnSw/
But why can't you color the row? It is also easy:
for (var i = 0; i < $("tr").length; i++) {
    $("tr:eq(" + i + ")").css("background-color", Colors[i]);
}

Comment: Just about to place that answer! +'d you :)
Comment: @Zuul, I know we have been thinking in the same direction, +'d your comment ;)
Comment: I should of mentioned I am using only td within a specific tr (defined by a class name) as in " tr.specific td " and using your code breaks my page for some reason. the page is data heavy and therefore contains several different table rows which are outside of what I am trying to edit. you can see here http://jsfiddle.net/WhnSw/1/
Comment: It works, but on my page it does not (stuff breaks), even though my first code doesnt break anything....is there any variation I can use perhaps? thank you so much for help
Comment: Could you provide a link to your page? It is hard to give a reason not observing your code.
Comment: Sorry, I cannot, you'd need to sign a NDA then, lol (it's a client)
Comment: Okay I got it to work, sorta, now I need to figure out other things. Thank you for help


Notation for graph weights over vertices *and* edges?

Question: Consider a graph $G=(V,E)$ in which both vertices and edges have a weight. I define it as a function $\omega$ from $V \cup E$ to the weight set.
This seems fine to me, but some colleagues are disturbed by the fact that $\omega$ takes elements from $V$ and $E$. They see them as two different sets, which is correct, but $V \cup E$ also is a set, and it seems to me that $\omega$ may take elements in it.
May you please tell me if my definition is ok? Is it awkward? Is there a better one?
I would like to avoid having two different weight functions, like for instance $\omega$ and $\omega'$ or $\omega_V$ and $\omega_E$. I find them awful in the expressions I deal with, like:
$$\omega(v) \longleftarrow \sum_{u\in N(v)} \frac{\omega(u,v)}{\sum_{w\in N(u)} \omega(u,w)}\cdot\omega(u)$$
Comment: @Matthieu Latapy: To understand what you're trying to do, it would help if you would edit your question to show the actual definition of your proposed weight function, as well as a theorem to illustrate its usefulness.
Comment: It could make sense to "mix" $V$ and $E$, if it wasn't actually $V$ but more like "hyperedges of size one". If the way you use your weighting functions afterwards is coherent with this interpretation, maybe you can make it more natural than two different weight functions.
Comment: "avoid having two different weight functions" saves a little for you at the cost of an awkard interpretative requirement for your prospective readers.
Comment: That's the actual question: is my notation awkward? It seems that some of us think so, which surprises me. I'll edit the question to make it explicit.
Comment: @quasi: Thanks for your suggestions. The definition is simply a function from $V\cup E$ to $\mathbb{R}$, which I then use in many contexts. I added a typical instance of expressions I deal with (although it is among the simplest).
Comment: @Matthieu Latapy: Now that you've provided more context, I withdraw the objection I made in my initial comment. But rather than define the weight function on $V\cup E$, instead use Misha Lavrov's suggested compromise: Define two functions $\omega_V$ and $\omega_E$, and then simply declare that $\omega$ will denote either of them (the number of arguments will make clear which one is intended).
Answer: There are two objections to be made.
First, people think of vertices and edges as different types of things, and want the inputs to a function to always be the same type. This is not a distinction set theory makes, but it's a distinction many people's intuition makes, especially when coming from a CS background.
Second, there are some rare, implausible cases where the definition could have unexpected results. Say your graph has three vertices: $a$, $b$, and $\{a,b\}$. If all three possible edges are present, and you think of an edge as a set of two vertices, then the edges are $\{a,b\}$, $\{a, \{a,b\}\}$, and $\{b, \{a,b\}\}$. You'll notice that the last vertex is the same set as the first edge; so your function $\omega : V \cup E \to \mathbb R$ would have to assign the same weight to that vertex and that edge. Again, it is rare for this issue to come up, and most people would not even consider this behavior - but it is inelegant for the definition to depend on what sort of objects the vertices are, or on how exactly the edges are defined.
Anyway, both of these issues are solved if you define two functions, say $\omega : V \to \mathbb R$ and $\omega' : E \to \mathbb R$. Since you do not like this, one compromise is to say:

The graph has two weight functions: a vertex weight function $V \to \mathbb R$ and an edge weight function $E \to \mathbb R$. In a slight abuse of notation, we will use $\omega$ to denote both functions.

This allows you to keep using the notation you wanted for the rest of your write-up.
Comment: This is interesting, thank you. There is indeed an ambiguity when $E\cap V \neq \emptyset$, and I clearly need to avoid it. Then, the _compromise_ you propose seems sound to me.
Comment: I would relate the first part of your answer to polymorphism, though. It is nowadays common to have programming language functions that take different types as inputs, and this is actually very old: arithmetic operators typically take integers and floats as inputs and then handle them differently. This leads some language to have different operators for integers and floats (like + and +. for instance), while others use the same operator. It seems there is no definitive answer.


How to work around open files limit when demuxing files?

Question: I frequently have large text files (10-100GB decompressed) to demultiplex based on barcodes in each line, where in practice the number of resulting individual files (unique barcodes) is between 1K and 20K.  I've been using awk for this and it accomplishes the task.  However, I've noticed that the rate of demuxing larger files (which correlates with more unique barcodes used) is significantly slower (10-20X).  Checking ulimit -n shows 4096 as the limit on open files per process, so I suspect that the slowdown is due to the overhead of awk being forced to constantly close and reopen files whenever the total number of demuxed files exceeds 4096.
Lacking root access (i.e., the limit is fixed), what kinds of workarounds could be used to circumvent this bottleneck?
I do have a list of all barcodes present in each file, so I've considered forking multiple awk processes where each is assigned a mutually exclusive subset (< 4096) of barcodes to search for.  However, I'm concerned the overhead of having to check each line's barcode for set membership might defeat the gains of not closing files.
Is there a better strategy?
I'm not married to awk, so approaches in other scripting or compiled languages are welcome.

Specific Example
Data Generation (FASTQ with barcodes)
The following generates data similar to what I'm specifically working with.  Each entry consists of 4 lines, where the barcode is an 18 character word using the non-ambiguous DNA alphabet.
1024 unique barcodes | 1 million reads
cat /dev/urandom | tr -dc "ACGT" | fold -w 5 | \
awk '{ print "@batch."NR"_"$0"AAAAAAAAAAAAA_ACGTAC length=1\nA\n+\nI" }' | \
head -n 4000000 > cells.1K.fastq

16384 unique barcodes | 1 million reads
cat /dev/urandom | tr -dc "ACGT" | fold -w 7 | \
awk '{ print "@batch."NR"_"$0"AAAAAAAAAAA_ACGTAC length=1\nA\n+\nI" }' | \
head -n 4000000 > cells.16K.fastq

awk script for demultiplexing
Note that in this case I'm writing to 2 files for each unique barcode.
demux.awk
#!/usr/bin/awk -f
BEGIN {
    if (length(outdir) == 0 || length(prefix) == 0) {
        print "Variables 'outdir' and 'prefix' must be defined!" > "/dev/stderr";
        exit 1;
    }
    print "[INFO] Initiating demuxing..." > "/dev/stderr";
}
{
    if (NR%4 == 1) {
        match($1, /.*_([ACGT]{18})_([ACGTN]{6}).*/, bx);
        print bx[2] >> outdir"/"prefix"."bx[1]".umi";
    }
    print >> outdir"/"prefix"."bx[1]".fastq";

    if (NR%40000 == 0) {
        printf("[INFO] %d reads processed\n", NR/4) > "/dev/stderr";
    }
}
END {
    printf("[INFO] %d total reads processed\n", NR/4) > "/dev/stderr";
}

Usage
awk -v outdir="/tmp/demux1K" -v prefix="batch" -f demux.awk cells.1K.fastq

or similarly for the cells.16K.fastq.
Assuming you're the only one running awk, you can verify the approximate number of open files using
lsof | grep "awk" | wc -l

Observed Behavior
Despite the files being the same size, the one with 16K unique barcodes runs 10X-20X slower than the one with only 1K unique barcodes.
Comment: sorting the file first will eliminate the problem (only one file will be open at a time).  Not sure the gain will be more than the cost of sorting.  Another alternative is some kind of partitioning (cheaper than sort) so that partitions are mutually exclusive (e.g. `C++` partition)
Comment: It's applicable. We don't know what your code is actually doing so we're guessing at what that might be while you're guessing at what's causing the problem and we're guessing at solutions without knowing what your input looks like, what output you need, etc.. All it'd take is a [mcve] to remove most of the guesswork and give us a starting point.
Comment: Post a [mcve] with concise, testable sample input and expected output and you'll probably get some answers. See [ask] if that's not clear.
Comment: Install a proper dbms. Load the data into tables. Do the work in SQL.
Answer: Without seeing any sample input/output or the script you're currently executing it's very much guesswork but if you currently have the barcode in field 1 and are doing (assuming GNU awk so you don't have your own code managing the open files):
awk '{print > $1}' file

then if managing open files really is your problem you'll get a significant improvement if you change it to:
sort file | '$1!=f{close(f};f=$1} {print > f}'

The above is, of course, making assumptions about what these barcoode values are, which field holds them, what separates fields, whether or not the output order has to match the original, what else your code might be doing that gets slower as the input grows, etc., etc. since you haven't shown us any of that yet.
If that's not all you need then edit your question to include the missing MCVE.

Given your updated question with your script and the info that the input is 4-line blocks, I'd approach the problem by adding the key "bx" values at the front of each record and using NUL to separate the 4-line blocks then using NUL as the record separator for sort and the subsequent awk:
$ cat tst.sh
infile="$1"
outdir="${infile}_out"
prefix="foo"

mkdir -p "$outdir" || exit 1

awk -F'[_[:space:]]' -v OFS='\t' -v ORS= '
    NR%4 == 1 { print $2 OFS $3 OFS }
    { print $0 (NR%4 ? RS : "\0") }
' "$infile" |
sort -z |
awk -v RS='\0' -F'\t' -v outdir="$outdir" -v prefix="$prefix" '
BEGIN {
    if ( (outdir == "") || (prefix == "") ) {
        print "Variables \047outdir\047 and \047prefix\047 must be defined!" | "cat>&2"
        exit 1
    }
    print "[INFO] Initiating demuxing..." | "cat>&2"
    outBase = outdir "/" prefix "."
}
{
    bx1   = $1
    bx2   = $2
    fastq = $3

    if ( bx1 != prevBx1 ) {
        close(umiOut)
        close(fastqOut)
        umiOut   = outBase bx1 ".umi"
        fastqOut = outBase bx1 ".fastq"
        prevBx1  = bx1
    }

    print bx2   > umiOut
    print fastq > fastqOut

    if (NR%10000 == 0) {
        printf "[INFO] %d reads processed\n", NR | "cat>&2"
    }
}
END {
    printf "[INFO] %d total reads processed\n", NR | "cat>&2"
}
'

When run against input files generated as you describe in your question:
$ wc -l cells.*.fastq
4000000 cells.16K.fastq
4000000 cells.1K.fastq

the results are:
$ time ./tst.sh cells.1K.fastq 2>/dev/null

real    0m55.333s
user    0m56.750s
sys     0m1.277s

$ ls cells.1K.fastq_out | wc -l
2048

$ wc -l cells.1K.fastq_out/*.umi | tail -1
1000000 total

$ wc -l cells.1K.fastq_out/*.fastq | tail -1
4000000 total

$ time ./tst.sh cells.16K.fastq 2>/dev/null

real    1m6.815s
user    0m59.058s
sys     0m5.833s

$ ls cells.16K.fastq_out | wc -l
32768

$ wc -l cells.16K.fastq_out/*.umi | tail -1
1000000 total

$ wc -l cells.16K.fastq_out/*.fastq | tail -1
4000000 total

Comment: Very nice! Testing on input with 1K versus 16K output files, this definitely makes the execution times comparable between the two.  It might be worth noting for others that sorting alone and letting the OS (CentOS) manage closing the previous files still results in significant slow down - i.e., one must explicitly close files as you've indicated.  Unfortunately, much of the data I have involves multiline blocks (one barcode per block), so it's not such a straightforward sorting operation, but still doable.
Comment: Of course. Assuming you're right about the number of open files being the issue, sorting doesn't help the execution time it just creates an input order such that we can in the awk script only have 1 output file open at a time and THAT is what helps the execution time. wrt multi-line input blocks - I added a shell script to my answer to show how to handle that.


Trigger to update another table to refer top most row in updated table

Question: This is my Meter Table.
MeterID int
LatestMeterReadingID int refers MeterReading

MeterReadingTable
MeterID int
MeterReadingID int
Reading bigint
ForDate date
isInactive bit

I want a trigger on the meter reading table to put the latest meter reading on the meter table when updated / inserted

New meter reading
Update Existing meter reading, date or meter (affects old meter too) or Inactivate meter reading

I also want the higher meter reading if the dates are equal.
I have done this via my ORM but want to transfer this logic to the database. I can do the insert part of it but not sure how to handle the update.
By top most I mean this
 Where isInactive=0 Order By ForDate DESC, Reading DESC

Comment: It sounds like `Meter` could be replaced by a *view* with those columns. Why do you need to *store* the information about which is the latest reading rather than *calculate* it?
Comment: It's faster when reporting on 1000+ meters? Running top 1 on that query 1000 times would be slow.
Answer: I think this is what you need. It may need some tweaking:
CREATE TRIGGER dbo.MeterReadingTrigger
   ON  dbo.MeterReading
   AFTER INSERT,UPDATE
AS
BEGIN
    SET NOCOUNT ON;

  ;WITH cte AS
  (
    SELECT
      ROW_NUMBER() OVER (PARTITION BY MeterID ORDER BY MeterID, ForDate DESC, Reading DESC) AS Row,
      mr.MeterID,
      mr.MeterReadingID,
      mr.Reading,
      mr.ForDate,
      mr.IsInactive
    FROM
      dbo.MeterReading AS mr
    JOIN Inserted i ON mr.MeterId = i.MeterID
    WHERE
      mr.IsInactive = 0
  )
  UPDATE m
   SET m.LatestMeterReadingId = cte.MeterReadingId
  FROM
    cte JOIN
    INSERTED i
      ON cte.MeterID = i.MeterId
    JOIN Meter m
      ON m.MeterID = i.MeterId
  WHERE
    Row = 1

END
GO

Answer updated with a JOIN to INSERTED in the CTE.
Comment: Can you explain to me how it updates the meter table if I set the isInactive flag to 1 on an existing row or how it works if I change the meterID on an existing row. Shouldn't it update 2 rows in the Meter table in the latter case?
Comment: @Dasiths. Did you bother to test this answer? I have tested the trigger and it correctly updates the meter table with the latest "active" MeterReadingID. As far as your second question, I can update the answer to do this, but it seems an unusual thing to do: that is update the MeterID!
Comment: Sorry if it came out rude. I would like to know how it works. Like why the use of partitions and some comments so I can learn and reuse it later on for other triggers. As for the second part of the question. Consider I'm updating a row where MeterID was 1 to 2. Shouldn't it update the latest meter reading for both meter 1 and 2 considering both of those meters could be affected?
Comment: After doing some more reading on how the partition and row_number() works I have come to the understanding that the trigger updates all meters on the table? Please correct me if I'm wrong. Would it not have an undesired performance impact? I will accept this as the answer as it works but would I be better served to just do it via the ORM for only the specific meters that need updating?
Comment: Notice that the MeterReading table is joined to the INSERTED virtual table which is automatically available inside a trigger (as well as a DELETED table). The INSERTED table has a row for each MeterReading row that was updated. We also JOIN the Meter table to INSERTED ON MeterID. That means that only Meters that need updating are referenced. So the performance impact is minimised.
Comment: BTW. I agree with the comment from @Damien_The_Unbeliever that a view is probably a better solution for a number of reasons. One being that if you change the logic behind your requirement no data updates would be needed. If you have the appropriate indexes in place the performance would be instant for the number of rows you have mentioned. This would mean that the Meter table is unnecessary. Lets say the view was called "Meters_View" then all you would need to do is SELECT * FROM Meters_view. If you like I will add another answer that uses a view. That would deal with the 2nd problem as well.


String JSON - Duas casas decimais

Question: Estou lendo um objeto JSON, e consigo obter os dados nos TextView.
Porém, no valor da String Preco quando o preço possui um zero na segunda casa decimal, ele não imprime no TextView.
Por exemplo: 9.20 exibe 9.2. Tentei algumas funções, porém, sem sucesso.
Alguém pode me ajudar ?
    @Override
    protected void onPostExecute(String[] jsonWinthor) {
        txtCodProd.setText(jsonWinthor[0]);
        txtDescProduto.setText(jsonWinthor[1]);
        txtValor.setText(jsonWinthor[2]);
        txtSifrao.setVisibility(View.VISIBLE);
    }
}

Objeto JSON lido:
{"Codigo":78,"Descricao":"BANDEJA PINT.TIGRE 1308","Preco":9.20,"CodigoBarra":"78"}

Comment: Jovem, por favor, poste seu código em texto e não em imagem.
Comment: Ok. Feito alteração.
Answer: Você pode primeiro converter para double e para usar o format. Veja:
double value = Double.parseDouble(jsonWinthor[2]);
String strValue = String.format("%.2f", value );

Veja nessa respostas outras maneiras de como fazer isso.
Comment: Valeu ! Funcionou perfeitamente. Obrigado ! Abç.

String JSON - Duas casas decimais


Azure DevOps REST API returns a 403 when using the system OAuth token during a build

Question: I'm running a script:
# Variables
$organization = "****"
$project = "****"
$repositoryId = "****"
$pullRequestId = $env:BUILD_PULLREQUEST_ID

$pat = "Bearer $env:System_AccessToken"
$featureReleaseUrl = "http://" + $env:prSourceBranchName + ".azurewebsites.net"

$body = @"
    {
        "comments": [
            {
                "content": "Link naar feature release $featureReleaseUrl"
            }
        ]
    }
"@

$createThreadInPRUrl = "https://dev.azure.com/$organization/$project/_apis/git/repositories/$repositoryId/pullRequests/$pullRequestId/threads?api-version=5.0"
if ($pullRequestId) {
    Invoke-RestMethod -Uri $createThreadInPRUrl -Headers @{Authorization = $pat} -Body $body -Method Post -ContentType 'application/json'
}

When it runs it returns a:

##[error]The remote server returned an error: (403) Forbidden.

I've created a Personal Access Tokens in my personal settings.
I've also created this script:
# Variables
$organization = "****"
$project = "****"
$buildId = $****

$pat = "Bearer $env:System_AccessToken"

if (!$env:Build_PullRequest_SourceBranchName) {
    $retrieveSourceBranchFromBuildURL = "https://dev.azure.com/$organization/$project/_apis/build/builds/$buildId" + "?api-version=5.0"
    $buildInformation = Invoke-RestMethod -Uri $retrieveSourceBranchFromBuildURL -Headers @{Authorization = $pat } -Method Get -ContentType 'application/json'
    $SourceBranchFromBuild = $buildInformation.sourceBranch.split('/')[-1]

    Write-Host "### no Build PullRequest SourceBranchName available ###"
    Write-Host "##vso[task.setvariable variable=prSourceBranchName;]"$SourceBranchFromBuild

}

And this runs fine. The difference between the first and second script is that the first is a POST and the second a GET. But they both use the $pat token.
Comment: No, the difference between the first and second is that they are **completely different APIs**. Does your build account have permissions to your Git repo(s)/PRs?
Answer: Even though the token you used is System.AccessToken, if you don't have access permission of Pull Request, you will also could not operate it.
Go Project Setting--> Repositories--> Repository you want to access, locate your account or the group you are in. Check the permission state of Contribute to pull requests.
You must have this Contribute to pull requests permission allowed, so that you can add the comment to PR.
Comment: Thanks a lot. Would have been helpful if Azure would return an error about not having permissions.
Comment: Aha, got this feeling. I would help you report this inconvenience message issue to relevant product team.


Couldnt not find function "cc" Knit PDF in R

Question: When trying to Knit PDF on R Studio i get this error. cc() is a function from package CCA(), which I've installed and loaded previously. It runs perfectly in the console as does this other function: plt.cc(), which needs the same package and library, too.
Quitting from lines 125-128 (Preview-ecc331f1f51.Rmd)  Error in eval(expr, envir, enclos) : could not find function "cc" Calls: <Anonymous> ... handle -> withCallingHandlers -> withVisible -> eval -> eval Execution halted

Need some help guys, i'm 'bout to crack my display !
Comment: Did you call `library(CCA)` in your document?
Comment: Expanding on @RichardScriven's comment, knitr runs in a _completely new/clean environment_. You have to load all packages/data/objects you need within the Rmd. It's one of the core components of knitr/Rmds being reproducible works.
Comment: @hrbrmstr What you write is often repeated, but it's not exactly/always true. It is true when the knitting is initiated using RStudio's "Compile PDF" button, but it is *not* true when calling `knit()` or `knit2pdf()` from the command line (with the default argument `envir = parent.frame()`.
Comment: @user2706569 it was fairly obvious the OP is pressing the button in RStudio
Comment: @hrbrmstr Yes. But thinking of future readers I think it's good to emphasize what applies in *this case* vs. *in general*. This is obvious for you and me but probably not for people who are new to the topic.
Comment: that's usually the case provided the OPs actually post complete/specific questions. SO isn't a `man` page repository.
Comment: I respectfully disagree. Yes, SO isn't a `man`page repository, but material on SO should be helpful for future readers. If an advice relies an something that is not clearly evident from the question, this should be mentioned, I think. I don't want to annoy you, I'd just like to create some awareness for this issue because I often read answers that seem to be misleading regarding the environment `knitr` works in.
Comment: Thank you Guys, very helpful. I´ll try to be more specific next time !
Answer: You should do this
require(CCA)
cc(SOMETHING) ...

Comment: Thank you!  I´ve just included require(CCA) inside the Chunk and runs all
Comment: you should only do that on the 1st chunk, other wise you are reloading the library without efficiency


C# Windows Service: The service did not respond to the start

Question: This is my first attempt at creating a Windows service that calls multiple threads.  What i did was create a console app, then added a windows service and installer to it.  My code is below.
partial class Service1 : ServiceBase
{
    public Service1()
    {
        InitializeComponent();
    }

    protected override void OnStart(string[] args)
    {
        System.Timers.Timer timer1 = new System.Timers.Timer();
        timer1.Interval = 10000;
        timer1.Start();
        timer1.Elapsed += new ElapsedEventHandler(timer1_Elapsed);
    }

    public static void timer1_Elapsed(object sender, EventArgs e)
    {
        Program.ThreadController();
    }

    protected override void OnStop()
    {
        // TODO: Add code here to perform any tear-down necessary to stop your service.
    }
}

class Program
{
    static void Main(string[] args)
    {
        ThreadController();
    }

    public static void ThreadController()
    {
        for (int i = 0; i < 3; i++)
        {
            new Thread(() => { processEventLogTesting(); });
        }
    }

    public static void processEventLogTesting()
    {
        EventLog.WriteEntry("ProcessThread1", "Process1 Works");
    }
}

I'm not sure why i'm getting this error, any hints would be awesome.
Thanks.
Answer: You have infinite recursion in processEventLogTesting (). That's why it's not responding. When I write a Windows service with a timer, all I do is wire the callback and start the timer in the OnStart. Then I sit back and let it do its thing. You are trying to do too much on startup.
Comment: okay, i took that out and it still didn't go, but what would you suggest to fix that
Comment: I really don't that's the issue, the error stop occurs on timer.Start()
Comment: It very well may be stopping there, but you have infinite recursion in `processEventLogTesting ()`. `timer.Start ()` is probably just where Windows gives up on the service. See my other answer below for how you should be doing this.
Answer: First I'd add in some capability to do debugging.  It's actually easy (when you know how!!!).
Add this to your class:
[DllImport("kernel32.dll")]
public static extern Int32 AllocConsole();

Then add this to your main function (and you CAN do this too when running as a service if you enable the service to access the user interface by the way).
for (int loop = 0; loop < args.Count; loop++)
{
  if (args[loop] == "debug")
    AllocConsole();
}

This gives you a console window and you can use Console.WriteLine to send output to it now. Yummy.
Now what you want to achieve, is for the service control manager to call your OnStart method, and then that method must RETURN the OK signal to let the SCM know that your app started up ok.  Otherwise, SCM thinks something is wrong and kills your app.  So OnStart is supposed to start your threads. So, make a Run() method that gets run by your thread (and by Main), and make OnStart return as fast as you can.
I notice I don't see any of this in your code example:
System.ServiceProcess.ServiceBase[] ServicesToRun;
ServicesToRun = new System.ServiceProcess.ServiceBase[] { new WinService1() };
System.ServiceProcess.ServiceBase.Run(ServicesToRun);
I suspect that your app isn't reporting to the SCM that it has successfully launched the "service" that is inside your executable.
You want your app to fire up, hand off the "service" to the SCM and let it know you are ok, and then return, while your service is now running with its own thread.
See this article for a great walk through:
http://msdn.microsoft.com/en-us/library/aa984464(v=vs.71).aspx
Comment: The answer is when you said "I notice I don't see any of this in your code example:"  I guess this wasn't added to Main() because I first made a console app then added a service
Answer:  partial class Service1 : ServiceBase
 {
    private Program program;

    public Service1()
    {
        InitializeComponent();
        program = new Program ();
    }

    protected override void OnStart(string[] args)
    {
        program.Start ();
    }

    protected override void OnStop()
    {
        program.Stop ();
    }
}

class Program
{
    private Systems.Timers.Timer timer;

    public Program ()
    {
        timer = new System.Timers.Timer ();
        timer.Interval = 10000;
        timer.Elapsed += new ElapsedEventHandler (timer_Elapsed);

    }

    public void Start ()
    {
        timer.Enabled = true;
    }

    public void Stop ()
    {
        timer.Enabled = false;
    }

    public static void timer_Elapsed (object sender, EventArgs e)
    {
        EventLog.WriteEntry ("ProcessThread1", "Process1 Works");
    }
}

Answer:
You don't start the Threads at your code, so:
for (int i = 0; i < 3; i++)
{
    new Thread(() => { processEventLogTesting(); }) { IsBackground = true }.Start();
}

At your processEventLogTesting() in addition to your wrong recursive call, you are creating a timer and then just override it by timer1 = new System.Timers.Timer(); so it will not fire.
public static void processEventLogTesting()
{
    System.Timers.Timer timer1 = new System.Timers.Timer();
    timer1.Interval = 10000;
    timer1.Elapsed += new ElapsedEventHandler(timer1_Elapsed);
    timer1.Start();
}

Answer: Windows services are notoriously difficult to debug. If you are not getting any other messages, try putting a pause on the onstart, and you should be able to attach to it, and then see what the problem is.
Nik has the answer here, but this may be useful for other windows services!
Comment: I always debug my services my having a console app project that will run the service logic in my debugger. That works pretty well for me.
Comment: Another way to debug is to create (e.g.) a Start() method in your ServiceBase. This method calls OnStart(null) and then sleeps with Timeout.Infinite. To use, call MyService.Start() instead of ServiceBase.Run(). Then you can just debug with F5.
Comment: I do place an extra sleep loop that work on debugger computers. Then I use attached debug mode to catch that line from VS debug environment.
`code`
            while (Environment.MachineName.ToUpper().StartsWith("CN"))
            {
                // Only on test debugger computers -- name like CNEMRE...
                Thread.Sleep(TimeSpan.FromSeconds(3));
            }
`code`