XCOM's don't work with PythonVirtualenvOperator airflow 1.10.6

Question: I'm using PythonVirtualenvOperator's in my airflow project an I need to pass parameters from one task to other, to test the xcoms i used this example and it works but when I change the pythonOperator to PythonVirtualenvOperator it have a prolem.
The code:
from airflow import DAG
from airflow.operators.python_operator import PythonOperator, PythonVirtualenvOperator

args = {
    'owner': 'Airflow',
    'start_date': airflow.utils.dates.days_ago(2),
    'provide_context': True,

}

dag = DAG('BAtatas', schedule_interval="@once", default_args=args)

value_1 = [1, 2, 3]
value_2 = {'a': 'b'}

def push(**kwargs):
    """Pushes an XCom without a specific target"""
    print(kwargs)
    kwargs['ti'].xcom_push(key='value from pusher 1', value=value_1)

def push_by_returning(**kwargs):
    """Pushes an XCom without a specific target, just by returning it"""
    return value_2

def puller(**kwargs):
    """Pull all previously pushed XComs and check if the pushed values match the pulled values."""
    ti = kwargs['ti']

    # get value_1
    pulled_value_1 = ti.xcom_pull(key=None, task_ids='push')
    assert pulled_value_1 == value_1

    # get value_2
    pulled_value_2 = ti.xcom_pull(task_ids='push_by_returning')
    assert pulled_value_2 == value_2

    # get both value_1 and value_2
    pulled_value_1, pulled_value_2 = ti.xcom_pull(key=None, task_ids=['push', 'push_by_returning'])
    assert (pulled_value_1, pulled_value_2) == (value_1, value_2)

push1 = PythonVirtualenvOperator(
    task_id='push',
    dag=dag,
    python_callable=push,
    requirements=['dill'],
    python_version='3.8',
    use_dill=True,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

push2 = PythonVirtualenvOperator(
    task_id='push_by_returning',
    dag=dag,
    python_callable=push_by_returning,
    requirements=['dill'],
    python_version='3.8',
    use_dill=True,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

pull = PythonVirtualenvOperator(
    task_id='puller',
    dag=dag,
    python_callable=puller,
    requirements=['dill'],
    python_version='3.8',
    use_dill=True,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

pull << [push1, push2]

the error:
[2019-11-29 17:58:04,745] {base_task_runner.py:113} INFO - Job 133: Subtask push Traceback (most recent call last):
[2019-11-29 17:58:04,745] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/bin/airflow", line 37, in <module>
[2019-11-29 17:58:04,746] {base_task_runner.py:113} INFO - Job 133: Subtask push     args.func(args)
[2019-11-29 17:58:04,747] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/utils/cli.py", line 74, in wrapper
[2019-11-29 17:58:04,748] {base_task_runner.py:113} INFO - Job 133: Subtask push     return f(*args, **kwargs)
[2019-11-29 17:58:04,748] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/bin/cli.py", line 551, in run
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     _run(args, dag, ti)
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/bin/cli.py", line 466, in _run
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     ti._run_raw_task(
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/utils/db.py", line 74, in wrapper
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     return func(*args, **kwargs)
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 930, in _run_raw_task
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     result = task_copy.execute(context=context)
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/operators/python_operator.py", line 113, in execute
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     return_value = self.execute_callable()
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/operators/python_operator.py", line 297, in execute_callable
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     self._write_args(input_filename)
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/operators/python_operator.py", line 337, in _write_args
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push     dill.dump(arg_dict, f)
[2019-11-29 17:58:04,749] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/dill/_dill.py", line 259, in dump
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     Pickler(file, protocol, **_kwds).dump(obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/dill/_dill.py", line 445, in dump
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     StockPickler.dump(self, obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 485, in dump
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     self.save(obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 558, in save
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     f(self, obj)  # Call unbound method with explicit self
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/dill/_dill.py", line 912, in save_module_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     StockPickler.save_dict(pickler, obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 969, in save_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     self._batch_setitems(obj.items())
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 995, in _batch_setitems
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     save(v)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 558, in save
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     f(self, obj)  # Call unbound method with explicit self
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/dill/_dill.py", line 912, in save_module_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     StockPickler.save_dict(pickler, obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 969, in save_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     self._batch_setitems(obj.items())
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 995, in _batch_setitems
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     save(v)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 558, in save
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     f(self, obj)  # Call unbound method with explicit self
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/dill/_dill.py", line 912, in save_module_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     StockPickler.save_dict(pickler, obj)
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 969, in save_dict
[2019-11-29 17:58:04,750] {base_task_runner.py:113} INFO - Job 133: Subtask push     self._batch_setitems(obj.items())
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 995, in _batch_setitems
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push     save(v)
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/usr/lib/python3.8/pickle.py", line 576, in save
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push     rv = reduce(self.proto)
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1194, in __getattr__
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push     self.var = Variable.get(item)
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/utils/db.py", line 74, in wrapper
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push     return func(*args, **kwargs)
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/models/variable.py", line 118, in get
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push     raise KeyError('Variable {} does not exist'.format(key))
[2019-11-29 17:58:04,751] {base_task_runner.py:113} INFO - Job 133: Subtask push KeyError: 'Variable __getstate__ does not exist'
[2019-11-29 17:58:07,380] {logging_mixin.py:112} INFO - [2019-11-29 17:58:07,379] {local_task_job.py:103} INFO - Task exited with return code 1

Any suggestion?
I'm using the PythonVirtualenvOperator to have my tasks individualized and have different requeremets/versions in each task.
Still not working with python3.7 the code is:
import airflow
from airflow import DAG
from airflow.operators.python_operator import PythonOperator, PythonVirtualenvOperator

args = {
    'owner': 'Airflow',
    'start_date': airflow.utils.dates.days_ago(2),
    'provide_context': True,

}

dag = DAG('BAtatas', schedule_interval="@once", default_args=args)

value_1 = [1, 2, 3]
value_2 = {'a': 'b'}

def push(**kwargs):
    """Pushes an XCom without a specific target"""
    print(kwargs)
    kwargs['ti'].xcom_push(key='value from pusher 1', value=value_1)

def push_by_returning(**kwargs):
    """Pushes an XCom without a specific target, just by returning it"""
    return value_2

def puller(**kwargs):
    """Pull all previously pushed XComs and check if the pushed values match the pulled values."""
    ti = kwargs['ti']

    # get value_1
    pulled_value_1 = ti.xcom_pull(key=None, task_ids='push')
    assert pulled_value_1 == value_1

    # get value_2
    pulled_value_2 = ti.xcom_pull(task_ids='push_by_returning')
    assert pulled_value_2 == value_2

    # get both value_1 and value_2
    pulled_value_1, pulled_value_2 = ti.xcom_pull(key=None, task_ids=['push', 'push_by_returning'])
    assert (pulled_value_1, pulled_value_2) == (value_1, value_2)

push1 = PythonVirtualenvOperator(
    task_id='push',
    dag=dag,
    python_callable=push,
    requirements=[],
    python_version='3.7',
    use_dill=False,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

push2 = PythonVirtualenvOperator(
    task_id='push_by_returning',
    dag=dag,
    python_callable=push_by_returning,
    requirements=[],
    python_version='3.7',
    use_dill=False,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

pull = PythonVirtualenvOperator(
    task_id='puller',
    dag=dag,
    python_callable=puller,
    requirements=[],
    python_version='3.7',
    use_dill=False,
    system_site_packages=True,
    op_args=None,
    op_kwargs=None,
)

pull << [push1, push2]

the error is:
[2019-12-04 14:08:59,579] {base_task_runner.py:113} INFO - Job 139: Subtask push     pickle.dump(arg_dict, f)
[2019-12-04 14:08:59,579] {base_task_runner.py:113} INFO - Job 139: Subtask push TypeError: cannot pickle 'module' object

and when I use use_dill=True,
the error is:
[2019-12-04 14:32:12,500] {base_task_runner.py:113} INFO - Job 141: Subtask push     return func(*args, **kwargs)
[2019-12-04 14:32:12,500] {base_task_runner.py:113} INFO - Job 141: Subtask push   File "/home/yk0l0diy/.local/share/virtualenvs/load_data-KWkJBdeu/lib/python3.8/site-packages/airflow/models/variable.py", line 118, in get
[2019-12-04 14:32:12,500] {base_task_runner.py:113} INFO - Job 141: Subtask push     raise KeyError('Variable {} does not exist'.format(key))
[2019-12-04 14:32:12,500] {base_task_runner.py:113} INFO - Job 141: Subtask push KeyError: 'Variable __getstate__ does not exist'
[2019-12-04 14:32:16,640] {logging_mixin.py:112} INFO - [2019-12-04 14:32:16,639] {local_task_job.py:103} INFO - Task exited with return code 1

Comment: Try with Python 3.7. We have not yet tested Airflow with Python 3.8 so it might work and it might now. Second thing, if you don't need dill try removing `dill` and `use_dill=False,`
Comment: still not working any other suggestion?
Answer: The problem lies in the serialization process when calling the virtual environment. The context contains things that pickle/dill are unable to serialize so in the latest version of Airflow the context is no longer provided.
That means you can not retrieve things from the context or push things into the context while executing something inside the virtual environment created by PythonVirtualenvOperator.
The only way to pass the virtual environment something from the context is to use templates_dict or op_kwargs to process the XCom during the rendering instead of trying to access the context from inside the virtual environment.
pull = PythonVirtualenvOperator(
    task_id='puller',
    op_kwargs={
        "result_push": "{{ ti.xcom_pull(task_ids='pusher') }}"
    },
)

And the only way of sending something from PythonVirtualenvOperator to Xcom is by using the standard output while inside the execution of the code inside the virtual environment (disclaimer: I don't recommend this at all. I don't know if I should be even saying this):
pusher = PythonVirtualenvOperator(
    task_id='pusher',
    do_xcom_push=True,
    ...
)

PythonVirtualenvOperator will try to do return self._read_result(output_filename) and the output of that script will be stored in XCom.
def push(**kwargs):
    print("test") # this value should appear inside the Xcom of that task if nothing else has printed something else and then you'll get a bunch of nothing.

Answer: This is fixed in airflow 2.1.0. The issue for this is here:
https://github.com/apache/airflow/issues/15335
Comment: I believe that the referenced fix only resolves the _push_ part of the overall "pyvenvop can't talk to xcom" issue  @jon


Partial Derivative theory question

Question:
So I think I have the association correct, but I am not positive on the 2nd question which could alternatively be read as: "Can you determine from the table if the change in speed or change in temperature has a greater effect on wind chill" I've calculated the changes when holding each item constant:

The boxed deltas are the average change. Not sure if this proves anything or whether I am just playing with numbers. Anyone have an idea?
Comment: You had the right idea to calculate the average change in wind chill that results from change in temperature and wind speed. Now, divide the average change in wind chill that resukts from change in speed by the total change in speed.
$$35.25/15$$
Answer: You had the right idea to calculate the average change in wind chill that results from change in temperature and wind speed. Now, divide the average change in wind chill that results from change in speed by the total change in speed.
$$35.25/15=2.35$$
Do the same with temperature. Notice, It looks like the average change in wind chill that results from change in temperature is something else, not $77.5$. I got $38.75$. So
$$38.75/40=0.96$$
So it looks like the speed has a greater effect on the wind chill. You follow why? Probably, you do already.

Partial Derivative theory question


Best way to parse a URL query string

Question: What is the best way to parse data out of a URL query string (for instance, data appended to the URL by a form) in python?  My goal is to accept form data and display it on the same page.  I've researched several methods that aren't quite what I'm looking for.
I'm creating a simple web server with the goal of learning about sockets.  This web server won't be used for anything but testing purposes.
GET /?1pm=sample&2pm=&3pm=&4pm=&5pm= HTTP/1.1
Host: localhost:50000
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost:50000/?1pm=sample&2pm=&3pm=&4pm=&5pm=

Comment: What's wrong with http://stackoverflow.com/questions/1349367/parse-an-http-request-authorization-header-with-python or http://stackoverflow.com/questions/4685217/parse-raw-http-headers.  You haven't given us enough info about what other approaches are lacking.  Do you have an example header or two?
Comment: Are you looking to write the parsing from scratch, or what?
Comment: Nothing is 'wrong' with either of these posts.  Based on the programming experiences I've head in the past, I'm inclined to do something similar like a regex expression in the second link.  However, I wanted to ask and see if there is a simpler way to do it since this is my first python program.
Comment: Looks to me like you're talking about URL query strings, not HTTP headers. You might want to update your question to reflect this.
Answer: Here is an example using python3 urllib.parse:
from urllib.parse import urlparse, parse_qs
URL='https://someurl.com/with/query_string?i=main&mode=front&sid=12ab&enc=+Hello'
parsed_url = urlparse(URL)
parse_qs(parsed_url.query)

output:
{'i': ['main'], 'enc': [' Hello '], 'mode': ['front'], 'sid': ['12ab']}

Note for python2: from urlparse import urlparse, parse_qs
SEE: https://pythonhosted.org/six/#module-six.moves.urllib.parse
Comment: And why are the values like this ```['value']``` ? ```dic['enc']``` gets ```['Hello']```  - how to get 'Hello'? with split?
Comment: @Suisse see https://stackoverflow.com/questions/11447391/ajax-why-jquery-replaces-with-a-space the values are in a list because multiple values can be encoded see : https://stackoverflow.com/questions/2571145/urlencode-an-array-of-values hope it helps
Answer: The urllib.parse module is your friend: https://docs.python.org/3/library/urllib.parse.html
Check out urllib.parse.parse_qs (parsing a query-string, i.e. form data sent to server by GET or form data posted by POST, at least for non-multipart data). There's also cgi.FieldStorage for interpreting multipart-data.
For parsing the rest of an HTTP interaction, see RFC2616, which is the HTTP/1.1 protocol specification.
Comment: For the amount of information he gave, that's all there is to say. Specifically, if you're actually referring to HTTP headers: is he using a webserver which actually allows you to get HTTP headers uninterpreted (via some stream)? Is he using WSGI (where HTTP-headers are interpreted by the framework)? Plain-old CGI, where you have to interpret the environment and hope for the best? Whatever.
Comment: I'm not writing the script for him. He specifically asked how to parse query data, at least that's what I read between the lines, even though those are not actually HTTP headers. But I didn't bother commenting on that.
Comment: I'm not suggesting that you should write the script for him, but urlparse is only a tiny piece of this puzzle.
Comment: urlparse looks like a great resource.  The header is pretty simple and I've added it to the original question.  As I'm sure you can guess, my initial idea is to parse the get line into an array of strings.
Comment: Are you trying to write a webserver? Or some form of packet inspection/inspector?
Comment: As posted this is a simple web server that serves a web page consisting of a form.  When the user clicks submit, the form inputs are appended to the URL.  My goal is to parse the appended url, retrieve what was entered into the form, and display it on the page.
Comment: Why not use a "proper" webserver to host your application? There's no need to reinvent the wheel (i.e., implement your own application server, which handles parsing the incoming request). Have you had a look at CherryPy or anything similar? I'm trying to discourage you, even as a pet/hobby project, to try to write anything resembling a web-server, HTTP/1.0+ are a PITA to implement correctly.
Comment: I'm only interested in writing my own, and finding the best way to parse a URL query string.
Comment: If it's just the URL-query-string to parse, check out the modules I referenced in the answer. If you need to parse the full HTTP client interaction/request, you're in for some reading of RFC2616 (http://www.w3.org/Protocols/rfc2616/rfc2616.html) which describes the HTTP protocol. There's nothing "premade" for this kind of parsing in the Python stdlib.
Comment: For Python 2, you're looking for `urlparse.parse_qs`.
Answer: If you need unique key from query string, use dict() with parse_qsl()
import urllib.parse
urllib.parse.urlparse('https://someurl.com/with/query_string?a=1&b=2&b=3').query
    a=1&b=2&b=3
urllib.parse.parse_qs('a=1&b=2&b=3');
    {'a': ['1'], 'b': ['2','3']}
urllib.parse.parse_qsl('a=1&b=2&b=3')
    [('a', '1'), ('b', '2'), ('b', '3')]
dict(urllib.parse.parse_qsl('a=1&b=2&b=3'))
    {'a': '1', 'b': '3'}

Comment: It's important to notice that the casting from tuple to dict result don't consider `b` to have two values, one which gets ignored. Wasn't aware of [`parse_qsl`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.parse_qsl), good addition.
Answer: built into python 2.7
>>> from urlparse import parse_qs
>>> parse_qs("search=quint&tags=python")
{'search': ['quint'], 'tags': ['python']}

Answer: only for one line quick prototyping CGI vars without imports, not the best obviously but could be useful.
agrs = dict(item.split('=') for item in env['QUERY_STRING'].split('&') if item)

Comment: This will break if any parameter in the query string is URL-encoded. "Manual parsing" of URLs is the source of many security issues.
Comment: indeed why the warning "only for prototyping" posted it to show case a quick parsing without any import
Comment: I wonder if every URL parser is a "manual parser"? At some point someone had to sit down and write it...
Answer: Based on this article, use can_ada to parse URLs in Python:
from their project:
import can_ada
urlstring = "https://www.GOoglé.com/./path/../path2/"
url = can_ada.parse(urlstring)
# prints www.xn--googl-fsa.com, the correctly parsed domain name according
# to WHATWG
print(url.hostname)
# prints /path2/, which is the correctly parsed pathname according to WHATWG
print(url.pathname)

import urllib.parse
urlstring = "https://www.GOoglé.com/./path/../path2/"
url = urllib.parse.urlparse(urlstring)
# prints www.googlé.com
print(url.hostname)
# prints /./path/../path2/
print(url.path)

Best way to parse a URL query string


Intersection of planes and calculating the equations

Question: Two planes:
4a - 6b + 8c = 3
-9a + 12b -3c = -10
What steps are there to calculate the equation of the line when the planes intersect?
Comment: set them equal.
Comment: What form of equation of a line do you want to end up with?
Answer: I think a simple way it's the following.
Let $z=t$ and solve the following system:
$$2x-3y=2-4t,$$
$$-3x+4y=-3+t$$ and $$z=t.$$
Can you end it now?
I got $$(1+13t,10t,t).$$
Answer: Multiplying the second equation by $4$ and adding to the first one  we get
$$x=1+\frac{13}{10}y$$
defining $$y=10t$$
we get $$x=1+13t$$ and plugging this in one of the equation above we get $$z=t$$ so our line is given by
$$[x,y,z]=1,0,0]+t[13,10,1]$$ where $t$ is a real number.


Frequency parameter of `ts()` and `findfrequency()` in R

Question: Is the result obtained in findfrequency() function of forecast package and the frequency parameter of ts() the same?
Answer: It probably should be, but there are no guarantees that it has to be.
findfrequency will return the period with the maximum spectral amplitude of the signal provided as an input. It is relative straight-forward calculation using the results from spec.ar where we estimate the spectral density associated with each candidate frequency of a time-series model from auto-regressive fit. Assuming that our raw signal, x, having assumed frequency 1, we use it as input to spec.ar:
specARresults <- spec.ar(x)

then then results from findfrequency are simply:
1 / specARresults$freq[which.max(specARresults$spec)]

i.e. the inverse of the frequency having the highest spectral density. And as the inverse of a frequency is a period we get our result. :) (Side-note: Why findfrequency returns a period eludes me.)
We can theoretically use this result as input to ts's argument about frequency but we do not have to. The frequency  we use to ts is probably better to relate to our actual problem. For example, we might have daily expenditure data that happen to have a strong weekly pattern (cause people spend more on weekends). If we do not care for a weekly pattern though, it might be more relevant for us to analyse them assuming a period of 365 days.
Comment: Welcome to the CV community!


Equation of Plane involving Intersection of planes

Question: Find a plane through $A$ $(2, 1, -1)$ and perpendicular to the line of
intersection of the planes $2x + y - z = 3$ and $x + 2y + z = 2$. Not too sure of what to do here, I know I need to find the point of intersection of the planes, and I found two points that I think are on the line of intersection, $B$ ($5/3$ ,$0$ ,$1/3$) and $C$ ($4/3$, $1/3$, $0$ ). However, I don't know where to go from there, and with the plane needing to be perpendicular.
Answer given is $x$-$y$+$z$=$0$ Any help is appreciated.
Answer: Hint:
A directing vector of the intersection line of the two planes is the cross-product $u=\vec n_1\times \vec n_2$ of the normal vectors of the planes.
Once you calculated its coordinates, the equation of the plane you're after is, in vector form
$$\vec u\cdot \overrightarrow{OM}=\vec u\cdot \overrightarrow{OA}.$$
Answer: The sum of the two planes equations gives
$$x+y=\frac 53$$
thus $$y=\frac 53-x$$
and
$$z=2-x-2y=2-\frac{10}{3}+x$$
the line intersection of the two planes has parametric equations :
$$x=0+x$$
$$y=\frac 53-x$$
$$z=\frac{-4}{3}+x$$
the vector director of this line is
$$(1,-1,1)$$
the plane perpendicular to this line has the equation
$$x-y+z=d$$
it contains the point $(2,1,-1)$ if $$d=0$$
the equation you are looking for is
$$\boxed{x-y+z=0}$$


How to convert int to double in a LINQ statement

Question: I have this line of code :
IEnumerable<double> katlar = ilanlarQuery.ToList()
                                         .Select(r => r.BulunduguKat.Value)
                                         .OrderBy(f => f);

I get error because BulunduguKat.Value is int, but katlar is double. How can I convert BulunduguKat.Value to double in this statement? Thanks.
Comment: ... by casting it?
Comment: Side-note: remove the `ToList`, it's inefficient to create a new List if you don't need it. If you need the result as list use `ToList` at the end. If `ilanlarQuery` is a database query and you want to use `Linq-To-Objects` use `AsEnumerable` instead.
Comment: @jason casting it is enough
Answer: Most straight forward way: just convert to double. By doing so you are telling that the TResult type of this Select operation is a double - making this an IEnumerable<double>
IEnumerable<double> katlar = ilanlarQuery.Select(r => (double)r.BulunduguKat.Value)
                                         .OrderBy(f => f);

Thanks to int being implicitly convertable to double this will work too:
IEnumerable<double> katlar  = ilanlarQuery.Select<YourType,double>(i => i.BulunduguKat.Value)
                                          .OrderBy(f => f)();

Also, as you see you can remove the .ToList() before
If indeed BulunduguKat is a Nullable<int> as Tim suggested then you should decide:
//If you don't want them in the result collection:
var katlar =  ilanlarQuery.Where(r => r.BulunduguKat.HasValue)
                          .Select(r => (double)r.BulunduguKat.Value)
                          .OrderBy(f => f)

//If you want them with some default value:
var katlar =  ilanlarQuery.Select(r => (double)(r.BulunduguKat.HasValue ? r.BulunduguKat.Value : 0))
                          .OrderBy(f => f);

Comment: Btw, if the type is a `Nullable` it could be `null`, then the cast will not succeed but give an `InvalidOperationException`. You should handle the case that `!r.BulunduguKat.HasValue`, for example with the conditional operator.
Comment: First answer gives error, second answer doesn't have `BulunduguKat.Value`
Comment: @jason - corrected second. What is the erro of the first?
Comment: Added dealing with the case that it is a nullable type
Answer: ds.Tables["#KAMPANYA"].AsEnumerable().Select(s => Convert.ToInt32(s["KAMNO"])).ToList()

Comment: While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value.


Amazon S3 Missing Files

Question: We're working on developing user widgets that our members can embed on their websites and blogs. To reduce the load on our servers we'd like to be able to compile the necessary data file for each user ahead of time and store it on our Amazon S3 account.
While that sounds simple enough, I'm hoping there might be a way for S3 to automatically ping our script if a file is requested that for some reason doesn't exist (say for instance if it failed to upload properly). Basically we want Amazon S3 to act as our cache and it to notify a script on a cache miss. I don't believe Amazon provides a way to do this directly but I was hoping that some hacker out there could come up with a smart way to accomplish this (such as mod_rewite, hash table, etc).
Thanks in advance for your help & advice!
Answer: Amazon doesn't currently support this, but it looks like it might be coming. For now, what you could do is enable logging and parse the log for 404s once an hour or so.
It's certainly not instant, but it would prevent long-term 404s and give you some visibility about what files are missing.
Comment: Understood that there doesn't seem to be an actual answer to this today but lets hope you're right and Amazon is going in that direction. Thanks!

Amazon S3 Missing Files


using drush for upgrade drupal 6 to 7 - create alias problem

Question: I want to use drush for upgrade my drupal 6 to drupal7. i read this document. i have a problem with create alias. i have been doing this
1- create a subdomain in my plesk (create sub.domain.com)
2- go to /var/www/vhosts/domain.com/httpdocs and download this file and rename this to "aliases.drushrc.php"
3- remove aliases.drushrc.php content and replace with
<?php
$aliases['sub'] = array(
  'root' => '/var/www/vhosts/domain.com/sub',
  'uri' => 'sub.domain.com',
);
?>
4- run command: drush @sub status and i get this error:
The drush command '@sub status' could not be found.  Rundrush cache-clear drushto clear the commandfile cache if you have installed new extensions
Answer: Although the Drupal root is a valid place to store alias files, Drush will not be able to find the alias records described there until the Drupal root as been specified.  Therefore, drush @sub status has a chicken-and-egg problem; you want to use @sub to specify the Drupal root, but Drush won't load the alias file that defines @sub until the Drupal site it is defined in is bootstrapped.
To resolve this problem, move your aliases file to $HOME/.drush/aliases.drushrc.php.  Once you do that, then your status command will work.
You may define your 'onward' alias either in $HOME/.drush/aliases.drushrc.php, or in /var/www/vhosts/domain.com/httpdocs/aliases.drushrc.php, per your preference.
Answer: Drush Site Upgrade needs to be installed first.
You can download the module and upload it to drush/commands/core folder.
Now you will be able to run this command
If you are getting some problems when Drush tries to create a new database, create it first and enter it in the alias file.
Comment: The error message here is deceptive; although Drush is saying "command not found", what it is really reporting is that @sub is not found.  Once the problem reported in the question has been resolved per my answer, then you are correct: `drush dl drush_sup` should be used to download Drush Site Upgrade to $HOME/.drush.  Using drush dl to install a Drush extension will always clear the Drush commandfile cache; if you install an extension manually (e.g. via wget or cp), then you should run `drush cc drush`) afterwards.


Criar sub-listas em python

Question: Criei o seguinte código:
    self.buckets = [[], []]

    for i in self.keys:
        for ii in self.lista:
            if i % len(lista) == ii:
                self.buckets[ii].append(i)

Que adiciona os elementos de uma lista no caso "self.keys", em outra lista "self.buckets", de acordo com o resto da divisão pelo "len(lista)". Ou seja, digamos que o elemento da vez seja "self.keys = 10" e o "lista = 0", como o resto da divisão seria 0 esse elemento deve ser adicionado no slot 0 da lista "self.buckets".
A questão é que o código acima funciona, mas não é tão prático, já que preciso criar manualmente todas as sub-listas, ex:
self.buckets = [[], []]

Devido a isso, queria se tem como automatizar essa criação de sub-listas, ou algo do tipo.
Answer: Pelo que entendi, você precisa criar uma lista com n sublistas, todas vazias.
Você pode fazer usando compreensão de listas:
numeroDeSublistas = 4
self.buckets = [[] for _ in range(numeroDeSublistas)]

O que retorna:
[[], [], [], []]

Como desejado.
Comment: Era exatamente isso, vlw mesmo !

Criar sub-listas em python


Find f '($π\over2$) by solving integrals

Question: Hello can you help me please with this problem about integrals, I don't know how to  solve g(x), or, how can I find f '($π\over2$). Here is the problem
Thanks in advance.
Comment: Please use MathJax (LaTeX) for formatting. And what is $\operatorname{sen}$?
Comment: Sorry for that, sen means sin
Answer: Let $\;F,G\;$ be s.t. $\;F'=f\,,\,\,G'=g\;$ (primitive functions), then:
$$f(x)=F(g(x))-F(0)\;,\;\;g(x)=G(\cos x)-G(0)\implies$$
$$f'(x)=F'(g(x))\,g'(x)=f(g(x))\,g'(x)\;,\;\;g'(x)=G'(\cos x)\,(\cos x)'=g(\cos x)\,(-\sin x)$$
Now put all the above together, justify things and end the solution.
Answer: Note that $f(x)=F(g(x))$ where
$$F(x)=\int_0^x \frac{1}{ \sqrt{1+t^3} }dt$$
and $g(x)=G(\cos x)$ where, again, $G$ is defined by
$$G(x)=\int_0^x \big( 1+\sin (t^2) \big)dt$$
By the FTC we have that
$$F'(x)=\frac{1}{ \sqrt{1+x^3} }$$
and that $G'(x)=1+\sin (x^2)$.
Now, by the chain rule, we have that
$$g'(x)=G'(\cos x)\cos' x = -G'(\cos x)\sin x = -\big( 1+\sin (\cos^2 x) \big) \sin x$$
Therefore
$$f'(\tfrac{\pi}{2})=F'(g(\tfrac{\pi}{2}))g'(\tfrac{\pi}{2})=\frac{1}{ \sqrt{1+(g(\tfrac{\pi}{2}))^3} } g'(\tfrac{\pi}{2})=\frac{1}{ \sqrt{1+0^3} } (-1)=-1$$
Comment: Thank you friend, that's correct, -1 is the answer, but how do you know that g(π/2) is   0, solving without an calculator?
Comment: $g(\tfrac{\pi}{2})=G(\cos \tfrac{\pi}{2})=G(0)$ and $G(0)$ is zero since $$\int_0^0 \big( 1+\sin (t^2) \big)dt=0$$ and in general $$\int_a^a f(x)dx=0$$
Comment: Now I understand. Thank you so much


Is it possible to integrate my existing Firebase project(iOS and Android) to separate google analytic account without losing current data

Question: I have a Firebase project already setup for iOS and Android platforms. Now I want to integrate with Google analytic account not from the same firebase account I am using but the other separate account where we have already integrated web.
Is it possible to do that? if yes how can I setup new google analytics for my existing project? and how it will work?
Appreciate the feedback
Comment: To Me, No you can't because of "GoogleService-Info.plist" that it is unique for each Account.
Answer: No you can't because of "GoogleService-Info.plist" it is unique for each Account.


Proving modular statements

Question: Show that for any even number $k$,
$$2^{nk} \equiv 1 \pmod {2^n + 1} $$
I tried using induction with base case $k = 0$, but I am unsure how to proceed.
Comment: $$2^n\equiv-1\pmod{2^n+1}$$
Answer: $2^{n}$ is congruent to $(-1)$ modulo $2^n+1$.
So $2^{nk}$ is congruent to $(-1)^k=1$ because $k$ is even.
Answer: Let $$2^n \equiv −1\, {(\mod2^n+1)}\implies {(2^n)}^k \equiv -1^k \pmod {2^n + 1}\implies$$
$$2^{nk} \equiv (-1)^k \pmod{2^n + 1} $$
Therefore
$$ 2^{nk} \equiv 1 \pmod {2^n + 1} \iff k := \{2x\,|\, x\in \mathbb{Z}\}$$

Proving modular statements


Find the smallest square matrix in which some objects fit following some rules

Question: I have to put some objects in a matrix. The data of these objects is given in another matrix in which each line contains an object, and the first column represents its width, and the second its height.
I want to know which is the smallest matrix that can contain all these objects, with the following rule: objects can't be adjacent to each other, except for diagonals. See the following example:
<code>|1 3|
|1 2|
</code>
In this case, we need to populate the main matrix with two objects: one that is 1x3 units in the main matrix, and another that is 1x2.
One possible example to fill it would be (not the smallest possible):
<code>|x x x o|
|o o o x|
|o o o x|
|o o o o|
</code>
This following example is incorrect:
<code>|x x x o|
|o x o o|
|o x o o|
|o o o o|
</code>
Because two objects collide. In this example, the answer would be the size of the smallest possible matrix in which all objects can fit; that is:
<code>|x o o|
|x o x|
|x o x|
</code>
The answer would be 3. So I want a function that receives the first matrix ([[1,3], [1,2]]) and returns 3.
I've tried for some time to do this, but it seems impossible to me. I don't know much of matrixes beyond determinants, and I see no way in which what I know relates to this. I think the main problem is that the ship may create an exclusive area, but only if it is not in the corners (and two or more ships may share the same area). I made a program that would try all possibilities and choose the smallest one, but it takes forever to do it for the amount of data I have. So, I was wondering, is there another way, purely mathematical (without testing everything)?
Comment: I believe it is just a typo in the text, do the dimensions of the first ship need to be 1x3 and not 2x3?
Comment: It was, indeed, a typo; I've already corrected the text. Thanks!
Answer: The short answer is that you cannot (theoretically) do much better than brute force searching.
While I do not have a proof, I would be extremely surprised if this problem is not NP-complete. Your problem is closely related to rectangle packing, a well studied problem.
Your best option would be to use a combinatorial search system, based on either SAT or Constraint Programming.These systems still do possibly exponential search, but have many tricks and heuristics to improve performance in practice.
I work on the minion constraint solver minion. How big are your biggest problems? I could give some guidance on how achievable they are.


Inequalities using AM-GM

Question: Use the AM-GM inequality to prove $(5xy + 6y)^3$ ≥ $1215xy^3$ for all real numbers $x, y > 0.$
Not sure if I was on the right track but so for my understanding is:
since there is a power of 3 on the left we need 3 numbers
so you have $5xy + 3y + 3y$ ≥ $3(5x \times 3y \times 3y)^3$
bit stuck after this, or am I completely on the wrong track?
Comment: Note that you can cancel out the $y^3$.
Answer: $5xy+3y+3y \geq 3(5xy*3y*3y)^{1/3}$ now if you raise both sides to the power 3 we get the result.
Comment: if i continue on that i would get $1215xy^2$ which isnt the right answer
Comment: Sorry i lost one $y$ from $xy$ now it should be ok.

Inequalities using AM-GM


Angular: multiple reciepients of data from endpoint

Question: Precondition
I have the following situation:

on the backend I have an endpoint, which delivers data. Data is changed very-very seldom.
on the frontend I have several components, which need the data from p.1 right after the loading.

Question
How to avoid multiple requests to the backend ( = how to share the result of the first request to all the other callings of a service method).

Source code - service
@Injectable()
export class ConfigurationService {

  private ConfigurationUrl: string = 'api/configurations/';
  private httpHeaders: HttpHeaders;
  // ...
  constructor(private httpClient: HttpClient) {
  }
  // ...
  loadCommonSettings(): Observable<CommonSettings> {
    return this.httpClient.get<CommonSettings>(this.ConfigurationUrl + "common");
  }
  // ...
}

Source code - components
export class HomeComponent {
  constructor(public dialog: MatDialog,  private configurationService: ConfigurationService, private _snackBar: MatSnackBar) {
  }

  isNewDisabled: boolean = true;
  commonSettings: CommonSettings;

  ngOnInit() {
    this.loadCommonSettings();
  }
  //...

  loadCommonSettings() {
    this.configurationService.loadCommonSettings().subscribe(
      res => {
        this.isNewDisabled = false;
        this.commonSettings = res;
      },
      err => {
        this.openErrorDialog("Error", "Error on data loading");
      }
    );
  }
  //...
}

Problem
Each calling of loadCommonSettings service function fires a new request to the service.
Supposes

I thought to store data in the service as in the cache, but anyway I have 2 almost simultaneous calls of a service method.
Maybe Subjects may help in this situation, but I'm completely not sure about it (because it seems, that Observables are also absolutely Ok for such calls).
Comment: are you familiar with service-workers? if not, read this: https://angular.io/guide/service-worker-intro
Answer: Caching is a senseful choice here as you already thought.
In the following an example for your service. You can also use a BehaviorSubject instead of the ReplaySubject and initialize it with undefined.
@Injectable()
export class ConfigurationService {

  private ConfigurationUrl: string = 'api/configurations/';
  private httpHeaders: HttpHeaders;
  private commonSettings$: Subject<string>;

  // ...
  constructor(private httpClient: HttpClient) {
  }
  // ...
  loadCommonSettings(): Observable<string> {
    if (!this.commonSettings$) {
      this.commonSettings$ = new ReplaySubject<string>(1);
      this.httpClient.get<string>(this.ConfigurationUrl + "common")
        .subscribe(data => this.commonSettings$.next(data));
    }
    return this.commonSettings$;
  }
  // ...

Here only the fist call to the service is starting a http-call. And all calls including the first one are only returning the subject.
Comment: Thanks a lot. It works well.
I guess, now I need to learn more about the ReplaySubject.
Thanks a lot again :-)
Answer: Below are the solution without using Subjects
Option 1: Using ShareReplay
loadCommonSettings(): Observable<CommonSettings> {
    return this.httpClient.get<CommonSettings>(this.ConfigurationUrl + "common").pipe(
        take(1),
        shareReplay(1)
    );
 }

Option 2 : Using local-Variable to hold result
// declare variable to hold cache
private cachedSetting: CommonSettings = null;

loadCommonSettings(): Observable<CommonSettings> {
    // if cachedSetting already available then return same
    if(cachedSetting){
        return of(this.cachedSetting);
    }

    return this.httpClient.get<CommonSettings>(this.ConfigurationUrl + "common").pipe(
        tap(response => this.cachedSetting = response)
    );
 }

Imports used:
import { shareReplay, take, tap } from 'rxjs/operators';
import { of } from 'rxjs';

Comment: thanks a lot for the answer. I've tried both options, and it seems that something is missing. I still see the duplicated request. I'm going to learn more about ReplaySubject and I'll try to fix both solutions. It seems it's pretty similar to the comment of a BozziDev. Thanks again.


Why do I get a compilation error when I try to have two methods with the same name and parameter type?

Question: If i change the byte to int I get a compiler error. Could you explain the problem?
public class A {
   protected int xy(int x) { return 0; }
}

class B extends A {
   protected long xy(int x) { return 0; } //this gives compilor error
   //protected long xy(byte x) { return 0; } // this works fine
}

Comment: Because the method signature of `A.xy` is the same as `B.xy`: same name, same amount of parameters and same type of parameters.
Comment: Your methods as they are posted in the text are absolutely fine because they have **different** parameter types. `int` is **not** the same as `byte`
Comment: @GermannArlington: If you just read the title and code, it's a bit confusing, but the opening line of the *question* makes it clear: *"f i change the byte to int I get a compiler error."*
Answer:
If i change the byte to int I get a compiler error.

If you do that, you have this:
public class A {
   protected int xy(int x) { return 0; }
}

class B extends A {
   protected long xy(int x) { return 0; }
}

...and the only difference in the xy methods is their return type. Methods cannot be differentiated solely by their return types, that's the way Java is defined. Consider this:
myInstance.xy(1);

Which xy should that call? long xy(int x) or int xy(int x)?
If your goal is to override xy in B, then you need to make its return type int in order to match A#xy.
Answer: You can't have two methods with same signature in a Class though the methods are placed separately in different classes in the same inheritance tree. i.e. Base class and sub class
Note : Just return types can't make the compiler understand the difference in methods. return type is NOT included in the method signature as well
Answer: You are trying to write two methods with the same name and input parameters, that is not possible.
Look at the following two methods:
float met(){
  return 4.5;
}

double met(){
  return 5.4;
}

If we would write this line then
int x = (int)met();

what method would be called?
It is not possible to decide, therefore this situation is not permitted.
Answer: That's because if You change byte to int You will have method with the same signature in base and sub class (same method name and parameter type) and therefore return type should be the same as well. Because is not (int and long) it will give You error
Answer: You can differentiate methods by parameters and names only.
     both methods are in same class B

     b.xy(byte x) or b.xy(int x) is called for input xy(0) or xy(1)

Answer: Overiding method should return a type that can be substituted for the type returned by overriden method
Answer: Because when you change the byte into int the Instance which calls the method doesn't know which method is meant. The overwritten one or the old one?
Thats why it's not allowed to have the same signatures (method name, type of parameters and amount of parameters). Just as an information: The return type is not part of the signature.


Sum of dice bet

Question: You have a 50 sided die and a 100 sided die. You bet on the sum of the dice and receive points equal to the sum of the dice. What is the sum that you choose to bet on to maximize your earnings.
I was able to do this for a 6 sided die and a 10 sided die.
My approach was to calculate the probability for 9,10,11 and 12 as the sum and found that P(9) = P(10) = P(11) = 1/10 > P(12) and hence I choose 11.
However I'm unable to calculate the same for large numbers like 50 and 100. Is there a way to calculate this easily? This is an interview question (both parts)
Comment: With a 6-die and a 10-die, you can see that we have 3 groups, Total from 2 to 7, from 7 to 11, and from 11 to 16. Values 7 and 11 are at the frontier, and can be consider in one or another group. All numbers from 7 to 11 have same probability. If we transpose to 50-die plus 100-die, 7 becomes 51, and 11 becomes 101.
Answer: Go for $101$. There are $50$ ways to make it, which is the maximum possible, and $102\cdot49=4998<5050=101\cdot50$.

Sum of dice bet


Slow upload speeds with pfsense virtual appliance

Question: I have a pfSense virtual appliance set up in front of a Windows server. The pfSense appliance has been configured with two L2L IPSec VPN sites and not too much else. The appliance has two vNics which both exist on the same VLAN, but one is "WAN" and the other is "LAN."
When I run speedtest.net on my Windows server when I have configured it to use a static WAN address and gateway, I get great speeds - maybe around 50 down, 15 up. However, when I configure it with a private IP address, I get similar download speeds but terrible upload speeds - around 2 or 3 Mbps consistently. I used Wireshark to see what gives but there didn't appear to be too much helpful information there, or I just could not find it.
Besides the L2L VPNs, other configurations include:

Automatic Outbound NAT
Virtual P-ARP IP for the Windows Server
WAN Firewall rule to allow * to * on RDP
WAN Firewall rule to allow * to * (enabled this just for testing... didn't help!)
No DHCP or any other services besides IPSec VPN
No Errors LAN or WAN
No collisions LAN or WAN

I would be happy to post the full config file if it would help. I've been scratching my head at this one all day!
Comment: I'm not that familiar with pfSense, but most likely, having two vNics is a bad idea. You should just have one with two addresses/netmasks. My bet is that you're dropping packets because they were received on the "wrong" vNic.
Comment: I am seeing a lot of ARP messages relating to a packet being sent on em0 but received on em1, etc... but pfSense requires two interfaces to work...
Answer: You are doing "NAT on a stick". pfSense, unfortunately, does not have good support for this configuration yet. Your configuration of two different interfaces will not work -- on the same VLAN, they'll see each other's packets.
Here's a horrible hack if it's VLAN 1. Set the outside vNic to tagged VLAN and the other to untagged. If that doesn't solve the problem, flip the ports. (Note that you should do this locally, not remotely, as you may lose contact.)
Comment: If I went ahead and configured two VLANs (two labels), as in Internet <----- vlan0 -----> pfSense <----- vlan 1 -----> Host would that work?
Comment: Yes. Then it would know which interface to associate each packet with by VLAN.
Comment: Ok I've went ahead and assigned the LAN interface to a disjoint and private vLAN but there is no noticeable improvement - even with a router reset.
Answer: Not clear exactly what you have there with two NICs on the same broadcast domain, you have a public and private subnet on the same broadcast domain? Don't do that in general, really bad practice, separate those out. One vswitch (using ESX terminology, replace as appropriate for your hypervisor) for the public IP subnet, one for the internal network.
In general there isn't anything to keep that from working properly, it's fully supported and works fine in pfSense (assuming the network in general is sane, not enough here to tell that).


How can I create a transparent WPF window?

Question: I got progress bar inside the MainWindow in WPF. Is possible to display only this control? I tried to set Window's visibility to hidden but it doesn't work (don't display anything).
And the second question:
I set progress bar's border rounded, but when it's loading on first few percents the animation is outside the bar, how can I fix this?

Code:
<Window x:Class="WpfApp2.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:local="clr-namespace:WpfApp2"
    mc:Ignorable="d"
    Title="MainWindow" Height="80" Width="800" Visibility="Visible"
    ContentRendered="Window_ContentRendered" WindowStyle="None">
<Window.Resources>
    <SolidColorBrush x:Key="ProgressBar.Progress" Color="#FF06B025"/>
    <SolidColorBrush x:Key="ProgressBar.Background" Color="#FFE6E6E6"/>
    <SolidColorBrush x:Key="ProgressBar.Border" Color="#FFBCBCBC"/>

</Window.Resources>
<Grid>
    <ProgressBar HorizontalAlignment="Left" Height="49" Margin="5,5,5,5" VerticalAlignment="Top" Width="772" Minimum="0" Maximum="100" Name="pbStatus" Style="{DynamicResource ProgressBarStyle1}"/>
</Grid>

Comment: @UweKeim - any reason you kept the `c#` tag?
Comment: Perhaps the question should be rephrased as `How can I create a transparent WPF window?`
Comment: Question changed, thanks
Answer:
Is possible to display only this control?

Try to set the following properties:
<Window ...
    Title="MainWindow"
    WindowStyle="None"
    AllowsTransparency="True"
    Background="Transparent"
    SizeToContent="WidthAndHeight">

As for your second question - please only ask one question per post - you could use the ClippingBorder class from here and put the track and the indicator inside it in the template:
<ControlTemplate TargetType="{x:Type ProgressBar}">
    <Grid x:Name="TemplateRoot">
        <Grid.RowDefinitions>
            <RowDefinition Height="*"/>
            <RowDefinition Height="0*"/>
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="0*"/>
            <ColumnDefinition Width="0*"/>
            <ColumnDefinition Width="*"/>
            <ColumnDefinition Width="0*"/>
        </Grid.ColumnDefinitions>
        <VisualStateManager.VisualStateGroups>
            <VisualStateGroup x:Name="CommonStates">
                <VisualState x:Name="Determinate"/>
                <VisualState x:Name="Indeterminate">
                    <Storyboard RepeatBehavior="Forever">
                        <DoubleAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[0].(ScaleTransform.ScaleX)" Storyboard.TargetName="Animation">
                            <EasingDoubleKeyFrame KeyTime="0" Value="0.25"/>
                            <EasingDoubleKeyFrame KeyTime="0:0:1" Value="0.25"/>
                            <EasingDoubleKeyFrame KeyTime="0:0:2" Value="0.25"/>
                        </DoubleAnimationUsingKeyFrames>
                        <PointAnimationUsingKeyFrames Storyboard.TargetProperty="(UIElement.RenderTransformOrigin)" Storyboard.TargetName="Animation">
                            <EasingPointKeyFrame KeyTime="0" Value="-0.5,0.5"/>
                            <EasingPointKeyFrame KeyTime="0:0:1" Value="0.5,0.5"/>
                            <EasingPointKeyFrame KeyTime="0:0:2" Value="1.5,0.5"/>
                        </PointAnimationUsingKeyFrames>
                    </Storyboard>
                </VisualState>
            </VisualStateGroup>
        </VisualStateManager.VisualStateGroups>
        <local:ClippingBorder BorderBrush="{TemplateBinding BorderBrush}"
                                    BorderThickness="{TemplateBinding BorderThickness}"
                                    Background="{TemplateBinding Background}"
                                    CornerRadius="20" Grid.ColumnSpan="3" Grid.RowSpan="1">
            <Grid>
                <Rectangle x:Name="PART_Track" Grid.ColumnSpan="3" Grid.RowSpan="1"/>
                <Grid x:Name="PART_Indicator" HorizontalAlignment="Left" Grid.RowSpan="1" Grid.ColumnSpan="3">
                    <Grid.RowDefinitions>
                        <RowDefinition Height="16*"/>
                        <RowDefinition Height="33*"/>
                    </Grid.RowDefinitions>
                    <Rectangle x:Name="Indicator" Fill="{TemplateBinding Foreground}" RadiusX="20" RadiusY="20" Grid.RowSpan="2"/>
                    <Rectangle x:Name="Animation" Fill="{TemplateBinding Foreground}" RenderTransformOrigin="0.5,0.5" Grid.RowSpan="2" RadiusX="20" RadiusY="20">
                        <Rectangle.RenderTransform>
                            <TransformGroup>
                                <ScaleTransform/>
                                <SkewTransform/>
                                <RotateTransform/>
                                <TranslateTransform/>
                            </TransformGroup>
                        </Rectangle.RenderTransform>
                    </Rectangle>
                </Grid>
            </Grid>
        </local:ClippingBorder>
    </Grid>
    <ControlTemplate.Triggers>
        <Trigger Property="Orientation" Value="Vertical">
            <Setter Property="LayoutTransform" TargetName="TemplateRoot">
                <Setter.Value>
                    <RotateTransform Angle="-90"/>
                </Setter.Value>
            </Setter>
        </Trigger>
        <Trigger Property="IsIndeterminate" Value="true">
            <Setter Property="Visibility" TargetName="Indicator" Value="Collapsed"/>
        </Trigger>
    </ControlTemplate.Triggers>
</ControlTemplate>

Answer: <Window
    Title="temp"  Height="400" Width="400" ResizeMode="NoResize" AllowsTransparency="True" WindowStyle="None" ShowInTaskbar="False" WindowStartupLocation="CenterScreen" Background="Transparent" BorderThickness="0">
<Grid>
    <ProgressBar/>
</Grid>
</Window>

Answer: I would animate the width and not scale the x axis as you are squashing the rectangle.
At the same time I would keep the corner radius of the background and the progress bar the same.


Fundamentals and maths required for algorithms

Question: I have been working on RTOS and Linux driver development for quite some time. Now I am  interviewing at semiconductor companies and failing to answer questions about algorithms on strings, and time and space complexity. I have not studied discrete maths and algorithms during my as I have an electronics background.
How can I overcome this gap?
Comment: There is a great introductory definition for time complexities in SO only http://stackoverflow.com/questions/487258/plain-english-explanation-of-big-o and http://stackoverflow.com/questions/107165/big-o-for-eight-year-olds
Answer: Start with something simple like: Algorithms in a Nutshell (good starting point for interview like questions)

Or Algorithms For Interviews
When you feel you know the above book, then you can think of diving into introduction to Algorithms.
Comment: As one of the authors of "Algorithms of For Interviews" I agree with Paul that this book is not a substitute for a textbook. But I also partially disagree. jammulak has already worked on complex software projects and AFI could be a fun way for him to teach the material to himself and get a quick interview prep. Of course (s)he may need a way to look things up on the net or in a textbook when he is stuck.
Comment: Algorithms for Interviews is not a good way to get started with Algorithms and complexity. It's written for people who've had a couple lectures on this material, worked through the problems, and forgot it all a while ago.
Answer: You need to review most of the course material for a Data Structures and Algorithms class. In order to answer those types of interview questions, you don't generally need the material covered in Discrete Math.
If you want to take the long way around (and actually understand the topic), I recommend you actually work through the class material and listen to the lectures. Since UC Berkeley posts some of their lectures online, you can watch just such a class (with an excellent instructor) here:
http://webcast.berkeley.edu/course_details.php?seriesid=1906978343
Pay particular attention to the lectures on Big O notation, and the sorting and searching algorithms. Those tend to be the bits that people ask about for job interviews.
If you can't be bothered to actually spend the 30 hours watching the lectures and many more working through the problems, you should at least consult the book that course uses:
Goodrich and Tamassia's Data Structures and Algorithms in Java
Answer: Introduction to Algorithms is a great algorithms book (and also happens to be listed 6th on the great influential book question)
Answer: I would suggest you Introduction to Algorithms(by CLRS) and Algorithm design manual for algorithms and complexity(by steven skienna) for understanding the Algorithms and complexity.Other than that there are quite a few good tutorials on algorithms and complexity at top coder ( site : topcoder.com/tc ) , you can check them out too.


Talend + FreeMarker

Question: Is there any way to use FreeMarker template to generate output in Talend OpenStudio (free version)? I want to read data from RDBMS and invoke FreeMarker Template to generate RDF data.
Thanks in advance for your help.
Answer: Talend Open Studio as well as the enterprise version do not have any Freemarker support. You can check Talend exchange to see if the community provides some custom component, which supports Freemarker templates or some other templating framework.
To solve your problem, you could either write your own Talend component (there are various tutorials that teach how to write a component - see links below) or you can add the Freemarker jar by using the tLibraryLoad component and write some Java code in within the job. For example: You could simply use a tJavaFlex component and receive some data flow. In the begin part of the tJavaFlex component, you initialize the Freemarker configuration and load the template from String or classpath. In the main part of the component, you collect the data of the incoming flow within a model, using custom logic. Finally In the end part of the tJavaFlex component, you process the template with the help of your model and write it into a file.
http://www.powerupbi.com/talend/componentCreation_1.html
https://help.talend.com/display/KB/How+to+create+a+custom+component

Talend + FreeMarker


Trazer Registros Existentes e Não Existentes

Question: Tenho a seguinte cena:
$consulta   = Dealer::find(1);

No Model
# Serviços
public function dealer_servicos(){
    return $this->hasMany('App\DealerService', 'id_concessionaria');
}

Daí se eu faço:
@foreach($consulta->dealer_servicos as $value => $res)
   {!! $res->servico !!}
@endforeach

Me traz todos os serviços que a concessionária TEM.
Tem como eu usar a mesma metodologia para trazer os serviços que a concessionária NÃO TEM ? Ou seja, simular o LEFT OUTER JOIN.
Preciso de todos os registros na mesma consulta.
Comment: Possível duplicata de [Como obter registros de uma tabela quando não há relacionamento com outra tabela em Laravel?](http://pt.stackoverflow.com/questions/127252/como-obter-registros-de-uma-tabela-quando-n%c3%a3o-h%c3%a1-relacionamento-com-outra-tabela)
Comment: Tem,  se você criar uma tabela com todos os serviços que a concessionaria deveria ter e depois selecionar todo esse itens e retirar os que a concessionária já tem.
Comment: Isso já existe, meu filho.
Answer: Estou percebendo que você está usando um hasMany de Dealer para DealerService. Creio que provavelmente você terá um relacionamento para Dealer dentro de DealerService.
Então, você pode usar o método whereDoesntHave para fazer isso.
$dealer = Dealer::find(1);

$dealerServices = DealerService::whereDoesntHave('dealers', function ($query) use($dealer)
{
        $query->where(['id' => $dealer->id]);

})->get();

Se notar, vai perceber que a pequena diferença é que essa consulta deve ser feita "fora do relacionamento".
Expressando em palavras, o método whereDoesntHave tem como finalidade fazer um "Quando DealerService não tiver relacionamento com Dealer de id 1".
Comment: Mas ele vai trazer todos na mesma consulta ?
Comment: Sim, mas se quiser usar o Paginate não tem problemas
Comment: Eu vou testar, se não trazer tudo vou até MG dá um murro no meio da sua boca.


In what type of application a small training set would be important?

Question: I have always heard that the more the samples for training a model the better.
But a lot of samples means a lot of computation time while creating the model.
Is there any application where using few samples for training a model, is justified? Is it a good practice?
Thank you
Comment: The more samples the better to generalize, yes, unless you can show there's an asymptotic limit to the improvements with training set size. This has been done time and time again in the literature with learning curves. Consider the application of multiplicative interactions on the MNIST dataset, which allows for acceptable error rates with a few thousand examples only (I saw this on a talk by Geoffrey Hinton, don't remember which one).
Answer: Generally speaking, the more data is available, the better. In fact, papers have found that training a model on 300 million images is better than only 30 million -- it's not the case that our models can be fully "saturated" with just a few million images.
However, obtaining ground truth labels for millions of images can be very expensive. Increasing the sample efficiency of models is a very active area of research. For example, there is lots of work in classifying images with only one sample.
The best approach, from a pragmatic point of view, is to gather as much data as possible, since it generally leads to better results.
Answer: Although this is the type of question where the most correct answer is probably "it depends," I will provide an answer by highlighting an application where a small number of samples is used for training.
Data streams are a challenging learning environment. Faria et al. offer the following definition (based off of Aggarwal et al.'s work):

A data stream $S$ is a massive sequence of multi-dimensional examples $x_1, x_2,..., x_N,…$, which is potentially unbounded, arriving in time stamps $T_1,T_2,...,T_N,…$ Each example $x_i$ is described by an n-dimensional attribute vector.

One specifically challenging aspect of data streams is that they can represent non-stationary environments: the underlying probability distributions from which the instances are drawn and what constitutes 'normal behaviour' can change over time. This is called "concept drift."
With the (potentially) infinite size of the data stream, it is not possible to keep all of the instances in memory. Therefore, right from the start, the algorithm must consider a limited number of instances. Without getting too far into the literature on concept drifting data streams, there are a few aspects which illustrate this (from Gama et al.):

Approaches that retrain do so by discarding their old model and building a completely new one on the basis of the instances being received from the data stream;
Approaches that increment do so by combining their model and/or a summary of past data with what can be learned from the new data;
Approaches that update their model based on change detection do so by collecting a number of instances from the newly detected concept before training; and
Approaches that use a sliding window as a forgetting mechanism inherently limit themselves to considering a limited number of instances from the data stream.

In each of these scenarios, the ideal size of the window/number of instances to consider is variable, but a common consideration is that it should be small enough to allow the algorithm to react to concept drift in a timely manner (while being large enough to be stable when there is no concept drift).
To summarize, the time dimension of the data stream environment produces a pressure on algorithms to be able to learn from a small number of instances (at least initially) in order to adapt to a non-stationary environment in a timely manner.


Training a neural network with uniform random inputs

Question: I'm playing with a little prediction project using a multi-layer perception (MLP) with robust back propagation.  I have a variety of variables which correlate to the single output I'm attempting to predict.
On a whim, I wanted to try providing the neural network with a single input of random uniform values (scaled -1 to 1).  The output surprised me.
A simple plot of the target values vs. the predicted values shows that the neural network learned the data to a surprising degree.
The images below describe what I'm observing.  The scale of the response makes the plot hard to see, but it should seem obvious that the training and test data perform similarly.  I can understand both results having the same distribution and scale, but cannot explain why the test data doesn't show more randomness in it's curve.
The neural network had 125 neurons in a single hidden layer and was iterated 800 times.  If anything, I might have expected overfitting on the training data, which should have exacerbated the randomness in the test data's response even more.
I had hoped to use this as a baseline metric to gauge the meaningfulness of other variables in the data set, though I'm not sure how to proceed from here.
I'm sure I've stumbled upon a well-documented phenomenon / technique, here...  Any thoughts?
Comment: What's an MLP? What's r-prop? Substantively, neural networks are prone to overfitting, so the relevant question is how well does the neural network do against data that it hasn't seen before? I suspect the performance will decline significantly when you apply the model to new data.
Comment: MLP = multi-layer-perceptron.  r-prop is an enhanced algortihm for back-propagation.  That's what had me confused though.  I plotted an X-Y plot of the target values (ordered from lowest to highest) and the corresponding prediction the NN made on the test data.  The predictions tracked rather nicely with the targets.  I suppose I could expect the output's distribution to get "trained" into the network, but I wouldn't have expected it to be so consistent with the target values.
Comment: I've made NNET models that perfectly predict the training data! It's best to train neural networks, and statistical models generally, to optimize out-of-sample performance. Usually this is accomplished with repeated $k$-fold cross-validation.
Comment: I may try that.  Thus far, I've implemented a stratified sampling scheme, using a 70/30 split between the training and test data.  The data's sampled randomly without replacement from 4 strata to help ensure consistent sampling across an otherwise long-tailed distribution.  Right now, I just need to be able to expose an uncorrelated (or poorly-correlated) input for what it really is.
Comment: Are you now observing correlation on new test data, that were not used for training?
Comment: I should expand a bit:  My training and test data do not overlap.  Both are randomly sampled from the total set of data divided into 4 strata.  When I trained  / tested on the data, my only input was a uniformly distributed random value.

The result showed correlation on both training and testing.  The perplexing aspect was that the testing data appears to have the same level of consistency as the training data.

Will try to post a couple x-y plots demonstrating my point...
Answer: It would appear there's something wrong with my current methodology.  Haven't figured out what, exactly, but implementing k-fold immediately demonstrated this to me, as a single random input for a value results no meaningful learning occurring in the network.
Back to the drawing board....
Answer: Perhaps your data has duplicate instances, or at least very similar instances, so when taking some instances for testing, you leaving other similar instances in the training data, this situation is as if your are testing and training on the same data.
You may use Weka to remove the duplicate instances and do the experiment again and see if you still get the same results.


How to get the best signal through several amplifiers?

Question: I have an audio source with internal amplifier. Should I max out its amplification, or keep it around middle, or as silent as possible, before feeding it to another amplifier?
Answer: Each stage must receive the RIGHT level.  Drive it until the peaks start overloading, then back off a notch or two.  The final amplifier, the one driving the speakers, should be turned fully up - the exception being when the system is grossly over-powered for the size of room, then it is acceptable to turn a 1kW system into a 100W one by turning down the final volume control.
Where every item has an input level meter, or at least an overload light, this is straightforward.  Otherwise it can become more of an art, particularly if you're trying to squeeze maximum volume out of a system, or if one component is noisy (not so much of an issue as it used to be).
The term you want to Google for is "Gain Staging".  There's lots of advice out there.


jQuery is not defined

Question: I just started my project which I haven't touched in a while. I got the most recent develop version of it using git and I up'ed my docker. Once I opened my website I saw, that something is off. Console is saying that "Drupal is not defined" and if I visit pages-> --any page-- -> content, the layout is off.

In the console here it says ReferenceError: jQuery is not defined.
What I already tried to do:

Rebuild my project
Resave configuration of jQuery update
Clearing cache (lol)
Checking if JS is enabled on my browser

Can someone help me out please?
Comment: @MassoudiMalek Thank you, but I tried this solution before posting here, sadly nothing helped. :/
Comment: give this a shot : https://www.drupal.org/node/111596
Answer: For someone who might experience this, for me Minify module was causing this, I think I lacked some configuration to use minified JS, so my localhost couldn't load JS, after disabling Minify module, everything was fine. Not the best solution, but for fast results its okay.
P.S. DON'T FORGET TO ENABLE IT IF YOU TWEAK SOME JS CODE AND EXPORT IT AS FEATURE.

jQuery is not defined


padlock on safari mobile not showing persistently

Question: I'm having a strange issue with an HTTPS site where the padlock sometimes disappears on safari mobile (both iphone and ipad). Some pages are more consistent than others, but some times the same page would show a padlock and another time it will be gone...
example pages:

https://m.frau-vintage.com (mostly showing padlock)
https://m.frau-vintage.com/shopping-cart/ (mostly not showing padlock)

I tested those pages with https://www.whynopadlock.com (all green), https://www.jitbit.com/sslcheck/ (no issues). SSLLabs gives an A- score. I was also using Firefox / Chrome (incl. mobile chrome) - and the padlock does seem to appear there consistently with no issues. I also traced all network connections using Chrome developer tools and there are no http connections, only https... I'm wondering what else could cause the padlock to disappear intermittently (and how to resolve this).
Answer: Maybe there is an unsafe script being injected dynamically by one of the libraries you are using specifically on these pages. The script might use http and not https.
This might help you: Find out what resources are not going over HTTPS
Comment: the padlock seems to appear / disappear early on when the page is loaded. It could be some script, but when debugging this on Chrome or Firefox I didn't see any request to non-SSL resource. I couldn't get debugging working on the iphone unfortunately... Also - wouldn't an unsafe script remove the padlock on other browsers? why only Safari?
Comment: whilst this didn't solve the problem, it was the most plausible direction, so I awarded the bounty... Unfortunately my question still remains unanswered :(
Answer: It seems the https://m.frau-vintage.com and the shopping-cart URL are going to different HTTPS servers (or atleast presenting different certificates).
Please check the certificate on the shopping cart server is matching the one on the on the https://m.frau-vintage.com.
Comment: it's a sub-folder on the same domain / server. How can it use a different certificate? (even with SNI, as far as I'm aware, the certificate is tied to the domain name, not a sub-folder)
Comment: Yes, certificates are tied to a host or host+domain only, or a wildcard of a domain. On the net Im on I can only see that one of the URL has certificate that doesnt match the URL. Maybe you have a redirect pointing to non-HTTPS url och different site.
Comment: Maybe you're using an old browser that doesn't support SNI? As I mentioned, I've run various tests, including for the SSL certificate and it seems to be ok... I don't think the problem is with the certificate, but happy to be proven wrong if it means this will resolve the issue with mobile safari.


How do I find if a taxonomy term is used?

Question: I'm using drupal6 taxonomy.The content type 'B' is using a tag field which can use the title of the some of nodes of the content type 'A'.If any node of 'B' tagged a name or title of a node of content type 'A', I want to display one message in the respective node of 'A'.
Alternatively, is there any way to find if www.mysite/category/tag/term doesn't gives the "Page not found" error? This is to find if the term is used, or not.
Answer: If you have the term name, you can use the following code.
$nodes = array();

if ($term = taxonomy_get_term_by_name($term_name)) {
  $result = taxonomy_select_nodes(array($term->tid), 'or', 0, FALSE, 'n.sticky DESC, n.created DESC');
  while ($row = db_fetch_object($result)) {
    $nodes[] = $row->nid;
  }
}

if (!empty($nodes)) {
  // The taxonomy term is used from at least a node.
}

Answer: I think that you could use the function drupal_http_request (in a module or in a template).
This functions returns the response of the request (200 if it's ok, and so on).
Comment: @arrubiu.Thanks for your response,if i want to check this link www.mysite.com/category/tags gives a page not found or not,how to use this drupal_http_request? If my question sounds bad please forgive me.
Answer: Try using taxonomy_get_term_by_name('term name'). It should return an empty array if no term is found, so just check with the empty() function.
Answer: First of all I want to thank whom all gave response to my post.And my answer is,I get the title from the url of content type 'A'(wwww.sitename.com/xxx/yyy/title-raw i get the value of title-raw that is the title of that node.)
And the taxonomy term page of content type 'B' has the url as www.sitename.com/category/content_type_b_machine_name/term name.
so i checked the last term of the taxonomy term page in its url with the title-raw of content-type 'A'
If the www.sitename.com/category/content_type_b_machine_name/title-raw-con-type-A doesnot gives a "400" (page not found error)page then the title is used as a taxonomy term.
Comment: If anyone know a better answers than mine,Please don't forget to share with us.


Bash -- Using a character-based graphical utility to store output

Question: I'm using bash to script a remote dialing program in an Ubuntu(14.04) server and I am having some significant trouble trying to read the output from it. I would like the program to execute as I would normally have it and then I have the timeout command kill it, like so:
#!/usr/bash

echo "begin"

timeout 60s minicom Cisco -d Boston >> log.txt

I am able to run the above command alone on the console without any sort of hiccups however when I run it through bash nothing really seems to happen, it goes through the timeout, but I just end up with these characters in log.txt:
^[[0m^O

I'm a bit green when it comes to scripting in bash; is there perhaps a rule about appending output from a graphical utility to output? I'm really quite confused as to why it is doing this.
Answer: From man minicom
   -C, --capturefile=FILE
        filename.  Open capture file at startup.

So, simply do this:
#!/usr/bash

echo "begin"

timeout 60s minicom Cisco -d Boston -C log.txt

Comment: When testing this solution I found that nothing was written to the specified logfile.
Comment: Shouldn't you be echoing "begin" to `Cisco` ? Like so: `echo "begin" >> Cisco`


Como usar uma função dentro de outra função em python3?

Question: Eu queria pegar o resultado do input da função nome e usar dentro da função intro, porém da forma que estou tentando, obtenho apenas um erro.
código:
def nome():
    name = str(input('Poderia nos dizer seu nome? '))
    return name

def intro():
    nome()
    print('Olá {}, seja bem vindo!'.format(name))

intro()

E este é o erro que obtive como resultado:

sou iniciante na área,agradeço desde já.
Answer: Se na função nome() você retorna o valor lido, você precisará armazenar esse valor em uma variável. A variável name que você definiu dentro de nome() não existe dentro de intro(). São escopos diferentes.
def nome():
    name = input('Poderia nos dizer seu nome? ')
    return name

def intro():
    name = nome()
    print('Olá {}, seja bem vindo!'.format(name))

intro()

Perceba que removi o str() do input também, pois isso é redundante; o retorno do input sempre será uma string.
Comment: Muito obrigado, funcionou como eu queria. No momento tenho que esperar para aceitar a resposta, mas assim que possível o farei.
Answer: Para que seu codigo funcione faça assim:
def nome():
    name = input('Poderia nos dizer seu nome? ')
    return name

def intro(name):
    print('Olá {}, seja bem vindo!'.format(name))

name = nome()
intro(name)

Observe que a variavel name esta sendo passada como um parametro para a fução intro()
Também, no python é permitido a criação de funções aninhadas e o seu código poderia ficar assim:
def intro():
    name = nome()
    print('Olá {}, seja bem vindo!'.format(name))

    def nome():
        name = input('Poderia nos dizer seu nome? ')
        return name

# aqui a chamada da função intro
intro()

E seja feliz...
Comment: Eu ia fazer outra pergunta, pq da forma que coloquei o código, sempre que eu quisesse a função nome dentro de outra função, ficaria pedindo para digitar o nome novamente, da forma que você fez o código funciona. Muito obrigado! pena eu só poder aceitar uma das respostas.


iphone app - dynamic functionality allowed?

Question: I re-framed this question from before to be a little simpler:
Is it allowed to have dynamic functionality in your iphone app i.e. connect to our servers and see what functionality should be downloaded and run on this instance of the app for a specific customer?
Please ask questions if you don't understand what we are after here. Thanks.
Comment: are you talking about something like checking for available speech recognition grammars in an ASR application?  that is probably OK, but I'm pretty sure that downloading any code and then, say, running it inside of an interpreter would be a no-no. this sounds like it might be quite iffy though, so you'd be best off contacting apple and trying to get an answer from them directly.
Comment: Thanks for the comment.
What I mean is this: say you have an app that you log on to. having logged on, the app connects to our servers to see what functions should be available to you, then allow those functions to be run. Like an App Framework if you will, where depending on what your user account says you have access to, you will have access to.
Answer: You are only allowed to downlad code or functionality through the In-App Purchase mechanism. The code or functionality enablers must reside on your own servers, or in the iOS itself, but the availability check must go through the In-App system.
It is however also possible to enable certain features in your application based on external subscriptions, etc. This can for instance be seen in the TripIt application, where "Pro" account holders have access to more functionality. If you decide to go down that path you should thread very carefully since Apple may decide to reject your application if for instance buttons appears dimmed because the user have not been granted access to a certain feature. The way TripIt seems to have "circumvented" this is simply to completely hide this information to the non-Pro user.
Comment: Thanks very much for this answer. I just want to make sure I have understood you correctly:

1) This implies that there's no way of having "plugins" to iphone apps i.e. small chunks of code that gets downloaded from our servers when running the app. Correct?

2) With your approach, the problem is that if you have ALL functions in your ONE app, as soon as one of the functions need an update, you need to update the app. That can lead to a LOT of updating.

Thanks much for taking the time with this. It is much appreciated.


Confusion as to the meaning of `cid` in webform.module

Question: I'm working on a custom module that will return a single question at a time from an existing webform. For example, from the path mymodule/(webform_nid)/(question).
This is my first encounter with cid.
From the documentation on webform_components_form(), it appears that cid represents a component id. (Elsewhere in the Drupal API documentation and on forums, I have seen it referred to as comment id).
Quickly analyzing some test data, it appears that, as a child element of a webform nid, I can reliably use it to sequentially retrieve single questions from a given webform nid.
Am I correct that cids are unique and sequential per nid? I assume the implementation of cid was done to maintain variable name convention.
Or is a cid a unique value assigned to each 'component' across the entire site, as its implementation as comment id would suggest (because you can load an existing comment without loading its node, it would need to be a globally-unique value like nid)?
Answer: A quick look at the webform_component table in the database, the description for that field is: "The identifier for this component within this node, starts at 0 for each node."
In my test, however, the first component has a cid of 1.  AND - keep in mind that if you delete a component, the next cid is incremental - so you can't assume 1 - 6 or whatever.
Comment: I didn't think to check the table comments; they do tend to be decent in Drupal modules. A subtle assumption I made is that component refers only to questions (and not to other things on the webform node).


Guidance in buying a tenor saxophone

Question: I want to buy a tenor saxophone.
The thing is that I used to play a bit on a very old (& nasty) one and now I want to make an upgrade.
I was thinking of buying it from the thoman superstore (btw, I live in europe, so that's like my single option).
I'm not a professional (actually kind of a newbie at sax), I just want to play for myself and with my friends, so I don't really want to make a big investment. My desired price would be somewhere below 700€.
Ok, now for the actual question(s) :

Should I trust smaller brands like "thomann" or "startone"?
Are there any good tips for buying a tenor sax?
Comment: The reputable manufacturers for beginner/intermediate saxophones are companies like Selmer, Yamaha, Jupiter. "Thomann" or "Startone" that you mention are likely to be made-in-China garbage that most music stores will not be able to repair for you if something goes wrong. If Thomann themselves have a repair shop and they'll support your purchase, that would change my mind, but otherwise, find the shop that will be supporting your instrument, and ask them what their beginner/intermediate saxophone is.
Answer: I picked up an excellent second hand one for £200, so I would suggest looking in pawn shops and second hand shops as well as new.
Things to look for in a 2nd hand one:

all the keys work smoothly
each note is in tune
no dents or dings
consistent tone across the full range

That's about it really - I tried 3 different tenor saxophones, which were all fine, before settling on the one I picked up. My decision was on quality of finish - I got the one that looked newest.

Guidance in buying a tenor saxophone


linked datepicker com Jquery

Question: Tenho um formulário com um período de data ('Data inicio' e 'Data fim'). A data de início não pode ser menor que a data final e a data final não pode ser menor que a data de início. Esses inputs estão utilizando o 'datepicker' do JQuery UI (JQueryUI 1.11.0 e JQuery 3.1.1).
Procurei na documentação do Jquery UI e encontrei o exemplo abaixo:
$(function () {
    var dateFormat = "dd/mm/yy",
        from = $("#from")
        .datepicker()
        .on("change", function () {
            to.datepicker("option", "minDate", getDate(this));
        }),
        to = $("#to").datepicker()
        .on("change", function () {
            from.datepicker("option", "maxDate", getDate(this));
        });

    function getDate(element) {
        var date;
        try {
            date = $.datepicker.parseDate(dateFormat, element.value);
        } catch (error) {
            date = null;
        }

        return date;
    }
});

Existe alguma outra forma de fazer isso?
Answer: Existe muitas outras formas de se fazer isso, por ex:
utilizando html e jQuery

$( document ).ready(function() {
  $("#dtaInicio, #dtaFinal").change(function(){
   var inicio = document.getElementById("dtaInicio").value;    var fim = document.getElementById("dtaFinal").value;
    console.log(inicio);
    console.log(fim);
   if (new Date(inicio) > new Date(fim)) {
     alert("inicio maior que fim return false");
     return false;
   }

  });

});

Data Inicial:
<input id="dtaInicio" type="date" value="" size=10 />

Data Final:
<input id="dtaFinal" type="date" value="" size=10 />

Experimente colocar uma data inicial maior do que a final, vai retornar false, no exemplo eu coloquei essa verificação na mudança dos inputs, voce pode colocar ao submit de um formulario

linked datepicker com Jquery


l/bin/ld: cannot find -lstdc++: No such file or directory

Question: When I tried to add the the python dependencies curl-cffi>=0.5.9 in alpine 3.18, shows error like this:
#12 59.10 gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#12 59.10 -Ilibbrotli -Ilibbrotli/include -I/root/visa/.venv/include
#12 59.10 -I/usr/local/include/python3.10 -c build/temp.linux-x86_64-cpython-310/_brotli.c
#12 59.10 -o
#12 59.10 build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/_brotli.
#12 59.10 o
#12 59.10 gcc -shared
#12 59.10 build/temp.linux-x86_64-cpython-310/build/temp.linux-x86_64-cpython-310/_brotli.
#12 59.10 o -L/usr/local/lib -Lbuild/temp.linux-x86_64-cpython-310 -llibbrotli -lstdc++
#12 59.10 -llibbrotli -o build/lib.linux-x86_64-cpython-310/brotli/_brotli.abi3.so
#12 59.10 /usr/lib/gcc/x86_64-alpine-linux-musl/12.2.1/../../../../x86_64-alpine-linux-mus
#12 59.10 l/bin/ld: cannot find -lstdc++: No such file or directory
#12 59.10 collect2: error: ld returned 1 exit status
#12 59.10 error: command '/usr/bin/gcc' failed with exit code 1
#12 59.10 add curl-cffi failed:
#12 59.10 Traceback (most recent call last):
#12 59.10   File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
#12 59.10     result = self.fn(*self.args, **self.kwargs)
#12 59.10   File
#12 59.10 "/usr/local/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line
#12 59.10 286, in install_candidate
#12 59.10     self.manager.install(can)
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pdm/installers/manager.py", line
#12 59.10 33, in install
#12 59.10     installer(str(prepared.build()), self.environment, prepared.direct_url())
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pdm/models/candidates.py", line
#12 59.10 414, in build
#12 59.10     self.wheel = Path(builder.build(build_dir,
#12 59.10 metadata_directory=self._metadata_dir))
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pdm/builders/wheel.py", line 28,
#12 59.10 in build
#12 59.10     filename = self._hook.build_wheel(out_dir, config_settings,
#12 59.10 metadata_directory)
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pyproject_hooks/_impl.py", line
#12 59.10 209, in build_wheel
#12 59.10     return self._call_hook('build_wheel', {
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pyproject_hooks/_impl.py", line
#12 59.10 311, in _call_hook
#12 59.10     self._subprocess_runner(
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pdm/builders/base.py", line 261,
#12 59.10 in subprocess_runner
#12 59.10     return log_subprocessor(cmd, cwd, extra_environ=env)
#12 59.10   File "/usr/local/lib/python3.10/site-packages/pdm/builders/base.py", line 107,
#12 59.10 in log_subprocessor
#12 59.10     raise build_error(e) from None
#12 59.10 pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines
#12 59.10 of the build output:
#12 59.10       |          curl_easy_perform
#12 59.10 gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
#12 59.10 -Icurl_cffi/include -Icurl_cffi/ffi -I/root/visa/.venv/include
#12 59.10 -I/usr/local/include/python3.10 -c curl_cffi/ffi/shim.c -o
#12 59.10 build/temp.linux-x86_64-cpython-310/curl_cffi/ffi/shim.o
#12 59.10 curl_cffi/ffi/shim.c: In function '_curl_easy_setopt':
#12 59.10 curl_cffi/ffi/shim.c:9:16: warning: unused variable 'opt_value'

I have searched from internet and tried to add libstdc++ into alpine, but still could not fixed this problem. This is the docker file look like:
FROM python:3.10-alpine3.18

MAINTAINER jiangxiaoqiang (firstname.lastname@example.com)

ENV LANG=en_US.UTF-8 \
    LC_ALL=en_US.UTF-8 \
    TZ=Asia/Shanghai

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime \
    && echo $TZ > /etc/timezone \
    && mkdir -p /root/visa \
    && apk update \
    && apk add gcc wget curl vim musl-dev libstdc++ curl-dev

RUN pip install pdm
ADD . /root/visa/
WORKDIR /root/visa/
RUN pdm install
ENTRYPOINT exec /root/visa/scripts/startup-app.sh

Am I missing something? what should I do to fixed this issue? This dockerfile works fine with my local macbook pro with M1 chip but could not work on the cloud server. I also tried to add libffi-dev:
RUN apk add libffi-dev

Answer: Finally I found install g++ will fix this issue:
RUN apk add g++

this is the new command:
apk add gcc wget curl vim musl-dev libstdc++ curl-dev libffi-dev g++


PostgreSQL 10 partition by list using enums

Question: I am trying to partition my table using enum column and I am encountering a somewhat strange behaviour.
create type positivity as enum (
    'POSITIVE',
    'NEGATIVE'
);

create table test (id int, polarity positivity) partition by list (polarity);
create table test_1 partition of test for values in ('POSITIVE');
create table test_2 partition of test for values in ('NEGATIVE');

explain select * from test where polarity = 'NEGATIVE';

For this code this is what I get for output of explain:

My question is why is it working like that and is this a bug or a feature in postgres?
EDIT: Adding constraints manually improves the query plan and that way I've got what I wanted (but I am still wandering about the behaviour explained above):
alter table test_1 add constraint test_1_check check(polarity='POSITIVE');
alter table test_2 add constraint test_1_check check(polarity='NEGATIVE');

New query plan for the same explain command:
Comment: You could post this question on the Postgres general mailing list (rather then generating a "real" bug report) - if it turns out to be a bug (and not a case of "not-yet-implemented") I'm pretty sure the developers will pick that up
Comment: Thank you for suggestion, I submitted the issue, seems like there is similar issue concerning arrays also, if you are interested look at:
https://www.postgresql.org/message-id/2b20c220-50cb-3fae-da57-92e8cb3675dd%40lab.ntt.co.jp
Comment: Unrelated, but: it's usually better to post execution plans as plain (formatted!) text  and in most cases it's better to use **`explain (analyze, buffers)`** instead of a simple `explain`
Comment: Might be a bug with enums. It works as expected if `polarity` is defined as `text`
Comment: @a_horse_with_no_name, yeah, sorry for not putting text :( It seems like a bug to me too, but i am to lazy to do everything they require of me just to report this bug
Answer: For anyone looking at this in mid 2018, the fix hasn't made it into release yet; I'm still seeing it happen on 10.4.
Here's the commit for the fix, which appears it is being reviewed this month -
https://commitfest.postgresql.org/17/1591/
Comment: Fix for this will appear in 10.5, which will be released later this week.

Here is the development version of release notes, which mentions an item for the fix for this issue ("Fix handling of partition keys whose data type uses a polymorphic btree operator class, such as arrays"): https://www.postgresql.org/docs/devel/static/release-10-5.html