htaccess 301 redirect in Magento 2

Question: I have trouble to get the 301 redirects working.
My goal is it to redirect domain one with sub directory to domain two without sub directory but with all the attributes.
FA. http://example.com/foo/bar -> http://example.org/bar
I've tried different approaches. The latest is
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{REQUEST_URI} ^/foo/(.*)$
RewriteRule ^(.*) http://example.org/%1 [R=302,NC]

The full .htaccess looks like this:
############################################
## overrides deployment configuration mode value
## use command bin/magento deploy:mode:set to switch modes
#   SetEnv MAGE_MODE developer
############################################
## uncomment these lines for CGI mode
## make sure to specify the correct cgi php binary file name
## it might be /cgi-bin/php-cgi
#    Action php5-cgi /cgi-bin/php5-cgi
#    AddHandler php5-cgi .php
############################################
## GoDaddy specific options
#   Options -MultiViews
## you might also need to add this line to php.ini
##     cgi.fix_pathinfo = 1
## if it still doesn't work, rename php.ini to php5.ini
############################################
## this line is specific for 1and1 hosting
    #AddType x-mapp-php5 .php
    #AddHandler x-mapp-php5 .php
############################################
## default index file
    DirectoryIndex index.php
<IfModule mod_php5.c>
############################################
## adjust memory limit
    php_value memory_limit 756M
    php_value max_execution_time 18000
############################################
## disable automatic session start
## before autoload was initialized
    php_flag session.auto_start off
############################################
## enable resulting html compression
    #php_flag zlib.output_compression on
###########################################
## disable user agent verification to not break multiple image upload
    php_flag suhosin.session.cryptua off
</IfModule>
<IfModule mod_php7.c>
############################################
## adjust memory limit
    php_value memory_limit 756M
    php_value max_execution_time 18000
############################################
## disable automatic session start
## before autoload was initialized
    php_flag session.auto_start off
############################################
## enable resulting html compression
    #php_flag zlib.output_compression on
###########################################
## disable user agent verification to not break multiple image upload
    php_flag suhosin.session.cryptua off
</IfModule>
<IfModule mod_security.c>
###########################################
## disable POST processing to not break multiple image upload
    SecFilterEngine Off
    SecFilterScanPOST Off
</IfModule>
<IfModule mod_deflate.c>
############################################
## enable apache served files compression
## http://developer.yahoo.com/performance/rules.html#gzip
    # Insert filter on all content
    ###SetOutputFilter DEFLATE
    # Insert filter on selected content types only
    #AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript application/javascript application/x-javascript application/json image/svg+xml
    # Netscape 4.x has some problems...
    #BrowserMatch ^Mozilla/4 gzip-only-text/html
    # Netscape 4.06-4.08 have some more problems
    #BrowserMatch ^Mozilla/4\.0[678] no-gzip
    # MSIE masquerades as Netscape, but it is fine
    #BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
    # Don't compress images
    #SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
    # Make sure proxies don't deliver the wrong content
    #Header append Vary User-Agent env=!dont-vary
</IfModule>
<IfModule mod_ssl.c>
############################################
## make HTTPS env vars available for CGI mode
    SSLOptions StdEnvVars
</IfModule>
############################################
## workaround for Apache 2.4.6 CentOS build when working via ProxyPassMatch with HHVM (or any other)
## Please, set it on virtual host configuration level
##    SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
############################################
<IfModule mod_rewrite.c>
############################################
## enable rewrites
    Options +FollowSymLinks
    RewriteEngine on
############################################
## you can put here your magento root folder
## path relative to web root
    #RewriteBase /magento/
############################################
## workaround for HTTP authorization
## in CGI environment
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
############################################
## TRACE and TRACK HTTP methods disabled to prevent XSS attacks
    RewriteCond %{REQUEST_METHOD} ^TRAC[EK]
    RewriteRule .* - [L,R=405]
############################################
## redirect for mobile user agents
    #RewriteCond %{REQUEST_URI} !^/mobiledirectoryhere/.*$
    #RewriteCond %{HTTP_USER_AGENT} "android|blackberry|ipad|iphone|ipod|iemobile|opera mobile|palmos|webos|googlebot-mobile" [NC]
    #RewriteRule ^(.*)$ /mobiledirectoryhere/ [L,R=302]
############################################
## never rewrite for existing files, directories and links
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l
############################################
## rewrite everything else to index.php

    RewriteCond %{REQUEST_URI} ^/shop/(.*)$
    RewriteRule ^(.*) https://example.org/%1 [R=302,NC]
    RewriteRule .* index.php [L]

</IfModule>
############################################
## Prevent character encoding issues from server overrides
## If you still have problems, use the second line instead
    AddDefaultCharset Off
    #AddDefaultCharset UTF-8
    AddType 'text/html; charset=UTF-8' html
<IfModule mod_expires.c>
############################################
## Add default Expires header
## http://developer.yahoo.com/performance/rules.html#expires
    ExpiresDefault "access plus 1 year"
    ExpiresByType text/html A0
    ExpiresByType text/plain A0
</IfModule>
###########################################
## Deny access to root files to hide sensitive application information
    RedirectMatch 403 /\.git
    <Files composer.json>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files composer.lock>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files .gitignore>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files .htaccess>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files .htaccess.sample>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files .php_cs.dist>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files .travis.yml>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files CHANGELOG.md>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files COPYING.txt>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files Gruntfile.js>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files LICENSE.txt>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files LICENSE_AFL.txt>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files nginx.conf.sample>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files package.json>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files php.ini.sample>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files README.md>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files auth.json>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
    <Files magento_umask>
        <IfVersion < 2.4>
            order allow,deny
            deny from all
        </IfVersion>
        <IfVersion >= 2.4>
            Require all denied
        </IfVersion>
    </Files>
# For 404s and 403s that aren't handled by the application, show plain 404 response
ErrorDocument 404 /pub/errors/404.php
ErrorDocument 403 /pub/errors/404.php
################################
## If running in cluster environment, uncomment this
## http://developer.yahoo.com/performance/rules.html#etags
    #FileETag none
# ######################################################################
# # INTERNET EXPLORER                                                  #
# ######################################################################
# ----------------------------------------------------------------------
# | Document modes                                                     |
# ----------------------------------------------------------------------
# Force Internet Explorer 8/9/10 to render pages in the highest mode
# available in the various cases when it may not.
#
# https://hsivonen.fi/doctype/#ie8
#
# (!) Starting with Internet Explorer 11, document modes are deprecated.
# If your business still relies on older web apps and services that were
# designed for older versions of Internet Explorer, you might want to
# consider enabling `Enterprise Mode` throughout your company.
#
# https://msdn.microsoft.com/en-us/library/ie/bg182625.aspx#docmode
# http://blogs.msdn.com/b/ie/archive/2014/04/02/stay-up-to-date-with-enterprise-mode-for-internet-explorer-11.aspx
<IfModule mod_headers.c>
    Header set X-UA-Compatible "IE=edge"
    # `mod_headers` cannot match based on the content-type, however,
    # the `X-UA-Compatible` response header should be send only for
    # HTML documents and not for the other resources.
    <FilesMatch "\.(appcache|atom|bbaw|bmp|crx|css|cur|eot|f4[abpv]|flv|geojson|gif|htc|ico|jpe?g|js|json(ld)?|m4[av]|manifest|map|mp4|oex|og[agv]|opus|otf|pdf|png|rdf|rss|safariextz|svgz?|swf|topojson|tt[cf]|txt|vcard|vcf|vtt|webapp|web[mp]|webmanifest|woff2?|xloc|xml|xpi)$">
        Header unset X-UA-Compatible
    </FilesMatch>
</IfModule>

I assume that some of the other rewrite rules are getting "accepted" so that the desired rule gets kind of bypassed
Answer: You can test this:
Options +FollowSymLinks
RewriteEngine on
RewriteRule ^foo/(.*)$ https://example.org/$1 [R=301,L]

Is works for me.

htaccess 301 redirect in Magento 2


What does keras.preprocessing.image.load_img do during resizing?

Question: What does keras.preprocessing.image.load_img do during image resizing?
In the following statement, target size is set to 128x128.
image1 = img_to_array(image.load_img(img, target_size=(128,128))) / 255

What if I load 100x100 size image?
Will it leave the image as it is or will it zoom to 128x128 size?
Answer: It will actually resize it to the target size!
If the image size is smaller than the target size it will be stretched to fit the desired size.
Comment: It depends to the target size, so it will preserve when they share the same aspect ratio.
Comment: does it maintain the aspect ratio?
Comment: @LahiruKarunaratne No, it does not care about the aspect ratio. It just resizes to the desired size. But I am wondering how exactly. Can anyone in the group answer it? So if it wants to stretch the image when the actual size is less than the desired size, does it do some kind of pixel duplication?


Remove key from array of json using circe

Question: I have a json as:
[
  {
    "a": "a",
    "b": "b",
    "c": "c"
  },
  {
    "a": "a1",
    "b": "b1",
    "c": "c1"
  }
]

Now, I want to delete the key for c. So the result would be:
[
  {
    "a": "a",
    "b": "b"
  },
  {
    "a": "a1",
    "b": "b1"
  }
]

I am using circe, what I have done until now is:
parse(entityAs[String]) match {
  case Right(value) =>
    val json = value.hcursor
    val result = json.downArray.downField("c").delete
    println(">>>>>>>>>>>>>>>>>")
    println(result.right.as[List[Json]])

  case Left(_) =>
}

There is some problem with this line
val result = json.downArray.downField("c").delete

What would be the correct line to delete the key c and then convert it back to list of json using circe?
Answer: delete returns a cursor at the position of the thing the element was deleted from. You want to get back the whole document, you can navigate back to the top with top. You'll get back the modified document. Given
val doc = parse(entityAs[String])

going to the top with
doc.hcursor.downArray.downField("c").delete.top

will give you
Some([
  {
    "a" : "a",
    "b" : "b"
  },
  {
    "a" : "a1",
    "b" : "b1",
    "c" : "c1"
  }
])

But as I understand, you want to remove the next "c" too. That would be
doc.hcursor.downArray.downField("c").delete.right.downField("c").delete.top

or, to delete them all
def removeCs(element: ACursor): ACursor = {
  val deleted = element.downField("c").delete
  val focus = deleted.success.getOrElse(element)
  focus.right.success match {
    case Some(next) => removeCs(next)
    case None => focus
  }
}

removeCs(doc.hcursor.downArray).top


using variable number of arguments in a function call and delegating them to another function

Question: I have got a global function which has the following signature:
void Systemfehlerprotokollieren(BYTE quelle,WORD fehlercode,WORD subfehlercode,
    BYTE klassifizierung, BYTE status,BYTE kanalnummer,DWORD detailfehler,
    WORD modulnummer,WORD location,WORD wLenZusatzText,char *pcZusatztext);

This function I want to simplify in two ways.

Use variable arguments like in printf instead of pcZusatztext
And thus get rid of parameter wLenZusatzText
Use the original signature in a local context

So my external function (the last parameters) would look like:
ext_Systemfehlerprotokollieren(WORD location, char *form, ...);
this function then should call void Systemfehlerprotokollieren(. . . ) with its above mentioned parameters as before.
Right now I have the following code parts:
void vSystemfehlerprotokollierenText(BYTE quelle,WORD fehlercode,WORD subfehlercode,
    BYTE klassifizierung,BYTE status,BYTE kanalnummer,DWORD detailfehler,
    WORD modulnummer,WORD location,va_list args)
{
    int ret;
    char zepuf_printf_mode_lokal[ZELE];
    memset(&zepuf_printf_mode_lokal[0],0x00,ZELE);
    ret = vsnprintf_s(zepuf_printf_mode_lokal, ZELE-1,_TRUNCATE, "%s",args );
    if (ret != -1)
    {
        if (ret < 0)
        {
            Systemfehlerprotokollieren(quelle,fehlercode,subfehlercode,
               klassifizierung,status,kanalnummer,detailfehler,
               modulnummer,location,0,NULL);
            return;
        }
    }
    Systemfehlerprotokollieren(quelle,fehlercode,subfehlercode,
        klassifizierung,status,kanalnummer,detailfehler,
        modulnummer,location,strlen(zepuf_printf_mode_lokal),zepuf_printf_mode_lokal);
}

in above function Systemfehlerprotokollieren will be called as usual
and
void SystemFehlerKG(WORD wFehlerCode, WORD wSubFehlerCode,BYTE KanalNummer,
    DWORD detailinfo2, DWORD detailinfo3, WORD programmstelle, char *form, ... )
{
    va_list args = NULL ;
    if (form != NULL)
    {
        va_start(args, form);
        vSystemfehlerprotokollierenText(SYS_FEHL_QUELLE_FLEXOS,wFehlerCode,wSubFehlerCode,
            SYS_FEHL_KLASS_FEHLER, SYS_FEHL_STATUS_FEHLER_KOMMT_GEHT, KanalNummer,
            (WORD)detailinfo2, (WORD)detailinfo3, programmstelle,args);
        va_end(args);
    }
    else
        vSystemfehlerprotokollierenText(SYS_FEHL_QUELLE_FLEXOS,wFehlerCode,wSubFehlerCode,
            SYS_FEHL_KLASS_FEHLER, SYS_FEHL_STATUS_FEHLER_KOMMT_GEHT, KanalNummer,
            (WORD)detailinfo2, (WORD)detailinfo3, programmstelle,"");
}

This works fine, when i call the second function like this:
SystemFehlerKG(5000+TCPIP_SYS_ERROR_FKT_SEND,0,SYS_FEHL_KANAL_ALLG,
    iRet,0,SFPROG_00000,"%s","");

but I don't know what to change to get the same result with
SystemFehlerKG(5000+TCPIP_SYS_ERROR_FKT_SEND,0,SYS_FEHL_KANAL_ALLG,
    iRet,0,SFPROG_00000);

... where the last two parameters are omitted...
[Edit #1]:
I see, that printf(); doesn't work either, it must be at least printf("");
so the closest possible approach would be
SystemFehlerKG(5000+TCPIP_SYS_ERROR_FKT_SEND,0,SYS_FEHL_KANAL_ALLG,
    iRet,0,SFPROG_00000,"");

[Edit #2 as suggested by @Jan Krüger]:
void SystemFehlerKG(WORD wFehlerCode, WORD wSubFehlerCode,
    BYTE KanalNummer, DWORD detailinfo2, DWORD detailinfo3,
    WORD programmstelle, char *form, ... )
{
    va_list args = NULL ;
    va_start(args, form);
    vSystemfehlerprotokollierenText(SYS_FEHL_QUELLE_FLEXOS,wFehlerCode,
        wSubFehlerCode, SYS_FEHL_KLASS_FEHLER,
        SYS_FEHL_STATUS_FEHLER_KOMMT_GEHT, KanalNummer,
        (WORD)detailinfo2, (WORD)detailinfo3, programmstelle,args);
    va_end(args);
}

I dont have the code at hand right now, but i think, that this threw an exception when form was not a valid (format) string (being NULL or "").
I will tell tomorrow.
[Edit #3:]
I made the changes to my code as in edit#2. Did not work.
But I found the error:
void vSystemfehlerprotokollierenText(BYTE quelle,WORD fehlercode,WORD subfehlercode,
    BYTE klassifizierung,BYTE status,BYTE kanalnummer,DWORD detailfehler,
    WORD modulnummer,WORD location,va_list args)
needs to be
void vSystemfehlerprotokollierenText(BYTE quelle,WORD fehlercode,WORD subfehlercode,
    BYTE klassifizierung,BYTE status,BYTE kanalnummer,DWORD detailfehler,
    WORD modulnummer,WORD location,char * form, va_list args)
the function SystemfehlerKGhas to be changed also to reflect the parameter form.
One thing still isnt very clear to me:
If I am using the ... in my top most function and want to call another function that uses the ... signature, what must i be aware of?
Comment: Change language? C doesn't allow for optional parameters. You have to supply all parameters before hyphens (ie `SystemFehlerKG(5000+TCPIP_SYS_ERROR_FKT_SEND,0,SYS_FEHL_KANAL_ALLG,iRet,0,SFPROG_00000, "");`, or `SystemFehlerKG(5000+TCPIP_SYS_ERROR_FKT_SEND,0,SYS_FEHL_KANAL_ALLG,iRet,0,SFPROG_00000, NULL);` if `NULL` is allowed for `char *form`).
Comment: It depends on how the function works. If documentation says that a `NULL` in format is considered as no other argument is ok. `printf` use the format string to print base text (i.e. `printf("hello!\n");`) and also to check if there are other arguments. So format **must be not null**.
Comment: I cannot change language, it must be C, because its part of a greater project written in plain C.
Comment: I cant get what you wanted to say with your second comment...??
Comment: What I ment is already well expressed in Jan Kruger answer in reference to `printf`. But if you are writing your own function you could have something like: `if (format != NULL) { ... //use variadic part}` checking the value allows you to work or not on the variadic part. `printf` assumes that format always point to memory that can be read, assigning NULL to it will result in a memory access error. In any case the last parameter before the hyphens is a standard parameter and you can't omit it.
Answer: Your edit is well-spotted. In printf and friends, the format string is not part of the variadic part of the function's signature. You have to pass it separately as a normal argument.
The other important bit: you always have to pass on the va_list structure to vsnprintf_s (via your vSystemfehlerprotokollieren), even if you're sure there are no extra arguments. So, don't make the va_start and va_end conditional.
Comment: @WolfgangRoth to answer your final question about calling `...` functions from `...` functions, I believe the simple answer is: you can't. At least not if you want to pass on the variadic arguments. As I said, this is the reason stuff like `vprintf` exists in the first place... it's the only way to pass on your variadic args to printf. Of course, if you're not actually trying to pass your variadic args (e.g. calling `printf` with unrelated arguments), there's nothing special to consider.


Regarding .htaccess

Question: I am using Cent OS hosted on VPS. I want to have  the feature of  "Use URL rewriting of joomla". On enabling this in joomla, it gives a message "Rename .htaccess.txt to .htaccess".
I have renamed the file but on opening the website, browser throws 404 error.
The rewrite module is perfectly loaded and for testing redirection, when I add the below in the httpd.conf, the redirection works perfectly. I think, the htaccess file is not being read.
I have also added the AllowOverride All in the directory setting of my virtual host but nothing changes.
Code Added to httpd.conf:
RewriteEngine On
Options +FollowSymLinks
RewriteRule ^joomla\.html http://www.joomla.org/? [R=301,L]

My question: Where exactly do we need to add AllowOverride All in httpd.conf ? In the directory /var/www/html or my joomla site document root which is /home/admin/mysite
Also does joomla use .htaccess to enabling url rewriting? What if we dont want to use .htaccess and only use httpd.conf
Thanks,
Vineet
Answer: To allow .htaccess you need add this in httpd.conf but under the module which is configured to server your website.
AllowOverride All

Editing root httpd.conf not advisable but on the other hand allowing htacess is also not advisable too.you can use vhost configuration and add configuration regarding your website there.It will add extra layer of security on server.

Regarding .htaccess


Are the theorems of P. Hall and M. Hall equivalent?

Question: Let $A$ be a set together with and indexed collection $\{A_{i}:i\in I\}$ of (not necessarily distinct) subsets of $A$.

A system of distinct representatives of $\{A_{i}:i\in I\}$ is a collection of elements $a_{i}\in A_{i}$ for each $i\in I$ such that, $a_{i}\neq a_{j}$ whenever $i\neq j$.

Phillip Hall's theorem (PH) - [P. Hall, On representatives of subsets, J.London Math. Soc. vol. 10, pp. 26-30 (1935)] is the following statement:

Let $A$ be any set (possibly infinite) together with a finite collection $\mathcal{A} = \{A_{1},\ldots,A_{n}\}$, $n\in\mathbb{N}$, of subsets of $A$. Then a necessary and sufficient condition for the existence of a system of distinct representatives is:

(Condition H) For each $1\leq k\leq n$, any choice of $k$ of the sets in $\mathcal{A}$ contains between them at least $k$ elements of $A$. That is, for every choice of $k$ distinct indices $1\leq i_{1},\ldots,i_{k}\leq n$, $|A_{i_{1}}\cup\ldots\cup A_{i_{k}}|\ge k$.

The folklore statement goes as follows: If there are $n$ women $\{1,\ldots,n\}$, each with a set $A_{i}$ of men they like (i.e. $A_{i}$ is the set of possible matches of woman $i$), there is a way to give all of them distinct husbands they like if and only if condition H holds. That is, if and only if any $k$ women like at least $k$ different men.
Marshall Hall's theorem (MH) - [M. Hall, Distinct representatives of subsets, Bull. Amer. Math. Soc. 54(10), pp. 922-926 (1948). Also Chapter 5 of M. Hall's book "Combinatorial Theory", 2nd. ed. John Wiley & Sons, 1986 has both theorems. They are Theorem 5.1.1 (P. Hall) and Theorem 5.1.2 (M. Hall), respectively.] is the following statement:

Let $A$ be any set (possibly infinite) together with a (possibly infinite) collection $\mathcal{C} = \{A_{i}:i\in I\}$ of finite subsets of $A$. Then a necessary and sufficient condition for the existence of a system of distinct representatives is:

(Condition H*) For each $k\ge 1$, any choice of $k$ of the sets in $\mathcal{C}$ contains between them at least $k$ elements of $A$. That is, for every choice of $k$ distinct indices $i_{1},\ldots,i_{k}\in I$, $|A_{i_{1}}\cup\ldots\cup A_{i_{k}}|\ge k$.

The folklore statement goes as follows: If there is a (possibly infinitely) set of women $I$, each with a finite set $A_{i}$ of men they like (i.e. $A_{i}$ is the finite set of possible matches of woman $i$), there is a way to give all of them distinct husbands they like if and only if condition H* holds. That is, if and only if any $k$ women like at least $k$ different men.
Note: These theorems have widespread graph theoretic formulations. See R. Diestel's book "Graph Theory", for example.
Are PH and MH equivalent statements over $\mathsf{ZF}+\mathsf{BPI}$? (where $\mathsf{BPI}$ denotes the Boolean Prime Ideal Theorem, which is equivalent over $\mathsf{ZF}$ to propositional compactness)
Comment: @MishaLavrov It makes sense. I added references.
Comment: P. Hall is obviously [Philip Hall](https://en.wikipedia.org/wiki/Philip_Hall), but who is M. Hall? It could plausibly be [Marshall Hall](https://en.wikipedia.org/wiki/Marshall_Hall_(mathematician)); do you have a source that says that these two versions of the theorem are due to different Halls?
Comment: (I don't mean to be confrontational, I am just curious about the math history here.)
Answer: In order to remove the question from the unanswered list I will post what I found out.
They are not equivalent over $\mathsf{ZF}$ since PH does not require any choice principle, while MH requires some form of choice. For example, MH implies $\mathsf{AC}_{\text{fin}}$, the axiom of choice for families of finite sets (see Theorem 2.3 in E. Tachtsis, Dilworth's decomposition theorem for posets in $\mathsf{ZF}$, Acta Math. Hungar. 159, p.603–617, 2019)

MH is equivalent (via the compactness theorem for propositional logic or Tychonoff's product theorem for compact Hausdorff spaces, for example - all equivalent to the Boolean prime ideal theorem $\mathsf{BPI}$ over $\mathsf{ZF}$) to the statement of MH with $I$ finite. That is, when the collection $\mathcal{C}$ in the statement of MH is finite (in which case condition H* is simply condition H). Let us call the modified statement MHF [See Paul R. Halmos and Herbert E. Vaughan, The marriage problem, American Journal of Mathematics, Vol. 72, No. 1, pp. 214-215 (1950) for a proof of MH. They use induction on the size of $I$ when the index set is finite, and Tychonoff's theorem when $I$ is infinite].
PH is equivalent to the statement of PH when each of the sets in the collection $\mathcal{A}$ is finite (which would be the case, when the set $A$ in the statement of PH is finite). Let us call the modified statement PHF.
PHF is clearly equivalent to MHF.

Therefore, it is enough to prove PHF, in fact, enough to prove PH for the case of $A$ finite, which can be done by induction on the cardinality of $A$. (Although this does not seem to be any simpler than proving PH directly by induction of size $n$ of the collection $\mathcal{A}$, or via Dilworth's theorem).
Perhaps there is a better way to express the equivalence of PH and MH.
Note: This has a nice exposition on the relationship between MH and the axiom of choice.
Comment: +1. But a small correction: you wrote that Kőnig's infinity lemma is equivalent to BPI over ZF. Assuming that by Kőnig's infinity lemma you mean "every infinite finitely branching tree has a path", this statement is much weaker than BPI.
Comment: I believe Kőnig's lemma is sufficient to prove that MHF implies MH when the set $I$ is countable. But for general infinite $I$, I think you need something stronger (BPI suffices).
Comment: @AlexKruckman Thank you for your feedback. Indeed, I found out that Kőnig's infinity lemma, meaning "every infinite finitely branching tree has an infinite path" is equivalent to the axiom of choice for countable families of finite sets (See Forms 10 and 10F, p.10 & 20 in "Consequences of the Axiom of Choice" by P. Howard and J. Rubin).
Comment: Interestingly, Form 14M (same refernce text) states that R. Cowen's generalization of Kőnig's infinity lemma is $\mathsf{ZF}$-equivalent to $\mathsf{BPI}$. The statement of Cowen's generalization is: "Let $T$ be a collection of
locally finite trees such that for any finite set of levels of $T$, there is a consistent
set of vertices piercing those levels. Then there is a consistent set of vertices
piercing the entire set of levels of $T$.


How do I dynamically modify a block's mark up at render time, in Drupal 7?

Question: I have a page with the following structure:

Header zone

Header One Area (block)

Content zone

Content Area (node)

The rendered node has an image attached to it. What I need to have happen is to make the attached image render inside the "Header One Area" block instead of as part of the content.
This is proving difficult, since blocks and their #markup are rendered before a node is processed. The better idea I had is to create a preprocess function for the page, through code figure out the node_id being rendered, grab its images, and then dynamically modify the blocks markup.
I have tried creating a MYTHEME_preprocess_page() method inside my template.php and then accessing the block via:
if ($block = block_get_blocks_by_region('header_one_area')) {
   // what code do I need here to make a #markup modification stick ???
   $block['block_4']['#markup'] = "
TEST
";
   print kpr($block);
}

I'm not quite sure how to actually modify its markup dynamically before it is rendered here.
Do you have any suggestion?
Answer: I think you are looking for hook_block_view_alter(). It provides both the alterable $data, as well as $block, which contains the module and delta.
To determine which blocks are available, you could start off with:
function MYTHEME_block_view_alter(&$data, $block) {
  kpr($block->module);
  kpr($block->delta);
}

Then once you find the module:
/**
 * Implements hook_block_view_alter().
 */
function YOURTHEME_block_view_alter(&$data, $block) {
  if ($block->module == 'block' && $block->delta == 1) {
    $data['content'] = '
New content
';
  }
}

Or even:
/**
 * Implements hook_block_view_MODULE_DELTA_alter().
 */
function YOURTHEME_block_view_block_4_alter(&$data, $block) {
  $data['content'] = '
New content
';
}

Comment: @MarkoBlazekovic you are mistaken. It says right on that page under $data:

> content: Either a string or a renderable array representing the content of the block.
Comment: Thanks Tim! This is a good solution and noted for future reference! My requirements though brought another solution over at the drupal.org boards. All I had to do was create a Views Block, unformatted field displaying the image with a Contextual Filter set to Content:Nid based on the Content URL ID. Thanks for the response!
Comment: This is wrong, $data['content']  needs to be array and not html

https://api.drupal.org/api/drupal/modules!block!block.api.php/function/hook_block_view_alter/7.x
Answer: if you can delegate the changes to the browser client, you can use Jquery to manipulate the html. See this functions (with examples) to manipulate (change/move/detach/attach/etc) your html http://api.jquery.com/category/manipulation/
http://www.w3schools.com/jquery/jquery_html.asp is another interesting resource.
here is one example to move one html element inside other (in this example the div with id="one" will be inserted inside div id="six"using the appendTo() Jquery method):

text

text

note that you can use jquery in the header (like previous example) to put inside your page.tpl.php or you can use jquery after render your html target:

text

text

Comment: -1 for linking to w3fools, -1 for suggesting altering output with JS.
Comment: Hello Tim, why linking to w3schools.com is an -1 and why use Jquery methods designed to alter html is -1? this can not be an alternative?.
Comment: http://w3fools.com, and because you should use the right tool for every job. When preventing the HTML from ever being generated is so straightforward given the right function, why waste the time having PHP generate it, and then jQuery go through and process it?
Comment: Hello Tim, I agree with use the best alternatives, also I agree in try to make this from the backend, but if an user do not know how to use/get the best, maybe you will be useful to know other possibility. I am offering a possible alternative if the user can not achive the best alternative.
Answer: The accepted answer will overwrite the block's content altogether.  #prefix and #suffix can be used to add markup to the block before or after the content.
function mymodule_block_view_alter(&$data, $block) {
  $data['content']['#prefix'] = '
';
  $data['content']['#suffix'] = '
';
}

The #prefix and #suffix will be honoured for any Drupal render array when it's passed through render() (or drupal_render()).


cs50 tideman lock_paiors function issue

Question: hi guys im having problome with my lockpairs functinog on pset3 tideman would love some feedback ty
bool checkcycle(int from, int to)
{
   if(from == to)
   {
       return true;
   }
   int i;
   for (i = 0; i < candidate_count; i++)
   {
       if(locked[from][i])
       {
           checkcycle(i,to);
       }
   }
   return false;
}

void lock_pairs(void)
{
   for (int i = 0; i < candidate_count; i++)
   {
      if(!checkcycle(pairs[i].winner , pairs[i].loser))
      {
          locked[pairs[i].winner][pairs[i].loser] = true;
      }
    return;
   }
}

:( lock_pairs locks all pairs when no cycles
lock_pairs did not lock all pairs
:( lock_pairs skips final pair if it creates cycle
lock_pairs did not correctly lock all non-cyclical pairs
:( lock_pairs skips middle pair if it creates a cycle
lock_pairs did not correctly lock all non-cyclical pairs
Answer: Your checkcycle function just need a little adjustment. I would change from to winner and to to loser. I think it would be easier to understand. Given a pair, you will call checkcycle(winner, loser). After checking if winner == loser, you should iterate over all pairs checking if loser is the winner, and calling checkcycle(winner, loser), passing the same original winner, and the loser of the loser

cs50 tideman lock_paiors function issue


Principle of superposition after using seperation of variables

Question: To solve a Partial Differential Equation with solution $ u(x,y) $, I separated $u$ into two variables, $u(x,y) = h(x)g(y)$.
For background: the resulting differential equations were:
$ \frac{h''}{h} = -\frac{g''}{g} = \lambda$
My solution got:
For h:
$\lambda = 0 \implies h = c_1$ where $c_1$ is any constant.
$\lambda < 0 \implies -\lambda = (\frac{n\pi}{L})^2$ with corresponding eigenfunction $h_n = \cos{(n \pi x / L)} $ for $ n = 1, 2, 3, ...$
For g:
$\lambda = 0 \implies$ eigenfunction is $ g = y$
$\lambda < 0 \implies$ eigenfunction is $ g_n = \sinh(n \pi y / L)$
How do I now use the principle of superposition to come up with the next step of a solution? I had thought the answer would be:
$u(x,y) = \sum_{n = 1}^{\infty} a_n \cos{(n \pi x / L)} \sinh(n \pi y / L) + \sum_{n = 1}^{\infty} b_n \cos{(n \pi x / L)} y  $
But my textbook gives the next step as:
$u(x,y) = c_0 y + \sum_{n = 1}^{\infty} a_n \cos{(n \pi x / L)} \sinh(n \pi y / L) $
Why is this?
Comment: What are the initial and boundary conditions?
Comment: Full qn: Solve Laplace's Eqn inside a rectangle $ 0 \leq x \leq L, 0 \leq y \leq H$, with Boundary Conditions: $u_x(0,y) = 0, u_x(L,y) = 0, u(x,0) = 0, u(x, H) = f(x)$
Answer: By looking at $h$ you have found what the eigenvalues are:
$$
\lambda_0=0
,\qquad
\lambda_n=-(n\pi/L)^2 \quad (n \ge 1)
.
$$
To each eigenvalue you have a corresponding function $h_n(x)$.
Then for each eigenvalue you solve
$$
g_n''(y)=-\lambda_n g_n(y)
,\qquad
g_n(0)=0
,
$$
to find $g_n(y$).
If you combine $h_n$ and $g_n$, you get a separated solution to the PDE
for each $n \ge 0$:
$$
u_n(x,y)=h_n(x) \, g_n(x)
.
$$
These separated solutions are then put together in a linear combination
$$
u(x,y) = \sum_{n \ge 0} c_n \, u_n(x,y)
,
$$
where the constants $c_n$ are chosen such that $u(x,t)$ satisfies the last boundary condition $u(x,H)=f(x)$.


restoring problems

Question: I tried to use the canvas tag to allow users to draw shapes, but drawing a new line causes all other lines to disappear. Try It (copy+paste to the textarea and click the "Edit and Click Me >>" button). I should mention that this problem exists in all 5 most popular browser (including IE7,8 and IE9beta).
The code:
<!DOCTYPE html>

    <canvas id="cnvs" style="position:absolute;top:0;left:0;border:1px black solid;" onmousedown="beginLine(event.clientX, event.clientY);"></canvas>

(click and drag the mouse to draw a line)

It's probably a save/restore mistake, but I can't find it.

I have tried 2 different forums:

w3schools
MSDN

but nobody could tell what the problem is.
Answer: You misunderstood the meaning of save and restore methods. They save canvas state not content.
Meaning, when you issue canvas.save() you are saving fillStyle, strokeStyle, lineWidth and lineJoin.
Simplest solution to your problem is to keep another canvas of the same size in memory and then once mouse button is released, clearRect and use drawImage method. Or you can push lines to an array and redraw them each time (which might be faster than drawing another canvas over existing one).
Additionally, another tip regarding clearRect. It turns out, this method is incredibly slow. In your case you are not using it often enough to have some significant impact on performance but it is much faster to assign same width and height to canvas object.
Comment: You might especially mention `save` saving transforms (rotate, translate, and scale).

restoring problems


Groupby funciton in Python to output weighed survey value

Question: I'm looking to produce a weighted value of the following variable:
df.satisfaction.value_counts()

Satisfied 74
Dissatified 33
Don't know 4
Name: satisfaction, dtype: int64

I've used the groupby function as follows:
df.groupby('satisfaction').agg('sum')

However, this outputs all of the variables. Is there a way of selecting just the weight variable?
Or is there an alternative that I should know about?
Thanks.
Comment: `df.groupby('satisfaction')['weight'].sum()`?
Comment: Looks good, thanks!
Answer: This method seems to work: df.groupby('satisfaction')['weight'].sum()


Compact connected Lie groups have no continuous square roots

Question: Let $G$ be a group. We say that $f : G \to G$ is a square root of $G$, if $f(x)^2=x$ for all $x \in G.$

Prove that a compact connected Lie group $G$ has no continuous square root.

What if, instead of a Lie group we consider a compact connected group with more that one element. Does the statement remain true?
Comment: Are there any compact connected Hausdorff topological groups that aren't Lie groups? I thought not.
Comment: The locally compact Hausdorff topological groups were classified by Yamabe. I don't understand the full classification, but my memory was that basically there were two types: Things that look like Lie groups, and things that look like the p-adics, plus mixes of the two. Things that look like the p-adics aren't connected, so that's why I think that the groups you are interested in are simply the Lie groups. But, since I don't know the details of Yamabe's result, don't rely on that.
Comment: @David Speyer: what's your intuition about it? why it shouldn't exist?
Comment: Any idea for the main problem?
Answer: Nudged by ehsanmo's comment, I returned to an idea I had a while ago and got it to work.
By hypothesis, $G$ is a compact connected manifold. Notice also that it is an oriented manifold: Choose any orientation on the tangent space at the identity and use left multiplication to translate it to the rest of the group. Let $\delta$ be the doubling map: $g \mapsto g^2$. For any continuous map of compact connected manifolds of the same dimension, we can define the degree of the map. The degree of a composition of two maps is the product of their degrees. So, if $\epsilon$ were a continuous square root of $\delta$, we would have $\delta \circ \epsilon = \mathrm{Id}$ and thus $\deg(\delta) \deg(\epsilon) = 1$.
I will compute that $\deg(\delta) = 2^r$, where $r$ is the dimension of a maximal torus of $G$. Hence, unless $G$ is trivial, $\deg(\delta)$ does not divide $1$ and there is no continuous square root.

Let $T$ be a maximal torus of $G$, with $T \cong (S^1)^r$, and let $W$ be the Weyl group. We will use the description of degree as the number of preimages of a generic point, counted with sign. Let $t$ be an element of $T$ such that the only way to write $t$ as $g u g^{-1}$, with $u \in T$ is to take $g \in W$. A generic element of $T$ will have this property. We claim that $g$ has $2^r$ square roots, and that $\delta$ is orientation preserving near each of them.
Consider a hypothetical square root of $t$. We can write it in the form $h u h^{-1}$ for $u \in T$, so $(h u h^{-1})^2 = h u^2 h^{-1} = t$. By our choice of $t$, we must have $h \in W$ and, replacing $(h,u)$ by $(\mathrm{Id}, h^{-1} u h)$, we may assume that $h = \mathrm{Id}$. In other words, we have shown that every square root of $t$ is in the torus. There are clearly $2^r$ square roots of $t$ in $T$.
Inside the tangent space to $G$ at $\mathrm{Id}$, choose a complement to $T$ and translate it around $T$ by left action. So the tangent bundle to $G$, restricted to $T$, splits as a direct sum of the tangent bundle to $T$ and a complementary trivial bundle. Using the formula $\delta(h u h^{-1}) = h u^2 h^{-1}$, we see that $\delta$ acts on the tangent bundle to $T$ by multiplication by $2$, and acts trivially on the chosen orthogonal complement. So the eigenvalues of the Jacobian of $\delta$ are all $2$'s and $1$'s, and we see that $\delta$ is orientation preserving. We have verified that the $\delta$-preimage of a generic point has $2^r$ elements, each counted with sign $+1$, as desired.
Answer: I think the following toplogical argument will also work:
I will denote $\delta: G \to G$, $\delta(x) = x^2$ and the hypothetical continuous square root $\epsilon$. Since $\delta \circ \epsilon = \mathrm{id}$ we have that $\epsilon$ is injective. By invariance of domain this implies that $\epsilon$ is open. By compactness of $G$ it follows that $\epsilon$ is also closed; hence we see that, because of connectedness of $G$, the square root $\epsilon$ must be a homeomorphism with inverse $\delta$.
So, in particular, $\delta$ needs to be injective. But looking at tori in $G$, this is not the case.
Comment: To conclude that $\delta$ is an inverse, I think you also have to show that $\epsilon \circ \delta = Id$.
Comment: But this holds very generally: Any left inverse $g: G \to G$ of a bijective map $f:G\to G$ must also be a right inverse: Suppose we are given $x,y$ s.t. $g(x) = g(y)$. By surjectivity of $f$ there are $a,b$ such that $f(a) = x$, $f(b) = y$. Hence $a = g(f(a)) = g(f(b)) = b$ implies $x = f(a) = f(b) = y$. So $g$ is injective. Surjectivity is clear.
Comment: Nice! I believe this works.


Is Elmer's rubber cement a viable choice for assembling Table Tennis Racket?

Question: So I'm facing a dilemma, on one end my friend assembled his by looking on amazon for the highest rated water-based glue. On the other end, my other friend uses and recommends Elemer's rubber cement, because he claims it minimizes the damage done to the blade, which would be nice.
Wanted to see if Elmer's rubber cement, is a viable choice for gluing rubber on the blade.
Side note, did some further searching and started wondering...
is rubber cement equivalent to speed glue?
Answer: Elmer's Rubber Cement is the classic glue for Table Tennis Blades.  But it is no longer legal for tournaments.  If you glued within a week you will likely fail the sniff test for VOCs (solvents).
Water based glues are what you are supposed to use now.  Tear Mender, YES Paste and Elmer's no Drip School Glue are common choices now.


Automatic creation of users

Question: I have a big problem. I don't know why, but every day are created users in my site. It's not a normal registration because I have disabled the block for login/registration in my site. The users' status is locked and they are many.
What can I do?
Comment: are you using drupal 6 or 7 ?
Comment: how do you login then ?
Comment: @PremAnand Using `/user` *page*? That's why disabling *block* can't help, probably.
Comment: @PremAnand you can access through example.com/user
Answer: Disabling block is pointless. You need to truly disable registration itself. Go to Administration » Configuration » People and under Who can register accounts? select Administrators only. If that's not enough, make sure you have all security updates installed.
Note: exact path to this setting may be a bit different in Drupal 6 or 8, but you should be able to find it easily enough.
Comment: /user is enabled because is used from admin for access at the siete and edit it. In /user/register the access is denied. Thx very much
Comment: if you do not need registration at all, test the /user/register url with an anonymous user, it must be denied, with @Molot answer it should be the good way
Comment: D6 path: admin/user/settings; D7 path: admin/config/people/accounts
Answer: It's probably spam. You should install a captcha module on the user registration form, like CAPTCHA module
Comment: Good idea if OP wants to keep user registration possible. I was under impression he does not. Well, it's his call ;) btw as far as I know traditional, math, re and blank field captchas are already pretty well useless against better spambots. Riddle and game ones are still OK.
Comment: My answer keep the registration possible

Automatic creation of users


Getting count of records in child table using select statement

Question: I have a stored procedure in which i am trying to select all the columns of a table Table 1. There is another table which uses Table1 primary key as foreign key. I want to count number of records in this foreign key table with that select like this:
SELECT *, count(*) VacancyCount
    FROM Table1 hc
    LEFT JOIN Table2 hv
    on hc.CompanyID = hv.CompanyID
    WHERE hc.Deleted = 0
    group by hc.CompanyID
    ORDER BY NameLang1

but it gives error:

Column 'dbo.Table1.NameLang1' is invalid in the select list because it
  is not contained in either an aggregate function or the GROUP BY
  clause.

Please suggest how to fix this?
Answer: Please try:
select
    *,
    (select COUNT(*) from Table2 hv where hv.CompanyID=hc.CompanyID) VacancyCount
from Table1 hc
where
    hc.Deleted = 0
order by hc.NameLang1, VacancyCount desc

for ordering using the new column
select * from(
    select
        *,
        CONVERT(NVARCHAR(100), (select COUNT(*) from Table2 hv where hv.CompanyID=hc.CompanyID)) VacancyCount
    from Table1 hc
    where
        hc.Deleted = 0
)x
Order by CASE WHEN @OrderByParam = 1 THEN NameLang1 ELSE VacancyCount END

Provided column NameLang1 and VacancyCount are of same datatype.
Comment: Can we order by results using count ?
Comment: Yea... sure... U can add `VacancyCount desc` or `VacancyCount asc`.
Comment: techdo: I am trying this Order by
 CASE WHEN @OrderByParam = 1 THEN NameLang1 ELSE VacancyCount END but it says invalid column VacancyCount
Comment: Inside the case condition, you wont get the alias column name `VacancyCount`. For that you need to make it as a block.
Comment: I used it but it says: Msg 245, Level 16, State 1, Procedure hr_Companies_GetList, Line 13
Conversion failed when converting the nvarchar value 'my company1aa' to data type int.   and i see that my company1aa is a value of column used in order by clause NameLang1
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/29978/discussion-between-techdo-and-dotnetsparrow)
Comment: This adding an inline select count statement causes serious performance issues for me.  It now takes several minutes for a statement that worked in 0.003 seconds previously.
Comment: Regarding efficiency, this will most likely result in sub-select on `Table2` being executed for every row of `Table1`, which is bad if `Table1` is large.
Answer: You're doing grouping wrong. You need to use all the columns from Table 1 in SELECT instead of '*' and in GROUP BY clause as well.
Or you can try a different approach like this:
SELECT *
FROM Table1 hc
LEFT JOIN (SELECT CompanyID, COUNT(*) cnt FROM Table2 GROUP BY CompanyID) hv
on hc.CompanyID = hv.CompanyID
WHERE hc.Deleted = 0
ORDER BY NameLang1

Comment: Can we order by results using cnt ?
Comment: Of course, it becomes an ordinary column when wrapped as a subquery and can be used in any clause (including order by).
Comment: I am using techdo answer but when i use Order by
 CASE WHEN @OrderByParam = 1 THEN NameLang1 ELSE VacancyCount END it says invalid column VacancyCount
Comment: Because it's not inside the subquery (the alias) in his answer, but it is in my one, that's why you're getting an exception. Oracle doesn't know about the column name (read alias) before completing the query and since subqeries are performed before main ones, it's allowed to use aliases used in subqueries as names of tables' columns in order by clause.
Comment: And by the way, the way which was suggested by @techdo is not as efficient in performance terms, there for each row in Table1 a subquery is performed, but in my example there's only one subquery execution and one join operations which's much faster.
Comment: much faster than outer apply
Answer: You will have to list every column in the GROUP BY clause
These columns are those in the SELECT * bit.
This would be correct ANSI SQL anyway.
SELECT * itself is bad anyway: it is always better to explicitly list columns
Answer: Try in this way include column list in group by
 SELECT column1,column2,column3..,NameLang1,count(*) VacancyCount
FROM Table1 hc
LEFT JOIN Table2 hv
on hc.CompanyID = hv.CompanyID
WHERE hc.Deleted = 0
group by column1,column2,column3
ORDER BY NameLang1


Why I am that unsuccessful with predicting with Generalized Lasso (genlasso {genlasso})?

Question: I try to utilize Generalized Lasso genlasso {genlasso} function to incorporate additional penalty matrix into estimation process.
I started with a "hello world" example: I put a diagonal matrix as a penalty matrix in a model. I realized the predictions I got from the model are very far from what I expected:

predictions are negative whereas my response variable is positive.

Regular Lasso from glmnet model gives me at least reasonable (positive) predictions, so my question is: what do I miss in function call or what I do not understand that I end up with such unintuitive predictions from genlasso {genlasso} model.
Reproducible example.
Step. 1. Simulate data based on covariance and mean estimates from my real data (correct simulation result for negative response variable values).
# Read objects describing simulation structure
sim.cov   <- dget(url("https://raw.githubusercontent.com/martakarass/so_q/master/genlasso/sim_cov"))
sim.means <- dget(url("https://raw.githubusercontent.com/martakarass/so_q/master/genlasso/sim_means"))

# Simulate X (explanatory variables) and Y (response) matrices
# from multivariate normal
library(MASS)
set.seed(1)
sim.sample <- mvrnorm(n = 150, mu = sim.means, Sigma = sim.cov)
sim.sample.x <- sim.sample[, 1:66]
sim.sample.y <- sim.sample[, 67]

# Correct for negative response variable value
sim.sample.y[sim.sample.y < 0]
# [1] -0.8409024 -1.8470264 -1.8503388
sim.sample.y[sim.sample.y < 0] <- abs(sim.sample.y[sim.sample.y < 0])
summary(sim.sample.y)
# Min.  1st Qu.   Median     Mean  3rd Qu.     Max.
# 0.01549  7.39000 10.93000 10.72000 13.58000 24.77000

Step 2. Simulate train and test subset indices
train.idc <- as.logical(rbinom(n = nrow(sim.sample), size = 1, prob = 0.8))
test.idc <- !train.idc

Step 3. Build regular Lasso model. Predict values with the use of best lambda value from cross-validation.
# LASSO
# Cross-validate on a train set
library(glmnet)
cv.glmnet.out <- cv.glmnet(x = sim.sample.x[train.idc, ],
                           y = sim.sample.y[train.idc],
                           nfolds = 10, type.measure = "mse", alpha = 1)
lambda.cv <- cv.glmnet.out$lambda.min
model.lasso <- glmnet(sim.sample.x[train.idc, ], sim.sample.y[train.idc],
                      alpha = 1)
model.lasso.predicted <- predict(model.lasso, s = lambda.cv,
                                 newx = sim.sample.x[test.idc, ])

# Investigate predictions from a model
summary(as.vector(model.lasso.predicted))
#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
#   3.412   9.235  10.850  10.960  12.760  17.950

Step 4. Build Generalized Lasso model. For convenience, use same best lambda as in regular LASSO model.
# LASSO `genlasso`: diagonal penalty matrix
p <- ncol(sim.sample.x)
library(genlasso)
genlasso.diag.out <- genlasso(y = sim.sample.y[train.idc],
                              X = sim.sample.x[train.idc, ],
                              D = diag(1, p))
model.genlasso.diag.predicted <- predict(genlasso.diag.out, lambda=lambda.cv,
                                         Xnew = sim.sample.x[test.idc, ])$fit

-> investigate predicted values, which are negative
summary(as.vector(model.genlasso.diag.predicted))
#     Min.  1st Qu.   Median     Mean  3rd Qu.     Max.
# -27.2800  -6.0070   1.2580   0.3049   6.2230  24.4700

Comment: The fact that negative values are predicted is not really problematic, but it *is* apparent that the predictions are far worse. I notice in the package help that the authors strongly suggest not fitting LARS with `genlasso`. Perhaps there's some numerical instability of the algorithm?
Comment: I wouldn't be surprised by a few of them being negative, but *a half* of them, with median so close to 0 and `1st Qu.` = -6, is something I do not get. Perhaps some numerical instability of the algorithm might cause the problem - if yes, then it would be pretty bad as I believe it is "simple" n>p problem.
Answer:
There are two issues:

glmnet includes an intercept by default while genlasso does not.
To disable the intercept in glmnet, use intercept=FALSE; it will reduce the performance significantly.  To instead add an intercept in genlasso, cbind a column of ones onto your X matrix, and change D accordingly.  If you choose to penalize it, you might need to set the corresponding value in the p+1 by p+1 D matrix more carefully than in the example code below.  By setting penalize.intercept to FALSE in the code below, it is not penalized by using a p by p+1 penalty matrix D which is an identity matrix with the row corresponding to the intercept cut off.
the lambda values for glmnet and genlasso are on different scales.
Note that glmnet
optimizes
$\frac{1}{2N}\sum_{i=1}^N(y-\beta_0-x_i^T\beta)^2+\lambda\left[(1-\alpha)\left\|\beta\right\|_2^2+\alpha\left\|\beta\right\|_1\right]$, while genlasso optimizes $\frac12\left\|y-X\beta\right\|_2^2+\lambda\left\|D\beta\right\|_1$; both use the $L^1$ norm for the lasso penalty, but glmnet's error term is equivalent to the $L^2$ norm divided by $2N$, while genlasso's formulation divides by just $2$.  The $\lambda$ values from one must be rescaled to be applied to another.  This is also an issue when using the CV'd $\lambda$ values for fitting a model with significantly different $N$, which is ignored in the code below.
We can either scale lambda.cv to the appropriate value for genlasso, or use cross-validation to select an appropriate value directly.  The code below takes the latter approach.  Note that glmnet provides two methods of selecting the lambda value from cross-validation: the one giving the minimum CV loss (lambda.min), and one from the 1-standard-error rule, which adds some more regularization in exchange for a bit higher CV loss (lambda.1se); the 1se rule produces better out-of-sample performance in this case.

Code for glmnet without intercept, genlasso with intercept and CV-selected lambda:
set.seed(42) # set RNG seed for reproducibility
n.train <- sum(train.idc) # number of training points
n.folds <- 10L # number of CV folds
foldid <- sample(rep_len(seq.int(n.folds), n.train)) # fold number for each training point
## (creates folds with roughly the same number of points)

train.x <- sim.sample.x[train.idc, ]
train.y <- sim.sample.y[train.idc]
test.x <- sim.sample.x[test.idc, ]
test.y <- sim.sample.y[test.idc]

## Call cv.glmnet specifying the foldid's:
cv.glmnet.fit <- cv.glmnet(x = train.x,
                           y = train.y,
                           foldid = foldid,
                           type.measure = "mse", alpha = 1)
## Choose the lambda with the minimum CV-estimated loss:
cv.glmnet.lambda.min.pred <- predict(cv.glmnet.fit,
                                     s = cv.glmnet.fit$lambda.min, #$
                                     newx = test.x)
## Using the 1-standard-error lambda selection rule:
cv.glmnet.lambda.1se.pred <- predict(cv.glmnet.fit,
                                     s = cv.glmnet.fit$lambda.1se, #$
                                     newx = test.x)
## with no intercept:
cv.glmnet0.fit <- cv.glmnet(x = train.x,
                            y = train.y,
                            foldid = foldid,
                            type.measure = "mse", alpha = 1,
                            intercept=FALSE)
cv.glmnet0.lambda.min.pred <- predict(cv.glmnet0.fit,
                                      s = cv.glmnet0.fit$lambda.min, #$
                                      newx = test.x)
cv.glmnet0.lambda.1se.pred <- predict(cv.glmnet0.fit,
                                      s = cv.glmnet0.fit$lambda.1se, #$
                                      newx = test.x)

## Get lambda sequence for genlasso on all training data:
D <- diag(1, p+1)
penalize.intercept <- FALSE
if (!penalize.intercept) D <- D[-1,]
genlasso.fit <- genlasso(y = train.y,
                         X = cbind(1,train.x),
                         D = D)
## Evaluate each lambda on each fold:
fold.lambda.losses <- tapply(seq_along(foldid), foldid, function(fold.indices) {
  fold.genlasso.fit <- genlasso(y = train.y[-fold.indices],
                                X = cbind(1,train.x[-fold.indices,]),
                                D = D)
  ## length(fold.indices)-by-length(cv.genlasso.fit$lambda) matrix, with
      ## predictions for this fold:
      ## $
  fold.genlasso.preds <- predict(fold.genlasso.fit,
                                 lambda = genlasso.fit$lambda, #$
                                 Xnew = cbind(1, train.x[fold.indices,]))$fit #$
  lambda.losses <- colMeans((fold.genlasso.preds - train.y[fold.indices])^2)
  return (lambda.losses)
})
## CV loss for each lambda:
cv.lambda.losses <- colMeans(do.call(rbind, fold.lambda.losses))
cv.genlasso.lambda.min <- genlasso.fit$lambda[which.min(cv.lambda.losses)] #$
## Caution, this lambda may need rescaling based on the ratio of the full training set size to the fold training set sizes.

## Predict:
cv.genlasso.lambda.min.pred <- predict(genlasso.fit,
                                       lambda = cv.genlasso.lambda.min,
                                       Xnew = cbind(1,test.x))$fit #$

summary(as.vector(cv.glmnet.lambda.min.pred))
summary(as.vector(cv.glmnet.lambda.1se.pred))
summary(as.vector(cv.glmnet0.lambda.min.pred))
summary(as.vector(cv.glmnet0.lambda.1se.pred))
summary(as.vector(cv.genlasso.lambda.min.pred))

mean((cv.glmnet.lambda.min.pred-test.y)^2)
mean((cv.glmnet.lambda.1se.pred-test.y)^2)
mean((cv.glmnet0.lambda.min.pred-test.y)^2)
mean((cv.glmnet0.lambda.1se.pred-test.y)^2)
mean((cv.genlasso.lambda.min.pred-test.y)^2)

Comment: Thanks for pointing out the typos.  The regularization terms are indeed the same, but the error terms are not: the $L^2$ norm in genlasso does not divide by $N$ like glmnet's formulation does.   Based on this observation, the ratio of equivalent lambdas between the two should theoretically be $N$; however, the CV-selected lambda's don't seem to follow this rule...  This brings up another issue: the CV-selected genlasso lambda may need scaling since the CV training sets are smaller than the full training set.
Comment: **locoboro**, thank you for your comprehensive answer! I would like to kindly ask you to clarify my thinking about where does "a scaling factor difference" comes from. I thought that in this particular example ($\alpha = 1$ and $D$ being a diagonal) I should end up with a regular Lasso formula (as $\lambda[(1-\alpha)||\beta||^2_2 + \alpha||\beta||_1]$ narrows down to $\lambda||\beta||_1$ and $\lambda||D\beta||_1$ narrows down to $\lambda||\beta||_1$), shouldn't I?
Comment: (Besides, I would also suggest correcting a few typos ("\" appearing before `$` and changing name of the parameter in `predict` function from `lambda =`  to `s =`.)


A question on convergence of Fourier series and the derivative of the function

Question: Consider a function $f_p : \mathbb{R} \to \mathbb{R}$ which is continuously differentiable in $(0,2\pi)$ except at two points $x = x_c$ and $x = x_o$. At $x = x_c$, $f_p(x)$ has a jump discontinuity. At the point $x = x_o$, $f_p'(x_o^+)$ and $f_p'(x_o^-)$ exist and are not equal.
Now define a function $f$ equal to $f_p$ on $(0,2\pi)$ and let it be a periodic function with period $2\pi$. Let $\hat{f}_k$ be the Fourier series coefficients of $f$. What I would like to know is whether the Fourier series defined by the coefficients $ik\hat{f}_k$ converge to $$\frac{f'(x_o^+)+f'(x_o^-)}{2}$$ at $x = x_o$ ?
Comment: $g=f_p'$ is piecewise continuous and the Fourier series of $g$, when evaluated at $x_0$, should converge to the average of its left and right limits at $x_0$. So you are essentially asking if $is\hat{f}(s)=\hat{g}$. But if this is true, we should also have $is\widehat{(f+H)}(s)=\hat{g}$, where $H$ is the Heaviside function, because $f'=(f+H)'$. This implies that $\hat{H}(s)=0$, which is weird. So I guess the answer to your question is negative.
Comment: @user1551 : you are probably right. I remember reading that it diverges at such points. I don't have the reference book in hand right now (Trig. Series by Zygmund) and hence this question...someone pleas confirm in form of an answer, even without any proof.
Answer: It will diverge, but if I am not mistaken, can be Cesàro summed.
Your function can be written as a sum of a Lipschitz function that is continuously differentiable except at (at most) two points, and a sawtooth wave. The first component has Fourier series that converges, and has the property that the Fourier series of the derivative is $ik\hat{f}_k$. The second component, however, gives the problem. A sawtooth wave has Fourier series expansion (up to a normalising constant) $\hat{f}_k = \frac{(-1)^k}{k}$. So the main issue is handling the sum of the form
$$ \sum (-1)^k\sin(k x) $$
This of course is not absolutely convergent. But away from the singularity, you can take the Cesàro sum: the partial sum of the above expression is more or less the Dirichlet kernel (shifted by a constant). And so the partial Cesàro sum is (roughly) just the Fejér kernel, which is known to converge point-wise away from the singularity.
Comment: It is Cesaro summable $(C,r)$ away from the singularity, only for $r > 1$. (given in the book Trignometric Series by Walker Page no. 55 section 3.5)...but by imposing a condition that the singularity to be that of a jump type, would we be able to make it $(C,1)$ summable ? could there be any such possibility ?


Add new Google sheet row when new value exists

Question: I have two Google spreadsheets, A and B: A gets replaced every day when a scheduled report copies over it and B remains the same (and is the one I want to update).
On a scheduled basis, I want to take email addresses that appear in A and see if they also exist in B. If they don't, then I want them added to a new row in B.
I have tried coming up with a way to automate this - either through macros or Zapier and I'm coming up short. All I want to know is if this is possible and what the best way to accomplish this would be. (I'm thinking of paying a contractor to do this, but I don't really know what I'd be asking for at this stage.)
Comment: share a copy/sample of your sheet with example of desired output
Answer: You can create a script in one of your sheet,
In your sheet open the script editor, by clicking on tool > Script editor.
create a function let's say :
<code>function saveAtoB(e){
}
</code>
then on the left menu you have some clock icon, where you can add triggers.
click on add trigger button (right bottom).

select your saveAtoB function,
select HEAD
select trigger on time
select daily.

then save.
The automation part is Done, quite easy.
Now it is quite easy to write the code inside the saveAtoB function, if your are familiar to code in JavaScript. Else it could be little tricky.
If you share a copy of your sheet I might help.
(I suggest not to pay a contractor to do this since it probably would take less that one hour).
Comment: Thanks! Good to know this is possible. I am not familiar with JavaScript unfortunately


How to get the root record in Oracle database

Question: Starting from the generic node (or leaf) how to get the root node?
SELECT
    ID,MSG_ID,PARENT_ID
FROM
    TABLE_X
CONNECT BY PRIOR PARENT_ID = ID;

ID  MSG_ID                              PARENT_ID
4   3                                   NULL
5   93bea0f71b07-4037-9009-f148fa39bb62 4
4   3                                   NULL
6   6f5f5d4ab1ec-4f00-8448-7a6dfa6461b2 4
4   3                                   NULL
7   3                                   NULL
8   7e0fae569637-4d29-9075-c273eb39ae8e 7
7   3                                   NULL
9   8a3e7485b3e8-45b1-a31d-c52fd32111c0 7
7   3                                   NULL
10  fcc622d5af92-4e61-8d7c-add3da359a8b 7
7   3                                   NULL

How to get the root msg_id?
Comment: Am I reading this right? There are two roots, and the `msg_id` for both is just `'3'`?
Comment: Yes in the db i can have two roots.
Answer: You can use a recursive query:
with cte (id, msg_id, parent_id) as (
    select id, msg_id, parent_id, from mytable where id = ?
    union all
    select t.id, t.msg_id, t.parent_id
    from mytable t
    inner join cte c on c.parent_id = t.id
)
select * from cte where parent_id is null

Comment: There is no need to scan `cte` a second time - and there is no need for `lvl`. The `where` clause should be `where parent_id is null`.
Answer: You were on the right path, but you are missing two essential ingredients.
First, to indicate the starting point, you need to use the start with clause. Obviously, it will be something like start with id = <input value>. You didn't tell us how you will provide the input value. The most common approach is to use bind variables (best for many reasons); I named the bind variable input_id in the query below.
Second, you only want the "last" row (since you are navigating the tree in the opposite direction: towards the root, not from the root; so the root is now the "leaf" as you navigate this way). For that you can use the connect_by_isleaf pseudocolumn in the where clause.
So, the query should look like this: (note that I am only selecting the root message id, as that is all you requested; if you need more columns, include them in select)
select  msg_id
from    table_x
where   connect_by_isleaf = 1  -- keep just the root row (leaf in this traversal)
start   with id = :input_id    -- to give the starting node
connect by prior parent_id = id
;


Invoke transactional method within the same SFSB

Question: Is it possible to invoke a transactional method from within a Stateful EJB itself? To speak more clearly:
<code>@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
@Stateless
public class MyService {

    @Resource
    SessionContext ctx;

    public void myMethod()  {
        // do something...

        // invoke method from the same class

        // As expected - this doesn't work as it's a regular local-call,
        // it's not aware of EJB nature of this call.
        save();

        // Doesn't work (although it worked with SLSB)
        ctx.getBusinessObject(MyService.class).save();
    }

    @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
    public void save() {
        // do something...
    }
}
</code>
Now what I want to achieve is to let the user call myMethod(); I want to be sure that this method is executed without the JTA transaction. After this call I want to invoke a save(); method which will be run within a transaction.
If I use the ctx.getBusinessObject(-) method I get:

WARNING: A system exception occurred during an invocation on EJB
  MyService method public void com.test.MyService.save()
  javax.ejb.IllegalLoopbackException: Illegal Reentrant Access : Attempt
  to make a loopback call on method 'public void
  com.test.MyService.save() for stateful session bean MyService

Are the inner calls not supported for SFSBs?
I'm running Glassfish 3.1.1.
Comment: You can solve this doing the following: Inject the same Service in  your class like this @EJB private MyService self; and then in myMethod() call the save methos like this self.save(). I have tried this in wildfly8.
Answer: It might be a bug in the Glassfish EJB implementation. It doesn't just happen when you call a method with a different transaction attribute, it happens with every reentrant call to a stateful session bean.
Just try putting a simple test method in your stateful bean and call it via the business object proxy. You'll get the same exception.
On JBoss AS 7, reentrant calls to stateful beans are allowed. Incidentally, a similar bug was present in OpenEJB some time ago: https://issues.apache.org/jira/browse/OPENEJB-1099
Comment: Thanks Arjan. Yeah, I've tested it without different Tx attributes. I will post this on the Glassfish JIRA and see what will happen.
Comment: Do you have an URL to your JIRA entry? I reproduced the issue [here](https://github.com/MartinanderssonDotcom/java-ee-concepts/blob/master/src/test/java/com/martinandersson/javaee/ejb/sessionbeans/tests/StatefulTest.java) (see last test method). You may clone this Arquillian test repository and test the project as is really easily. Default Maven profile uses Arquillian's remote adapter for GlassFish, so have that server running. I made this test pass for both WildFly (former JBoss) and GlassFish as I can not declare a winner/looser using the EJB specification only.
Comment: @MartinAndersson In the mean time I've committed a test for this to the [Java EE 7 Samples project](https://github.com/javaee-samples/javaee7-samples. Maybe it's an idea for you to contribute your tests to that project as well? It's more convenient to have all tests in one place (I had a separate suite of JASPIC tests that I contributed as well)
Comment: @PiotrNowicki Did you ever created a GlassFish JIRA for this?
Comment: Arjan, was at first thinking of becoming a contributor but favor my current project setup where everything is just one Maven module with test files and everything is done programmatically. Also, my long term goal is to not just prove or disprove Java EE concepts but also superawesome docs that explain as much as possible with quotes from the specifications.
Comment: Will look into your new test case within a week :) Many thanx to both of you two for your hard community work.


Filtering Data in CSV file

Question: I have CSV file in this format
a,b,c,d,e,f,no disk detected
a,b,c,d,e,f,disk run into error
a,b,c,d,e,f,no memory in the server
a,b,c,d,e,f,memory has correctable errors

in the last column, I need to search for the word
disk and replace it with  disk error
memory and replace it with a memory error
That part I already figure out
 cat filename.csv |awk -F "," '{print $NF}' |sed 's/^.*disk.*$/disk error/'  |sed 's/^.*memory.*$/memory error/'

Now the part I need help to when sed replace this string, can it's possible to write into the same file(filename.csv) or generate a new file with all columns + updated column, so new file will look like this
 a,b,c,d,e,f,disk error
 a,b,c,d,e,f,disk error
 a,b,c,d,e,f,memory error
 a,b,c,d,e,f,memory error

Answer: It is easier to do with sed:
sed -E 's/^(.+,).*(disk|memory).*$/\1\2 error/' file.csv

a,b,c,d,e,f,disk error
a,b,c,d,e,f,disk error
a,b,c,d,e,f,memory error
a,b,c,d,e,f,memory error

To make changes inline in same file use:
sed -i.bak -E 's/^(.+,).*(disk|memory).*$/\1\2 error/' file.csv

== Details ==
Search Regex:

^: Start
(.+,): Greedy Match till last comma and capture it in group #1
.*(disk|memory): Match 0 or more characters before matching disk or memory and capture it in group #2
.*$: Match 0 or more characters before end

Replacement Pattern:

\1: Back-reference to group #1 to place text till last comma back
\2 error: Append disk error or memory error
Answer: An awk solution:
awk -F, '{if ($NF~/disk/) $NF="disk error"; if ($NF~/memory/) $NF="memory error";}1' OFS=, file

With GNU awk v4.1.0+, you can add -i inplace switch to change the file inplace.
Otherwise, try awk .... file | tee file.
So above command itself does not require GNU awk, but if you have GNU awk, you can do it with this more concise way:
awk -F, '{match($NF,"(disk|memory)",m);$NF=m[1] " error";}1' OFS=, file

NF means column number, $NF means the last column.
-F, set the FS field separator to comma.
OFS=, set the outout field separator to comma.
Comment: We can do without the very last semicolon.
Comment: @ClaesWikner Yep, it can be saved. Thanks for mention it. I added it here is to have some kind of *closure*, just feeling better :)

Filtering Data in CSV file


Image of the intersection is contained in the intersection of images; why are not these equal?

Question: If there are any minor mistakes in my proof, it would be great if they were pointed out - but let it not be the central discussion. I'm rather concerned why the answer is $\subset$ instead of $=$ since I have proved it both ways. It's not immediately clear to me.

Prove $f(\cap \scr{C}) \subset \cap f(\scr{C})$.

Here are the conditions:

Let $y \in f(\cap\scr{C}$).
Then $\exists x \in \cap\scr{C}$ $: f(x) = y$.
Then $\forall C \in \scr{C}$ $: x \in C$.
Then $\forall C \in \scr{C}$, $y \in f(C)$.
Then $y \in \cap f(\scr{C})$.

For the RHS $\Longrightarrow$ LHS, I proved it just going backwards. I haven't thought of a contradiction where this fails.

Let $y \in \cap f(\scr{C})$.
Then $\forall C \in \scr{C}$, $y \in f(C)$.
Then $ \exists x \in C \hspace{2 pt} [\forall C \in \scr{C}$ $: f(x) = y]$.
Then $x \in \cap \scr{C}$.
So $y \in f(\cap\scr{C})$.
Comment: This question is very much related to http://math.stackexchange.com/q/502404/11994 from the same user.
Answer: I'll take things in order. The second line of the first argument isn't right: $f$ need not be injective, so $f^{-1}(y)$ need not even make sense. You could write $x\in f^{-1}[\{y\}]$, but it would be better and clearer to say simply that there is an $x\in\bigcap\mathscr{C}$ such that $f(x)=y$. The remainder of the first argument, however, is fine.
The second argument is simply wrong, and the conclusion of it is false. Specifically, the third line is false: for each $C\in\mathscr{C}$ there is an $x_C\in C$ such that $f(x_C)=y$, but it might be a different $x_C$ for each $C\in\mathscr{C}$. Here, as in the erroneous step in the first argument, you seem to be unconsciously assuming that $f$ is injective, so that a given $y\in Y$ can be the image of only one point of $X$.
To see specifically how this can happen, imagine that $X=\{0,1\}$, $Y=\{0\}$, $f(0)=f(1)=0$, and $\mathscr{C}=\{\{0\},\{1\}\}$. $\bigcap\mathscr{C}=\varnothing$, so $f\left(\bigcap\mathscr{C}\right)=\varnothing$, but $0\in\bigcap f\left(\mathscr{C}\right)$.
Comment: @Don: No, you're not: nothing in that argument depends on $f$ being onto, since you're considering only points $y$ that are in the range of $f$.
Comment: I'm actually assuming $f$ is onto in the second argument.
Answer: For the simplest case take two subsets $A$ and $B$ of a set $E$ and $f:E\longrightarrow F$ a map and then it is already true that $f(A\cap B)\subset f(A)\cap f(B)$ but not the converse. Here is a counterexample. Let $E=\{x_1,x_2\}$ and $F=\{y\}$ and $$f:E\rightarrow F;\; x_1\mapsto y\; \text{and}\;  x_2\mapsto y$$
Let $A=\{x_1\}$ and $B=\{x_2\}$.  We have that $f(A\cap B)=f(\emptyset)=\emptyset$ while $f(A)\cap f(B)=\{y\}\cap \{y\}=\{y\}$. Hence the result.


How do i avoid runtime type checks in this code?

Question: As part of the assignment, I wrote the below code to encode the 2d-array into Doubly linked list.
RunLengthEncoding class takes care of encode/decode functionality. Amidst implementation of this class, there was a necessity to typecast some variables to push the type check(String) at runtime.
Below are the lines that does typecasting:
class RunLengthEncoding {
  private DList2 list;

  private String nextRun() {
      Object obj = null;
        .....
      obj = this.list.nTh(this.sizeOfRun);
      return (String)obj;
  }

  public void check() {
          ....
      while(node != this.list.sentinel){
         int nodetype = Utility.regexChecker("\\D{1,}", (String)node.runObject, 1);
      }
  }

}

====================
typecast is done because, because doubly linked list(class DList2{..}) is a list of DListNode2's which accepts and returns object of type Object.
class DListNode2{
   Object runObject;
   DListNode2 prev;
   DListNode2 next;
}

My question:
How should i avoid type casting in RunLengthEncoding class to avoid runtime type checks, when i use DList2 class?
Note: I am a java beginner.
Comment: How does the standard java.util.LinkedList class solve that problem? How is it declared and used?
Comment: You need to use generics to avoid run time type checking.
Answer: You should use generics. For example:
class DList<T> {
    DListNode2<T> head; // Or whatever properties your class contains
}

class DListNode2<T> {
   T runObject;
   DListNode2<T> prev;
   DListNode2<T> next;
}

And in your RunLengthEncoding class declare list like this: DList<String> list;. That way you can define the type of the parameters in your list and won't have to do casting when you retrieve your object from the list.
Take a look at the standard Java Collections API classes, for example LinkedList. They do this the same way.
Comment: i see `LinkedList` instead of `LinkedList` in java.util. what is the difference?
Comment: is this the only reason why we use generics?
Comment: You can use E, T, ... it does not matter. But general convention is to use single uppercase letter for type parameters. And as for the second question, the reason we use generics is to create classes which can be used with different data types without having to write classes for each type or having to cast Object types to desired types.
Comment: so this example is a complete example to find all reasons to use generics?


OAuth2 SSO with Spring Boot without the authorization screen

Question: I have resource, authorization and _ui applications written using Spring Boot 1.5.3, OAuth2 and MongoDB.
The resources are going to be accessed from mobile apps as well as a couple of web applications (one for regular users and the other one for admins). The apps are quite similar to the samples from the guides by Dave Syer. What different is that the users are stored in the database and the clients are stored in an xml file located in the resources folder of the authorization server.
I am struggling with the logon experience for the web users. Following the guides for the JWT based OAuth app, after the login page, the user is redirected to the authorization screen, which is not the desired behavior. I.e., I don't want my authorization server to ask if the user trusts my web application to access its resources. Instead, I want users redirected to the ui pages right after login, as one would expect.
I found this project on GitHub (very similar to the apps from the guide) which behaves exactly as I want, but once I start customizing it by adding my authentication and authorization implementation, it reverts back to using the authorization screen. Apparently, I am missing something, but I was not able to figure out what exactly.
authorization/src/main/resourcs/application.yml
security:
  oauth2:
    client:
      client-id: trusted-app
      client-secret: secret
      scope: read, write
      auto-approve-scopes: .*
  authorization:
      check-token-access: permitAll()
server:
  port: 9999
  context-path: /uaa
mongo:
  db:
    name: myappname

authorization/src/main/resourcs/client-details.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:oauth="http://www.springframework.org/schema/security/oauth2"

   xsi:schemaLocation="http://www.springframework.org/schema/beans
                    http://www.springframework.org/schema/beans/spring-beans.xsd
                    http://www.springframework.org/schema/security/oauth2
                    http://www.springframework.org/schema/security/spring-security-oauth2.xsd">

<oauth:client-details-service id="client-details-service">

    <oauth:client
            client-id="trusted-app"
            secret="secret"
            authorized-grant-types="authorization_code, password,refresh_token"
            authorities="ROLE_WEB, ROLE_TRUSTED_CLIENT"
            access-token-validity="${oauth.token.access.expiresInSeconds}"
            refresh-token-validity="${oauth.token.refresh.expiresInSeconds}"/>
    </oauth:client-details-service>
</beans>

authorization/src/main/java/AuthorizationApplication.java
@SpringBootApplication
@RestController
public class AuthorizationApplication extends AuthorizationServerConfigurerAdapter {

    @RequestMapping("/user")
    @ResponseBody
    public Principal user(Principal user) {
        return user;
    }

    @Configuration
    static class MvcConfig extends WebMvcConfigurerAdapter {
        @Override
        public void addViewControllers(ViewControllerRegistry registry) {
            registry.addViewController("login").setViewName("login");
            registry.addViewController("/").setViewName("index");
        }
    }

    @Configuration
    @Order(-20)
    static class LoginConfig extends WebSecurityConfigurerAdapter {
        @Override
        protected void configure(HttpSecurity http) throws Exception {
            http
                .formLogin().loginPage("/login").permitAll()
            .and()
                .requestMatchers()
                .antMatchers("/", "/login", "/oauth/authorize", "/oauth/confirm_access")
            .and()
                .authorizeRequests()
                .anyRequest().authenticated();
        }
    }

    @Configuration
    @EnableAuthorizationServer
    @ImportResource({"classpath*:client-details.xml"})
    protected static class OAuth2AuthorizationConfig extends AuthorizationServerConfigurerAdapter {

        @Autowired
        private AuthenticationManager authenticationManager;

        @Resource(name="client-details-service")
        private ClientDetailsService clientDetailsService;

        @Override
        public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
            clients.withClientDetails(clientDetailsService);
        }

        @Override
        public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
            endpoints
                .authenticationManager(authenticationManager)
                .accessTokenConverter(jwtAccessTokenConverter());
        }

        @Bean
        public JwtAccessTokenConverter jwtAccessTokenConverter() {
            JwtAccessTokenConverter converter = new JwtAccessTokenConverter();
            return converter;
        }
    }

    @Bean
    PasswordEncoder passwordEncoder(){
        return new StandardPasswordEncoder();
    }

    public static void main(String[] args) {
        SpringApplication.run(AuthorizationApplication.class, args);
    }

}

authorization/src/main/java/mypackage/UserService.java
@Service
public class UserService implements UserDetailsService {

    private UserAccountRepository userAccountRepository;

    @Autowired
    public UserService(UserAccountRepository userAccountRepository){
        this.userAccountRepository = userAccountRepository;
    }

    @Override
    public UserDetails loadUserByUsername(String s) throws UsernameNotFoundException {

        UserAccount userAccount = userAccountRepository.findByEmail(s);

        if (userAccount != null) {
            return userAccount;
        } else {
            throw new UsernameNotFoundException("could not find the user '" + s + "'");
        }
   }
}

ui/src/main/resources/application.yml
auth-server: http://localhost:9999/uaa
server:
  port: 8080
spring:
  aop:
    proxy-target-class: true
security:
  oauth2:
    client:
      clientId: trusted-app
      clientSecret: secret
      access-token-uri: ${auth-server}/oauth/token
      user-authorization-uri: ${auth-server}/oauth/authorize
      scope: read, write
    resource:
      token-info-uri: ${auth-server}/oauth/check_token

ui/src/main/java/UiApplication.java
@SpringBootApplication
@EnableOAuth2Sso
public class UiApplication extends WebSecurityConfigurerAdapter{

    public static void main(String[] args) {
        SpringApplication.run(UiApplication.class, args);
    }

    @Bean
    OAuth2RestTemplate oauth2RestTemplate(OAuth2ClientContext oauth2ClientContext, OAuth2ProtectedResourceDetails details) {
        return new OAuth2RestTemplate(details, oauth2ClientContext);
    }
}

Answer: From http://www.springframework.org/schema/security/spring-security-oauth2.xsd Element client-details-service > complexType client > attribute autoaprove

Scopes or scope patterns that are autoapproved (comma-separated), or
  just "true" to autoapprove all.

Just add the autoapprove="true" attribute to your trusted-app in client-details.xml. That way the authserver will not request user's confirmation to access the resources.
Here is an example of how to implement this behaviour directly in your Java configuration.
Answer: Additionally, if your clients are provided by a database, actually DataSource, then the AUTOAPPROVE column of related client should be set to 'true' in the table OAUTH_CLIENT_DETAILS.


Windows Server: VPN clients cannot see LAN clients

Question: I'm trying to set up a server running Windows Server 2012 R2 to host VPN (RRAS). I have gotten VPN connections to work with certificates (SSTP). Now I need the VPN clients to be able to see LAN clients, and LAN clients to see VPN clients. I haven't been able to figure out how to do this. I've bought a 2nd NIC because I read somewhere that this was needed. All VPN clients get IPs in the range 192.168.2.2-192.168.2.10 while LAN clients get IPs like 192.168.1.x.
Illustration:
VPN Client <--Internet--> Personal Router <--LAN--> VPN Server
From the outside the VPN connections are port forward to the VPN Server. Connections are established and work fine. But a connected VPN Clients cannot connect to the LAN clients nor the internet when connected through VPN.
I haven't been able to figure this out. Does anyone have any tips or links to instructions on how to do this? I don't actually know what to search for in this instance.
Jens
Answer: Solved this by creating a static route in the router like this:
192.168.2.0 MASK 255.255.255.0 192.168.1.2


How to use Firebase ML-Kit text recognition in java on windows

Question: I want to use the Firebase ML-Kit text recognition in my JavaFX application but I am not sure how to. I've used other text recognition libraries such as tesseract in Java, but the difference is phenomenal, which is why I want to use the Firebase ML-Kit's Text Recognition. Can anyone guide me?
Answer: firebaser here
ML Kit is only available for iOS and for Android. It is not available for Windows at the moment.
If you want to cast your vote for ML Kit being made available on Windows too, file a feature request.


"ValueError: Input 0 is incompatible with layer gru1: expected ndim=3, found ndim=4"

Question: I am new to both python and neural networks.I am trying to build a model with CNN+RNN for gesture classification in the video.Each video consist of 30 Frames (batch of 30 images). For CNN layer,I am using Conv2D layer in CNN layer and GRU in RNN layer. The images are of size 84*84 and are RGB images(Channel = 3). I am getting the "ValueError: Input 0 is incompatible with layer gru1: expected ndim=3, found ndim=4" error when I am trying to add the GRU layer. Below is my code:
<code>    model1 = Sequential()
    model1.add(Conv2D(64, (3,3), strides=(1,1), padding='same', input_shape=(84,84,3),name='c2D1'))
    model1.add(BatchNormalization())
    model1.add(Activation('elu'))
    model1.add(MaxPooling2D(pool_size=(2,1), strides=(2,1)))

    model1.add(Conv2D(128, (3,3), strides=(1,1), padding='same',name='c2D2'))
    model1.add(BatchNormalization())
    model1.add(Activation('elu'))
    model1.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))

    model1.add(Conv2D(256, (3,3), strides=(1,1), padding='same',name='c2D3'))
    model1.add(BatchNormalization())
    model1.add(Activation('elu'))
    model1.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))

    model1.add(Conv2D(256, (3,3), strides=(1,1), padding='same',name='c2D4'))
    model1.add(BatchNormalization())
    model1.add(Activation('elu'))
    model1.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))

    model1.add(GRU(units=50,input_shape=(256,84,84),return_sequences=True,name='gru1'))
    model1.add(layers.Dense(nb_labels))

    model1.add(Flatten())
    model1.add(Dropout(0.5))
    model1.add(Dense(512, activation='elu'))
    model1.add(Dropout(0.5))
    model1.add(Dense(5, activation='softmax'))
</code>
Kindly let me know what should be the correct value for the input_shape for the GRU layer.
Comment: Does this answer your question? [Keras ValueError: Input 0 is incompatible with layer conv2d\_1: expected ndim=4, found ndim=5](https://stackoverflow.com/questions/47665391/keras-valueerror-input-0-is-incompatible-with-layer-conv2d-1-expected-ndim-4)
Comment: Does this answer your question? https://stackoverflow.com/questions/44583254/valueerror-input-0-is-incompatible-with-layer-lstm-13-expected-ndim-3-found-n?rq=1
Comment: Does this ansewr your question? https://stackoverflow.com/questions/54877516/valueerror-input-0-is-incompatible-with-layer-conv2d-5-expected-ndim-4-found?rq=1
Comment: Does this answer your question? https://stackoverflow.com/questions/54118069/valueerror-input-0-is-incompatible-with-layer-conv1d-1-expected-ndim-3-found?rq=1
Comment: Does this answer your question? https://stackoverflow.com/questions/56859738/how-to-fix-valueerror-input-0-is-incompatible-with-layer-flatten-expected-mi?rq=1
Comment: Does this answer your question? https://stackoverflow.com/questions/58165813/how-to-fix-valueerror-input-0-is-incompatible-with-layer-lstm-2-expected-ndim?rq=1
Answer: Your code in no way does what you say you want it to do. Those convolutional layers expect a batch of images but what you describe is a batch of sequences of images. That GRU layer should (for it to work) be given a tensor of shape (batch_size, sequence_length, features) but instead it's getting a (batch_size, 5, 10, 256) with its input_shape parameter pointlessly set to (256,84,84).
To get what you want, make the convolutional part time-distributed (https://www.tensorflow.org/api_docs/python/tf/keras/layers/TimeDistributed), then flatten everything after the second dimension (using https://www.tensorflow.org/api_docs/python/tf/keras/layers/Reshape), and only then apply the GRU. You don't have to tell the GRU layer the correct input shape since it will infer that automatically.
Answer: Conv2D takes a 3 dims image in input (width x height x channels). If you want to trick keras into processing a color video you should use Conv3D (at least in the first/first two layers). Conv3D accepts takes as input width x height x depth x channels. You could keep your color channels as "channels" and use the depth dimension as the time one.


Does $\sum_{j=0}^n\sum_{k=0}^n\binom{j}{a}\binom{k}{b}\binom{n-j-k}{c}=\binom{n+1}{a+b+c+2}?$

Question: I'm working on a problem,

(a) Let $a,b,n\geq1$ with $a+b\leq n$. By considering choosing $a+b+1$ numbers from the set $\{0,1,...,n\}$, and the possibilities for the number in position $a+1$ when the chosen numbers are listed in increasing order, show that $$\binom{n+1}{a+b+1}=\sum_{k=0}^n\binom{k}{a}\binom{n-k}{b}.$$
(b) Hence, or otherwise, express $$\sum_{j=0}^n\sum_{k=0}^n\binom{j}{a}\binom{k}{b}\binom{n-j-k}{c},$$ where $a+b+c\leq n$, as a single binomial coefficient.

Does the following argument make sense for (b)? (I leave a certain amount implicit.)

Let \begin{align}
& j && \text{ be the } (a+1)^\text{th} && \text{number chosen}, \\
& j+1+k && \text{ be the } (a+1)+(b+1)=(a+b+2)^\text{th} && \text{number chosen;}
\end{align}
Then there are \begin{align}
& \binom{j}{a} && \text{ways of choosing the first $a$ numbers from } [0,j-1], \\
& \binom{k}{b} && \text{ways of choosing the next $b$ numbers from } [j+1,j+k], \\
& \binom{n-j-k}{c} && \text{ways of choosing the next $c$ numbers from } [j+1+k,n];
\end{align}
therefore the given expression is equal to $$\binom{n+1}{a+b+c+2}.$$
Comment: Two thumbs up! Looks alright!
Comment: Pretty sure the sum should require $j+k \leq n$, as otherwise the negative terms will get you back away from the goal.
Answer: We can also derive a closed formula of the double sum by applying the identity
\begin{align*}
\sum_{k=0}^n\binom{k}{a}\binom{n-k}{b}=\binom{n+1}{a+b+1}\tag{1}
\end{align*}
twice.

We obtain
\begin{align*}
\color{blue}{\sum_{j=0}^n}&\color{blue}{\sum_{k=0}^{n-j}\binom{j}{a}\binom{k}{b}\binom{n-j-k}{c}}\tag{2}\\
&=\sum_{j=0}^n\binom{j}{a}\sum_{k=0}^{n-j}\binom{k}{b}\binom{(n-j)-k}{c}\tag{3}\\
&=\sum_{j=0}^n\binom{j}{a}\binom{n-j+1}{b+c+1}\tag{$\to$ (1)}\\
&=\sum_{j=0}^{n+1}\binom{j}{a}\binom{(n+1)-j}{b+c+1}\tag{4}\\
&\,\,\color{blue}{=\binom{n+2}{a+b+c+2}}\tag{$\to$ (1)}\\
\end{align*}
and observe that lower and upper index of the result have a summand $2$.
Comment:

In (2) we set the upper limit of the inner sum to $n-j$ since the upper index of $\binom{n-j-k}{c}$ is non-negative.

In (3) we make a rearrangement as preparation to apply (1).

In (4) we set the upper limit of the sum to $n+1$ which does not change the sum, since $\binom{0}{b+c+1}=0$. We observe we can apply (1) again.