Emails delivered to Microsoft/Outlook successfully, but do not arrive in inbox/spam folders

Question: Struggling with this for awhile.  Have searched everywhere, read every post/thread I can find and have a thread started on Microsoft's "Answers" site, but the replies there are so slow.
I am running Plesk w/ PostFix and emails are being successfully sent to Microsoft servers, however the emails never arrive in the Inbox (or Junk folder, or anywhere else) but also do not bounce.  They are simply silently dropped somewhere.
I have SPF, DomainKeys and DomainKey Signing all working properly:
Response from Port 25:
==========================================================
Summary of Results
==========================================================
SPF check:          pass
DomainKeys check:   pass
DKIM check:         pass
Sender-ID check:    pass
SpamAssassin check: ham

Line from maillog:
Aug 27 14:49:15 server postfix/smtp[5303]: DEC011C2013: to=<email@example.com>, relay=mx4.hotmail.com[18.104.22.168]:25, delay=1.8, delays=0.21/0.02/0.98/0.57, dsn=2.0.0, status=sent (250  <firstname.lastname@example.com> Queued mail for delivery)

MX Toolbox Diagnostic Results
220 server.eaglehomes.ca ESMTP Postfix

SMTP Banner Check   OK - 184.108.40.206 resolves to server.eaglehomes.ca
SMTP Reverse DNS Mismatch   OK - Reverse DNS matches SMTP Banner
SMTP TLS    OK - Supports TLS.
SMTP Connection Time    0.811 seconds - Good on Connection time
SMTP Open Relay OK - Not an open relay.
SMTP Transaction Time   3.167 seconds - Good on Transaction Time

main.cf
readme_directory = /usr/share/doc/postfix-2.8.17/README_FILES
virtual_mailbox_domains = $virtual_mailbox_maps, hash:/var/spool/postfix/plesk/virtual_domains
virtual_alias_maps = $virtual_maps, hash:/var/spool/postfix/plesk/virtual
virtual_mailbox_maps = , hash:/var/spool/postfix/plesk/vmailbox
transport_maps = , hash:/var/spool/postfix/plesk/transport
smtpd_tls_cert_file = /etc/postfix/postfix_default.pem
smtpd_tls_key_file = $smtpd_tls_cert_file
smtpd_tls_security_level = may
smtpd_use_tls = yes
smtp_tls_security_level = may
smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt
smtp_use_tls = yes
smtpd_timeout = 3600s
smtpd_proxy_timeout = 3600s
disable_vrfy_command = yes
mynetworks = , 127.0.0.0/8, [::1]/128
smtpd_sender_restrictions = check_sender_access hash:/var/spool/postfix/plesk/blacklists, permit_sasl_authenticated
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org
smtp_send_xforward_command = yes
smtpd_authorized_xforward_hosts = 127.0.0.0/8 [::1]/128
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
virtual_mailbox_base = /var/qmail/mailnames
virtual_uid_maps = static:30
virtual_gid_maps = static:31
sender_dependent_default_transport_maps = hash:/var/spool/postfix/plesk/sdd_transport_maps
virtual_transport = plesk_virtual
plesk_virtual_destination_recipient_limit = 1
mailman_destination_recipient_limit = 1
mailbox_size_limit = 0
virtual_mailbox_limit = 0
myhostname = server.eaglehomes.ca
message_size_limit = 51200000
non_smtpd_milters =

# OpenDKIM
milter_default_action = accept
milter_protocol = 6
smtpd_milters = , inet:127.0.0.1:12768, inet:127.0.0.1:8891
non_smtpd_milters =

Email functions fine otherwise - the server does hundreds, if not thousands of emails per day and has no issues sending to/from any other service.
I'm at a loss as to what to check/try next and would love some help.
Comment: Do you receive mails if you send to another account on your same server? What about sending to a non microsoft account? what's your postfix configuration? Is the postfix queue empty?
Comment: Hi Luke, sorry for the late reply. I checked your DNS and DKIM records and all seems fair. I had quite a lot of trouble in the past with MS servers for what ended up being my dkim key not long enough, but yours is 1024bit already. Maybe you can ask to Microsoft, like @mfinni suggests? Keep us updated.
Comment: Thanks for double checking the settings.  I figured they were correct too.  I have finally heard back from Microsoft and they were able to make some changes on their end that helped, but didn't specify what.
Comment: I'd bet your mails were all caught by some weird rule in their spam filter.. well, good to know that they answered and even solved the problem! Honestly, I've never had a good experience with MS's support, maybe they're getting better
Comment: Really?  A downvote with no comments or suggestions?
Comment: I've updated the question with main.cf file.  Send email to/from a non-microsoft account works fine (the server handles hundreds, sometimes thousands of emails per day).  The queue is empty.
Answer: Run get-messagetracking on the Exchange server. Or have the admin of that system do it, if you can't. If it's Microsoft, open a ticket with them.
SMTP 250 is a success code, so something in their system is dropping the ball, from your perspective. There's literally nothing you can do about it, from a technical perspective, if you don't manage the system that gave a 250 to your sendmail.
Comment: Thanks for the reply.  That is what I figured as well and did finally hear back from Microsoft.  They made a change that corrected the issue, but won't tell me what it was.  Strange.
Comment: I suspect that they use a whitelist.  If you run your own mailserver, it would then be blocked by default.
Comment: A link to the appropriate microsoft help desk would really improve this answer.
Comment: @Keith - I'm not sure what you mean. Are you suggesting that I should have included instructions on how to open a ticket with Microsoft for mail delivery issues to outlook.com? I'm sure that's changed since 2014, so it probably wouldn't improve the answer much.
Comment: That's true.  I figure they have several help desks and I want to post my ticket in the right place.  I did find a [digital ocean forum thread](https://www.digitalocean.com/community/questions/website-email-arrives-at-gmail-and-other-domains-but-not-hotmail-com-or-outlook-com) with some promising links at the bottom.
Answer: Follow the steps here to make it work.

Setup a SPF record
Setup DKIM
Setup a DMARC record
Setup a reverse DNS record
HELO must be a host name not an IP
Make sure that your IP is not on a blacklist
Use a static IP address for your server
Sign up for JMRP and SNDS


How to cache custom controller markup for logged-in users?

Question: How do we set up a controller so the rendered output is cached?
Blocks are cached for logged in users by default, and the build function doesn't even run once the block is cached for logged in users.
I have some custom controllers for landing pages where I would like to cache the output for logged-in users.
I guess I could put everything within those controllers in a block or in several blocks, but is there a relatively simple way to make a controller behave like a block and not even run the build function once it is rendered for a logged in user?
Answer: One thing that you can do is the same as the entity view builder is doing, use a #pre_render callback.
If you look at \Drupal\Core\Entity\EntityViewBuilder::viewMultiple(), you can see it doesn't really do anything except prepare the render array and add a #pre_render callback to the whole thing.
The reason for that is that the pre_render callback will only be called on a cache miss. So you return your array with the cache keys/tags/contexts/max-age, the renderer service will attempt to load from the render cache. On a cache hit, it will return the previously cached HTML, on a miss, it will call the pre_render callback and you can do the things you want to do there.
Comment: +1 for the great info. One question, what's the difference, if viewMultiple() would do a complete build right away. Would that make the caching not prevent a rebuild on a cache hit?
Comment: I don't really understand that question. If you want to elaborate in a separate question then I'm happy to answer there.
Comment: Never mind. My question was about this and I found the answer there: https://dev.acquia.com/blog/drupal-8-performance-render-caching. Thanks again for the insight, still have some trouble to completely grasp the entity view builder.
Answer: You can make any render array or part of it cachable by setting cache keys. See this example from the entity view builder:
<code>// Cache the rendered output if permitted by the view mode and global entity
// type configuration.
if ($this->isViewModeCacheable($view_mode) && !$entity->isNew() && $entity->isDefaultRevision() && $this->entityType->isRenderCacheable()) {
  $build['#cache'] += array(
    'keys' => array(
      'entity_view',
      $this->entityTypeId,
      $entity->id(),
      $view_mode,
    ),
    'bin' => $this->cacheBin,
  );
</code>
The variable cacheBin is set to <code>'render'</code>.
You can choose your own schema for the cache keys, dependent on the data you build the render array with. The cid to store the cache item is the result of the keys and has to be unique.
Comment: The concept is, you cache those parts of the render array, which are expensive to build, like in this example an entity. Then it doesn't matter, if the controller is executed or not.
Comment: This caches the rendered output, but the controller will still rebuild the build array.  I would like to return the rendered output if cache is set for it.  I guess I should use $cache = \Drupal::cache()->get($cid)), but I'm not sure how to get the cid from the keys.
Comment: Yeah, I'm starting to figure out that you have to leave rendered entities alone.  You can do view and then build and pull individual fields out in a controller, but then you lose the benefit of the cacheable part from the view render array. I'm using custom controllers to display data from config_page entities, and I'm realizing that I should have been altering the build of the config pages view mode, and returning the render array of that rather than pulling items out of the entities build array and assembling in a custom controller and a custom theme function.


Cannot deserialize the current JSON array (e.g. [1,2,3])

Question: So im working on the Overwatch API, and I've been struggling on displaying the achievements from this JSON:
{
totalNumberOfAchievements: 67,
numberOfAchievementsCompleted: 57,
finishedAchievements: "57 / 67",
- achievements: [
{
name: "?",
finished: true,
image: "https://overwatch-a.akamaihd.net/img/icons/0x0E6000000000015D-0e20595aad6b8ccf2de427df797ea1a78ba7dbe412cc54a0e5add73aaaca80bc7d5322859afa7e4da33816fe8ed89ee2c2f0c97c0ae87e5e5f32f29df989e21f.png",
description: "...",
category: "General"
},
{
name: "Centenary",
finished: true,
image: "https://blzgdapipro-a.akamaihd.net/game/achievements/0x0E60000000000156.png",
description: "Win 100 games in Quick or Competitive Play.",
category: "General"
},

LIST CONTINUES...
I get this error:

Additional information: Cannot deserialize the current JSON array (e.g. >[1,2,3]) into type 'Overwatch.Models.Achievements' because the type requires a >JSON object (e.g. {"name":"value"}) to deserialize correctly.

When trying to deserialize to this object:
public class OWAchievements
    {
        public int totalNumberOfAchievements { get; set; }
        public int numberOfAchievementsCompleted { get; set; }
        public string finishedAchievements { get; set; }
        public Achievements achievements { get; set; }
    }

public class Achievements
    {
        public IEnumerable<Achievements> achievements { get; set; }
        public string name { get; set; }
        public bool finished { get; set; }
        public string image { get; set; }
        public string description { get; set; }
        public string category { get; set; }
    }

Through my controller:
[HttpPost]
        public ActionResult Index(string profileId)
        {
            var json = new WebClient().DownloadString($"https://api.lootbox.eu/pc/eu/{profileId}/achievements");
            var data = JsonConvert.DeserializeObject<OWAchievements>(json);
            ViewBag.Tag = profileId;
            return View(data);
        }

And finally displaying it in my view:
@model IEnumerable<Overwatch.Models.OWAchievements>

@{
    ViewBag.Title = "Index";
}

Enter battle-tag

        @using (Html.BeginForm())
        {
            @Html.TextBox("profileId", null, new { @class = "form-control" });

            <button class="btn btn-primary" type="submit"> Find Player </button>
        }

@if (Model != null)
{

                @Html.DisplayNameFor(model => model.totalNumberOfAchievements)

                @Html.DisplayNameFor(model => model.numberOfAchievementsCompleted)

                @Html.DisplayNameFor(model => model.finishedAchievements)

        @foreach (var item in Model)
        {

                    @Html.DisplayFor(modelItem => item.totalNumberOfAchievements)

                    @Html.DisplayFor(modelItem => item.numberOfAchievementsCompleted)

                    @Html.DisplayFor(modelItem => item.finishedAchievements)

                    @Html.ActionLink("Edit", "Edit", new { /* id=item.PrimaryKey */ }) |
                    @Html.ActionLink("Details", "Details", new { /* id=item.PrimaryKey */ }) |
                    @Html.ActionLink("Delete", "Delete", new { /* id=item.PrimaryKey */ })

        }

}
else
{
    ViewBag.Text = "Nothing";
}

Sorry for the long and annoying post, but I've been struggling for some time now and I really hope someone can help me.
Comment: Error Message is clear, you want to deserialize a json array to a object. Change your code as `JsonConvert.DeserializeObject>(json);`
Comment: Still not working boss, I've tried that at least 10 times :D
Answer: Provided JSON has missing closing ] and }.
After I fixed this I tried with following classes:
public class Rootobject
{
    public int totalNumberOfAchievements { get; set; }
    public int numberOfAchievementsCompleted { get; set; }
    public string finishedAchievements { get; set; }
    public Achievement[] achievements { get; set; }
}

public class Achievement
{
    public string name { get; set; }
    public bool finished { get; set; }
    public string image { get; set; }
    public string description { get; set; }
    public string category { get; set; }
}

Please not hierachy between RootObject and Achievment.
Answer: {
    "totalNumberOfAchievements": 67,
    "numberOfAchievementsCompleted": 57,
    "finishedAchievements": "57 / 67",
    "achievements": [{
        "name": "?",
        "finished": true,
        "image": "https://overwatch-a.akamaihd.net/img/icons/0x0E6000000000015D-0e20595aad6b8ccf2de427df797ea1a78ba7dbe412cc54a0e5add73aaaca80bc7d5322859afa7e4da33816fe8ed89ee2c2f0c97c0ae87e5e5f32f29df989e21f.png",
        "description": "...",
        "category": "General"
    }, {
        "name": "Centenary",
        "finished": true,
        "image": "https://blzgdapipro-a.akamaihd.net/game/achievements/0x0E60000000000156.png",
        "description": "Win 100 games in Quick or Competitive Play.",
        "category": "General"
    }]
}

This will the your class structure:
public class Achievement
{
    public string name { get; set; }
    public bool finished { get; set; }
    public string image { get; set; }
    public string description { get; set; }
    public string category { get; set; }
}

public class Achievements
{
    public int totalNumberOfAchievements { get; set; }
    public int numberOfAchievementsCompleted { get; set; }
    public string finishedAchievements { get; set; }
     public List<Achievement> achievements { get; set; }
}

Your Json is also incorrect, I have corrected it


Autocompletar com > após inserir sinal - no objeto

Question: Eu utilizava o NetBeans 8.0 e, codificando em PHP, após colocar a variável do objeto e o sinal de menos (-), automaticamente o editor incluia o sinal > para mim.
<code>$myClass = new MyClass();
$myClass- // neste momento, o editor completava o código, ficando da seguinte forma:

$myClass-> // e o editor exibia as opções de métodos e propriedades da classe
</code>
Depois que atualizei para a versão 8.2, este autocompletar parou de funcionar e não estou encontrando a opção nas configurações da ferramenta.
Alguém sabe onde eu consigo restaurar esta funcionalidade?
Answer: Encontrei o plugin PHP Enhancements, que, entre outras pequenas melhorias, faz o trabalho perfeitamente.


PhoneGap + jQuery Mobile + cakePHP

Question: I have made a full application in JQM using cakePHP as the server side language. Application at this stage is only available via the browser.
I have used standard SQL in PHP to pull all data and I am not using JSON at all. My question is do I have to pretty much ditch cakePHP to get my application working on iPHONE and adopt a more JSON approach?
I am quite new to JSON and have issues in regards to how i'll handle access control and controlling what data is visible to certain users.
My thoughts so far would be to build a template for Phonegap so the app works offline and offers relevent error messages and my other app could be loaded into it once the user choose to login?
Any ideas or nudge in the right direction would be tops.
Comment: I answered below about using JSON and PHP.  For the part about "build a template for Phonegap so the app works offline", you could look at frameworks that have some functionality provided for doing MVC (like Backbone.js or a million others). This would work as long as the page was not reloaded. The data is stored in memory, synced via Ajax with the server when it can, and if it can't then you can register an error callback to handle that case.
Answer: PHP can use json via the built-in functions json_encode and json_decode.  So you can continue using CakePHP as the backend if you want, the client should neither know nor care.
For PHP JSON documentation, see: http://us.php.net/manual/en/function.json-encode.php
The use of JSON is a choice of what format you send data in between the client and server.  JSON happens to be convenient on the client side which is written in Javascript.  You don't need to use JSON to make an iPhone-friendly web site / web app.
My general approach is to have a toJSON function on my PHP data objects.  I have a function that I use to send JSON to the client, and it first looks for the toJSON method before sending the object.  If it finds that, it calls it and sends the result of that.  This gives the object a chance to present a minimal set of data members as opposed to sending everything.  This is the same basic approach used in Ruby on Rails too.
Here's an example.  Perhaps CakePHP has their own equivalent of this:
function sendJSON($data) {
    header('Content-Type: application/json');
    header('Cache-Control: no-cache, must-revalidate');
    header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');

    echo json_encode_custom($data);
    //echo '(' . json_encode($data) . ')';
}

function json_encode_custom($data) {
    if( is_object($data) && method_exists($data, 'toJSON') )
            $data = $data->toJSON();
    return json_encode($data);
}

Answer: You can always keep you cakePHP site.
Since phonegap can only have html, css and javascript, you need a remote server to provide an API service so that the app can request data from the remote server via javascript. Then the remote server can answer the request by encoding the data in json format.

PhoneGap + jQuery Mobile + cakePHP


multiple search field on form using php and mysql

Question: I stuck with this problem for a few days already.
The logic of the system i want to develop is here

This is the code for the image
    <?php
session_start();
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

    <title>Search</title>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

    <form action="process/searchprocess.php" method="GET">

                    Client Ic
<input type="text" name="client_name" />

                    Client Ic
<input type="text" name="client_ic" />

                    Client Address
<input type="text" name="client_add" />

<input type="submit" value="Search" align="center" />

    </form>

This is my process code
    <?php
    session_start();
    mysql_connect("localhost", "root", "") or die("Error connecting to database: ".mysql_error());

    mysql_select_db("waveevo") or die(mysql_error());
    ?>

        <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

            <title>Search results</title>
            <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

    <?

php
    $query = $_GET['client_name'];
    $query2 = $_GET['client_ic'];
    $query3 = $_GET['client_add'];

    if ($query == null && $query2 == null && $query3 == null)
    {
        echo "Please at least insert one the value";
    }

    else
    {
        $query = htmlspecialchars($query);
        $query2 = htmlspecialchars($query2);
        $query3 = htmlspecialchars($query3);

        $query = mysql_real_escape_string($query);
        $query2 = mysql_real_escape_string($query2);
        $query3 = mysql_real_escape_string($query3);

        $raw_results = mysql_query("SELECT * FROM client WHERE ('client_name' LIKE '%".$query."%') OR ('client_ic' LIKE '%".$query2."%') OR ('client_add_1' && ' ' && 'client_add_2' && ' ' && 'client_add_3' && ' ' && 'client_add_4' LIKE '%".$query3."%')") or die(mysql_error());;

        if(mysql_num_rows($raw_results) > null){ // if one or more rows are returned do following

            while($results = mysql_fetch_array($raw_results)){
                ?>

                        ID
                        Name
                        IC
                        Mobile
                        Address
                        Marital Status
                        Race
                        Asset Type
                        Bank
                        Amount
                        Nationality
                        Limit

                        <?php echo $results["client_id"]; ?>
                        <?php echo $results["client_name"]; ?>
                        <?php echo $results["client_ic"]; ?>

<?php echo $results["client_mobile_1"]."
".$results["client_mobile_2"]."
".$results["client_mobile_3"]; ?>
                        <?php echo $results["client_add_1"]."
".$results["client_add_2"]."
".$results["client_add_3"]."
".$results["client_city"]."
".$results["client_postcode"]; ?>
                        <?php echo $results["client_marital_status_id"]; ?>
                        <?php echo $results["client_race_id"]; ?>
                        <?php echo $results["client_asset_type_id"]; ?>
                        <?php echo $results["client_bank_id"]; ?>
                        <?php echo $results["amount"]; ?>
                        <?php echo $results["client_nationality_id"]; ?>
                        <?php echo $results["client_limit"]; ?>

            <?php
            }

        }

        else{ // if there is no matching rows do following
            echo "No results";
        }
    }
?>

So I try enter value for client ic, for example 1234, there  is no data in the database that match the value that i entered just now but the result still show have, can i know why because i already don't have way to solve this
Comment: The 10 year old `ext/mysql` extension was deprecated in 5.5 and has been removed in 7.0 - you should learn to use newer APIs, like [PDO](http://wiki.hashphp.org/PDO_Tutorial_for_MySQL_Developers) or MySQLi.
Comment: You're using wildcards, so basically there might be a row with a value that is '12345' or '51234' and it will match because of the `%` characters around your query parameters. MySQL can't make up data, so if you get a result it means it somehow matched.
Comment: Also, if you submit an empty field, you'll basically run a query that says select everything where this column starts with anything, and ends with anything. so that could be problematic as well
Comment: the data in the database for client_ic is 1234, and i search 99, there is a result shown. my database only have one data
Comment: You're using `OR` which means that it will return all rows which match any of the where statements separated with the `OR` so, if you only give it `client_ic = 1234` and `client_name='%%'` it won't match on the '1234' but it will match on the `client_name`
Comment: @FMashiro all the column have data, s0 '%%' should not function right?
Comment: @penglipurlara in my opinion it shouldn't but it does.
Answer: Thatś because when you use wildcards LIKE '%".$query."%' if your variable $query is empty you are just getting all because you compare with everything  LIKE '%%'.
You need to change this sentence:
$raw_results = mysql_query("SELECT * FROM client WHERE ('client_name' LIKE '%".$query."%') OR ('client_ic' LIKE '%".$query2."%') OR ('client_add_1' && ' ' && 'client_add_2' && ' ' && 'client_add_3' && ' ' && 'client_add_4' LIKE '%".$query3."%')") or die(mysql_error());;

whith something like this
$sql_query="SELECT * FROM client WHERE ";
$other=false;

if ($query != null and $query!="") {
    $sql_query=$sql_query."('client_name' LIKE '%".$query."%')";
    $other=true;
}

if ($query2 != null and $query2!="") {
    if ($other) {
        $sql_query=$sql_query." OR ";
    }
    $sql_query=$sql_query."('client_ic' LIKE '%".$query2."%')";
    $other=true;
}
if ($query3 != null and $query3!="") {
    if ($other) {
        $sql_query=$sql_query." OR ";
    }
    $sql_query=$sql_query."('client_add_1' && ' ' && 'client_add_2' && ' ' && 'client_add_3' && ' ' && 'client_add_4' LIKE '%".$query3."%')";
    $other=true;
}
if ($other) {
    $raw_results = mysql_query($sql_query) or die(mysql_error());
}

Comment: it still show the result even there is no result should be shown
Comment: Try to set if ($query != null and $query!="") for $query, $query2 & $query3 to avoid the "" value


A space is Hausdorff iff...

Question: Here's an exercise from Marco Manetti's "Topologia" book (ex. 3.59 in the italian version) that i'm stuck on:

Prove that a topological space $X$ is $\mathrm{T2}\iff\{x\}=\bigcap\limits_{U\in\,\mathcal{I}(x)}\kern{-7pt}\overline{U}\;\;\;\forall x\in X$

Partial proof:
$(\Rightarrow)$ Let $x\in X$, suppose that $y\in X$ and $x\neq y$. They have disjoint neighborhoods because $X$ is $\mathrm{T2}$. So $\exists A,B\subset X$ open and disjoint, such that $x\in A$ and $y\in B$, and as they're disjoint $A\subset B^{\kern{1pt}\mathrm{c}}$ and $B^{\kern{1pt}\mathrm{c}}$ is a closed neighborhood of $x$, choosing such a $B_y$ for each $y\neq x$ yields a family of closed neighborhoods of $x$, and $\bigcap\limits_{y\neq x}\kern{-3pt}\overline{B^{\kern{1pt}\mathrm{c}}_y}=\bigcap\limits_{y\neq x}\kern{-3pt}B^{\kern{1pt}\mathrm{c}}_y=\{x\}$.
$(\Leftarrow)$ My initial idea was to prove that if in a space $X$ with the property on the right we assume that given a pair of points $x,y\in X$ they have no $A\in\mathcal{I}(x),B\in\mathcal{I}(y)$ open and disjoint we have a contradiction, but all my ideas seemed ineffective, how should i proceed?
Answer: Suppose that $X$ is $T_2$. If $x,y\in X$, there are $U\in\mathcal{I}(x)$ and $V\in\mathcal{I}(Y)$ such that $U\cap V=\emptyset$. Let $O$ be an open set such that $y\in O$ and $O\subset V$. Then $O^\complement$ is a closed set that containes $U$. Therefore, $\overline U\subset O^\complement$ and therefore $y\notin\overline U$. Sinse this occurs for every $y\in X$, $\bigcap_{U\in\mathcal{I}(x)}\overline{U}=\{x\}$.
Now, suppose that $X$ is not $T_2$. Take $x,y\in X$ such that, for any $U\in\mathcal{I}(x)$ and any $V\in\mathcal{I}(y)$, $U\cap V\neq\emptyset$. It's not hard to prove that $y\in\bigcap_{U\in\mathcal{I}(x)}\overline{U}$.
Comment: Thanks for the suggestion!
Comment: Then the proof follows from lemma 3.21 of the book: _"x is in the closure of a subset B iff each neighborhood of x has non-empty intersection with B"_
Comment: Indeed it does.
Answer: Starting the proof with "let $x,y\in X$" is not the right way.
Suppose $X$ is Hausdorff and let $x\in X$. Suppose $y\in X$ and $y\ne x$. Then there exist $U\in\mathcal{I}(x)$ and $V\in\mathcal{I}(y)$ with $U\cap V=\emptyset$; in particular $y\notin\bar{U}$. Thus $y\notin\bigcap_{U\in\mathcal{I}(x)}\bar{U}$.
Suppose $X$ is not Hausdorff and let $x,y\in X$, with $x\ne y$, such that $U\cap V\ne\emptyset$, for every $U\in\mathcal{I}(x)$ and $V\in\mathcal{I}(y)$. In particular, $y\in\bar{U}$, for every $U\in\mathcal{I}(x)$.
Comment: @Ladooscuro Well, that's a *very* important point. Also you want to show something about any $x\in X$. So it's better to start with $x$ and introduce the auxiliary $y$ later.
Comment: Care to explain why that's not the right way? except for _not assuming x \neq y_
Comment: Thanks, i'll keep it in mind

A space is Hausdorff iff...


OnClientClick OnClick

Question: In my asp page, I take a screenshot of the client's desktop with an applet and ask them to send it to the server with a single click of a LinkButton. I use runApplet() function to call my applet to capture the screen and assign the strings value to a hidden value. (picture is stored as base64 string) Until here, everything works perfect! However, SendLinkButton_Click doesnt seem to be executing!
This is my link button.
<asp:LinkButton ID="SendLinkButton"
          OnClientClick="runApplet(); return false;"
          OnClick="SendLinkButton_Click"
          Visible="false"
          CssClass="portal-arrow portal-button"
          runat="server">Send</asp:LinkButton>

This is my Javascript function
   function runApplet() {

      var msg = document.capture.capture();
      var hiddenControl = '<%= inpHide.ClientID %>';
      document.getElementById(hiddenControl).value = msg;
   }

and this is what's inside of SendLinkButton_Click
protected void SendLinkButton_Click(object sender, EventArgs e)
{
    Server.Transfer("Preview.aspx", true);
}

when I put the javascript function to a LinkButton's OnClientClick, and execute this "SendLinkButton_Click" with another LinkButton. It works perfect! But I want them to work with just one click!
Please help!
Comment: runApplet() :) just runApplet() sorry about the mistake
Answer: Your client click is returning false so no postback will be made to the server after this point.
Try changing:
OnClientClick="runApplet(); return false;"

To
OnClientClick="runApplet();"

Answer: Remove return false from the OnClientClick attribute. If you return false from it, the postback will not execute.

OnClientClick OnClick


Appropriate statistical model to evaluate the quality of measurements from different instruments?

Question: I have a dataset where a number of samples (>600) were weighed by three different instruments, called V G and P. Each instrument was used three times to weigh each sample so there are nine values total per sample. One of the instruments (V) is considered a "standard," so the comparisons of interest are really between the other two instruments, G and P.
This is what the data look like (each sample is identified by a unique combination of the identifiers Trial, Year, and Plot). As you can see there is variability between samples, between instruments within a sample, and often within instruments within a single sample as well.
     Trial Year Plot   G1   G2   G3   P1   P2   P3   V1   V2   V3
  1:    H6 2020  101 55.8 55.8 55.8 56.3 56.1 56.9 56.3 56.0 56.2
  2:    H6 2020  102 56.3 56.3 56.3 57.2 57.4 57.2 56.3 56.5 56.6
  3:    H6 2020  103 55.7 55.7 55.7 58.0 57.8 57.9 57.7 57.3 57.0
  4:    H6 2020  104 57.5 57.5 57.5 57.9 58.5 58.8 57.1 57.6 57.6
  5:    H6 2020  107 58.3 58.2 58.2 57.9 57.1 57.2 56.2 56.5 56.6
 ---
642:    U8 2020  311 57.4 57.4 57.4 57.3 57.0 57.3 56.8 56.2 56.2
643:    U8 2020  312 57.6 57.6 57.6 57.5 57.0 57.5 56.4 56.6 56.3
644:    U8 2020  313 54.9 54.9 54.9 55.7 55.3 55.6 54.9 54.2 54.8
645:    U8 2020  314 57.1 57.0 57.0 56.3 56.1 56.1 55.0 54.9 54.9
646:    U8 2020  315 54.4 54.4 54.4 55.5 55.6 55.6 54.7 54.5 54.4

I have two goals:

To assess the relative quality of G and P. For example do either of them have a systematic bias or does one produce more variable measurements than the other on the same sample?
To create a calibration equation relating each of the readings from G and P to the standard V, so that we can in the future correct readings from them.

I thought the best way to do this would be with a latent variable confirmatory factor analysis model where we assume there is an unobserved true value for each sample, and we observe 9 different values each time. This is how I set up the model in lavaan in R:
cfa_model <-
  'W_true =~ V + G + P
   V =~ V1 + V2 + V3
   G =~ G1 + G2 + G3
   P =~ P1 + P2 + P3'

There are four latent variables, the "true" weight W_true, and a variable for each of the three instruments.
I fit the model like this:
cfa(model = cfa_model, data = mydata)

This is what the model DAG looks like:

After fitting the model I tried to get a prediction for the latent variables but they are given as factor scores and I am not sure how to convert those back to the same scale as the data (i.e. I would expect the unobserved true weight to be around 56 for the first row). So I'm not sure how to use this to get a calibration equation to use later. I am concerned that I am thinking about this wrong and this is the wrong way to be doing this. Can anyone point me in the right direction?
Answer: You can estimate bias as the difference between the mean or median weight measurements. Variability is more challenging as it has two components:

Intra-variability quantifies the repeatability of measurements taken with the same instrument.
Inter-variability quantifies the disagreement between measurements taken with two different instruments.

NB Intra- and inter-measurement variability is discussed in a lot more detail in Chapter 16 of Biostatistics for Biomedical Research course notes. Available online.

First let's look at the weight measurements for the 10 samples you've provided.

We learn a lot from this plot. The two most interesting observations are: instrument G has almost no intra-variability and instrument P is consistently over-biased.
Let's put these observations into numbers. I'll use the procedure described in BBR. The R code is attached at the end.
First we consider intra-variability. The standard instrument, V, measures the weight of sample 1 as [56.3, 56, 56.2]. We pair the measurements and compute the absolute differences: |56.3 - 56| = .3, |56.3 - 56.2| = .1, |56 - 56.2| = .2. The average (.3 + .1 + .2) / 3 = .2 is a measure of variability known as Gini's mean difference.
Below I've plotted the intra-instrument variability for each sample. Take the average across samples to estimate the overall intra-varibility.

#>   instrument    intra-variability
#>            G               0.0133
#>            P               0.293
#>            V               0.280

Second we consider inter-variability. The standard instrument, V, measures the weight of sample 1 as [56.3, 56, 56.2] while the alternative instrument, P, measure its weight as [56.3, 56.1, 56.9]. We pair each V measurement with each P measurement and compute the absolute differences: 0, 0.2, 0.6, 0.3, 0.1, 0.9, 0.1, 0.1, 0.7. The average 0.33 is measure of the disagreement between V and P about the weight of sample 1.
Below I've plotted the inter-instrument variability between V and either P or G, for each sample. Again, take the average across samples to estimate the overall inter-variability.

#>   instrument1 instrument2     inter-variability
#>             V           P                 0.85
#>             V           G                 0.883
#>             P           G                 0.867

And finally let's estimate weight as the median measurement by an instrument.

#> bias   G - V   P - V
#>         0.39   0.830

The computations confirm and elaborate on the observations we made at the start by plotting the raw measurements:

Instrument G has little intra-variability (0.013) while instruments V and P have about the same intra-variability (0.28 and 0.29 respectively).
Instrument G has less bias than instrument P (0.39 and 0.83 respectively).
Instrument G has slightly higher inter-disagreement with instrument V (0.88) than instrument P (0.85).
On the given 10 samples, instrument P is consistently over-biased while instrument G has two "modes": it has low bias for samples 1,2,4,8,10 and high bias for samples 3,5,6,7,9.

R code in all its gory details.

data <- data.frame(
  Trial = c("H6", "H6", "H6", "H6", "H6", "U8", "U8", "U8", "U8", "U8"),
  Year = c(2020L, 2020L, 2020L, 2020L, 2020L, 2020L, 2020L, 2020L, 2020L, 2020L),
  Plot = c(101L, 102L, 103L, 104L, 107L, 311L, 312L, 313L, 314L, 315L),
  G1 = c(55.8, 56.3, 55.7, 57.5, 58.3, 57.4, 57.6, 54.9, 57.1, 54.4),
  G2 = c(55.8, 56.3, 55.7, 57.5, 58.2, 57.4, 57.6, 54.9, 57, 54.4),
  G3 = c(55.8, 56.3, 55.7, 57.5, 58.2, 57.4, 57.6, 54.9, 57, 54.4),
  P1 = c(56.3, 57.2, 58, 57.9, 57.9, 57.3, 57.5, 55.7, 56.3, 55.5),
  P2 = c(56.1, 57.4, 57.8, 58.5, 57.1, 57, 57, 55.3, 56.1, 55.6),
  P3 = c(56.9, 57.2, 57.9, 58.8, 57.2, 57.3, 57.5, 55.6, 56.1, 55.6),
  V1 = c(56.3, 56.3, 57.7, 57.1, 56.2, 56.8, 56.4, 54.9, 55, 54.7),
  V2 = c(56, 56.5, 57.3, 57.6, 56.5, 56.2, 56.6, 54.2, 54.9, 54.5),
  V3 = c(56.2, 56.6, 57, 57.6, 56.6, 56.2, 56.3, 54.8, 54.9, 54.4)
)

library("tidyverse")
iccmlr::theme_set_amazon()

GiniSd <- function(x, y = NULL, na.rm = TRUE) {
  if (is.null(y)) {
    # Adapted from Hmisc::GiniMd
    if (na.rm) {
      k <- is.na(x)
      if (any(k)) {
        x <- x[!k]
      }
    }
    n <- length(x)
    if (n < 2) {
      return(NA)
    }
    w <- 2 * ((1:n) - (n - 1) / 2)
    sum(w * sort(x - mean(x)))
  } else {
    sum(outer(x, y, function(xi, yi) abs(xi - yi)))
  }
}

Aggregate <- function(variabilities, ...) {
  variabilities %>%
    group_by(...) %>%
    summarise(
      variability = sum(N * variability) / sum(N),
      N = sum(N)
    )
}

Estimate <- function(sample, instrument, measurement) {
  measurements <- tibble(
    sample,
    instrument,
    measurement
  )
  measurements %>%
    group_by(
      sample,
      instrument
    ) %>%
    summarise(
      measurement = median(measurement)
    ) %>%
    ungroup() %>%
    pivot_wider(
      id_cols = sample,
      names_from = instrument,
      values_from = measurement
    )
}

IntraVariability <- function(sample, instrument, measurement) {
  measurements <- tibble(
    sample,
    instrument,
    measurement
  )
  intra_var <- measurements %>%
    group_by(
      sample,
      instrument
    ) %>%
    summarise(
      N = choose(n(), 2),
      variability = GiniSd(measurement) / N
    ) %>%
    ungroup() %>%
    drop_na()
  list(
    intra_var = intra_var,
    by_sample = Aggregate(intra_var, sample),
    by_instrument = Aggregate(intra_var, instrument)
  )
}

InterVariability <- function(sample, instrument, measurement) {
  measurements <- tibble(
    sample,
    instrument,
    measurement
  )

  nested <- measurements %>%
    chop(measurement)

  inter_var <-
    inner_join(
      nested %>% rename(instrument1 = instrument, x = measurement),
      nested %>% rename(instrument2 = instrument, y = measurement),
      by = "sample"
    ) %>%
    filter(
      instrument1 > instrument2
    ) %>%
    rowwise() %>%
    mutate(
      N = length(x) * length(y),
      variability = GiniSd(x, y) / N
    ) %>%
    select(
      -c(x, y)
    ) %>%
    drop_na()
  list(
    inter_var = inter_var,
    by_sample = Aggregate(inter_var, sample),
    by_instrument_pairs = Aggregate(inter_var, instrument1, instrument2)
  )
}

plot_measurements <- function(measurements, measurement) {
  measurements %>%
    arrange(
      sample, instrument, {{ measurement }}
    ) %>%
    mutate(
      id = cumsum(sample != lag(sample, default = ""))
    ) %>%
    group_by(
      sample, instrument
    ) %>%
    mutate(
      h = cumsum({{ measurement }} == lag({{ measurement }}, default = 0)),
      y = id + 0.1 * h
    ) %>%
    ggplot(
      aes({{ measurement }}, y,
          shape = instrument,
          color = instrument
      )
    ) +
    geom_hline(
      aes(yintercept = y),
      linetype = 3,
      color = "gray",
      size = 0.5,
      data = tibble(y = seq(1, 10, 1))
    ) +
    geom_point(
      stroke = 1,
      size = 2
    ) +
    scale_shape_manual(
      values = c(0, 1, 2)
    ) +
    theme(
      axis.title = element_blank(),
      axis.text.y = element_blank(),
      axis.ticks.y = element_blank(),
    )
}

measurements <- data %>%
  unite(
    "sample", c(Trial, Year, Plot)
  ) %>%
  pivot_longer(
    -sample,
    names_to = "instrument",
    names_transform = list(instrument = ~ str_sub(., 1, 1))
  )

estimates <- Estimate(
  measurements$sample,
  measurements$instrument,
  measurements$value
)
estimates %>%
  summarise(
    `G - V` = mean(G - V),
    `P - V` = mean(P - V)
  )

intra_var <- IntraVariability(
  measurements$sample,
  measurements$instrument,
  measurements$value
)
intra_var$by_instrument

inter_var <- InterVariability(
  measurements$sample,
  measurements$instrument,
  measurements$value
)
inter_var$by_instrument_pairs

# Make scatterplots

p <- plot_measurements(measurements, value)
p + ggtitle("Raw measurements")

p <- plot_measurements(
  estimates %>% pivot_longer(c(G, P, V), names_to = "instrument", values_to = "estimate"),
  estimate
)
p + ggtitle("Estimated weight")

p <- plot_measurements(intra_var$intra_var, variability)
p + ggtitle("Intra variability")

p <- plot_measurements(
  inter_var$inter_var %>% filter(instrument1 == "V") %>% rename(instrument = instrument2),
  variability
)
p + ggtitle("Inter variability (V - ?)")

Comment: @qdread This is kind. Thanks for the acknowledgement and letting me know about it.
Comment: I wanted to follow up and say that this was a very valuable and helpful post. I thanked you in the acknowledgments section of a MS we are submitting partly based on this.


How to correctly set up my mixed-effect model?

Question: I have data on days in which the greening of trees happen across America in 2015. This includes meteorological and topography data etc. I want to predict the day of greening happens through a linear mixed-effect model by meteorological data and topography data being fixed effects while the states of America is the random effect. I have looked into how to conduct the linear mixed-effect model in R, and I tried to perform this, but the output looks strange. I have looked at many examples and this 'cheat sheet' on how to perform linear mixed-effect models in R.
Right now, when I for example plot the relationship between relative humidity and day of greening I get strong positive relationships in many states which makes a lot of sense in this study.

But, in my overall linear mixed-effect model I get a negative estimate for relative humidity, while a simple linear regression generates a positive estimate for relative humidity as predictor. The models are below:
The number 15 represents the year 2015
Doy: Day of the year, greening occurs
PostC15: Carbon content in the vegetation
PostMC15: Mean carbon content in the state
PostPAR15 photosynthetically active radiation
PostElev15: Elevation
PostPR15: Precipitation
PostRH15: Relative humidity
PostTemp15: Air temperature
PostAsp15: Aspect

lm.doy <- lm(doy ~ postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15, data = df)

    lmm.doy <- lme(doy ~ postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15, data = postSM, random = ~1+postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15|states, method = 'ML', control = lmeControl(opt = "optim", msMaxIter=1000, maxIter = 1000, msMaxEval = 1000))

Below is the summary of both:

    summary(lm.doy)
    Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
    (Intercept) -2.549e-16  1.286e-02   0.000 1.000000
    postC15     -3.567e-02  1.636e-02  -4.626 3.83e-06 ***
    postMC15     4.209e-02  2.225e-02   3.689 0.000228 ***
    postPAR15   -5.008e-02  3.016e-02  -1.660 0.096960 .
    postElev15   6.690e-02  1.588e-02   2.323 0.020217 *
    postPR15     7.233e-02  1.871e-02   2.797 0.005178 **
    postRH15     8.407e-02  2.556e-02   3.289 0.001012 **
    postTEMP15   3.635e-01  2.043e-02   8.004 1.52e-15 ***
    postAsp15   -3.278e-01  1.477e-02  -8.655  < 2e-16 ***

    summary(lmm.doy)
                     Value  Std.Error   DF   t-value p-value
    (Intercept) -0.2376308 0.10220873 4500 -1.248727  0.2118
    postC15     -0.1541045 0.01906032 4500 -2.313944  0.0207
    postMC15    -0.1168753 0.02265520 4500 -1.186277  0.2356
    postPAR15   -0.2255873 0.03026647 4500 -3.818988  0.0001
    postElev15  -0.1322979 0.01637984 4500 -1.361303  0.1735
    postPR15    -0.1411053 0.01815212 4500 -1.713592  0.0867
    postRH15    -0.1729018 0.03824127 4500 -1.644868  0.1001
    postTEMP15   0.1462955 0.04477048 4500  0.810701  0.4176
    postAsp15    0.1557790 0.01890807 4500  2.421137  0.0155

I am not sure that I set up my linear mixed-effect model correctly. In simple terms, I thought by defining:
    random = ~1+postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15|states

I more or less compute small linear regression for each state and eventually 'sum' it up to one global estimate of intercept and slope for each individual variable. Or should I basically do the following:
    doy ~ (1|states) + postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15 + (0+postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15|states)

To get what I want...the effect of each and every variable for each and every state?
How do I set up a linear mixed-effects model in R to predict the day of greening for each individual state from meteorological and topographical data?
Comment: Can you be more explicit about what each predictor name means? Looking at this I can surmise what part of the problem may be, but the naming of the predictors doesn't help unless you provide some explanation of what they represent.
Comment: Hi @ShawnHemelstrand. Thanks for your reply. I have edited the question to included brief descriptions of each variable included. Does it help?
Comment: Did this model converge? Im less familar with `lme` but typically that many random effects terms will cause extreme convergence issues with `lmer`.
Comment: Not once I used `lmer`, but if I use `nlme` with `control = lmeControl(opt = "optim", msMaxIter=1000, maxIter = 1000, msMaxEval = 1000)` it does converge. At least I dont get any converge error.
Answer: As @shawn-hemelstrand commented, I am pretty surprised that this code converged for you. nlmer::lme() will sometimes allow users to fit models that they do not have enough data for. If you are new to mixed models, I would strongly recommend starting with lme4::lmer()(also using the lmerTest library). For example, I might first fit this model as:
lmm.doy.lmer <- lmerTest::lmer(doy ~
   postC15+postMC15+postPAR15+postElev15+postPR15+postRH15+postTEMP15+postAsp15 +
   (1|states),
   data = postSM)

I think the broader question is, what is the best way to analyze this data? You have non-independent measurements within-state. But there's more! The states themselves are non-independent. Some states are closer than others, and their measurements are therefore going to be more similar. I also imagine that your measurements from within each state relate to different geographic regions - counties maybe? So the same applies. Basically, you have a massive spatial auto-correlation issue (your observations are non-independent in a structured fashion). You'll need to use an analysis that specifically accounts for spatial auto-correlation. See here for a decent primer.
Comment: Your situation may be an example of [Simpson's paradox](https://statisticsbyjim.com/basics/simpsons-paradox/) . Your lm model gives your the overall relation between postRH15 and doy ignoring the possible differences between states, whereas lme / lmer give you the average of each state's slope (and also each state's separate slopes if you extract them from the model). These two can have opposite signs, as also illustrated for instance [here](https://stats.stackexchange.com/questions/226001/inconsistent-results-between-linear-model-and-linear-mixed-effects-model).
Comment: If I do as stated, I still get that `postRH15` is of negative influence on `doy`, also if I set it to `(1+postRH15|states)`. Should I just come to terms with the fact that `postRH15` influences the `doy` negatively?
Comment: Agree with @Sointu re simpson's paradox. Again, I do think that a standard mixed effect model (e.g., random intercepts, maybe random slopes) does not sufficiently account for the structure of your data.


Is there a symplectomorphism (with some conditions) which is not a Hamiltonian isotopy?

Question: The answer is definitely no in general.
But what if we require some additional conditions?
Let $F:(M,\omega)\to(M,\omega)$ be a symplectomorphism, i.e. $F^*\omega=\omega$. We further assume $F$ is isotopic to $\mathrm{id}_{M}$ in $\mathrm{Diff}(M)$ or even in $\mathrm{Diff}(M,\omega)$.

Question: Is it necessary that $F$ must be a Hamiltonian isotopy? If not, is there any counter-example?
Answer: There are two ways to read your question since you use the phrase "Hamiltonian isotopy", rather than the related notion of "Hamiltonian symplectomorphism"-i.e. a symplectomorphism which Hamiltonian isotopic to the identity.
V1. Is a symplectic isotopy a Hamiltonian isotopy?
No, see Proposition 9.19 of "Introduction to symplectic topology" (Second edition).
V2. Is a symplectomorphism which is isotopic (or even symplectically isotopic) to the identity a Hamiltonian symplectomorphism?
The answer is also no. See see the symplectomorphism of $T^*{S^{1}} $constructed here: Showing that some symplectomorphism isn't Hamiltonian. It is symplectically isotopic to the identity by $F_{t}(x,\xi) = (x,\xi + t)$, but it is not a Hamiltonian symplectomorphism as is proven there.


Reference project not match the real one

Question: I have two project in different solutions,
and in one of them (let call it A) there is a refernce to the other one (let call it B).
it was good, untill I decide I need another Ctor in one of the classes in B.
But in A I can't use this Ctor
I tried close and open the visual studio,
shut down the computer,
wait few hours
delete and readd the refernce in A to B.
but I still can't use this Ctor...
what can I do?
Thanks!
Comment: how did you add the reference?
Comment: right click - > add refernce - > browes -> find the project folder - > bin - > choose the dll file named as the project
Comment: Did you *build* the first solution? Did you build the correct variation of it? DEBUG or RELEASE, depending on which reference you added?
Comment: Is your constructor for the class in B public?
Comment: Yes, I build them both, and it is public.
it was good at the beginning, but I just can't edit it
Comment: is the class public?
Comment: yes, it's public.
I already used ather methids in this class, except the new Ctor
Comment: Is there any reason you cannot have both of the projects in the same solution?
Comment: it posible, yes,
but I prefered that the project wount be in the same solution, if possible
Answer: This is the solution :
rebuild B.
(Build - > build solution)


ESIs with Varnish 2.1.5 for Anonymous Users

Question: We're trying to configure our sites running D7 to use ESIs with Varnish 2.1.5, and having a hard time figuring this out.
We have a file /test.php that is being included in the template via esi so it appears on all pages of our site. The call to the file is through (in the template)
<esi:include src="/test.php" />

Relevant vcl conf is:
sub vcl_fetch {
...
  // enable esi
  esi;
  // if url is test.php, set ttl to 15 s
  if (req.url == "/test.php") {
    unset beresp.http.set-cookie;
    set beresp.ttl = 15s;
  }
}

The text from test.php appears when users are logged in, but does not appear when they are logged out -- which is what we're after. This is without the ESI module as that is for authenticated users.
Comment: "This is without the ESI module as that is for authenticated users." The ESI module works with anonymous and authenticated users; just be aware that all anonymous users are treated the same.
Comment: ESI in its current form is there to display fragments of dynamic content. It really doesn't care if your logged in or not as it treats user 0 (anonymous) as one user. So if all anonymous users are going to get the same experience you shouldn't have any issues with ESI.
Comment: @mikeytown2: Thanks for the clarification, the 1st line on project page: "ESI - or Edge Side Includes - is a **high performance caching** solution for **Authenticated users**" thew me off.
Comment: @mikeytown2 My understanding is that with the ESI module one can cache pages when users are authenticated, and also display fragments of dynamic content. Is my understanding amiss?
Answer: Try turning off Anonymous caching and see what results you get (caching should be handled by Varnish).

Go to admin/config/development/performance
Uncheck Cache pages for anonymous users
Comment: Does anonymous caching on that page imply caching at the Drupal level?
Comment: Yes, it caches at the Drupal level
Comment: Thanks this works! To better my understanding: A combination of "flush all caches" and varnish restart should clear all anonymous caches, right? I tried this, but the esi content was only visible when users were logged in. Why would that be?
Comment: I'm not real savvy on ESI, but I would check if Varnish is actually processing pages that are cached by Drupal, or if it's Apache that's processing it.  In other words, maybe Apache is handling the request instead of Varnish, and Apache cannot handle the .  Lots of assumptions in this comment!  Just throwing out some ideas.
Comment: For clearing the Varnish cache you should consider using https://drupal.org/project/purge and https://drupal.org/project/flush_page_cache which helps you flush the drupal cache/ varnish cache for a particular page when used along with purge.
Answer: I have a working instance with Drupal7, Varnish and ESI in working state[Th ESI tags are getting replaced for both Anonymous and logged in users].
I can confirm that ESI tags work well with Drupal Page Cache. I don't see a reason you need to disable the Drupal Page cache for ESI tags to work.
And regarding the behavior that you were experiencing, I think the anonymous page cache was built you added your ESI tag and hence it was not displaying. I think clearing Drupal Cache and Varnish Cache could have just fixed your issue.
Comment: You can also check out my blog http://foobar.azrisolutions.com/drupal/drupal7-varnish-and-esi-the-holy-grail-of-caching-for-the-authenticated-users/ IF you want to know how to use the ESI module.


How to open and update WPF window by threading

Question: I have Two WPF window Window1.xaml and window2.xaml.
In window2.xaml.cs i have one function which does some complex calculation and display intermediate resuls on a TextBlock.
Now What i want.
by clicking on button of Window1.xaml i want to open Window2.xaml as a dialog box and want to execute complex function of windows2.xaml.
If i call complex button on Window2.xaml' load event then dilog box apear after execution of complex function.
How to do this by Threading.
Answer: You need not open the window in a thread. Once you've opened Window2, start a thread for the complex function in the Window2.Loaded event handler. This will leave your UI undisturbed.
Comment: private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            Thread t = new Thread(ComplexFunction);
            t.Start();
        }

and in
ComplexFunction()
{
// doing something
UpdateStatus("adasd");
// doing something
UpdateStatus("dsad");
}

It gives runtime error.
The calling thread cannot access this object because a different thread owns it
UpdateStatus(string status)
{
StatusText.AppendText(status + "\r");
}
Comment: @Jeevan Bhatt: If you're accessing any element created in UIThread then you may need to ask the dispatcher to do the favour for you. `myDispatcher.BeginInvoke(DispatcherPriority.Background, new Action(() => { // Code that accessess UIElement }));` where myDispatcher is a reference to Dispatcher.CurrentDispatcher Set this before calling the thread.
Answer: Opening a Window in a thread that is not UI thread is not recommended at all. You can open a popup window using Show() (modeless) method or ShowDialog() (modal) method. In Window2's load method, you can start a new thread which does a complex operation and once it is done with the operation, it can show a dialog.
Now, while in a different thread than UI thread, if you want to show any UI (dialogs, windows) or change existing UI (textblock, etc), you must use Dispatcher.Invoke method.


Every conservative vector field is irrotational

Question: I have done an example where I needed to show that every conservative $C^2$ vector field is irrotational. However, there is something unclear in the solutions: Namely, I am uncertain what does the following sentence at the end of the solution mean:
"since second partial derivatives are independent of the order (for smooth functions)", and I was wondering how does that imply that the equality before that is 0?
Comment: Note that $\partial_y \phi_z = \frac{\partial^2 \phi}{\partial y\partial z}$.
Answer: This is normally called symmetry of second derivatives, or Clairaut's theorem. It means that for functions that have continuous second partial derivatives,
$$ \frac{\partial^2f}{\partial x \partial y} - \frac{\partial^2 f}{\partial y \partial x} = 0. $$
But each term in the line above is of precisely this form (Presumably the document's notation has $\phi_x = \partial_x \phi$ and so on).
Comment: Yes, that should be sufficient. Presumably it's just to make it consistent with the second part, which does need $C^2$ since 2 derivatives really are taken.
Comment: Thanks! Also, why does it have to be a $C^2$ field, would not the same thing apply for $C^1$ field, since we are calculating derivatives of the potential, and in the end only calculate the 1st derivatives of the field?
Comment: But, regarding the 2nd part, first a derivative of potential of F is taken, and then, in fact, a derivative of F, which means that we only require $C^1$,  since only the 1st derivative of F is calculated(in other words: 2nd derivative of potential of F) or?
Comment: I mean part (b) of the question.


How to calculate de average from user input numbers

Question: So basically, I'm new to programming and can't figure it out how to add certain numbers generated by the user. I want a program, capable of collecting for n days the temperature, and at the end, inform what the peak heat occurred, as well as the average of the temperatures collected.
<code>#include <iostream>
using namespace std;

int main()
{
    int days, day0 = 0, degrees, degrees_max = 0;
    float degrees_average;

    cout << "Days of the study" << endl;
    cin >> days;

    while (days > day0)
    {
        cout << "How many Degrees?" << endl;
        cin >> degrees;

        if (degrees < 999)
        {
            day0++;
        }

        if (degrees > degrees_max)
        {
            degrees_max = degrees;
        }
    }

    degrees_average = degrees / days;

    cout << "The max temperature was: " << degrees_max << "C"
         << " And the average was : " << degrees_average << endl;

}
</code>
SO the only thing left is how to calculate the average.
All the help is appreciated! Thanks
Comment: An average is calculated by adding up all the values, then dividing it by the number of values. The shown code handles each value, as its entered, and it can be trivially adjusted to add them up. What exactly is your question? You already have everything needed to do this.
Comment: The great secret of computer programming is how you solve a problem with a pencil and paper will still work. It may not be the fastest solution, but it always gets you moving forward
Comment: For example: I say there are 3 days in the study (where the program asks for "..days..") , the program asks for the first temperature, the second and a third. But it only shows correctly the max temperature of these three but it fails when adding all the temperatures and diving by 3, cause there isn't any code for that. Hope I can make myself understand
Comment: The solution is simple: one has just to add the code for that.
Comment: In C++, you should be using `std::vector` for creating a dynamically sized list (if you wanted to _track_ the temperatures). Otherwise, since you're only going for the statistics here, you can just do them as you go. But as it is now, `degrees` is just your "currently read degrees", not the "sum of degrees". You need another variable (for "current sum"), or apply different math so you could continually update `degrees_average`
Answer: <code>#include <iostream>

using namespace std;

int main()
{
    int days, max=-1;
    double current, total = 0;
    cin >> days;
    for (int i=0; i<days; i++){
        cin >> current;
        if (current > max) max = current;
        total = total + current;
    }
    cout << "max: " << max << endl;
    cout << "average: " << total/days << endl;

    return 0;
}
</code>
Comment: The use of ```using namespace std;``` is [not recommended](https://stackoverflow.com/questions/1452721/why-is-using-namespace-std-considered-bad-practice). Solution works fine anyway.


What is the meaning of admissibility within a class, that every decision rule in a class is admissible in that class?

Question: Suppose that I have that $X$ is a Poisson random variable with mean $\lambda$. Suppose a decision rule is to estimate $\lambda$ by using $\delta(Y) = aY$. Now, let $K$ be the class of all decision rules where $0< a<1$. I would like to see that using the squared loss function, every decision rule in $K$ is admissible in $K$. However, I am not sure what it is meant to be admissible in $K$. It seems that for example, If I take $a = \frac{1}{2}$, then $\delta(Y) = \frac{1}{2}Y$, but I can always find another $a$ within $0< a<1$ such that the loss is less. In other words, it seems that to satisfy this criteria, the only way this can happen is if the risk were equal among all rules within the class. Can anyone help me see what is going on? thanks!
Comment: Given the question, I would suggest you add a `self-study` tag. Have you looked at the decision theory books like [Berger's](http://amzn.to/1Wf6TLm) or [mine's](http://amzn.to/1SU6XdZ)?
Answer: To truly see what's going on, graphics help.
Let's begin with the definitions.  I will present them both mathematically and in a symbolic computing language Mathematica (to show how concrete and practical they are).

An estimator $t$ of a parameter $\theta$ is a procedure to convert any sample $X$ into a number $t(X)$.  In this situation you are comparing a set of estimators indexed by a number $a$.  Let us call them $t_a$.  You are told that $t_a(X) = aX$.  Code:
estimator[sample_, a_:] := a sample

The loss $\Lambda$ penalizes discrepancies between the result of an estimator and the true parameter value.  Squared loss is the function $\Lambda(t, \theta) = (t-\theta)^2$. Code:
loss[estimate_, parameter_] := (estimate - parameter)^2

The risk $r$ is the expected loss for any given parameter value $\theta$ when using a particular estimator $t$; that is, $r_t(\theta) = \mathbb{E}_\theta(\Lambda(t(X), \theta)).$  Code:
risk[parameter_, estimator_, loss_, family_] := Module[{x},
  Expectation[loss[estimator[x], parameter], x \[Distributed] family[parameter]]]

Risk is explicitly a function of the parameters.  Comparing functions is tricky, so let's reason it out. To compare estimators $t$ and $t^\prime$, consider a parameter value $\theta$.  If the risk of using $t$ is less than the risk of using $t^\prime$, then obviously you would want to use $t$: its expected loss is smaller.  Unfortunately you don't know the value of $\theta$--that's why you're estimating it!  Thus, you would want to survey the landscape by comparing $t$ to $t^\prime$ for all plausible values of $\theta$.  In the language of risk, you want to graph the functions $r_{t}(\theta)$ and $r_{t^\prime}(\theta)$.  In most cases, for some values of $\theta$ the risk of $t$ will be smaller and for other values of $\theta$ the risk of $t^\prime$ will be smaller.  However, if--given a procedure $t$ you are thinking of using--you can find another procedure $t^\prime$ whose risk at any $\theta$ is no worse than the risk of $t$ and sometimes is definitely better, then why use $t$ at all?  Such a $t$ is called inadmissible.  All other $t$ are admissible.
Graphically, the plot of risk versus parameter for an inadmissible $t$ will lie above (or on) the plot for some other $t^\prime$.  Thus, to assess admissibility you need to plot the risk functions of all the procedures and check whether any two of them cross.  After all, if it's always the case that the plot for $t$ and the plot for $t^\prime$ cross, no matter what $t$ or $t^\prime$ might be, then neither one is always superior to the other.
Let's plot the risks for a careful selection of the procedures $t_a$ in the question.  I chose $a \in \{1/1000, 1/3, 1/2, 6/7\}$ as being good representatives of the whole family.  Since the parameter is any positive number obviously I can't plot the entirety of each risk function, but by looking at a good range of parameter values I can at least get a sense of what's going on.  The range from $0$ to $3$ works well.  Code:
Plot[Evaluate@Table[risk[parameter, estimator[#, a] &, loss, PoissonDistribution],
  {a, {1/1000, 1/3, 1/2, 6/7}}], {parameter, 0, 3}]

Now you can see that the risk functions all overlap.  For small $a$ they rise quickly and attain truly awfully high risks, but nevertheless for very small values of the parameter they still appear to have lower risk than other risk functions.  This is intuitive: a small value of $a$ is steering the estimate to low values and this will work well when the parameter is small.  A larger value of $a$ will work better for larger values of the parameter.
It would be nice to prove that.  To that end, you have to knuckle down and do the calculation.  Code:
risk[\[Lambda], estimator[#, a] &, loss, PoissonDistribution]

$(1-a)^2 \lambda^2 + a^2 \lambda$

All that remains is to show that any two distinct such functions cross at some positive value of the parameter $\lambda$.  That's a matter of elementary algebra of little interest here.  I will point out, though, that since all the risks are multiples of the parameter $\lambda$, you may factor it out, reducing the problem to comparing the graphs of $\lambda \to a^2 + (1-a)^2 \lambda$, which is a family of (mutually crossing) lines:

Each value of $a$ produces a line showing $1/\lambda$ times the risk function for $t_a$.  The colors between the lines graduate from red through blue as $a$ increases from $0$ to $1$.
Answer: If you consider the Poisson problem, $X\sim \mathcal{P}(\lambda)$, under quadratic loss, if $\delta_a(x)=ax$, then
$$\begin{align*}
R(\lambda,\delta_a)&=\mathbb{E}_\lambda[(aX-\lambda)^2]\qquad\qquad\qquad\qquad\\
&=a^2\text{var}_\lambda(X)+(\mathbb{E}_\lambda[aX]-\lambda)^2\\
&=a^2\lambda+(a\lambda-\lambda)^2\qquad\qquad\\
&=\lambda\{a^2+\lambda(a-1)^2\}
\end{align*}$$
which should help you compare the estimators $\delta_a$. Namely,
$$\begin{align*}R(\lambda,\delta_a)-R(\lambda,\delta_b)&=\lambda\{a^2+\lambda(a-1)^2-b²-\lambda(b-1)²\}\\&=\lambda\{[a²-b²]+\lambda[(a-1)^2-(b-1)²]\}\\&=\lambda\{(a-b)(a+b)+\lambda(a-b)(a+b-2)\}\\&=(a-b)\,\lambda\{(a+b)+\lambda(a+b-2)\}\end{align*}$$


What shall we name our chat room?

Question: I hope it's not too early to start asking what to name our chat room.
One name per answer, please.
Answer: Lost in Translation
A sort of play on how things can be confusing in a new land, so you come on here to find out what it means :)
Comment: I like this one a lot
Comment: This seems more appropriate for Linguistics to me :)
Comment: Linguists never get lost in translation d-;
Answer: The Embassy
Much less 'official' than my previous suggestion, yet surely familiar and welcoming to expats :)
Comment: Embassies aren't official? ;-)
Answer: The Consulate
Much like The Embassy, I just like saying the word more :)
Comment: "Consulate" is also more often used in other languages than "Embassy" I think.
Comment: "Consulate" is used a lot more than "Embassy" in general, as there is ever, at most, one Embassy per foreign country, but there are often multiple consulates.
Answer: The Outpost
Just sounds remote and kinda cool - the place to find others like yourself.
Comment: Sounds more like a Fort on a Frontier then expat related... Somehow this always associates with Mos Eisley for me...
Answer: Without Borders
Referring to the fact that the chat is open, and borders don't matter. (I originally wanted to use the Chat Sans Frontières, which would refer to Jeux Sans Frontières, international TV competitions, but I found out that in English, the French title is used, and that would be confusing.)
Comment: Oh, how I miss that show
Answer: The other side
Comment: This reminds me too much of [John Edward](http://en.wikipedia.org/wiki/John_Edward). :P
Answer: The Neutral Zone
Answer: Papers Please
In the style of yester-year
Answer: Over There
Because it's about asking questions about that distant place rather than where you're from.
Answer: The New Life
Most move for a particular reason, for a better job, life, health etc...
Answer: Somewhere Else
Because you used to live somewhere but now you're going to live somewhere else.
Because we have the main site, the meta site, and the chat room is somewhere else.
Answer: The Between
Because we are
Comment: Or **In Between**, or just **Between**, then we can say "I'm in Between" :)
Answer: All Other Passports
Because that's the (usually extremely crowded, slow) line I have to get in at the airport whenever I come home - as opposed to the one (local passports - usually not at all crowded) that my wife gets to use.
Answer: Frontier
Because it means "border" literally, which separate countries. You'll cross one or more when you expatriate.
But it also means a new expanse of the world at its edge, the chatroom is where you'll discuss the new frontiers that the site might pursue, as in making decisions about scope and policy.
Answer: Exile
Because it's funny (-:
Unless you have no sense of humour.
Or unless you have actually been forcibly exiled.
But then again maybe you've been forcibly exiled but still have a great sense of humour (-:
There are also totally random reasons to like it: It's really short. It starts with the same first two letters as the site name.
Answer: We're not in Kansas any more
Not to plug my home state... I'm actually rather ashamed that this is the only thing anyone anywhere knows about Kansas...  But it seems appropriate.
Comment: I don't get the reference. It seems odd to chose some (presumably) US-centric cultural reference for a site about living abroad.
Comment: @GaëlLaurans Wizard of Oz.
Comment: LOL - excellent choice, except for those who are relocating there of course.
Comment: I think just about all native English speakers would get the reference, except perhaps the very young. It's not US-centric any more than the language here being English is England-centric.
Comment: Except I would make it "You're not in Kansas any more"
Comment: @Karlson: The direct quote from the movie is [We're not in Kansas any more](https://www.youtube.com/watch?v=vQLNS3HWfCM).
Answer: The Adjustment Bureau
I do like "The Embassy" but embassies serve a much broader purpose and clientele than the expatriate community.
This is an effort to capture that "special expat twist" - the inevitable adjustments that come with a new context.
Answer: Trailblazer :)
A person who is the first to do something; an innovator!!!
Answer: Destination Abroad

What shall we name our chat room?


How do I fetch a specific version of file to /tmp with git in linux?

Question: That is, I don't want to replace the work space,
but fetch a older version of file to /tmp?
Is that possible ?
Answer: git show $commitid:$filename > /tmp/$filename

Writes version $commitid of file $filename to /tmp/$filename.
Answer:
You can't extract single file from remote repo
You can use git-archive in order to get minimail possible part or tree for needed changeset and remove unwanted files
You can git checkout cs in order to return to changeset, copy file out of tree and return back to HEAD git checkout

git man is your best friend


transpose a columns to row in a list of dataframes and write it to a new dataframe in R

Question: I have a list of dataframes and need to transpose each column in a dataframe and create a new dataframe.  Here is a reproducible example. So I need to create 4 dataframes of size 500 rows and 100 columns by transposing 4 columns for each dataframe in the list.
  # simulate list of 500 dataframes where each dataframe has 100 rows and 4 columns

    mat<-as.matrix(diag(c(1,2,3,4)))

  dat<-list()
     for (i in 1:500){
   dat[[i]]<-as.data.frame(mvrnorm(100,rep(0,4),mat))
                     }

 # I need an output that would look something like this - transpose first column in each of the 500
 dataframes and combine into a single dataframe. I am showing for 3 dataframes in a list.

 t(dat[[1]][,1])
 t(dat[[2]][,1])
 t(dat[[3]][,1])

dat.1<-rbind(t(dat[[1]][,1]),t(dat[[2]][,1]),t(dat[[3]][,1]))
 head(dat.1)

# Tried doing this function for transposing first column of each of the 500 dataframes in the list
 # but gives an error that "dim(X) must have positive length ".
    dat.1<-apply(dat,1,function(x){t(dat[[x]][,1])})

Answer: I think this is what you are going for:
do.call(rbind, lapply(dat, `[`, , 1L))

Answer: You can try :
result <- do.call(rbind,lapply(dat, `[[`, 1))

Answer: In tidyverse, we can use
library(dplyr)
library(purrr)
map_dfr(dat, `[`, 1)


Giving a single sub directory multiple names via htaccess

Question: Primary Scenario ( Answer Needed )
If my website http://website.com has multiple subdirectories:
/features/ . /articles/ . /projects/
Which all contain their any number of files and folders;
Is there a way to change the name of a leading directory?
For example:
http://website.com/content/features/( ... )

Becomes

http://website.com/public/features/( ... )

AND

http://website.com/private/features/( ... )

--
Former Scenario ( On Hold )
Lets say I have a website hosted on http://website.com.
And I would like to add a few personalised projects in a sub section of the site, for example /extra/.
So, lets say I have:
/extra/feature.php?subject=( a childrens cartoon )
/extra/article.php?subject=( fetish photography project )
/extra/ebook.php?subject=( local golfing borchure )

Is there a way to apply an alias to the extra directory, so that it becomes two or more other names? For example:
/extra/feature.php?subject=( a childrens cartoon )

Becomes

/public/feature.php?subject=( a childrens cartoon )

And

/private/feature.php?subject=( a childrens cartoon )

I assume this would be possible with htaccess?
Answer: In .htaccess you can use mod_rewrite to internally rewrite the URL from /public/ to /extra/.
For example, to rewrite only URLs of the form .../feature.php?subject=<something> you could do something like the following in the .htaccess file in the document root:
RewriteEngine On

RewriteCond %{QUERY_STRING} ^subject=[^&]+
RewriteRule ^(?:public|private)/(feature\.php)$ /extra/$1 [L]

This matches URLs of the form /public/feature.php?subject=<something> or /private/feature.php?subject=<something> and rewrites the request to /extra/feature.php?subject=<something>. This is an internal rewrite, the URL in the address bar does not change.
The ?: in (?:public|private) makes the subpattern non-capturing, since we don't need this in the substitution string (aka target URL).
(feature\.php) - This is a capturing subpattern and is referenced in the substitution using the $1 backreference. This simply saves typing (and potential errors). However, you could also group article.php and ebook.php together to handle all three instances with a single directive.
The RewriteCond directive is necessary in order to match the query string. The RewriteRule only matches the URL-path.
The query string from the requested URL is passed through to the target URL by default, so you do not need to do anything with the query string.
Comment: Thanks MrWhite. I'll be trying this out on my website later on. I can't give you an upvote as I don't have 15 reputation, but if all works well I'll accept your answer once tested
Comment: Thank you. Two questions though ... 1) Would the htaccess file be located in `extra`? ... 2) As you've used a `|` between `public` and `private` am I right in assuming that you could have multiple other file names? If so, would you recommend a limit to the number of names you use?
Comment: 1) These directives would need to go in the `.htaccess` in the document root, not in the `/extra` subdirectory. (I've updated my answer.) Assuming `/public` and `/private` don't exist as physical directories then these requests only reach your root directory, until they are rewritten. 2) Yes, you can have multiple other "virtual directory" names. The limit is really determined by readability. Too many and it's simply harder to read. This can be split into multiple conditions if you wish.
Comment: "used a `|` between `public` and `private`" - the whole argument is a regular expression (regex). The `|` (vertical bar) is used to express _alternation_ in regex. The surrounding parentheses `(`...`)` limits the alternation to just that group of characters. https://www.regular-expressions.info/alternation.html
Comment: Gave this a try but there's something wrong. When I visit `/extra/feature/example`, I'm shown the correct page. But when I change `extra` to `public` I'm shown `Not Found : The requested URL ... was not found on this server.`
Comment: I've also updated my scenario. Seems a bit easier to find a solution to but I'm not 100% sure.
Answer: The solution to Scenario 1 is simply 2 lines of code placed in a .HTACCESS file, within the root directory of http://website.com.
The code is:
RewriteEngine on
RewriteRule ^(?:public|private)/(.*) extra/$1

Now, visiting http://website.com/extra/(directory), http://website.com/public/(directory), or http://website.com/private/(directory) will all show the same destination files.
Any other /name/ will result in an error 404.


Exact algorithms (e.g. in coding theory, cryptography) using the field of rational numbers

Question: I noticed that most algorithms in coding theory or cryptography are based on the integers and some arithmetic results (e.g. RSA) or on the finite fields (e.g. Elliptic curve cryptography or BCH codes).
The field $\mathbb{Q}$ is of infinite cardinality, which I suppose makes it less interesting for most applications (e.g. elliptic curves may have an infinite number of points, we cannot easily control the memory required for exact representation, etc.).
Despite this, I was wondering if there were algorithms using rational numbers and their properties as a field (or some extension of $\mathbb{Q}$) with real usage?
Comment: I'm not sure whether this is what you want, but Arithmetic Coding (https://en.m.wikipedia.org/wiki/Arithmetic_coding) uses rationals.
Comment: The [number field sieves](https://en.wikipedia.org/wiki/General_number_field_sieve) (including QS) are one family of examples.
Comment: The number field sieve does not need non-integer numbers , so I do not think it adresses the question.
Answer: Not a direct answer to "algorithms over $\mathbb{Q}$", but there are constructions of error-correcting codes over fields of characteristic zero, which is the closest I know of. I'll throw some references below — but you are right that in general, if it's not a discrete set, it's difficult to utilize for information transmission applications (coding, cryptography, etc.).

D. Augot, P. Loidreau, and G. Robert, "Rank metric and Gabidulin codes in characteristic zero," in 2013 IEEE International Symposium on Information Theory, pp. 509–513, IEEE, 2013.
S. Muelich, S. Puchinger, D. M ̈odinger, and M. Bossert, "An alternative decoding method for Gabidulin codes in characteristic zero," in 2016 IEEE International Symposium on Information Theory (ISIT), pp. 2549–2553, IEEE, 2016.
S. Muelich, S. Puchinger, and M. Bossert, "Low-rank matrix recovery using Gabidulin codes in characteristic zero," Electronic Notes in Discrete Mathematics, vol. 57, pp. 161–166, 2017.
C. Sippel, C. Ott, S. Puchinger, and M. Bossert, "Reed-Solomon codes over fields of characteristic zero." in 2019 IEEE International Symposium on Information Theory (ISIT), pp. 1537–1541, IEEE, 2019.


Descobrir sequencia numerica em array javascript

Question: Olá!
dado o seguinte array:
let array = [3,6,9,10,11,12,16,19]

Preciso pegar os valores sequenciais desse array ou seja, somente os numeros 9 10 11 12 e guardar em outro array. como faria isso?
obrigado!
Comment: Caso a entrada seja `[3, 6, 9, 10, 11, 12, 16, 19, 20, 21, 22]` o que deverá ser retornado?
Comment: Deverá ser retornado todas as sequencias, porem isso que preciso fazer, é em um objeto que contem dia mês e ano que eu uso em um calendário no react native. vamos dizer que a logica seja assim: no meu calendário se estiver marcado 3 6 e 9 ele marca em uma lista os dias com sua descrição. caso seja sequencia ou um range 10 11 12 13 14, preciso saber que tenho uma sequencia que vai de 10 a 14 e adicionar a lista como somente um item que tem a mesma descrição, eu nao preciso criar esses dias pois a descrição deles é a mesma.
Answer: Uma alternativa é utilizar uma pilha para armazenar os números sucessivos.
Funciona assim:

Para cada elementos da entrada olhe para o próximo vizinho:

se o elemento for consecutivo o empilhe e olhe para o próximo vizinho repetindo esse passo sucessivamente se necessário.
se o elemento não for consecutivo descarregue a pilha dentro do resultado e aponte para o ultimo elemento verificado.

let arr = [3, 6, 9, 10, 11, 12, 16, 19, 20, 21, 22, 25];
let resultado = [];

//Para cada elementos da entrada...
for (let i = 0; i < arr.length; i++) {
  let pilha = [arr[i]];                        //inicializa a pilha com o elemento.
  let j = i + 1;                               //Aponta o primeiro candidato a consecutivo.
  //Para cada consecutivo...
  for (; arr[j] == pilha[pilha.length - 1] + 1; j++)
    pilha.push(arr[j]);                        //Empilha o consecutivo.
  if (pilha.length > 1) resultado.push(pilha); //Se a pilha possui mais de um elemento a descarrega no resultado.
  i = j - 1;                                   //Aponta para o ultimo elemento pesquisado.
}

console.log(resultado);

No comentário você apresentou outro requisito o de também exibir individualmente números que não formem listas. Para isso basta remover a comparação de tamanho da pilha antes de compor o resultado.

let arr = [3, 6, 9, 10, 11, 12, 16, 19, 20, 21, 22, 25];
let resultado = [];

for (let i = 0; i < arr.length; i++) {
  let pilha = [arr[i]];
  let j = i + 1;
  for (; arr[j] == pilha[pilha.length - 1] + 1; j++)
    pilha.push(arr[j]);
  resultado.push(pilha); //Descarrega a pilha no resultado sem fazer comparações de tamanho.
  i = j - 1;
}

console.log(resultado);
Answer: fiz assim, funciona.

let teste = [3, 6, 9, 10, 11, 12, 16, 19];

filtro = [];
anterior = '';
lock = 0;
for (x = 0; x < teste.length; x++) {
  atual = teste[x];
  if (anterior == '') {
    filtro[filtro.length] = atual;
    anterior = atual;
  } else if (atual == anterior + 1) {
    filtro[filtro.length] = atual;
    lock = 1;
    anterior = atual;
  } else if (lock) {
    break;
  } else {
    filtro = [];
    anterior = '';
  }
}
console.log(filtro);


Can the cohomology of the Grassmannian identified with the cohomology of a specific dense open subvariety?

Question: Let $(\mathbb{C}^{2p},Q)$ be a $2p$-dimensional complex vector space equipped with a nondegenerate symmetric bilinear form $Q$ where $p\geq 3$. Let $l\leq p-2$. You may assume that $l$ is odd if this helpful. Consider the Grassmannian $\bar{Y}=\mathbb{G}(2(l-1),2p)$ parametrizing $2(l-1)$-dimensional linear subspaces of $\mathbb{C}^{2p}$. Let $W$ be a nondegenerated $2(l-1)$-dimensional subspace. Let $G=\mathrm{SO}_{2p}$. The group $G$ acts on $\bar{Y}$ and we can consider the orbit of $W$ under the action of $G$, this is
$$
Y:=GW=\{V\in\bar{Y}\mid V\text{ nondegenerated}\}=G/Q
$$
where $Q$ is the stabilizer of $W$ in $G$ ($Q=\mathrm{Stab}_G(W)=\{g\in G\mid gW=W\}$). The homogeneous space $Y$ is a dense open subset of $\bar{Y}$ with respect to the Zariski topology.
My question is now the following: do we have an isomorphism in cohomology $H^*(Y,\mathbb{Z})\cong H^*(\bar{Y},\mathbb{Z})$ where the isomorphism is as rings, i.e. is supposed to preserve the cupproduct?
Is there maybe a general satement which implies this isomorphism? Or a specific argument to prove the isomorphism in this situation? There might be an obvious reason for this which I don't know.
I see that we cannot always hope to identify the cohomology of a space with a the cohomology of a dense open subspace, eg. $H^*(\mathbb{R},\mathbb{Z})\not\cong H^*(\mathbb{R}\setminus\{\text{pt}\},\mathbb{Z})$, but I hope that it is true at least in the situation described above.
Any help is appreciated! Thanks!
Comment: You might want to work with Borel-Moore Homology for this to be true.
Answer: Let $M$ be a closed connected manifold, and $Y\subset M$ a dense open subset, missing at least one point. Suppose Y is connected (otherwise $H_0$ distinguishes $Y$ and $M$). Then $H^n(M,\mathbb{Z}/2)=\mathbb{Z}/2$ but $H^n(Y,\mathbb{Z}/2)=0$ as it is a non-compact n-dimensional manifold. If one adds orientable to the assumptions of $M$ then we get this for $\mathbb{Z}$ coefficients.
Comment: It seems that you mean $Y$ instead of $M$ in the second occurence of $H^n$. So you say the isomorphism is just not true in my situation since the Grassmannian is orientable. Thanks!
Comment: Note: I'm not really familiar with the zariski topology, this is just differential topology


Aligning 9 divs in a 3X3X3 Layout With Link Box Under Each Div

Question: Edit #2: I am going to try to implement this:
https://jsfiddle.net/Lcb1eot7/4/
into my giant wall of code and see if I can duplicate the results.
END EDIT #2
EDIT: Alright here is a little fiddle demonstrating the scenario, I did not add many attributes to it so that it is easy to understand. What elements would I need to add to this fiddle in order to achieve my desired effect?
https://jsfiddle.net/Lcb1eot7/3/
END EDIT
I have here a stumper:
What I am trying to accomplish today is this, 9 divs, each div has an a href block underneath it, all aligned in a 3x3x3 div. Here is an example:
1 2 3
a a a

4 5 6
a a a

7 8 9
a a a

The issue I am encountering is that the link boxes are being pushed over to the right of each div, and not underneath it. If I use:
clear:both;

It breaks the format, however the link boxes will align properly underneath each div...
I am now experimenting with:
display: table;

But i still cant get everything aligned the way I want. Here is a screenshot of what my page looks like right now (not all divs appear on screenshot but that should give you the gist of what is going on):

Here is some relevant code:
HTML

More Info

More Info

More Info

More Info

More Info

More Info

More Info

More Info

More Info

CSS
.rowpics1 a {
margin-top: 20px;
position: relative;
overflow: hidden;
display: table;
table-layout: fixed;
}

.rowpics2 a {
margin-top: 20px;
position: relative;
overflow: hidden;
display: table;
table-layout: fixed;
}

.rowpics3 a {
margin-top: 20px;
position: relative;
overflow: hidden;
display: table;
table-layout: fixed;
}

.industryClickerLink {
/* Permalink - use to edit and share this gradient: http://colorzilla.com/gradient-editor/#b11733+0,b7171a+99 */
background: #b11733;
/* Old browsers */
background: -moz-radial-gradient(center, ellipse cover, #b11733 0%, #b7171a 99%);
/* FF3.6+ */
background: -webkit-gradient(radial, center center, 0px, center center, 100%, color-stop(0%, #b11733), color-stop(99%, #b7171a));
/* Chrome,Safari4+ */
background: -webkit-radial-gradient(center, ellipse cover, #b11733 0%, #b7171a 99%);
/* Chrome10+,Safari5.1+ */
background: -o-radial-gradient(center, ellipse cover, #b11733 0%, #b7171a 99%);
/* Opera 12+ */
background: -ms-radial-gradient(center, ellipse cover, #b11733 0%, #b7171a 99%);
/* IE10+ */
background: radial-gradient(ellipse at center, #b11733 0%, #b7171a 99%);
/* W3C */
filter: progid: DXImageTransform.Microsoft.gradient( startColorstr='#b11733', endColorstr='#b7171a', GradientType=1);
/* IE6-9 fallback on horizontal gradient */
display: table-cell;
width: 290px;
height: 30px;
position: absolute;
padding-bottom: 5px;
overflow: hidden;
float: left;
bottom: 0;
}
.row1a a {
float: left;
position: relative;
overflow: hidden;
display: table-cell;

}

.row1b a {
position: relative;
float: left;
overflow: hidden;
display: table-cell;
}

.row1c a {
display: table-cell;
position: relative;
float: left;
overflow: hidden;
}

There is more code but I think that should get the point across. So does anyone know whats going on here? What can I do to get these aligned properly and using a cross browser compatible method? Thank you for the help.
And to clarify, I am trying to get the "MORE INFO" boxes underneath each picture square.
Comment: I did not include the "fadein" div in the CSS but it looks like this (it is a JS image transition thing):                                                                    .fadein {
    position: relative;
    width: 290px;
    height: 190px;
    left: 0;
    top: 0;
    overflow: hidden;
}
Comment: A ***small*** demo just enough to demonstrate the problem would be of more use than the mega codedump.
Comment: Sure let me create a demo!
Answer: So... I finally found the issue.... the simplest thing that took me almost an entire day to find. I broke everything down and wrote it all from scratch and then boom, I saw what my mistake was. In the CSS I was calling all of the row1a,row1b,row1c, etc. as row1a a, row1b a, row1c a. Each box is a link but I did not realize I had placed the a tag alongside them in the CSS.
At the same time, I also cleaned up all of my code as I realized how redundant it was.
In summary row1a, row1b,row1c, etc. now look like this:
rowBox {
    width:290px;
    display:inline-block;
    float:left;
}

Answer: Here is my update to your 1st edits fiddle to achieve the effect you're asking about: https://jsfiddle.net/Lcb1eot7/5/
To break it down:
1) I gave each 'littleBoxOfFun#' div a common class of 'lbf'.

Becomes

2) In the stylesheet I targeted the lbf class and added a float:left, and a right margin of 10px for visibility.
.lbf{
    float:left;
    margin-right:10px;
}

3) Because every 3rd lbf div does not need any margin to the right I used an nth-child selector to target every 3rd lbf div and remove the margin.
.lbf:nth-child(3n+0){
    margin-right:0;
}

Hope this helps.
Comment: Awesome suggestion Justin, I may try this in a future project! Thanks for your help and time.


Display Search Query in URL PHP

Question: I have a textbox that allows users to search my website.
<form method="post" action="">
  <input type="text" name="search" id="search" />
  <input type="submit" name="submit" id="submit" value="Search" />
</form>

Upon the user clicking the "Search" button, how do create the following URL, and append the contents of the search box to end, whilst adding "_" in replace of any spaces?:
http://www.mysite.co.uk/find/this_is_my_search
Also, if somebody edits the search query in the URL, how do I update the search textbox to reflect this change, so the box contains the following
this is my search.
Comment: why would you do that? Do you really think that user will be editing search query in browser address field and replace all spaces to underscores?
Answer: You can change action URL with JavaScript before submit event.
You can Use onSubmit event or use jquery
$('form').submit(function(){
    $(this).attr('action', $('#search').val().replace(' ', '_'))
})
Comment: I'm looking to do this using PHP only, not JavaScript
Answer: You need to handle that url using you 404 page. You can use $_SERVER['REQUEST_URI'] to get that path. For instance you could do this in your custom 404 page:
<?php
    if (stripos($_SERVER['REQUEST_URI'], "/find/") == 0) //If url is yoursite.co.uk/find/
    {
        $searchterm = basename($_SERVER['REQUEST_URI']);
        //Search for $searchterm
        //Print the results
    }
    else
    {
        header("HTTP/1.0 404 Not Found");
    }
?>

Answer: This can be achieved several ways.
Javascript
See StarLight's answer. It's simple and does the job, but you need to check for illegal URL-characters.
PHP redirect
Example: Set form action to 'search.php'.
<?php
$urlstr = urlencode(str_replace(' ', '_', $_GET['search']));
header("Location: http://www.mysite.co.uk/find/$urlstr");
exit;
?>

To update the textbox you need the following code on the page showing the search box.
<?php
// First get $urlstr from url.
$searchstr = str_replace('_', ' ', urldecode($urlstr));
$textboxvalue = (empty($searchstr)) ? 'Search' : $searchstr;
?>

<input type="submit" name="submit" id="submit" value="<?=$textboxvalue?>" />

Answer: Change form method to get.. And also if you're using code igniter or other framworks you should be able to code like
  $search = Uri:segment(2);
  echo '<input type="submit" name="submit" id="submit" value="'.$search.'" />';

Comment: What do i need to do in the .htaccess file though, to make the url friendly: http://www.mysite.co.uk/find/this_is_my_search
Comment: You can use ;
    RewriteEngine On

    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.*)$ index.php/$1 [NC,QSA,L]
 . your url simly should look like index.php/find/blablabla .. So in index.php you should route find controller with search param. But it's another job.. Just use codeigniter framework it's easy and handle all this stuf for you

Display Search Query in URL PHP