Recommendations on ZFS on FreeBSD as a NAS box?

Question:
Please note that the answers on this page are from 2009, and should not be relied on as gospel. If you have a specific question about zfs then please click the Ask a Question button and ask a specific question.

I was thinking of building a home backup system using FreeBSD 7.2 and the ZFS file system. Has anyone had any experience with that file system?
Specifically:

Is it possible to boot from ZFS? (Would I want to?)
How easy is it to add a drive?
How well does it handles drives of different sizes?
Can you add new drives on the fly (or at least with just a reboot)?
Would I be better served by something off the shelf?

Any other thoughts and suggestions would be welcome.
Edit:
Just to be clear I have read the FreeBSD page on ZFS. I am looking for suggestions from people with practical experience with a similar setup to what I want.
Answer: I build a home FreeBSD file server using ZFS.
It is an AMD X2 3200+ with 3GB of RAM. It has a PCI Express Gig-E. The boot drive is an old 400GB and I have 4 750GB Seagte drives (one with a difference firmware version, just in case).
Booting from ZFS would have been nice (it would make install simpler), but I used the ZFSOnRoot instructions to setup the Root/OS drive with ZFS (if all the partitions are ZFS, then it doesn't need to do a fsck at boot to check the UFS filesystems). The reason that you would want this that you can then setup all of your partitions (/var, /usr, /tmp, etc.) with different options as required (such as noatime and async for /usr/obj, which will speed kernel compiles), but they will all share space from a common pool. Then you can setup a data drive and give each user a partition of their own (with different quotes and settings). You can then take snapshots (which are low cost on ZFS).
My home server has a df that looks like:
/dev/ad0s1a           1.9G    744M    1.1G    41%    /
devfs                 1.0K    1.0K      0B   100%    /dev
dozer/data            1.8T     62G    1.7T     3%    /data
dozer/home            1.7T    9.6G    1.7T     1%    /home
dozer/home/walterp    1.9T    220G    1.7T    11%    /home/walterp
tank/tmp              352G    128K    352G     0%    /tmp
tank/usr              356G    4.4G    352G     1%    /usr
tank/var              354G    2.2G    352G     1%    /var

Performance wise, copying files is real fast. The one thing that I would note is that I have been using ZFS on FreeBSD AMD64 systems that have 3-4GB and it has worked well, but from my reading, I'd be worried about running it on a i386 system that had 2GB or less of memory.
I ran of out of SATA ports on the motherboard, so I have not tried to add any new drives. The initial setup was simple, a command to create the RAIDZ and then the command to create /home, which was formatted in seconds (IIRC). I'm still using the older version of ZFS (v6), so it has some limitations (It doesn't require drives of an equal size, but unlike a Drobo, if you had 3 750GB drives and a 1TB drive, the end result will be as if you had 4 750GB drives).
One of the big reasons that I used ZFS with RAIDZ was the end-to-end checksums. CERN published a paper that documented a test they did where they found 200+ uncorrected read errors while running a R/W test over a period of a few weeks (the ECC in retail drives is expected to have a failure once every 12TB read). I'd like the data on my server to to be correct. I had a hard crash because of a power outage (someone overloaded the UPS by plugging a space heater to it), but when the system can back, ZFS came back quickly, without the standard fsck issues.
I like it, because I could then add CUPS to Samba to get a print server. I added a DNS cache and can add other software as I like (I'm thinking about adding SNMP monitoring to the desktops at my house to measure bandwidth usage). For what I spent on the system, I'm sure I could have a bought a cheap NAS box, but then I wouldn't have a 64-bit local Unix box to play with. If you like FreeBSD I'd say go with it. If you prefer Linux, then I'd recommend a Linux solution. If you don't want to do any administration, that is when I would go for the stand alone NAS box.
On my next round of hardware upgrades, I'm planning on upgrading the hardware and then installing the current version of FreeBSD, which has ZFS v13. V13 is cool because I have a battery backed up RAM disk that I can use for the ZIL log (this makes writes scream). It also has support for using SSDs to speed up the file server (the specs on the new Sun File Servers are sweet, and they get them from a ZFS system that using SSD to make the system very quick).
EDIT: (Can't leave comments yet).
I pretty much followed the instructions at http://www.ish.com.au/solutions/articles/freebsdzfs. The one major change that exists in 7.X since those instructions were written was that 7.2 came out and if you have 2+ GB, you should not have to add the following three lines in /boot/loader.conf:
vm.kmem_size_max="1024M"
vm.kmem_size="1024M"
vfs.zfs.arc_max="100M"

The instructions also explain how to create a mirror and how to put the system back into recovery mode (mount with ZFS). After playing with his instructions once or twice, I then used the ZFS Admin manual from Sun http://opensolaris.org/os/community/zfs/docs/zfsadmin.pdf to understand better what ZFS was. To create my data store, I used a modified version of the command on Page 91 to create ZFS pools. This being FreeBSD, I had to make a small change:
zpool create dozer raidz /dev/ad4 /dev/ad6 /dev/ad8 /dev/ad10

Where ad4-ad10 where found by doing dmesg |grep 'ata.*master', this are the names of the SATA hard drives on the system that will be used for the big data partition. On my motherboard, the first three ata ports (ad0-3) where the 4 PATA ports and then because each SATA port is a master, there are no old numbers.
To create the file system, I just did:
zfs create dozer/data
zfs set mountpoint=/data dozer/tank

The second command is required because I turned off default mountpoints for shares.
Comment: Do you by chance have notes of everything you did? And would you be willing to post them? :)
Comment: Great post on FreeNAS + ZFS. I've been curious about this myself. Thanks for your info!
Answer: Introduction:
I finally built my system, and here are my notes, in case it helps anyone else.
Goals:

Build a home NAS box which can also double as my source control and
internal web server.
Keep the cost under $1000

Specifications:

Must have at least one terabyte of storage
Must have data redundancy (RAID or something similar)
Must be able to replace my current aging source code control server

Design:

FreeBSD 7.2 (eventually to be upgraded to 8.0).
OS is on its own boot drive, in this case one IDE drive
Data is stored on six SATA drives.

We use ZFS as the file system, since it has gotten such favorable reviews.
        ZFS pretty much requires that we run a 64 bit OS, and likes a lot of
        memory, so I should get a minimum of 4Gb
Hardware:

ABS Aplus ABS-CS-Monolith Black SECC Steel ATX Full Tower Computer Case  - 1 @ $69.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16811215009
Western Digital Caviar Green WD5000AADS 500GB SATA 3.0Gb/s 3.5" Hard Drive 6 @ $347.94 ($57.99 ea)
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136358
XIGMATEK XLF-F1253 120mm 4 white LED LED Case Fan - 1 @ $8.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16835233015
LITE-ON Black IDE DVD-ROM Drive Model iHDP118-08 - 1 @ $19.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16827106275
Crucial 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) CT2KIT25664AA800  - 1 @ $45.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16820148160
GIGABYTE GA-MA74GM-S2 AM2+/AM2 AMD 740G Micro ATX AMD Motherboard - 1 @ $54.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16813128342
OKGEAR 18" SATA II Cable Model GC18ATASM12  - 6 @ $11.94 ($1.99 ea)
http://www.newegg.com/Product/Product.aspx?Item=N82E16812123132
AMD Athlon 64 X2 5050e Brisbane 2.6GHz Socket AM2 45W Dual-Core Processor Model ADH5050DOBOX - 1 @ $62.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16819103298
KINGWIN Mach 1 ABT-1000MA1S 1000W ATX / BTX Power Supply - 1 @ $199.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16817121037
Seagate 400GB IDE Drive - 1 @ (had in closet), but an 80GB IDE drives cost about $37.00,
    and 80GB is more than enough.
Adaptor Bracket for IDE hard drive to fit in a five inch bay - 1 @ ~ $17.00
Shipping - 1 @ ~ $35.00

Software:
    FreeBSD 7.2 - 1 @ $0.00
    http://www.freebsd.org/
Total Cost:
    $874.81
Hardware Setup:
    A basic computer build, with three minor issues.

The case I purchased had slots for 6 hard drives, and two 3.5 bays. I
assumed the IDE could fit in one of the 3.5 bays. This was a bad
assumption, and there was no reasonable way to make it work. I went and
bought an adapter at Fry's for ~ $17.00, and it worked fine.
The Sata cables I purchased had the 90 degree connectors, which was
nice, except with six drives, there was no way to make the work. Plugging
in one cable caused the inflexible part of the connector to hang over the
next hard drive. I had to go to Fry's and by 5 regular SATA cables. Sadly
the ones I bought at Newegg were so cheap, that it is not worth sending
them back.
The case points the back of the hard drives to the side of the case,
and the power cables from the power supply have a stiff connector which
stuck out over the edge of the case. This didn't allow me to slide the
side cover back into place. I had to play around with it a bit to get it
to work, and eventually ended up with the two of the modular power cables
(they have four SATA plugs on each) interleaved between the drives, so that
the first cable powered drives 0, 2, and 4, and the second powered 1, 3,
and 5. This allowed enough flex that I could zip tie them out of the way.

OS Setup:

Burned the FreeBSD 7.2 ISOs to CD. I could have used the single DVD,
but I didn't have any lying around.
Burned memtest86+ ( http://www.memtest.org/ ) onto a CD.
Powered up the freshly built computer, and went into the bios to make
sure it saw all 7 drives and the DVD-ROM. It did. Changed the boot order
to make the CDROM first.
Inserted the memtest86+ CD into the freshly built computer, rebooted it
up, and let it run over night. Passed with no errors.
Installed FreeBSD 7.2, if you are unfamiliar with this I recommend
reading the following: http://www.freebsd.org/doc/en/books/handbook/install.html
It does a much better job of explaining what to do than I can.
Here are my specific settings:

Did a Standard install
Used the entire IDE drive for the OS

used the default file system layout
left the 6 SATA drives untouched

Developer install without X-Windows, since the box will be
          headless
The system is not an NFS Client or Server
FTP and inetd disabled
SSH allowed
Added no packages (those would be added later).
Added one user

After install and reboot I noticed that only 4 of the 6 SATA drives
were detected. I went into the BIOS and Under Integrated Peripherals
change OnChip SATA Type to be AHCI, and OnChip SATA port 4/5 Type to be
"SATA" Saved settings and rebooted.
At this point FreeBSD detected all six drives as:
    ad4
    ad6
    ad8
    ad10
    ad12
    ad14
Get the latest from cvs using csup:
    csup -g -L 2 stable-supfile
I had already edited the file to use the host:
    cvsup11.us.FreeBSD.org
leaving all other information as is.
Rebuilt and installed the latest kernel and world as described here:
http://www.freebsd.org/doc/en/books/handbook/makeworld.html
    Customized my kernel (see ZFSNAS). I disabled a large set of devices,
    since I never plan on using SCSI, USB, PCMCIA, Serial, Parallel, etc.
added the following to /etc/make.conf:
    CPUTYPE=athlon64
    CFLAGS= -O2 -fno-strict-aliasing -pipe
make -j8 buildworld

NAS Setup:

Create the ZFS pool for our storage:
    zpool create storage raidz2 ad4 ad6 ad8 ad10 ad12 ad14
Create the home filesystem on the newly created storage:
zfs create storage/home
cp -rp /home/* storage/home
rm -rf /home /usr/home
zfs set mountpoint=/home storage/home

edit /etc.rc/conf and add the following:
zfs_enable="YES"

This mounts the ZFS file systems on bootup.
Created root, samba, and perforce directories
zfs create storage/root
cp -rp /root/* storage/root
rm -rf /root
zfs set mountpoint=/root storage/root
zfs create storage/fileshare
zfs create storage/perforce

Unless you need more file systems on your pool, you are pretty much done
with the ZFS part. See the following for more details:
http://www.freebsd.org/doc/en/books/handbook/filesystems-zfs.html
http://opensolaris.org/os/community/zfs/docs/zfsadmin.pdf

Ports installed:
/usr/ports/shells/bash
    make install
/usr/ports/editors/vim
    make install
/usr/ports/net/samba33
    make
    make install
    // Use all defaults un-check cups.
/usr/ports/devel/perforce
    make
    make install PERFORCE_PORT=XXXX PERFORCE_USER=p4user PERFORCE_GROUP=p4
    rm -rf /usr/local/perforce
    cd /storage/perforce/
    mkdir root
    mkdir log
    chown p4user:p4user *
    cd /storage
    chown p4user:p4user perforce

Edited /usr/local/etc/perforce.conf as follows:
    #
    # Perforce FreeBSD configuration file
    #
    #
    # $FreeBSD: ports/devel/perforce/files/perforce.conf.in,v 1.3 2005/01/18 15:43:36 lth Exp $

    #
    # Perforce ROOT
    #
    PERFORCE_ROOT="/storage/perforce/root"

    #
    # Perforce user (it is recommended to run p4d as a non-root user)
    #
    PERFORCE_USER="p4user"

    #
    # p4d/p4p port (default: 1666)
    #
    PERFORCE_PORT="XXXX"

    #
    # p4p cache directory
    #
    PERFORCE_PROXY_CACHE="/usr/local/perforce/cache"

    #
    # p4p target server (default: perforce:1666)
    #
    PERFORCE_PROXY_TARGET="perforce:1666"

    #
    # p4d options (see man p4d)
    #
    PERFORCE_OPTIONS="-d -p $PERFORCE_PORT -v server=1 -L /storage/perforce/logs/p4d.log"

    #
    # Uncomment this line to have the server started automatically
    #
    PERFORCE_START=yes

Users Added:
user1
user2

Groups created:
sambashare
    Added user1 and user2 as members

chgrp sambashare /storage/fileshare
chmod 775 /storage/fileshare
chmod g+s /storage/fileshare

Samba Configuration:
Samba configuration file:
#################
    [global]
       workgroup = USERLAN
       server string = ZFS NAS
       security = user
       hosts allow = 192.168.1. 127.
       log file = /usr/local/samba/var/log.%m
       max log size = 50
       passdb backend = tdbsam
       dns proxy = no

    [user1share]
       comment = user1 share
       path = /storage/fileshare
       valid users = user1 user2
       public = no
       writable = yes
       printable = no
       create mask = 0765
#################

pdbedit -a -u user1
    # followed prompts
pdbedit -a -u user2
    # followed prompts

Comment: Fantastic answer! Upvote... check!
Answer:

Is it possible to boot from ZFS? (Would I want to?)

I see no reason why you'd want to, I'd think the Snapshot support is only mature enough in OpenSolaris so that you can switch back to an older version and boot that (buth that's actually just wild guessing).

How easy is it to add a drive?

Add as in expand a striped pool? Just add a drive to the pool, that's about it. Consider the implications of your next question thou.

How well does it handles drives of different sizes?

You could use it as a stripe and tell ZFS to keep n copies of a file. So you could use the full storage availability you have and still get decent redundancy

Can you add new drives on the fly (or at least with just a reboot)?

Replacing Devices in a Storage Pool I guess this is the recommended solution, the easiest way to find out how well this works on FreeBSD is probably to give it a try.

Would I be better served by something off the shelf?

Have you considered FreeNAS (Roadmap) 0.70 seems about to be released and will support ZFS.
You will save yourself the hassle of all the framework and get a relatively nice to use GUI for free with it.
Comment: I'm using FreeNAS 0.7RC1, and it's been a breeze to setup with ZFS.
Answer: I have servers with FreeBSD+ZFS(on 7.2-STABLE and 8.0-CURRENT), not in production thou.
Booting from ZFS is described here
http://lulf.geeknest.org/blog/freebsd/Setting_up_a_zfs-only_system/
Adding drives on the fly is as easy as typing "zpool add mypool da7", new drive is usable right after this; you also can add whole bunch of drives in stripe, mirror, raidz(improved raid-5) or raidz2(improved raid-6)
Drives of different sizes can be placed in pool but can't be used in mirror/stripe/raid(if i recall correctly, only smallest hdd space will be usable then)
(Open)Solaris have support of ZFS right out of the box
Answer: There's a nice thread on building a home ZFS NAS over at ArsTechnica.
Answer: I've no experience with ZFS, but I just saw this in my RSS feeder:
http://www.freebsdnews.net/2009/06/10/zfs-version-13-freebsd-current/
Which suggests that it is maturing, but isn't ready for production (on FreeBSD).


Changed frame returns to default when view disappears and appears again

Question: I have a tab bar application using storyboard and in one view, I change the frame of a UIButton animated within a method. The thing is that the method is perfectly continuing while another tab bar item is presented (so the view controller is not dismissed but only disappeared) but when the first view appears again, the button whose frame had changed is in the default position again, as I set it in interface builder.
Does anybody know if there's an easy way to prevent this behaviour and keep the current frames when a view disappears?
Answer: Are you using Auto Layout? (Storyboard > Utilities (right panel) > File Inspector - Has a Use Auto Layout checkbox). If so, try to uncheck it. If it works right, then you need to change the Layout Constrains, not the frame.
Parameters like frame and center are ignored under Auto Layout. Instead, constrains have a parameter called constant that can be changed and animated.
Comment: @fabf98dev Sure thing. Run into this myself. Layout Constrains can be created in Interface Builder and Control+Drag as property into the code. You can then change the myConstrain.constant easily.


openldap ch_calloc core dump in docker containers

Question: With docker 24, in the debian docker image, running openldap results in a malloc segmentation fault. Even a simple version check:
$ docker run --rm -it --entrypoint bash debian
# apt update && env DEBIAN_FRONTEND=noninteractive apt install --yes slapd
# slapd -V
@(#) $OpenLDAP: slapd 2.4.57+dfsg-3+deb11u1 (May 14 2022 18:32:57) $
Debian OpenLDAP Maintainers <firstname.lastname@example.com>

slapd: ../../../../servers/slapd/ch_malloc.c:107: ch_calloc: Assertion `0' failed.
Aborted (core dumped)

Then obviously trying to launch the server with something like slapd -F /tmp/slapd -h ldap://:639/ -d -1 also fails with a core dump.
Is this an upstream bug or a docker misconfiguration?
Answer: Setting a smaller open files limit, like for instance executing ulimit -n 1024 before running slapd solves the issue.


Is it possible to get relation in Views to Commerce products?

Question: I made View by Taxonomy term.
Using Relations I got Content which has this Taxonomy term.
Now I want to see couple of Commerce fields (SKU and Price) which are referenced from Content type.
Is it possible from Taxonomy View go to Content (I have already done it) and then to Commerce fields from Content?
Comment: Yes you use the product relationship.
Answer: Make the relationship with commerce product then you will see the related fields in fields list.


A mathematical function f() that partitions integers into two sets, {1} and the rest of integers.

Question: To elaborate a little more,
I want an algebraic formula for a function f such that $f(1)=x$ and $f(y)=z$ where $y\neq 1$ and $z\neq x$ ($z$ being the same value for all $\Bbb Z\setminus\{1\}$).
If possible, I would like to have this solution.
$$f(1) = 0, \qquad f(y) = 1, \text{ where } y\neq1$$
This way, given the output of this function $f$, I will know the number passed to $f$ is not $1$.
I want to use it in an equation (containing variable $x$ and bunch of other variables) where I want to set the value of the equation to zero when $x=1$.
So, if I have equation $$p = (x-1)(x+1) + 2y$$
I want it to function properly except when $x=1$ in which case I want the output to be zero.
So, I will modify the equation to
$$p = ((x-1)(x+1)+2y)\cdot f(x)$$
The output won't change for any other number since $f(z)=1$.
Comment: Do you mean $y! =1$ or $ y \neq 1$ ? ... would you like me to edit ?
Comment: You already defined the function in your question. It is called the indicator function of $\mathbb{Z}\setminus \{1\}$. Pergams you are asking for an (algebraic) formula for this function? Then you should state that in your question
Comment: [Here's a MathJax tutorial](http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) :)
Comment: Why would you want an algebraic formula for this?
Comment: $(x-1)(x+1)+2y$ is not an equation
Comment: Apologies and thanks for the edits. Yes I wanted an algebraic formula for the given function. The answer given by @bof does the trick!
Answer: The expression
$$f(x)=1-0^{|x-1|}=0^{0^{|x-1|}}$$
does the trick, provided you're O.K. with $0^0=1.$
Alternatively, using the ceiling function:
$$f(x)=\left\lceil\frac{(x-1)^2}{(x-1)^2+1}\right\rceil$$
Comment: The first solution is really elegant. +1


How to link a new textview created on java with its xml characteristics in android studio?

Question: In Java, I have created a TextView whose text is the string written by the customer on an EditText in a previous activity. In the XML of the activity, I wrote the characteristics I want this TextView to have. My code creates a new TextView without those characteristics, anyone knows how to link them? I attach the java and XML codes. javacode xmlcodeThanks in advance!
Comment: Avoid taking screenshots or your code..post your code snippet in your question itself
Answer: You need to define the textview properly.
In oncreate method add
TextView text_message_1;
text_message_1=(TextView) findViewById(R.id.text_message_01);

Then set the text you get from other activity.
Answer: Since you are creating a TextView dynamically, so there is no need define a TextView tag in XML. You can simply set properties of that TextView in Java and add it to any ViewGroup or Table or anything you want.
And you can also try using findViewById() method for that TextView.
If you still face any problem, let me know.
I hope it answers well.


Breaking a monolith into microservices using templates for similar apps

Question: I have a huge monolith application, a ASP.NET MVC Framework service offering ~50 different "groups" of unrelated data, separated into 50 different controllers/business logics. I was tasked into breaking this monolith into 50 microservices (each one hosted in Docker containers).
What I'm planning to do is to create a "template application" using ASP.NET Core to serve as a reference project and, for each of those 50 applications, I will fork this "template" project, add the corresponding controller with the necessary business logic and may need or not to customize this template to add some specific features.
For example, one application may need to respond to requests using a CSV format instead of the JSON, which is the standard for ~45 apps. So, I would modify the "template project" to include this "CSV response" feature and it would be used by the other ~4 projects that would use it.
What I can see currently:
PROS

Developing a ASP.NET Core application is easy, but it requires some configurations and customizations (CORS, Cache, routes, etc.) that would be already defined into "my company template" project.

CONS

Forking and merging the template project can be troublesome. If we find a security issue in the template, it would be necessary to manually update all of the 50 projects and fix some potential merge conflicts.

Is there another option available in ASP.NET Core to make this "template" project easier for updates and maintenance? Or a design pattern for this?
Answer: Just use a class library. Common functionality and even base controllers and such can all go there and be referenced by the actual API projects. Your common config can be added to the class library as IServiceCollection extensions:
public static class IServiceCollectionExtensions
{
    public static IServiceCollection AddMyCors(this IServiceCollection services)
    {
        services.AddCors(...);
        return services;
    }

    ...
}

Then, any change to this library automatically propagates to your APIs because of the direct reference.
However, bear in mind, that pretty much the entire point of microservices is loose-coupling. You should really only share this kind of functionality for things that are relatively static and won't change. Making changes that requires every single one of 50 services to be updated, pretty much defeats the entire point.
If there's too much common, shared functionality, you have to ask yourself if you've truly decomposed the services into subdomains. A very common mistake in this way is not realizing that the actual data should be segregated as well. Ideally, each microservice should have its own data store (database, for example). If you are trying to make the microservices share the same data store, then you will inevitably end up with significant cross-over and it will be difficult if not impossible to segregate their domains. That means you loose niceties like foreign keys sometimes and you may need to even incur some data duplication. That's where CQRS, event sourcing, and other related patterns start to come into play.


I can't install mediawiki on the google cloud

Question: I installed everything. Followed the instructions and then once I finished I put LocalSettings.php on the same folder as index.php. That is site/wiki/ Yet it still says that LocalSettings.php is missing. I typed the address LocalSettings.php and it gives me a 404. Its almost as if LocalSettings.php is being blocked. How do I fix this?
Comment: Make sure that space in `LocalSettings. php` isn't there, too. It should be `LocalSettings.php`.
Comment: I saved it as LocalSettings.php I just wrote it like that because this site thought I was trying to type code for some odd reason.
Comment: Wrong file permissions, presumably?
Answer: https://www.mediawiki.org/wiki/Manual:LocalSettings.php

The LocalSettings.php file is not a wiki page and you cannot access it with your web browser.


Origins of measure and integration

Question: Calculating integrals is often called as "Computing an area or a volume". More formal, integrals are measures defined in a certain space (satisfying $\sigma$-additivity and sometimes invariance conditions like translational invariance). I am asking about the very origins of measures and integrals.
Here, I think that measures and integrals arise from counting distinct "squares" or "cubes". In a multidimensional space, every Dimension is different; therefore different combinations of hypercubes can be aligned. Counting all possible combinations of hypercubes that can be aligned in a $N$-dimensional space leads to the following volume formula $V=A^N$ with the number of cubes that span one Dimension $A$.
Is the origin of measures a combinatorial origin? Which logics lead to this concept?
Comment: Maybe not quite addressing your question, but still a great read: [A garden of integrals, Frank E. Burk](http://www.maa.org/publications/ebooks/a-garden-of-integrals)
Answer: Can't comment yet, but I think there might be ambiguity over whether you are talking about historical origins or an abstract framework that gives the topic a place in the tree of mathematical knowledge.
I'll assume you meant the latter.
My interpretation is that the $\sigma$-algebra gives an idea of the level of detail we want in a measure space. The simplest sigma algebras are partitions, which creates equivalence classes of points. This type of $\sigma$-algebra is very coarse in that it says that large classes of points are indistinguishable from our perspective.
Very fine $\sigma$-algebras allow us to distinguish between every point in the set.
If you already have a topology on the set, then Borel $\sigma$-algebras allow you to base the fineness of the measure on the topology. In this case, to know the value of the measure on a particular set in the $\sigma$-algebra, one only needs to know the values on the topological sets.
In the case that the topology is given by a metric we only have to know the value of the measure on balls of different radii.
The case of cubes that you mentioned are products of metric spaces, and this is where it all began historically.
The fineness of the measure becomes more important when we consider measurable functions. These are similar to measuring devices and the fineness of the $\sigma$-algebra is similar to the precision or tolerance of the measurements. An example from probability concerns two random variables $X,Y:(\Omega,\mathscr{F})\to\mathbb{R}$, where $\mathscr{F}$ is the $\sigma$-algebra on $\Omega$. One can consider the $\sigma$-algebras $\sigma(X)$ and $\sigma(Y)$ obtained by pulling back measureable sets in $\mathbb{R}$. Note that if we have given $\mathbb{R}$ a partition for a $\sigma$-algebra then $\sigma(X)$ and $\sigma(Y)$ can at most just be partitions of $\Omega$, and so won't contain much information.
Lets give $\mathbb{R}$ the Borel $\sigma$-algebra. Conditioning $X$ on $Y$, say by considering $Z=\mathrm{E}[X|Y]$ creates a third $\sigma$-algebra. Since $Y$ already contained information, we expect that $\sigma(Z)$ should be coarser than $\sigma(X)$, and indeed, this is the case. When the space $\Omega$ consists of square integrable functions we can think of $\mathrm{E}[X|Y]$ as a projection of $L^2(\Omega,\sigma(X))$ onto a subspace $L^2(\Omega,\sigma(Y))$.
So, as in a lot of mathematical areas, the concept of measure spaces started with a very specific set of measure spaces. At this point in time, measure was probably considered a specialized topic for people interested in integration and differential equations. Over the years the topic has been abstracted quite a bit, leading to the way that students (like me) are taught to think about it, which is much more general than its origins would suggest.
Comment: Comments on the conditional expectation example would be nice. I'm still learning this abstract version of conditionally defined variables and don't want to put out misinformation.
Comment: and thank you @kryomaxim for accepting this answer. I can now comment everywhere.

Origins of measure and integration


How to express "round up to the nearest 2" in mathematical notation?

Question: I'm trying to solve a problem I posted here on Math SE. Part of my solution is to round up to the nearest x (e.g. 4.5 rounds up to 6 if x = 2). How do I express that in mathematical notation?
Comment: I actually need a "ceiling that rounds to nearest `x`"
Comment: But $4.5$ is nearer from $4$ than from $6$. What do you exactly mean with "the nearest $2$?
Answer: Typically, we round to integers, and so we have good notation for doing so.  For example:

$\lfloor x \rfloor$ rounds $x$ down to an integer,
$\lceil x \rceil$ rounds $x$ up to an integer, and
$\lceil x \rfloor$ rounds $x$ to the nearest integer (with the borderline case $\ast.5$ rounding up or down according to some rule).

To round to a multiple of some given number, we can first divide by that number, round to an integer, then multiply by that number.  Thus if we want to round $x$ up to the nearest multiple of $m$, we can write
$$ m \left\lceil \frac{x}{m} \right\rceil.$$
To round down, or to round to the nearest multiple of $m$, replace $\lceil \cdot \rceil$ with the appropriate operation.
Answer: How about $2\lceil x/2 \rceil$?
EDIT: Yes, it does work.
Proof: We know that $0\le \lceil x/2 \rceil-x/2<1$. Then $0\le 2\lceil x/2 \rceil-x<2$. Since $2\lceil x/2 \rceil$ is an even integer, it is the least even integer that is not lesser than $x$.
Comment: I think that works. Thanks.


Redirect multiple https ports to multiple external ip & ports

Question: I'm not sure if this is possible or not.
I would like to accept https traffic on multiple ports (not just 443) and redirect them to external IP addresses and ports, for example:
mydomain.com:443 -> 188.8.131.52:10000
mydomain.com:444 -> 184.108.40.206:10001
mydomain.com:445 -> 220.127.116.11:20111

Basically multiple ports accepting https traffic, forwarded to external IP addresses on different ports.
I have been able to do this with http traffic using iptables but now I'm struggling to do it with https traffic. I have been experimenting with /ports.conf but I think I'm in the wrong place as this is more to do with using multiple ports for multiple websites on the same server, which is not what I want to do. I do not need any traffic to stay on the server and serve content, I just want to NAT the incoming https traffic onto its destination.
Should I be trying to do this in iptables? If so, how do I specify that it is the https traffic to redirect, not the http traffic. I've had a brief look at nginx, but I'm not familiar with it (I mainly use apache).
Thank you.
Comment: Many ISPs block port 445. So don't count on that working. As for the rest, why do you even want to use separate ports?
Answer: I've been able to resolve this using VirtualHosts in apache.
I listened for multiple http ports in ports.conf
<IfModule ssl_module>
    Listen 443
    Listen 444
    Listen 445
</IfModule>

And then simply added the following to my .conf file in sites-available
<VirtualHost *:444>
    ProxyPreserveHost On
    ProxyRequests Off
    ServerName xxx
    ServerAlias xxx
    ProxyPass / http://220.127.116.11:10000/
    ProxyPassReverse / http://22.214.171.124:10000/
    SSLCertificateFile /etc/letsencrypt/live/xxx/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/xxx/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>

Hope this helps someone.
Comment: I think it's better to `Listen 444 https` and/or to put `Listen` textually just before ``, because whenever you delete `` you will accidentally reveal your `DocumentRoot`. Also, you most likely don't need these additional ports, your true goal is something quite different.
Comment: @kubanczyk Thank you. But the my `Listen` directives are in the `ports.conf` file not in the .conf file under `/sites-available` folder where I am specifying the `VirtualHost` for the additional ports and redirection


does the following function have all directional derivatives?

Question: $$xy\sin(\frac{1}{xy})$$
the function has partial derivatives at every point , but i wanted to know whether this function had directional derivatives at every point? for $x=0$ the function is $y\sin(\frac{1}{y})$ and for $y=0$ the function is $x\sin(\frac{1}{x})$ and when both functions are $0$ the function has a value of $0$ .
Answer: The best thing here if you want to know if there are directional derivatives everywhere that you simply try to calculate for an arbitrary point the directional derivative in an arbitrary direction. Look when it is possible and when not. This gives you the answer.


How sum of squares is calculated by R ANOVA function fo non-factor variables in linear model

Question: > d = data.table(a = rnorm(40), b = rnorm(40), c = rnorm(40))
> summary(aov(a ~ b + c, d))
            Df Sum Sq Mean Sq F value Pr(>F)
b            1   1.17  1.1707   0.836  0.367
c            1   0.07  0.0677   0.048  0.827
Residuals   37  51.84  1.4011

I understand how it's done for factor variables, as those divide whole dataset into groups. But how is it calculated for numerical variables b and c?
Comment: It's the sum of squares of differences of predictions from measured. Generally one would be using the `lm()`-function rather than aov. Take a look at the help page for lm to see what values get returned.
Comment: As I understand you've described Residuals Sum Sq. But how are other two Sum Sq (for b and c) calculated?
Comment: There are actually different methods. One method (the easiest to grasp in one sentence) is to look at the increment in sums of squares due to regression when a covariate is added.
Answer: One method (the easiest to grasp in one sentence) is to look at the increment in sums of squares due to regression when a covariate is added. This is R's ANOVA (or AOV) strategy, which implies that the order of addition of variables is important:
> anova( lm(mpg ~ cyl, mtcars))
Analysis of Variance Table

Response: mpg
          Df Sum Sq Mean Sq F value    Pr(>F)
cyl        1 817.71  817.71  79.561 6.113e-10
Residuals 30 308.33   10.28
---

When we add another variable the regression sums of squares stays the same for the cyl variable:
> anova( lm(mpg ~ cyl+disp, mtcars))
Analysis of Variance Table

Response: mpg
          Df Sum Sq Mean Sq F value    Pr(>F)
cyl        1 817.71  817.71 87.5883 2.903e-10
disp       1  37.59   37.59  4.0268   0.05419
Residuals 29 270.74    9.34

If disp is added first, its SS-regression is maintained and the incremental SS-regression is attributed to the next covariate, this time to cyl.
> anova( lm(mpg ~ disp+cyl, mtcars))
Analysis of Variance Table

Response: mpg
          Df Sum Sq Mean Sq F value    Pr(>F)
disp       1 808.89  808.89  86.643 3.271e-10 ***
cyl        1  46.42   46.42   4.972   0.03366 *
Residuals 29 270.74    9.34

There is ongoing holy-war between the proponents of this method as default and the SAS authors who want to use a method that allocates sums-of-squares differently (and I don't think I can state in one sentence what they do do except to say that the sums of squares regression using so-called "type-III" ANOVA for each variable at any given level of complexity are not affected by the order of addition or removal of variables.)
The proponents of the R approach think that the theory-agnostic application of stepwise methods is bad statistics. They think you should be setting up yur models based on what is known or established by existing science and then adding variables that represent any new hypotheses. I'm not sure who invented the "typing" system for sums-of-squares strategies but R uses type II while SAS uses type III sums of squares in their respective default regression methods. There are R packages that can provide a type III calculation if that's what you need to attempt replication of SAS results. My memory is the the car package has an Anova function that will allow specification of the desired type.


Single-layer perceptron mathematical formulation

Question: I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then
$$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\sigma\left(\sum_{j=1}^dw_{ij}x_j + h_i\right)$$
So the initial $d$ is the dimension of the input vector, let's say $d=3$ and we have $N=4$ hidden neurons. This means that each single neuron $i$ in the hidden layer can be written as
$$\sum_{j=1}^3w_{ij}x_j = \sigma(w_{i1}x_1+w_{i2}x_2+w_{i3}x_3 + h_i) \, \, \, \, , \, \,  i = \{1,2,3,4\}.$$
But then what do coefficients $c_i$ actually represent here? Maybe they are the weights connecting the $i$-th neuron of the hidden layer to a single output? So maybe this formalism is describing a single-layer perceptron with a single final output?
And if yes how would I modify the above formulation to describe a multi-output perceptron? Maybe I could intend it coordinate-wise.
Comment: Draw out a small network with one predictor, two hidden neurons, and one output neuron, and see if you can figure it out there. // I believe a parameter is missing from your first equation.
Answer: This is not single layer perceptron, which doesn't have a hidden layer. That said, $c_i$ represents the weight of the $i$-th hidden neuron's output. You could have a multi-output perceptron if you remove the last accumulation, but it should be something your problem requires.
Comment: Apologize I meant single-hidden-layer
Comment: Ok, to have multiple outputs, you need to have multiple output neurons. Currently, you only have one. And, that should be something your problem can accommodate, i.e. what does the output of your model signify?
Comment: How should I mathematically modify the formulation that I have provided in order to accomodate multiple neurons? Let's say that maybe the output of my model signifies a multi-target regression, or a multi-label classification.
Comment: You need multiple output neurons and it would be easier with matrix notation, e.g. $y=W^Tx$, where $y$ would represent a vector of outputs.


Airy function's identity.

Question: Suppose that $z>0$ and let us write $z=v^2$ where $v>0$, the question is how to show that from $Ai(z)=\frac{1}{\pi}\int_0^\infty \cos(\frac{1}{3}t^3+zt)dt$ that:
$$Ai(v^2)=\frac{1}{2\pi i}\int_I \exp(v^2 s-1/3 s^3)ds$$
where $I$ is the imaginary axis from $-\infty i$ to $+\infty i$.
I first tried the identity: $\cos(\ldots)= \frac{1}{2} (\exp(i\ldots)+\exp(-i\ldots))$, but I don't see how to continue from there?
Thanks!
Comment: $\cos$ is an even function, while $\sin$ is odd, which means you can transform: $$\frac{1}{\pi}\int_0^\infty \cos(\frac{1}{3}t^3+zt)dt= \\ = \frac{1}{2\pi} \int_{-\infty}^\infty \left( \cos(\frac{1}{3}t^3+zt)+i \sin (\frac{1}{3}t^3+zt)  \right)dt$$
Comment: @YuriyS That's literally the answer.
Comment: @PeterForeman, I don't like posting short answers, so anybody is welcome to post their own ;)
Comment: @YuriyS yes it was trivial, as all of maths when done right. :-) cheers!
Answer: \begin{align*}
Ai(v^2)&=\frac1\pi\int_0^\infty \cos\left(\frac13t^3+v^2t\right)\,dt\\&=\frac1{2\pi}\int_\mathbb R\cos\left(\frac13t^3+v^2t\right)\,dt\\&=\frac1{2\pi}\int_\mathbb R\cos\left(\frac13t^3+v^2t\right)+i \sin\left(\frac13t^3+v^2t\right)\,dt\\&=\frac1{2\pi i}\int_\mathbb R e^{i (\frac13t^3+v^2t)}\,d(it)\\&=\frac1{2\pi i}\int_Ie^{v^2s-\frac13s^3}\,ds.
\end{align*}

Airy function's identity.


Where to include common scripts

Question: I am looking at adding a new javascript library to drupal 8. I want the ability to call this library from other modules / libraries.
The way WordPress does this is by the use of wp_enqueue_scripts. Using a common library would make sure that my library is loading just once.
An existing implementation of this in drupal is the way you can add dependencies into your the yml files
js:
 dependencies:
  - core/jquery

Is there a way I can add my own javascript library as the dependency that can be used throughout my drupal instance (all modules and themes should be able to use this as a dependency)
Comment: Have a look at https://www.drupal.org/docs/8/theming/adding-stylesheets-css-and-javascript-js-to-a-drupal-8-theme it explains it really well.
Answer: To add dynamic libraries as shared libraries, check the following
hook_library_info_build() for dynamic library definitions
You can add javascript libraries to a theme like the following
Adding Javascript to a D8 theme
You can add javascript libraries to a module like the following
Adding Javascript to a D8 module

Where to include common scripts


How to use two .cpp files in a project (class)?

Question: How to use two .cpp files in a project?
main.cpp:
#include "second.h"
int main () {
    secondFunction();
}

second.h:
void secondFunction();

second.cpp:
#include "second.h"
void secondFunction() {
   // do stuff
}

It works, but when I use class it doesn't work.
empoyee.h:
class employee {
public:
void paymentmethod();
};

employee.cpp:
 #include "employee.h"
 void employee::paymentmethod() {
 //code
   }

main.cpp:
#include "employee.h"
main()
{  employee em;
 em.paymentmethod()

}

I had figured out error in .cpp declaration.
Comment: It's hard telling what could be wrong without looking at the contents of the Makefile. Please post the contents of the Makefile.
Comment: Did you omit the inclusion of employee.h in main.cpp? Also please post the error message
Comment: Welcome to SO! First of all, "help me solve this" is not a question, but an inappropriate imperative. Second, it appears your post does not include the complete error message, that makes it difficult to tell what went wrong.
Answer: I'm not sure what it is you are trying to do but here are some tips:

Every header file should include all headers that you are going to use for that file or class.
Every cpp file that corresponds to a particular header file should include ONLY that header file.
Your main.cpp should include all headers that you are using for your program.

Your code should look something like this:
// main.cpp
#include "second.h"
#include "employee.h"

int main(){
//code
}

In your other files:
//second.h

//includes here
void function();
//---------------------------------------------

//second.cpp
#include "second.h"

void function(){ //code }

//---------------------------------------------

//employee.h

//includes here

class employee{
public:
     void payment();
//ctor, dtor, and other declarations
}

//---------------------------------------------

//employee.cpp

#include "employee.h"

void employee::payment(){ //code }

//rest of methods

Unfortunately, there will be build problems if you include the same header more than once in the same program. So for all header files, you should implement this guard:
#ifndef __MYCLASS_H__
#define __MYCLASS_H__

//includes

class MyClass{
//declarations
}

#endif

Comment: I disagree on points 2 and 3: **2** .cpp files may very well need things from other headers not needed in their corresponding header. **3** The main file should, like all others, only include what it *needs*.
Comment: I see your point about cpp files needing headers not in their corresponding headers, but I personally put everything I will need for my cpp in the corresponding header. The cpp should be free of declarations, includes, and any other preprocessor directives. The cpp should simply contain code. That's how I learned though. And the Google C++ Style guide doesn't seem to care how this is done particularly. It just says, "any includes present in the related header do not need to be included again in the related cc (i.e., foo.cc can rely on foo.h's includes)."


Need text after particular text and before other particular text

Question: I am trying parse out some text that is in an html file. The text I am trying to parse out it is the text that is always after the text developer and before the text >
So an example would be:
developerNeedThis1></textarea>

In this case I want the text: NeedThis1.
I try this:
(?<=name=developer).*
But this gets me:

NeedThis1></textarea>

Any tips on how to also add in to get the text before
></textarea>

Thanks.
Answer: Depends on what tool you're using, but here's sed:
sed -e 's/developer\([^>]*\)>.*/\1/'

Input:
developerNeedThis1></textarea>

Output:
NeedThis1

Comment: Thanks. I am using Jmeter which I believe is perl based.   I am testing in expresso and got this one to work (?<=developer).*(?=>`
Comment: link doesn't have exactly what I looking for.  your way also does not work unfortunately.  There'sa  regular expression tester in JMeter and when I test my way it works!  But, when I run the test it does not.  Very frustrating
Comment: Apache JMeter doc says: Note that (?<=regexp) - lookbehind - is not supported. Looks like I am goosed http://jmeter.apache.org/usermanual/regular_expressions.html


File creation in rails + jquery

Question: I am new to Rails. In my web application, the user can type the contents of a file in a textarea or a content Editable iFrame and when he saves the file, the contents should be written to the file with the file extension specified by the user. How can I do so??
Comment: Where do you want to save file, on client side or on server?
Answer: Where're you having a problem? Saving a file in ruby is easy. Something like this should work, assuming you have parameters file_type and text.
File.open("myfile.#{params[:file_type]}", 'w') { |f|
  f.write(params[:text])
}

File creation in rails + jquery


Somebody help me understand this ctypes python code?

Question: <code>hotkey_ = enable_shortcut       # Check to break the while loop

def proper_fctn():
    if hotkey_:
        if not user32.RegisterHotKey(None, shortcut_id, modifier, vk):
            pass
        try:
            msg = wintypes.MSG()
            while user32.GetMessageA(byref(msg), None, 0, 0) != 0:
                if msg.message == win32con.WM_HOTKEY:
                    if not hotkey_:
                        break
                    fctn_to_run()
                user32.TranslateMessage(byref(msg))
                user32.DispatchMessageA(byref(msg))
        except:
            pass
</code>
If somebody could help me understand the lines, so I could understand the process better.
Comment: That's like...all the lines. Can you be more specific about your questions?
Answer: These are Win32 APIs.  All of them are well-documented in Microsoft's MSDN documentation.  You've basically written a Windows application with a standard main loop.
Google takes you straight to their docs.
https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerhotkey
https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getmessage


Smooth infinitesimal analysis and law of excluded middle

Question: I read about smooth infinitesimal analysis and I have several questions:

How to prove that in SIA every function on $R$ is continuous? (Every function whose domain is $R$, the real numbers, is continuous and infinitely differentiable.)

What does "$ε.1$" and "$ε.0$" mean in this proof? (https://publish.uwo.ca/~jbell/basic.pdf , page 5-6)

For what purpose do we use Kock-Lawvere axiom when we deal with law of excluded middle? (https://www.fuw.edu.pl/~kostecki/sdg.pdf , page 21)

Thanks!
Comment: It looks as if "e" is $\varepsilon$ which is epsilon.
Comment: @soupless Thanks. I edited it.
Answer: It just means the product.  The point is that assuming the law of excluded middle would imply (by the argument given there) that
$$
(\forall \epsilon) \;\epsilon \cdot 1 = \epsilon \cdot 0.
$$
By the cancellation principle, we derive $1=0$.  This is the desired contradiction.  Therefore LEM cannot hold in the setting of synthetic differential geometry (a.k.a. smooth infinitesimal analysis).


Is the body of the ukulele structured for right-handed people?

Question: Is the body of the ukulele structured for right-handed people? Can left-handed people play the ukulele without discomfort?
Answer: It depends on how. the body of your ukulele is shaped. Most ukuleles look like smaller accoustic guitars, ie. symmetrical, so no problem there. Some custom ukuleles would have this problem, but in general no.
Additionally, if you want to switch the strings, the tensions are so similar that you don't have to worry about breaking the ukulele when restringing for lefty playing. Of course, one could simply hold it like a lefty and read chord diagrams backwards (I've heard imagining seeing through the neck to get to the fretboard helps, but I'm not lefty, so I don't know).
The ukulele is one of the easiest instruments to adapt for left-handers.
Answer: Since the back of most ukes are symmetrical, it won't make any difference whether it's played with the neck facing left or right. Unlike some electric guitars, which are built handed - but that doesn't stop some players! Even the cutaways don't cause too much grief (double, obviously!) The strings will be in the 'wrong' places if it's just turned round, but even that isn't a huge problem - unless you've already learned the shapes - albeit with the wrong hand now! Highly unlikely. And swapping strings round is no big deal.
A lot of lefties on all kinds of instruments will opt for a right-handed approach, due to several factors - better choice of less expensive models, ability to play someone else's instrument, the fact that one's dominant hand (mainly fingers) is /are playing  the more intricate stuff.
Bearing that in mind, I've taught far more lefties to play right (on guitar and bass mainly), than I have taught lefties left-handed. It's not a problem either way for me, but weighing up pros and cons often tips the balance. There's always the one or two who are entrenched in lefthandedness, and I respect that.
So basic answer is no difference, but why?!


Microsoft Exchange Transport is rejecting message submissions because the available disk space has dropped below the configured threshold

Question: This relates to a Windows SBS 2011 server and Exchange 2010.
We are seeing the following errors in Event Log:
15004:

The resource pressure increased from Normal to Medium.
The following resources are under pressure: Queue database path
  ("C:\Program Files\Microsoft\Exchange
  Server\V14\TransportRoles\data\Queue\mail.que") = 97% [Medium]
  [Normal=95% Medium=97% High=99%] Queue database logging path
  ("C:\Program Files\Microsoft\Exchange
  Server\V14\TransportRoles\data\Queue\") = 97% [Medium] [Normal=95%
  Medium=97% High=99%] Physical memory load = 88% [limit is 94% to start
  dehydrating messages.]
The following components are disabled due to back pressure: Inbound
  mail submission from the Internet Mail submission from Pickup
  directory Mail submission from Replay directory Content aggregation
The following resources are in normal state: Version buckets = 0
  [Normal] [Normal=80 Medium=120 High=200] Private bytes = 2% [Normal]
  [Normal=71% Medium=73% High=75%] Batch Point = 0 [Normal] [Normal=2000
  Medium=4000 High=8000] Submission Queue = 0 [Normal] [Normal=1000
  Medium=2000 High=4000]

and 15006, same message as above except the first line reads:

Microsoft Exchange Transport is rejecting message submissions because
  the available disk space has dropped below the configured threshold.

From my research on this it should point to lack of available disk space on the server. However, from the 494GB C: drive there is 64GB of free space. This should be plenty of space. Can anyone give me some idea on how to resolve this issue. It has happened about 4 times in the past week.
We restart the transport service or the entire server (we aren't a big company so this isn't too big a deal for us) and this resolves the issue.
Answer: This is known as "Back pressure" and it is a safety measure to ensure that the disk or partition that hosts the Exchange Transport database has sufficient free space to process incoming mail. Exchange monitors free space based on percentage and has the defaults described in the Event Log you shared: [Normal=95% Medium=97% High=99%]. Microsoft has an in depth article on TechNet describing back pressure in detail.
Exchange stores these files on the system drive by default. While it is possible to adjust the default thresholds, the typical (and more responsible) solution is to move the location of the Transport database to another drive.
After Exchange 2007 SP1, Microsoft included a PowerShell script to simplify and automate the process of moving the database to a different location. This script is located at C:\Program Files\Microsoft\Exchange Server\Scripts\Move-TransportDatabase.ps1 and should be run from within the Exchange Management Shell.
Of note is that there are 3 databases that can be moved by this script and I recommend moving all three. They are:

Transport Queue Database (Default: %ExchangeInstallDir%TransportRoles\data\Queue)
IP Filter Database (Default: %ExchangeInstallDir%TransportRoles\data\IpFilter)
Temporary Storage (Default: %ExchangeInstallDir%TransportRoles\data\Temp)

Each of these databases has an associated log that can (and should) be moved as well.
Each of the databases can be moved by specifying the appropriate parameter when running the script. The log associated with each database can be moved with its own parameter. You can identify the syntax for the script by running it with no parameters:
 [PS] C:\Program Files\microsoft\Exchange Server\scripts>.\Move-TransportDatabase.ps1

For a more reader-friendly description, Petri has a decent article on back pressure relating to Exchange 2007 as well as basic instructions for how to move the database if you do not have SP1 installed and instructions for running the script.
Comment: The `Move-TransportDatabase.ps1` script is described in a "Note" in this Microsoft Document https://docs.microsoft.com/en-us/exchange/mail-flow/queues/relocate-queue-database It is still supported on the latest versions of Exchange!


Where does this come from? or how do I derive it?

Question: $$\delta \vec{x} = \frac{\partial\vec{x}}{\partial r}\delta r + \frac{\partial\vec{x}}{\partial \theta}\delta \theta+\frac{\partial\vec{x}}{\partial \phi}\delta \phi$$
Answer: This is just the chain rule. You have that $\vec{x}$ is a function of $(r,\theta.\phi)$, so that if we take a derivative of $\vec{x}$ with respect to some variable $t$, we have
$$
\frac{d\vec{x}}{dt}=\frac{\partial\vec{x}}{\partial r}\frac{dr}{dt}+\frac{\partial\vec{x}}{\partial \theta}\frac{d\theta}{dt}+\frac{\partial\vec{x}}{\partial \phi}\frac{d\phi}{dt}
$$
Thus, the first order taylor expansion for $\delta \vec{x}$ is
$$
\frac{\partial\vec{x}}{\partial r}\delta r+\frac{\partial\vec{x}}{\partial \theta}\delta\theta+\frac{\partial\vec{x}}{\partial \phi}\delta\phi
$$
Comment: so like $\vec{x} + \vec{\delta x} = \vec{x} + \delta t \frac{d\vec{x}}{ dt} =\vec{x} + (\frac{\partial\vec{x}}{\partial r}\delta r+\frac{\partial\vec{x}}{\partial \theta}\delta\theta+\frac{\partial\vec{x}}{\partial \phi}\delta\phi) $?
Comment: Yeah, exactly like.


Application of the Pythagorean Theorem in Word Problem

Question:
When you follow the link above, you'll find that the image shows 2 poles set up in a room with two boxes set up to run along them. Pole 1 is parallel to the west and east walls while pole 2 is parallel to the north and south walls. All measurements of each are shown in the image. Box 1 starts at the end of Pole 1 and Box 2 starts at the end of Pole 2.

Find n (the distance between Box 2 and Box 1)
The boxes begin running along the poles at the same time at a speed of 0.3km/h. What is the direct distance between them after two seconds?
During the first 14 secs, calculate the smallest possible distance between the boxes while they run along the poles and the exact time at which this happens.

I calculated that the answer to the first question would be 38.4057 metres while the answer to the second question would be 25.9808 metres. However, I don't know how to work out the answer to the last question. Help would be much appreciated :)
Comment: Find position of box $1$ and box $2$ after $t$ seconds. (speed of each is 5/6 m/sec). Now find the distance between the boxes in terms of t. Differentiate. Find $t$. Use the $t$ that is less than or equal to 14 sec.
Comment: But are my answers to the first two questions correct?
Comment: @sku also, if I use your method, it won't let me find the exact time (to 2.d.p) as I will be working with whole numbers and essentially, using the guess and check method. Are there any other strategic approaches?
Answer: Let the segment joining top of box 1 and box 2 be AB.
Let the segment joining bottom of box 1 and box 2 be CD.
In time $t$, A and B would have moved $5t/6$.
Consider segment CD as a function of time $t$.
$CD^2(t) = (25 - 5t/6)^2 + (15-5t/6)^2$
So $AB^2(t) = (25 - 5t/6)^2 + (15-5t/6)^2 +5^2$ ... Eqn(1)
Plug $t =0$ for question $1$ and $t=2$ for Q2.
For $3$;  differentiate. $-125/3 + 25t/18 -75/3 + 25t/18 =0$
You get $t=24$
But we want min distance within $14$sec. So plug $t=14$ in the distance Eqn(1)
Comment: I don't really understand this method. Can you please elaborate?
Comment: Since it involves this complex picture, I dont know how to explain. Hopefully someone in the community can do a better job. However, let me know which part you dont understand.


Commerce Paypal Confirm Pane missing

Question: I have installed Drupal Commerce with latest version and also Commerce Paypal + Commerce Paypal Express Checkout.
EC is working so far, but I can´t find the Confirm-Pane on the checkout page/configuration.
Can somebody give me a tip what to do to make it visible on the checkout configuration.
It should be visible under "Confirm Order" and its called:
 "Express Checkout review and confirm (only to be used on the confirm order page)"
Answer: I fixed it...
I had a similar module installed (commerce_paypal_ec) and it was conflicting with commerce_paypal submodule for EC.
i misunderstood the concept, i tought i had to install submodules but those exist already in commerce_paypal.
in case somebody run into the same problem.
kind regards,
kevin

Commerce Paypal Confirm Pane missing


Como voltar à activity e executar um método?

Question: tenho a activity Perfil.class e la exibe minhas fotos, ao clicar nela, abre um fragmentDialog exibindo a foto em slideshow... mas ao deletar a foto, preciso q volte ao meu perfil e atualize as minhas fotos, executando fotos(myID);
Como fazer isso, se possivel?!
SlideshowDialogFragment.class
private void delete_photo(final String idPhoto){
        pDialog.setMessage(getResources().getText(R.string.wait));
        showDialog();
        String tag = "deletePhoto";
        pDialog.setMessage(getResources().getText(R.string.wait));
        showDialog();
        StringRequest req = new StringRequest(Request.Method.POST,
                Config.DEL, new Response.Listener<String>() {
            @Override
            public void onResponse(String response) {
                hideDialog();
                try {
                    JSONObject jObj = new JSONObject(response);
                    boolean error = jObj.getBoolean("error");
                    if (!error) {
                        new Alert().MessageSuccess(getActivity(), view,
                                getText(R.string.delete_photo_success).toString());
                        // Voltar para meu Perfil.class e executar fotos(myID);

                    } else {
                        new Alert().MessageError(getActivity(), view,
                                getText(R.string.delete_photo_error).toString());
                    }
                } catch (JSONException e) {
                    e.printStackTrace();
                }
            }
        }, new Response.ErrorListener() {
            @Override
            public void onErrorResponse(VolleyError error) {
                new Alert().MessageError(getActivity(), view,
                        getText(R.string.error_connection).toString());
                hideDialog();
            }
        }) {
            @Override
            protected Map<String, String> getParams() {
                Map<String, String> params = new HashMap<String, String>();
                params.put("me", myID);
                params.put("id", idPhoto);
                params.put("token", code);
                return params;
            }
        };
        CustomApplication.getInstance().addToRequestQueue(req, tag);
    }

Answer: Passo a passo:
1 - Crie uma interface a coloque dentro dela um método void resposta();
2 - Faça a activity implementar essa interface e programa dentro do método implementado a atualização da lista.
3 - Dentro do fragmento, declare uma variável do tipo da interface.
4 - Na hora de criar o fragmento, envie o this(a activity) e insira esse this na variável da interface que está no fragmento.
5 - Quando terminar o método de exclusão de fotos do fragmento(ou na última linha do método de exclusão), chame interface.resposta();
Esse comando final irá chamar o método resposta(); da activity(pq dentro a interface conterá o valor this da activity).
Comment: nao entendi o passo 3 e 4... Criei a interface PhotosRefreshMyProfile e no Perfil usei implements PhotosRefreshMyProfile. Tambem no Perfil criei o metodo @Override
    public void resposta() {
        fotos(myID);
    }  esta certo até aqui ?
Comment: consegui, muito obrigado :)


One random record from everyone month

Question: Say we have the following data frame:
    ShippedDate        OrderID  Subtotal
1   1996-07-16 00:00:00 10248   440.00
2   1996-07-10 00:00:00 10249   1863.40
...
25  1996-08-06 00:00:00 10272   1456.00
26  1996-08-12 00:00:00 10273   2142.40
...
51  1996-09-11 00:00:00 10298   3127.00
52  1996-09-13 00:00:00 10299   349.50
...
78  1996-10-14 00:00:00 10325   1497.00
79  1996-10-14 00:00:00 10326   982.00
80  1996-10-14 00:00:00 10327   2262.50
81  1996-10-17 00:00:00 10328   1168.00
etc

How to choose a record (random) for each?
For example

1   1996-07-16 00:00:00 10248   440.00
2   1996-08-12 00:00:00 10273   2142.40
3   1996-09-13 00:00:00 10299   349.50
4   1996-10-14 00:00:00 10327   2262.50

how to calculate the sum for each month?
For example

    1   1996-07 2303.4
    2   1996-08 3598.4
    3   1996-09 3476.5
    4   1996-10 5909.5
Answer: I would suggest you will create a year-month variable once and then use it in all the rest of the operations in order to avoid recalculating it over and over again.
Here's a possible data.table approach
Create variable
library(data.table)
setDT(df)[, YearMonth := paste0(year(ShippedDate), "-", month(ShippedDate))]

Randomly sample from each year-month
df[, .SD[sample(seq_len(.N), 1)], by=YearMonth]
#    YearMonth         ShippedDate OrderID Subtotal
# 1:    1996-7 1996-07-16 00:00:00   10248    440.0
# 2:    1996-8 1996-08-12 00:00:00   10273   2142.4
# 3:    1996-9 1996-09-11 00:00:00   10298   3127.0
# 4:   1996-10 1996-10-14 00:00:00   10326    982.0

Sum per year-month
df[, sum(Subtotal), by=YearMonth]
#    YearMonth     V1
# 1:    1996-7 2303.4
# 2:    1996-8 3598.4
# 3:    1996-9 3476.5
# 4:   1996-10 5909.5

Comment: Thank you so much for everything
Comment: Super, thx. But how get second column form
df[, sum(Subtotal), YearMonth]?
a <- df[,c(2)] stopped working
Comment: V1 column form df[, sum(Subtotal), YearMonth]
Comment: Oh, that's because its a `data.table` class now. You can either do `df[, "V1", with = FALSE]` or just convert it back to data.frame using `setDF(df)` and then use your syntax as normal. By the way, you can call `V1` whatever you want when aggregating, for example `df[, .(Sum = sum(Subtotal)), YearMonth]`
Comment: Also, I'd recommend looking into `data.table` vignettes as this is a great package I'd recommend you to get familiar with, see [here](https://github.com/Rdatatable/data.table/issues/944)
Answer: You can select a random sample of size n (1 in your case) per month and year as (if your original data is stored as df):
library(zoo)
library(dplyr)
df_g <- group_by(df, as.character(as.yearmon(ShippedDate)))

sample_n(df_g, n, replace = FALSE)

You can calculate the sums per month and year as follows (using the supplied data):
summarise(df_g, sum = sum(Subtotal))

Source: local data frame [4 x 2]

  year_mon    sum
1 Aug 1996 3598.4
2 Jul 1996 2303.4
3 Okt 1996 5909.5
4 Sep 1996 3476.5

Answer: I would select a random row for each month by:
df[tapply(1:nrow(df), substr(df$ShippedDate,1,7), FUN= sample, size= 1),]

To get the total of the Subtotal column for each month:
result <- tapply(1:nrow(df), substr(df$ShippedDate,1,7), FUN= sum)

If you need the "result" months and totals in a data.frame:
data.frame(names(result), result)

I really need to learn to use plyr, but for now I'm trying to learn plain R.

One random record from everyone month


Clojure error calling a recursive function -- most likely a parentheses issue

Question: I am VERY green to Clojure. But I am trying to learn the syntax by creating some functions. I am having issues with the parentheses and syntax in general...
This function is supposed to take a list and a position and return the list with that position removed--but I am getting an error that I don't fully understand. I have done some reading and it seems like its an issue with nested parentheses.. but I'm not sure how to fix it.
Any feedback would be greatly appreciated.
The error:

ClassCastException java.lang.Long cannot be cast to clojure.lang.IPersistentCollection clojure.core/conj (core.clj:83)

The code:
(defn delete-at
"accepts a list and position--returns the list with
 value at that position removed"
(
[L, pos]
(cond
  (empty? L) nil
  (zero? pos) (rest L)
  :else (
          delete-at (first L) (rest L) (- pos 1))
  )
)
([L-new, L2, pos]
(cond
  (zero? pos) (conj L-new (rest L2))
  :else (
          (delete-at (conj L-new (first L2)) (rest L2) (- pos 1))
          )
  )
)
)

Comment: The parentheses in Clojure, or any other Lisp for that matter, are not the same as curly braces in any C-family language. The way you format your code seems to indicate that you believe they are. Take a look here: http://hitchhikersclojure.com/blog/hitchhikers-guide-to-clojure/ for an introduction to Clojure syntax.
Answer: giving it a pass through emacs' formatter makes the problem jump out to me: an extra set of () around the last call to delete-at and the arguments to the first conj call are reversed.
(defn delete-at
"accepts a list and position--returns the list with
 value at that position removed"
([L, pos]
 (cond
  (empty? L) nil
  (zero? pos) (rest L)
  :else (delete-at (first L) (rest L) (- pos 1))))
([L-new, L2, pos]
 (cond
  (zero? pos) (conj L-new (rest L2))
  :else ((delete-at (conj L-new (first L2)) (rest L2) (- pos 1))))))

when reading Clojure, lots of stacked closing parens ))))) is normal and looks good to the eye (once you get used to it) and stacked opening parens (( jumps out as suspicious. It mainly comes up when you are calling a function that returns a function, and then you want to call the resulting function.
user> (defn delete-at
 "accepts a list and position--returns the list with
  value at that position removed"
([L, pos]
 (cond
  (empty? L) nil
  (zero? pos) (rest L)
  :else (delete-at (first L) (rest L) (- pos 1))))
([L-new, L2, pos]
 (cond
  (zero? pos) (conj (rest L2) L-new)
  :else (delete-at (conj L-new (first L2)) (rest L2) (- pos 1)))))
#'user/delete-at
user> (delete-at [1 2 3] 1)
(1 3)

It's well worth getting emacs setup with cider, clojure-mode, and paredit (paredit is very useful and getting used to it is a bit of an accomplishment) This is a good place to start learning, and many people choose to use a starter kit of some sort such as emacs starter kit or emacs live
Comment: just as a side note this can be written as `(concat (take (dec pos) l) (drop pos l))` amongst many other ways.


What are the popular production software deployment systems?

Question: I am curious to know what are the popular production deployment system i.e. popular ways in which things are moved from test/qa sandbox and final releases are pushed into production available to be used. I have heard of things like autosys,vcs,zapp. Are these industry standard production deployment systems? What do biggies like Google,Microsoft,Yahoo,Amazon etc use?
Answer: One tool used widely at Yahoo is pogo, which is a massively parallel ssh-based execution tool. It can be used to invoke commands in parallel on thousands of machines at once.  Pogo also contains a constraint mechanism to say things like "only run on 20% of this type of server at once".  There's also a pre/post hook and recipe mechanism in pogo to perform custom check and command before and after a push.  One way this is used frequently is to run health checks which verify that services on a machine come back up after a software push.
A similar tool that others use is Capistrano.  I don't know a lot about it but that tool gets mentioned a lot.
Many people use the remote execution facility in their configuration management tool to accomplish this task as well. This can be done for example with Puppet or Cfengine or Chef.  Those solutions tend to work ok at smaller scales.  At larger scale (thousands of servers) you should really look in to a more specialized tool like the ones I've mentioned above.
Of course you can also use tools like ClusterSSH to accomplish tasks like this, although I recommend against that approach because it's not really scriptable or repeatable.


cmake: Include directories root?

Question: I'm trying to set up cmake for a project I'm working on, but I have a problem which I can't resolve currently. My project has the following folder structure:
MotorEngine (root dir)
| CMakeLists.txt
| ThirdParty
|-| SDL2
|-|-| include (contains all header files for SDL2)
|-|-| lib
|-|-|-| x64
|-|-|-|-| SDL2.lib (the library file I need to link with)
| Source
|-| CMakeLists.txt
|-| main.cpp

The root CMakeLists.txt file:
cmake_minimum_required(VERSION 2.6)
project(MotorEngine)

# Set an output directory for our binaries
set(BIN_DIR ${CMAKE_CURRENT_SOURCE_DIR}/Binaries)

set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/Binaries)
set(THIRDPARTY_PATH ${CMAKE_CURRENT_SOURCE_DIR}/ThirdParty)

# Include SDL2
include_directories(${THIRDPARTY_PATH}/SDL2/include)

# Add the engine + third party subdirectory
add_subdirectory(Source)

The Source's CMakeLists.txt:
project(MotorEngine)
add_executable(MotorEngine main.cpp)
target_link_libraries(MotorEngine ${THIRDPARTY_PATH}/SDL2/lib/x64/SDL2.lib)

Now, I want to achieve the following, in the main.cpp I want to write
#include "SDL2/include/SDL2.h"

But currently I have to write
#include "SDL2.h"

Since there will be files with the same name later on, I need to distinguish them in their folders. So the easiest would be to add the "ThirdParty" folder as a root so I can use #include relative to that then, but doing
include_directories(${THIRDPARTY_PATH})

does not work. Any ideas? Thank you!
Comment: You may try to change `include_directories` to `target_include_directories` and specify `MotorEngine` as target explicitly.
Comment: @k.v. I'm a total noob when it comes to cmake terms, how would I do that?
Comment: `target_include_directories(MotorEngine, ${THIRDPARTY_PATH})` @iRedCraft
Comment: I looked the command up since I had some errors, modified it and now it works! I'll write an answer to my own question here.
Comment: You should create an imported library target and set the [`INTERFACE_INCLUDE_DIRECTORIES`](https://cmake.org/cmake/help/latest/prop_tgt/INTERFACE_INCLUDE_DIRECTORIES.html) to it.
Comment: @MizuxDev I'd like to do that, but it still doesn't include them, if I do the following: https://pastebin.com/mva5zYy8 (CMakeLists.txt inside the SDL folder)
Comment: @iRedCraft It should work ! Did you `add_subdirectory(ThirdParty/SDL2)` in your root CMakeLists.txt before `add_subdirectory(Source)`, then in `Source/CMakeLists.txt` a `target_link_libraries(MotorEngine PUBLIC SDL2)` should work to have the include path (you can use `set(CMAKE_VERBOSE_MAKEFILE TRUE)` to check). After if you want `"SDL2/include/SDL2.h"` you should create your imported target in ThirdParty/CMakeLists.txt and not in ThirdParty/SDL2 ...

After usually include are `foo.h` or `foo/foo.h` not `foo/include/foo.h` IMHO
Comment: @MizuxDev I have now the following setup: https://pastebin.com/gCJ2BLvC but still I can't include "SDL.h" nor "SDL2/SDL.h" nor "SDL2/include/SDL.h" :/
Comment: in your pastebin your root CMakeLists.txt should contains `add_subdirectory(${THIRDPARTY_PATH})` and not `add_subdirectory(${THIRDPARTY_PATH}/SDL2)` (need to remove the SDL2)
Comment: @MizuxDev Tried it, but it isn't added to the project's include paths, therefore I still can't include it
Answer: With help of k.v. I was able to sort this out. I needed to add the following in the CMakeLists.txt in the Source directory:
target_include_directories(MotorEngine PUBLIC ${THIRDPARTY_PATH})

cmake: Include directories root?


How do I quantify the decay in the initial condition of an AR process?

Question: I'm working on basic code that generates data from AR and VAR processes; the code generates enough observations to dampen the effect of the initial conditions. For example, if I want to generate 30 observations, the code generates 300 observations and only returns the final 30.
In an answer to my question about this code , a user recommended that instead of generating $N$ observations and only returning the last $T$, I pass a parameter that specifies

the desired decay of the initial condition.

I'm not sure how to interpret this. Take a simple univariate AR(1) process as an example:
\begin{align}
y_t = \rho y_{t-1} + u_t
\end{align}
where $\lvert \rho \rvert < 1$, $u \sim N(0, \sigma_u^2)$, and $y_0 = 0$. What does it mean to talk about the desired decay of this initial condition? I couldn't find any references that discussed this. For example, is this referring to calculating some number of observations after which the initial condition has decayed by, say, 90%? What exactly does that mean?
The actual processes I'm working with are slightly more involved, e.g.
\begin{align}
\begin{bmatrix}
y_{1,t} \\
y_{2,t}
\end{bmatrix}
&=
\begin{bmatrix}
0.02 \\
0.03
\end{bmatrix}
+
\begin{bmatrix}
0.5 & 0.1 \\
0.4 & 0.5
\end{bmatrix}
\begin{bmatrix}
y_{1,t-1} \\
y_{2,t-1}
\end{bmatrix}
+
\begin{bmatrix}
0 & 0 \\
0.25 & 0
\end{bmatrix}
\begin{bmatrix}
y_{1,t-2} \\
y_{2,t-2}
\end{bmatrix}
+
\begin{bmatrix}
u_{1,t} \\
u_{2,t}
\end{bmatrix}
\end{align}
but I'm just trying to get a general idea for now.
Comment: I think the user is just commenting on the choice of $N=10T$ in your code.  There, "10" is a magic constant.  I think he's saying you should instead calculate $N$ as a function of the desired "decay" of the initial condition.  For example, in your AR(1) process, $y_t = \rho^{t-1}y_0 + f(\mathbf{u}_{1,\ldots,t})$.  Waiting until $t=1+\log(0.01)/\log(\rho)$ would give you $y_t=0.01y_0+f(\mathbf{u}_{1,\ldots,t})$, washing out the effect of $y_0$ to one hundreth of its value, for example.
Comment: I'm starting to wonder if there are *any* references about this concept, if it's even a valid concept. After weeks of searching and no responses either on here or on Code Review, I'm increasingly skeptical.
Comment: For more complicated processes, rather than solve analytically, you might want to run a second process with no noise term in the recurrence relation.  Keep stepping both processes until the variables in the no-noise process appear to have converged to steady-state values.
Answer: I just remembered I'm supposed to give answers as an Answer, not a Comment, so I'll use this opportunity to offer some code in R that simulates your example process with a dynamically-determined burn-in period.  Red points are burn-in; blue are steady-state.
In case you can't run R, here's an example of the output.

And if you can run R, here's the code.
<code># parameters of the process
# rho is 0.99; each u_t is N(0,0.02**2)
rho = 0.99
u = function() rnorm(1,mean=0,sd=0.02)

# Y is the actual process, Z the no-noise cousin
# start each process at 1,1
Y1=Y2=Z1=Z2=c(1,1)
t=2

# say a sequence is "converged" if the last three
# terms are within a factor of 1+-1e-6 of each other
converged = function(Z) {
  n = length(Z)
  if (n<3) F else { L=log(Z[(n-2):n]); mu=mean(L); max(abs(L-mu))<1e-6; }
}

# step until converged
while (!converged(Z1) | !converged(Z2)) {
  t = t + 1
  Y1[t] = 0.02 + 0.5*Y1[t-1] + 0.1*Y2[t-1] + u()
  Z1[t] = 0.02 + 0.5*Z1[t-1] + 0.1*Z2[t-1] # no noise
  Y2[t] = 0.03 + 0.4*Y1[t-1] + 0.5*Y2[t-1] + 0.25*Y1[t-2] + u()
  Z2[t] = 0.03 + 0.4*Z1[t-1] + 0.5*Z2[t-1] + 0.25*Z1[t-2] # no noise
}

# step the same amount again for "steady state" data
t_burn = t
while (t <= 2*t_burn) {
  t = t + 1
  Y1[t] = 0.02 + 0.5*Y1[t-1] + 0.1*Y2[t-1] + u()
  Y2[t] = 0.03 + 0.4*Y1[t-1] + 0.5*Y2[t-1] + 0.25*Y1[t-2] + u()
}

# plot the results; red=burn-in period, blue="steady state" data
plot(c(1,t),range(c(Y1,Y2)),type="n",xlab="t",ylab="Y")
add_points = function(Y) {
  points(1:t,Y[1:t],col="grey",type="l")
  points(1:t_burn,Y[1:t_burn],col="red",pch=20,type="o")
  points((t_burn+1):t,Y[(t_burn+1):t],col="blue",pch=20,type="o")
}
add_points(Y1)
add_points(Y2)
</code>
Comment: This is quite helpful; thank you. Your explanation of the decay makes perfect sense, and I'll look over the code over the next couple of days.


How do you specify OrderBy clause on two columns

Question: We would like to orderBy 2 columns in the Seam EntityQuery interface as well as the JPA model. How do we do this?
@Entity
public class A{

@OrderBy(???) // should this be hardcoded here, is it database agnostic
List bobjects;
}

@Entity
public class B {

   public short startTimeHrs;
   public short startTimeMins;
}

@Name("bList")
public class B extends EntityQuery {
        setOrderColumn("startTimeHrs, startTimeMins"); // Is this correct?
        setOrderDirection("asc");
}

Answer: Yes you can use @OrderBy to order your queries.
@OrderBy("startTimeHrs, startTimeMins")
@OneToMany(...)
getBobjects() {
   return bobjects;
}

Now whenever you are saying A.getBobjects() they will be ordered. However, if you are using the EntityQuery way of retrieving the result you can override the getEjbql() and put the order by there.
@Name("bList")
public class B extends EntityQuery {
    @Override
    public String getEjbql() {
          return "select b from B b order by startTimeHrs, startTimeMins";
    }
}

Or you can @Override the getResultList() to manipulate the collection there in your entityLists
Comment: I wouldn't want to override the getResultList method here. Would you specify the orderBy direction as part of Ejbql or via the setOrderDirection(..) method?
Comment: That's up to you. You can do everything in the ejbql
Answer: If you are talking about javax.persistence.OrderBy, this annotation takes into parameter a list of comma separated properties (of the target entity) and order the collection accordingly. For example in your case, startTimeHrs asc, startTimeMins asc.
From the JPA 1.0 specification:

9.1.28 OrderBy Annotation
The OrderBy annotation specifies the
  ordering of the elements of a
  collection valued association at the
  point when the association is
  retrieved.
@Target({METHOD, FIELD}) @Retention(RUNTIME)
public @interface OrderBy {
String value() default "";
}

The syntax of the value ordering
  element is an orderby_list, as
  follows:
orderby_list::= orderby_item [,orderby_item]*
orderby_item::= property_or_field_name [ASC | DESC]

If ASC or DESC is not specified,
  ASC (ascending order) is assumed.
If the ordering element is not
  specified, ordering by the primary key
  of the associated entity is assumed.
The property or field name must
  correspond to that of a persistent
  property or field of the associated
  class. The properties or fields used
  in the ordering must correspond to
  columns for which comparison operators
  are supported.


Probability of geometric random variables in certain order

Question: I have been having some trouble with the following problem, and I have finally reached a solution that I think is correct but I would like to double check. The problem is the following:

Consider a sequence of multinomial trials with three outcomes: type 1, type 2, and type 3, each occurring with probability $p_1,p_2,p_3$, respectively (so $p_1+p_2+p_3=1$). Let $N_i$ be the first trial which had outcome $i,~i=1,2,3$. Compute $P(N_1<N_2<N_3)$.

There are other parts to the problem, and one of them is to describe the distribution of each $N_i$, which I said is geometric with success probability $p_i$. In order for $N_1<N_2<N_3$ we must have $N_1=1$, because there must be an outcome for the first trial. In another part of the problem I showed that, for $1<j<k$, we have $$(1)\qquad P(N_1=1,N_2=j,N_3=k)=p_1^{j-1}p_2(1-p_3)^{k-j-1}p_3$$ since the first outcome can occur $j-1$ times before outcome two, then any outcome other than three occurs $k-j-1$ times before outcome three. So I would think that the desired probability is just the sum over $j\geq 2$ and $k\geq j+1$ since $P(N_1<N_2<N_3)=P(N_1=1,~1<N_2<N_3)$. So using the geometric series formula, we have:
$$(2)\qquad \begin{align}P(N_1<N_2<N_3)&=\sum_{j=2}^\infty\sum_{k=j+1}^\infty p_1^{j-1}p_2(1-p_3)^{k-j-1}p_3\\&=\frac{p_2p_3}{p_1(1-p_3)}\sum_{j=2}^\infty\left(\frac{p_1}{1-p_3}\right)^j\sum_{k=j+1}^\infty (1-p_3)^k\\&=\frac{p_2p_3}{p_1(1-p_3)}\sum_{j=2}^\infty\left(\frac{p_1}{1-p_3}\right)^j\left(\frac{(1-p_3)^{j+1}}{p_3}\right)\\&=\frac{p_2}{p_1}\sum_{j=2}^\infty p_1^j\\&=\frac{p_1p_2}{1-p_1}\end{align}$$
My questions: Is my logic correct, and mainly: are equations $(1),(2)$ correct?
Answer: Evidently this approach is correct, as I received full marks on the assignment. Just posting this answer to close off the question.
Comment: Receiving full marks is not always indicative of correctness.  It depends on the patience, focus, and skill of the person determining the marks. Your equation (1) looks correct to me. The final answer is so simple that it makes one wonder if there isn't another method to reach it.  Indeed, $p_1$ is the chance that the first answer is type 1, and $\frac{p_2}{1-p_1}$ is the chance that the next answer-that-isn't-type-1 is type 2, so there you have it.  I would guess the person giving out the marks simply checked the final answer, and not your logic or equations (1) or (2).


Quais as diferenças entre as formas de manipular strings enormes com Python?

Question: Quando é necessário trabalhar com strings muito grandes é normal vermos abordagens que buscam otimizar o processo de alguma forma. Das abordagens que já vi são utilizando:

io.StringIO ou io.BytesIO
tempfile.TemporaryFile
tempfile.NamedTemporaryFile
tempfile.SpooledTemporaryFile

As quatro formas são objetos file-like e podem ser utilizados junto a um gerenciador de contexto.
with io.StringIO() as stream:
  ...

with tempfile.TemporaryFile() as stream:
  ...

with tempfile.NamedTemporaryFile as stream:
  ...

with tempfile.SpooledTemporaryFile as stream:
  ...

Qual é a diferença entre utilizar cada uma dessas soluções? Existem outras formas?
Answer: Observando a documentação referente ao módulo tempfile é possível sanar algumas de suas dúvidas.
Essa biblioteca é usada para a criação de arquivos e diretórios temporários, e para usos simples a diferenciação entre seus métodos a nível detalhado não é tão necessária. Contudo elas existem a nível de sistema operacional, como explico abaixo:

tempfile.TemporaryFile(mode, buffering, encoding, newline, suffix, prefixe, dir):

Esse método retorna um objeto file-like que pode ser usado como área de armazenamento. Agora olhando pelo lado do sistema operacional baseado em Unix, a entrada de diretório para o arquivo não é criada ou se é, é removida imediatamente após a criação do arquivo.

tempfile.NamedTemporaryFile(mode, buffering, encoding, newline, suffix, prefix, dir, delete):

Esse aqui, funciona da mesma forma do TemporaryFile(), com uma exceção, o arquivo terá um nome visível no sistema de arquivos(no Unix a entrada do diretório não é desvinculada). E esse nome pode ser descoberto com o atributo name() do objeto.

tempfile.SpooledTemporaryFile(max_size, mode, buffering, encoding, newline, suffix, prefix, dir):

Repare que nesse caso até um parâmetro é incrementado, ele tem o funcionamento igual ao TemporaryFile() com a exceção de que os dados são armazenados na memória até seu tamanho exceder o max_size, ou até o método fileno() for chamado, e então seu funcionamento segue igual ao do TemporaryFile().

io.BytesIO():

Retorna um objeto parecido com um arquivo na memória que não altera as novas linhas e é semelhante ao open (filename, "wb"). Lida com strings byte ().

io.stringIO():

Retorna um objeto semelhante a um arquivo na memória que altera novas linhas e é semelhante ao open(filename, "w"). Ele lida com strings unicode ().

Espero que tenha esclarecido alguma de suas dúvidas. Muito do que foi dito, foi encontrado na documentação que eu linkei acima. Abraços!
Comment: Sim, foi bem útil. Só para esclarecer, a ideia da pergunta era justamente dispor desse conteúdo, mesmo que resumido, em português. Eu já sabia as diferenças, mas agora temos uma referência inicial para utilizarmos de referência :D
Comment: Ótimo jeito de impulsionar a comunidade !


How do I change which pickup a specific Guitar Pro 7.5 track uses?

Question: I have a guitar part which uses the neck pickup. How do I switch the track to neck pickup so that it sounds more like the original track?
Answer: The following numbers correspond to different segments in the below screenshot:

Open the track editor (in the right-hand sidebar, shortcut: F6)
Expand the sound details ("Show/hide sound details" tooltip)
Click "Bank settings" (the headstock image)
Change the pickup using the "Position" dropdown menu


StdOut class not found

Question: I'm using netbeans 8.0.1 to write, compile and run my code buy I can't seem to find the "standard Libraries" like StdOut and StdStats that are referred examples that I find.
Example:
https://github.com/merwan/algs4/blob/master/percolation/src/PercolationStats.java
http://introcs.cs.princeton.edu/java/stdlib/javadoc/StdStats.html

I can use System.out.println and the like to print but the StdOut class isn't found.
Anyone else having the same issue and if so, how does one rectify it?
Comment: These are not standard classes. Like princeton.edu created their StdStats and provide source for it. If you want to use these classes - compile them and add to your classpath.
Comment: As explained in the second link, those classes were written by Robert Sedgewick and Kevin Wayne for a Java textbook.
Answer: The term "stdout" is just a shortening of the phrase "standard output". It's a language-agnostic term, but in Java it usually just refers to System.out. In other words, if you're looking for a StdOut class in the standard library, you won't find one, because it's not the name of a standard Java class.
Comment: I don't think that's right. I just added some example URLs to the question
Comment: @paullb See edit. It's quite possible for it to be the name of the class (I could easily create a `StdOut` class in about five seconds), but it's not the name of a class *in the standard Java library*.
Answer: Here is you standard library as a part of the princeton course .It also gives details of adding it
http://introcs.cs.princeton.edu/java/stdlib/
Comment: @paullb Only if you have good reason to use it rather than the available functionality in the Java standard library.
Comment: So I should be adding that jar to the project?
Comment: If you using some class in that library or some other class which uses that library then you should include that in your classpath.
Comment: Upon rereading the assignment, it looks like I should include it. Thanks for the quick response!

StdOut class not found


Sequelize Associantions

Question: Possuo 2 models:
user features:
const DataTypes = require('sequelize');
const database = require('../config/dbconnection');
const user= require('./user');

const user_features = database.define('user_features', {
    id: {
      type: DataTypes.INTEGER(6),
      allowNull: false,
      primaryKey: true,
      autoIncrement: true
    },
    user_id: {
      type: DataTypes.INTEGER(6),
      references: 'USERS',
      referencesKey: 'user_id',
      allowNull: true
    },
    {
      text:{
          type: "LONGBLOB",
          allowNull: true
      }
    },
    {
       timestamps: false
    });

    user_features.belongsTo(user);
    module.exports = user_features;

users:
const DataTypes = require('sequelize');
const database = require('../config/dbconnection');

const users = database.define('users', {
    id: {
      type: DataTypes.INTEGER(6),
      allowNull: false,
      primaryKey: true,
      autoIncrement: true
    },
    name {
      type: DataTypes.STRING(45)
    }
 },
 {
   timestamps: false
 });

 module.exports = users;

e no console mostra o seguinte erro:
Error: user_features.belongsTo called with something that's not a subclass of Sequelize.Model

Preciso fazer a associação entre essas tabelas para na regra de negocio eu poder fazer um select no banco de dados e trazer essas informações.
Atualmente eu estou fazendo o select assim:
model.sequelize.query('SELECT NAME, TEXT FROM USERS LEFT JOIN USER_FEATURES ON USERS.ID = USER_FEATURES.USER_ID')

mas quero padronizar o prejo e queria utilizar o ORM:
 getAll = (req, res, next) => model.findByPk(req.params.id,
            {
                attributes: ['NAME', 'TEXT']
            })
            .then((result) => {
                res.send(result);
            })
            .catch(error => console.log('repository getById error: ' + error));

Como poderia fazer essa associação?
Answer: Amigo, se fosse uma classe do tipo Model sequelize vc teria que extender ela pra Model, creio que por isso o Sequelize não está compreendendo que user_features é um model do sequelize. Segue um exemplo da documentação:

sequelize.define('modelName', {
  columnA: {
      type: Sequelize.BOOLEAN,
      validate: {
        is: ["[a-z]",'i'],        // will only allow letters
        max: 23,                  // only allow values <= 23
        isIn: {
          args: [['en', 'zh']],
          msg: "Must be English or Chinese"
        }
      },
      field: 'column_a'
  },
  columnB: Sequelize.STRING,
  columnC: 'MY VERY OWN COLUMN TYPE'
});

sequelize.models.modelName

Repare que no final existe o seguinte código: sequelize.models.modelName

Sequelize Associantions


Como manter o layout dos componentes criados pelo WindowBuilder no eclipse após a compilação?

Question: Criei um JFrame utilizando o plugin WindowBuilder do Eclipse. Foi definido espaçamentos, tamanhos e padrões. Entretanto ao compilar o projeto os padrões são perdidos. Abaixo as 2 fotos para comparação.
Aqui é o Preview do JFrame.

JFrame após compilação.

Gostaria de manter o layout e padrões da primeira imagem. Alguém poderia ajudar?
Answer: Adicione o código a seguir dentro do seu main():
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
        } catch (Throwable e) {
            e.printStackTrace();
        }

Por exemplo deixando assim:
public class Tela extends JFrame {
    private static final long serialVersionUID = 1L;
    private JPanel contentPane;
    /**
     * Launch the application.
     */
    public static void main(String[] args) {
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
        } catch (Throwable e) {
            e.printStackTrace();
        }
        EventQueue.invokeLater(new Runnable() {
            public void run() {
                try {
                    Tela frame = new Tela();
                    frame.setVisible(true);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
    }
    /**
     * Create the frame.
     */
    public Tela() {
        ...
    }
}

Adicionando o código em questão, você está aplicando o Look and Feel do seu sistema, caso contrário a aplicação escolherá o padrão que é o Metal.

CrossPlatformLookAndFeel—this is the "Java L&F" (also called "Metal") that looks the same on all platforms. It is part of the Java API (javax.swing.plaf.metal) and is the default that will be used if you do nothing in your code to set a different L&F.

Fonte: The Java Tutorials - How to Set the Look and Feel
Comment: Resolvi. Era só prestar um pouco mais de atenção nas grid! Obrigado!
Comment: Obrigado @Math. Resolveu em partes meu problema. O *look and feel* dos componentes foram mudados. Entretanto o problema dos espaçamentos ainda continua. :/
Comment: Espaçamento entre os componentes? Poderia [edit] sua pergunta e colocar um print pra entender como que fica? Pois não consigo ver essa diferença rodando um projeto meu.
Comment: Dê uma olhada na figura 1, os botões abaixo "limpar", "salvar" e "salvar e sair" estão alinhados, já na figura 2 após a compilação esses espaçamentos são perdidos.
Comment: Ah sim, agora eu percebi isso. Qual Layout Manager está utilizando no rodapé? Por exemplo, BorderLayout, GridBagLayout, MigLayout, etc..


Why use access modifiers

Question: I know the difference between the different access modifiers and when to use them, but I'm left with the why. I understand that it simplifies a project, but is there any other use e.g. security?
Comment: Access modifiers are a high level concept. It doesn't make your code any more secure. It's all about making it harder to make mistakes, and more straight-forward to code things correctly.
Answer: If everything was public it would make maintainability much harder because you would never know for sure whether a class/member could be modified/removed because anything external could be making use of it. If something is declared private/protected/internal, when you return to the code later to make changes you have a clear picture of what can access it and how you can modify without breaking other components.
It also reduces complexity to outside eyes (encapsulation), by hiding internal members.
Answer: As a name suggest they allows you to provide access. you can define a class member protected or private which makes them inaccessible for outside the class objects. thus they are more about access rights than security. You can read more here
Answer: Everything became much clearer in big team. Say, you gave someone your .dll and they don't have your source code. Would you like to write a lot of XML-comments with "USE IT SPARELY".
And sometimes - your .dll is your final product. You need to avoid mistakes
So, use cases which I see fitable:
To avoid usage of internal methods, which shouldn't be called independently. If you have sequence of private methods, - that are using shared properties/fields - you don't want someone to use them and raise unexpected exceptions, right?
To avoid mess with properties/fields, which used for internal purposes
and more...

Why use access modifiers