How to implement Android callbacks in C# using async/await with Xamarin or Dot42?

Question: How do you  implement callbacks in C# using async/await with Xamarin for Android? And how does this compare to standard Java programming for Android?
Answer: With Xamarin for Android version 4.7, at the time of this writing still in publicly  available beta, we may use .NET 4.5 features to implement 'async' methods and 'await' calls to them. It always bothered me, that if any callback is needed in Java, the logical flow of code in a function is interrupted, you have to continue the code in the next function when the callback returns. Consider this scenario:
I want to collect a list of all available TextToSpeech engines on an Android device, and then ask each of them which languages it has installed. The little "TTS Setup" activity that I wrote, presents to the user two selection boxes ("spinners"), one listing all the languages that all the TTS engines on this device support. The other box below lists all the voices available for the language selected in the first box, again from all available TTS engines.

Ideally all the initialization of this activity should happen in one function, e.g. in onCreate(). Not possible with standard Java programming because:
This requires two "disruptive" callbacks – first to initialize TTS engine – it becomes fully operational only when the onInit() is called back. Then, when we have an initialized TTS object, we need to send it an "android.speech.tts.engine.CHECK_TTS_DATA" intent, and await it result again in our activity callback onActivityResult(). Another disruption of logic flow. If we are iterating through a list of available TTS engines, even the loop counter for this iteration cannot be a local variable in a single function, but must be made a private class member instead. Pretty messy in my opinion.
Below I'll try to outline the necessary Java code to achieve this.
Messy Java code to collect all TTS engines and the voices their support
public class VoiceSelector extends Activity {
private TextToSpeech myTts;
private int myEngineIndex; // loop counter when initializing TTS engines

// Called from onCreate to colled all languages and voices from all TTS engines, initialize the spinners
private void getEnginesAndLangs() {
    myTts = new TextToSpeech(AndyUtil.getAppContext(), null);
    List<EngineInfo> engines;
    engines = myTts.getEngines(); // at least we can get the list of engines without initializing myTts object…
    try { myTts.shutdown(); } catch (Exception e) {};
    myTts = null;
    myEngineIndex = 0; // Initialize the loop iterating through all TTS engines
    if (engines.size() > 0) {
        for (EngineInfo ei : engines)
            allEngines.add(new EngLang(ei));
        myTts = new TextToSpeech(AndyUtil.getAppContext(), ttsInit, allEngines.get(myEngineIndex).name());
        // DISRUPTION 1: we can't continue here, must wait until  ttsInit callback returns, see below
    }
}

private TextToSpeech.OnInitListener ttsInit = new TextToSpeech.OnInitListener() {
@Override
public void onInit(int status) {
    if (myEngineIndex < allEngines.size()) {
        if (status == TextToSpeech.SUCCESS) {
            // Ask a TTS engine which voices it currently has installed
            EngLang el = allEngines.get(myEngineIndex);
            Intent in = new Intent(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
            in = in.setPackage(el.ei.name); // set engine package name
            try {
                startActivityForResult(in, LANG_REQUEST); // goes to onActivityResult()
                // DISRUPTION 2: we can't continue here, must wait for onActivityResult()…

            } catch (Exception e) {   // ActivityNotFoundException, also got SecurityException from com.turboled
                if (myTts != null) try {
                    myTts.shutdown();
                } catch (Exception ee) {}
                if (++myEngineIndex < allEngines.size()) {
                    // If our loop was not finished and exception happened with one engine,
                    // we need this call here to continue looping…
                    myTts = new TextToSpeech(AndyUtil.getAppContext(), ttsInit, allEngines.get(myEngineIndex).name());
                } else {
                    completeSetup();
                }
            }
        }
    } else
        completeSetup();
}

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == LANG_REQUEST) {
        // We return here after sending ACTION_CHECK_TTS_DATA intent to a TTS engine
        // Get a list of voices supported by the given TTS engine
        if (data != null) {
            ArrayList<String> voices = data.getStringArrayListExtra(TextToSpeech.Engine.EXTRA_AVAILABLE_VOICES);
            // … do something with this list to save it for later use
        }
        if (myTts != null) try {
            myTts.shutdown();
        } catch (Exception e) {}
        if (++myEngineIndex < allEngines.size()) {
            // and now, continue looping through engines list…
            myTts = new TextToSpeech(AndyUtil.getAppContext(), ttsInit, allEngines.get(myEngineIndex).name());
        } else {
            completeSetup();
        }
    }
}

Note that the line that creates a new TTS object with ttsInit callback, has to be repeated 3 times in order to continue looping through all the available engines if any exceptions or other errors happen. Maybe the above could be written a little bit better, e.g. I thought that I could create an internal class to keep the looping code localized and my loop counter to at least not be a member of the main class, but it's still messy. Suggestion for improvements of this Java code welcome.
Much cleaner solution: Xamarin C# with async methods
First, to simplify things I created a base class for my Activity that provides CreateTtsAsync() to avoid DISRUPTION 1 in the Java code above, and StartActivityForResultAsync() to avoid DISRUPTION 2 methods.
// Base class for an activity to create an initialized TextToSpeech
// object asynchronously, and starting intents for result asynchronously,
// awaiting their result. Could be used for other purposes too, remove TTS
// stuff if you only need StartActivityForResultAsync(), or add other
// async operations in a similar manner.
public class TtsAsyncActivity : Activity, TextToSpeech.IOnInitListener
{
    protected const String TAG = "TtsSetup";
    private int _requestWanted = 0;
    private TaskCompletionSource<Java.Lang.Object> _tcs;

    // Creates TTS object and waits until it's initialized. Returns initialized object,
    // or null if error.
    protected async Task<TextToSpeech> CreateTtsAsync(Context context, String engName)
    {
        _tcs = new TaskCompletionSource<Java.Lang.Object>();
        var tts = new TextToSpeech(context, this, engName);
        if ((int)await _tcs.Task != (int)OperationResult.Success)
        {
            Log.Debug(TAG, "Engine: " + engName + " failed to initialize.");
            tts = null;
        }
        _tcs = null;
        return tts;
    }

    // Starts activity for results and waits for this result. Calling function may
    // inspect _lastData private member to get this result, or null if any error.
    // For sure, it could be written better to avoid class-wide _lastData member...
    protected async Task<Intent> StartActivityForResultAsync(Intent intent, int requestCode)
    {
        Intent data = null;
        try
        {
            _tcs = new TaskCompletionSource<Java.Lang.Object>();
            _requestWanted = requestCode;
            StartActivityForResult(intent, requestCode);
            // possible exceptions: ActivityNotFoundException, also got SecurityException from com.turboled
            data = (Intent) await _tcs.Task;
        }
        catch (Exception e)
        {
            Log.Debug(TAG, "StartActivityForResult() exception: " + e);
        }
        _tcs = null;
        return data;
    }

    protected override void OnActivityResult(int requestCode, Result resultCode, Intent data)
    {
        base.OnActivityResult(requestCode, resultCode, data);
        if (requestCode == _requestWanted)
        {
            _tcs.SetResult(data);
        }
    }

    void TextToSpeech.IOnInitListener.OnInit(OperationResult status)
    {
        Log.Debug(TAG, "OnInit() status = " + status);
        _tcs.SetResult(new Java.Lang.Integer((int)status));
    }

}

Now I can write the entire code looping through the TTS engines and querying them for available languages and voices within one function, avoiding a loop run throughout three different functions:
// Method of public class TestVoiceAsync : TtsAsyncActivity
private async void GetEnginesAndLangsAsync()
{
    _tts = new TextToSpeech(this, null);
    IList<TextToSpeech.EngineInfo> engines = _tts.Engines;
    try
    {
        _tts.Shutdown();
    }
    catch { /* don't care */ }

    foreach (TextToSpeech.EngineInfo ei in engines)
    {
        Log.Debug(TAG, "Trying to create TTS Engine: " + ei.Name);
        _tts = await CreateTtsAsync(this, ei.Name);
        // DISRUPTION 1 from Java code eliminated, we simply await TTS engine initialization here.
        if (_tts != null)
        {
            var el = new EngLang(ei);
            _allEngines.Add(el);
            Log.Debug(TAG, "Engine: " + ei.Name + " initialized correctly.");
            var intent = new Intent(TextToSpeech.Engine.ActionCheckTtsData);
            intent = intent.SetPackage(el.Ei.Name);
            Intent data = await StartActivityForResultAsync(intent, LANG_REQUEST);
            // DISTRUPTION 2 from Java code eliminated, we simply await until the result returns.
            try
            {
                // don't care if lastData or voices comes out null, just catch exception and continue
                IList<String> voices = data.GetStringArrayListExtra(TextToSpeech.Engine.ExtraAvailableVoices);
                Log.Debug(TAG, "Listing voices for " + el.Name() + " (" + el.Label() + "):");
                foreach (String s in voices)
                {
                    el.AddVoice(s);
                    Log.Debug(TAG, "- " + s);
                }
            }
            catch (Exception e)
            {
                Log.Debug(TAG, "Engine " + el.Name() + " listing voices exception: " + e);
            }
            try
            {
                _tts.Shutdown();
            }
            catch { /* don't care */ }
            _tts = null;
        }
    }
    // At this point we have all the data needed to initialize our language
    // and voice selector spinners, can complete the activity setup.
    ...
}

The Java project, and the C# project, using Visual Studio 2012 with Xamarin for Android add-on, are now posted on GitHub:
https://github.com/gregko/TtsSetup_C_sharp
https://github.com/gregko/TtsSetup_Java
What do you think?
Learning how to do this with Xamarin for Android free trial was fun, but is it worth the $$ for Xamarin license, and then the extra weight of each APK you create for Google Play Store of about 5 MB in Mono runtimes we have to distribute? I wish Google provided Mono virtual machine as standard system component on equal rights with Java/Dalvik.
P.S. Reviewed the voting on this article, and I see that it gets also some down-votes. Guess they must be coming from Java enthusiasts! :) Again, suggestions on how to improve my Java code are also welcome.
P.S. 2 - Had an interesting exchange on this code with another developer on Google+, helped me to understand better what actually happens with async/await.
Update 8/29/2013
Dot42 also implemented 'async/await' keywords in their C# product for Android, and I tried porting to it this test project. My first attempt failed with a crash somewhere in Dot42 libraries, awaiting (asynchronously, of course :) ) for a fix from them, but there is an interesting fact they observed and implemented when it comes to 'async' calls from Android activity event handlers:
By default, if there is some activity "configuration change" while you're awaiting a result of a long async operation inside an activity event handler, such as e.g. orientation change, the activity is destroyed and re-created by the system. If after such change you return from an 'async' operation to the middle of an event handler code, the 'this' object of the activity is no longer valid, and if you stored some object pointing to controls within this activity, they are also invalid (they point to the old, now destroyed objects).
I hit this problem in my production code (in Java) and worked-around by configuring the activity to be notified, and not destroyed and recreated on such events. Dot42 came with another alternative, quite interesting:
var data = await webClient
             .DownloadDataTaskAsync(myImageUrl)
             .ConfigureAwait(this);

The .configureAwait(this) extension (plus one more code line in activity OnCreate() to setup things) ensures that your 'this' object is still valid, points to the current instance of activity, when you return from await, even if configuration change occurs. I think it's good to at least be aware of this difficulty, when you start using async/await with Android UI code, see more writeup on this at Dot42 blog: http://blog.dot42.com/2013/08/how-we-implemented-asyncawait.html?showComment=1377758029972#c6022797613553604525
Update on Dot42 crash
The async/await crash I experienced is now fixed in Dot42, and it works great. Actually, better than Xamarin code due to the smart handling of 'this' object in Dot42 between the activity destruction/recreation cycles. All of my C# code above should be updated to take into account such cycles, and currently it's not possible in Xamarin, only in Dot42. I'll update that code on demand from other SO members, for now it seems that this article does not get much attention.
Comment: @GregK. You use the TCS class to create a task that represents when the event is ready, but without blocking a thread.  Then you can `await` that task.  Then rather than setting an auto reset event you just set the result of the TCS.  That's what svick is suggesting.  That's the appropriate way of signaling using the `Task` model.  Using an auto reset event is designed for non-asynchronous blocking.
Comment: `await Task.Run(delegate { Event1.WaitOne(); });` I think this is really ugly. A much better solution would be to use `TaskCompletionSource`. And all those empty `catch`es also aren't a good practice.
Comment: Great, thank you, @svick! Will experiment with TaskCompetionSource and improve my code, while my free Xamarin trial lasts. I'm still learning C#, Java and Android, so all comments help me. Empty catches - in some places I really don't care if a buggy TTS engine, a 3rd party code after all, crashes, and still want to continue my looping. But you're right, it will be good to at least output the exception to the Debug log to have an idea what's going on.
Comment: @svick - would appreciate a quick code sample showing how I could use TaskCompletionSource in the context of my app, awaiting for callbacks. Note that we cannot block the current thread while waiting for callbacks return. Thanks!
Comment: @GregK. Just look at the documentation for TCS.  It's honestly a very simple class; just looking at it's API is generally enough to figure out how to use it.
Comment: @Servy - I looked and looked, and can't figure it out. All the examples of TCS are about starting new tasks, new thread etc. This is different, after creating a TTS object we must wait on the same (UI) thread until the OnInit() callback is called. I think that after my corrections today, isolating async calls in the base class, the code is clean and simple as is.
Comment: @Servy and svick- thank you! I think I figured out TCS and implemented it correctly now. The code in the article and my GitHub project already reflects this change. Let me know if anything else could be done better.
Answer: I use the following model to convert call backs to async:
SemaphoreSlim ss = new SemaphoreSlim(0);
int result = -1;

public async Task Method() {
    MethodWhichResultsInCallBack()
    await ss.WaitAsync(10000);    // Timeout prevents deadlock on failed cb
    lock(ss) {
         // do something with result
    }
}

public void CallBack(int _result) {
    lock(ss) {
        result = _result;
        ss.Release();
    }
}

This is very flexible and can be used in Activities, inside callback objects ect.
Be careful, using this the wrong way will create deadlocks ect. The lock prevents result changing after if the timeout runs out.


Conjugacy Classes of subgroups in GL(n,p)

Question: What is the conjugacy class of subgroups of order $p$ in $GL(n,p)$? (Are all subgroups of order $p$ conjugate in $GL(n,p)$?)
Comment: Take a look at the possible subspaces of $\mathbb{F}_p^n$ fixed by an element of order $p$ in $\mathrm{GL}(n, p)$. Do they all have the same dimension?
Comment: You mean "What are the conjugacy classes of subgroups of order $p$ in $GL(n,p)$?
Comment: @Alex: Should (s)he compute the order of $\mathrm{GL}(n, p)$ just to get to know the group, or does the order of the group help in any way for the question?
Comment: @Alex: I'm very curious about this. How does the order help considering that there is also a cyclic group of the same order?
Comment: @jug I'm sorry, I was oversimplifying things. Please ignore my comments.
Answer: Every subgroup of order p in GL(n,p) is conjugate to a subgroup generated by a matrix in Frobenius normal form.  Such a matrix is a direct sum of companion matrices of the polynomials (x−1)k where 1 ≤ k ≤ p, and is determined up to conjugacy by the partition of n defined by 1 ≤ k1 ≤ k2 ≤ … ≤ km ≤ p where n = k1 + k2 + … + km and all such partitions other than 1 ≤ 1 ≤ … ≤ 1 define subgroups of order p (k1 = k2 = … = kn = 1 defines the trivial subgroup).
These ki are calculated from the subgroup by considering the dimensions of subspaces fixed by the subgroup (eigenspaces of eigenvalue 1) and the associated generalized eigenspaces (nullspaces of those polynomials (x−1)k), so this is just a more detailed version of @jug's comment.
If we let I(n,p) be the number of conjugacy classes of subgroups of GL(n,p) of order p, then we have the bijective but slightly messy formula:

I(n,p) = number of partitions (disregarding order) of n into positive integers no greater than p.
  In particular, no, GL(n,p) contains non-conjugate subgroups of order p as soon as n ≥ 3 if p is odd, and as soon as n ≥ 4 if p = 2.

For p = 2, one gets the simple formula I(n,2) = floor(n/2).
I believe this is also equal to NrPartitions(n+p,p)−1, the number of partitions of n+p into at most p things (and subtracting one to remove the trivial subgroup).  This should follow by dual Young tableaux or otherwise by general combinatorics.
Answer: (Same as Jack's, different normal form!)
If $A\in\operatorname{GL}(n,p)$ has order $p$, then $A^p-I=0$. It follows that $A$ satisfies the polynomial $X^p-1$, so the minimal polynomial $m_A$ divides $X^p-1=(X-1)^p$. Since every eigenvalue of $A$ is a root of $m_A$, we conclude that the only eigenvalue of $A$ (in any algebraic closure of $\mathbb F_p$), is $1$.
It follows at once that both the Jordan form $J$ of $A$ and a matrix conjugating $A$ to $J$ are in fact both elements of $\operatorname{GL}(n,p)$. If we want to count the elements of $\{A\in\operatorname{GL}(n,p):A^p=I\}$ up to conjugation in $\operatorname{GL}(n,p)$, then, it is enough to count the conjugacy classes of matrices of $\operatorname{GL}(n,p)$ in Jordan canonical form with minimal polynomial dividing $(X-1)^p$.
This is a pretty simple excercise. The answer is: the number of partitions of $n$ with parts not larger than $p$.
Hmm, this is counting conjugacy classes of matrices and not of subgroups...*
Comment: In general conjugacy classes of elements and conjugacy classes of cyclic subgroups are similar (the latter are called **rational** conjugacy classes).  For GL (and p-elements) and "big" groups (like symmetric groups) they are the same.  In other words, all the generators of a subgroup of order p have the same generalized eigenspaces, so are all conjugate, so every conjugacy class (of elements of order p) is rational.


Do electronics bought airside have to be fully charged?

Question: With the recent regulations concerning electronic devices having to be charged, to take them on the plane, I'm wondering, do you also have to do this with electronics bought airside at the airport, which would presumably have very little charge when taken out of the box for the first time?
Or do they get wrapped in a special bag like liquids do?
Comment: I believe the rule is they need to be charged to take them through security. If you buy them after security, this is not an issue. However, if you have a connection that might require you to clear security again, you should plan how to charge them before that time.
Comment: Comment only: Yet another rule to guard against honest and everyday people. This "test" is so easily fudged that it is of no deterrence whatsoever to any "bad guys" who are aware of it in advance.
Answer: While it doesn't specify if it must be fully charged
TSA said:

As the traveling public knows, all electronic devices are screened by
  security officers. During the security examination, officers may also
  ask that owners power up some devices, including cell phones.
  Powerless devices will not be permitted onboard the aircraft. The
  traveler may also undergo additional screening.

It must at least turn on, so if you buy a new electronic device if it can turn on there should not be a problem, of course as always it depends on the security guard.
But as Gregory says if you bought after the security check there shouldn't be a problem
Comment: Also I'm guessing that if it's still sealed in its original packaging you should be OK.
Comment: Back in the pre-9/11 era I took a laptop through security with a very bad battery pack--it would die soon after the POST was completed.  I warned the agent what would happen, he accepted seeing it start the boot process and then die.  (I was only using it plugged in, the useless battery wasn't worth replacing.)  It's possible they are more strict now, though.


Mac OS X Snow Leopard 403 Forbidden

Question: After an update to my computer my server on Snow Leopard has become unstable. I continually get Forbidden 403 issues on every page that I try to access. I can upload my apache log files...just not sure where they are stored on Mac. Working on a project and need to get this rectified asap so any and all help is appreciated.
Answer: Did you check
<code>/etc/apache2/httpd.conf
/etc/apache2/users/whatever.conf
</code>
to make sure they didn't get moved or reset? Something broad like this has changed to cause 403 on all the pages.
Answer: Your starting point should, of course be checking the logs.
On OS X the Apache logs can be found in:
<code>/var/log/apache2
</code>
It would be my bet that there is a permission problem on a file or perhaps a missing folder, but a check of <code>/var/log/apache2/error_log</code> should give you the clues you need.
Answer: Just had to deal with that.  Here is the seminal link on that.
http://wiki.apache.org/httpd/13PermissionDenied

In order to serve files, Apache must have the proper permission granted by the operating system to access those files. In particular, the User or Group specified in httpd.conf must be able to read all files that will be served and search the directory containing those files, along with all parent directories up to the root of the filesystem.

Mac OS X Snow Leopard 403 Forbidden


How AsyncTask works in Android

Question: I want to know how AsyncTask works internally.
I know it uses the Java Executor to perform the operations but still some of the questions I am not understanding. Like:

How many AsyncTask can be started at a time in an Android app?
When I start 10 AsyncTask, will all tasks will run simultaneously or one by one?

I have tried with 75000 AsyncTask to test the same. I don't get any problem and seems like all the tasks will be pushed to stack and will run one by one.
Also when I start 100000 AsyncTasks, I start getting OutOfMemoryError.
So is there any limit of no of AsyncTask which can be run at a time?
Note: I have tested these on SDK 4.0
Comment: I have same Problem can you give any suggestion for My Stack Question http://stackoverflow.com/questions/17326931/download-bunch-of-files-from-server
Answer: AsyncTask has a rather long story.
When it first appeared in Cupcake (1.5) it handled background operations with a single additional thread (one by one). In Donut (1.6) it was changed, so that a pool of thread had begun to be used. And operations could be processed simultaneously until the pool had been exhausted. In such case operations were enqueued.
Since Honeycomb default behavior is switched back to use of a single worker thread (one by one processing). But the new method (executeOnExecutor) is introduced to give you a possibility to run simultaneous tasks if you wish (there two different standard executors: SERIAL_EXECUTOR and THREAD_POOL_EXECUTOR).
The way how tasks are enqueued also depends on what executor you use. In case of a parallel one you are restricted with a limit of 10 (new LinkedBlockingQueue<Runnable>(10)). In case of a serial one you are not limited (new ArrayDeque<Runnable>()).
So the way your tasks are processed depends on how you run them and what SDK version you run them on.
As for thread limits, we are not guaranteed with any, yet looking at the ICS source code we can say that number of threads in the pool can vary in range 5..128.
When you start 100000 with default execute method serial executor is used.
Since tasks that cannot be processed immediately are enqueued you get OutOfMemoryError (thousands of tasks are added to the array backed queue).
Exact number of tasks you can start at once depends on the memory class of the device you are running on and, again, on executor you use.
Comment: Wouldn't it throw a RejectedExecutionException (& not a OutOfMemory) if it filled the work queue though.
Comment: I will in case you run them with PARALLEL_EXECUTOR. But in case of a serial one the ArrayDequeue is used. Edited the answer.
Comment: Thank you Raman. Your answer clarifies my queries regarding AsyncTask.
Comment: The `LinkedBlockingQueue` capacity as of Android KitKat is now 128 for `THREAD_POOL_EXECUTOR`. The `ArrayDeque` for `SERIAL_EXECUTOR` still has no capacity limit.
Answer: let us dive deep into the Android's Asynctask.java file to understand it from a designer's perspective and how it has nicely implemented Half Sync-Half Async design pattern in it.
In the beginning of the class few lines of codes are as follows:
 private static final ThreadFactory sThreadFactory = new ThreadFactory() {
        private final AtomicInteger mCount = new AtomicInteger(1);

        public Thread newThread(Runnable r) {
            return new Thread(r, "AsyncTask #" + mCount.getAndIncrement());
        }
    };

 private static final BlockingQueue<Runnable> sPoolWorkQueue =
            new LinkedBlockingQueue<Runnable>(10);

    /**
    * An {@link Executor} that can be used to execute tasks in parallel.
    */
    public static final Executor THREAD_POOL_EXECUTOR
            = new ThreadPoolExecutor(CORE_POOL_SIZE, MAXIMUM_POOL_SIZE, KEEP_ALIVE,
                    TimeUnit.SECONDS, sPoolWorkQueue, sThreadFactory);

The first is a ThreadFactory which is responsible for creating worker threads. The member variable of this class is the number of threads created so far. The moment it creates a worker thread, this number gets increased by 1.
The next is the BlockingQueue. As you know from the Java blockingqueue documentation, it actually provides a thread safe synchronized queue implementing FIFO logic.
The next is a thread pool executor which is responsible for creating a pool of worker threads which can be taken as and when needed to execute different tasks.
If we look at the first few lines we will know that Android has limited the maximum number of threads to be 128 (as evident from private static final int MAXIMUM_POOL_SIZE = 128).
Now the next important class is SerialExecutor which has been defined as follows:
private static class SerialExecutor implements Executor {
       final ArrayDeque<Runnable> mTasks = new ArrayDeque<Runnable>();
       Runnable mActive;

       public synchronized void execute(final Runnable r) {
           mTasks.offer(new Runnable() {
               public void run() {
                   try {
                       r.run();
                   } finally {
                       scheduleNext();
                   }
               }
           });
           if (mActive == null) {
               scheduleNext();
           }
       }

       protected synchronized void scheduleNext() {
           if ((mActive = mTasks.poll()) != null) {
               THREAD_POOL_EXECUTOR.execute(mActive);
           }
       }
   }

The next important two functions in the Asynctask is
public final AsyncTask<Params, Progress, Result> execute(Params... params) {
        return executeOnExecutor(sDefaultExecutor, params);
    }

and
public final AsyncTask<Params, Progress, Result> executeOnExecutor(Executor exec,
            Params... params) {
        if (mStatus != Status.PENDING) {
            switch (mStatus) {
                case RUNNING:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task is already running.");
                case FINISHED:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task has already been executed "
                            + "(a task can be executed only once)");
            }
        }

        mStatus = Status.RUNNING;

        onPreExecute();

        mWorker.mParams = params;
        exec.execute(mFuture);

        return this;
    }

AS it becomes clear from the above code we can call the executeOnExecutor from exec function of Asynctask and in that case it takes a default executor. If we dig into the sourcecode of Asynctask, we will find that this default executor is nothing but a serial executor, the code of which has been given above.
Now lets delve into the SerialExecutor class. In this class we have final ArrayDeque<Runnable> mTasks = new ArrayDeque<Runnable>();.
This actually works as a serializer of the different requests at different threads. This is an example of Half Sync Half Async pattern.
Now lets examine how the serial executor does this. Please have a look at the portion of the code of the SerialExecutor which is written as
 if (mActive == null) {
                scheduleNext();
            }

So when the execute is first called on the Asynctask, this code is executed on the main thread (as mActive will be initialized to NULL) and hence it will take us to the scheduleNext() function.
The ScheduleNext() function has been written as follows:
protected synchronized void scheduleNext() {
            if ((mActive = mTasks.poll()) != null) {
                THREAD_POOL_EXECUTOR.execute(mActive);
            }
        }

So in the schedulenext() function we initialize the mActive with the Runnable object which we have already inserted at the end of the dequeue. This Runnable object (which is nothing but the mActive) then is executed on a thread taken from the threadpool. In that thread, then "finally "block gets executed.
Now there are two scenarios.

another Asynctask instance has been created and we call the execute method on it when the first task is being executed.
execute method is called for the second time on a same instance of the Asynctask when the first task is getting executed.

Scenario I : if we look at the execute function of the Serial Executor, we will find that we actually create a new runnable thread (Say thread t) for processing the background task.
Look at the following code snippet-
 public synchronized void execute(final Runnable r) {
           mTasks.offer(new Runnable() {
               public void run() {
                   try {
                       r.run();
                   } finally {
                       scheduleNext();
                   }
               }
           });

As it becomes clear from the line mTasks.offer(new Runnable), every call to the execute function creates a new worker thread. Now probably you are able to find out the similarity between the Half Sync - Half Async pattern and the functioning of SerialExecutor. Let me, however, clarify the doubts. Just like the Half Sync - Half Async pattern's Asynchronous layer, the
mTasks.offer(new Runnable() {
....
}

part of the code creates a new thread the moment execute function is called and push it to the queue (the mTasks). It is done absolutely asynchronously, as the moment it inserts the task in the queue, the function returns. And then background thread executes the task in a synchronous manner. So its similar to the Half Sync - Half Async pattern. Right?
Then inside that thread t, we run the run function of the mActive. But as it is in the try block, the finally will be executed only after the background task is finished in that thread. (Remember both try and finally are happening inside the context of t). Inside finally block, when we call the scheduleNext function, the mActive becomes NULL because we have already emptied the queue. However, if another instance of the same Asynctask is created and we call execute on them, the execute function of these Asynctask won't be executed because of the synchronization keyword before execute and also because the SERIAL_EXECUTOR  is a static instance (hence all the objects of the same class will share the same instance… its an example of class level locking) i mean no instance of the same Async class  can preempt the background task that is running in thread t. and even if the thread is interrupted by some events, the finally block which again calls the scheduleNext() function will take care of it.what it all means that there will be only one active thread running the task. this thread may not be the same for different tasks, but only one thread at a time will execute the task. hence the later tasks will be executed one after another only when the first task complets. thats why it is called SerialExecutor.
Scenario II: In this case we will get an exception error. To understand why the execute function cannot be called more than once on the same Asynctask object, please have a look at the below code snippet taken from executorOnExecute function of Asynctask.java especially in the below mentioned portion:
 if (mStatus != Status.PENDING) {
            switch (mStatus) {
                case RUNNING:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task is already running.");
                case FINISHED:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task has already been executed "
                            + "(a task can be executed only once)");
            }
        }

AS from the above code snippet it becomes clear that if we call execute function twice when a task is in the running status it throws an IllegalStateException saying "Cannot execute task: the task is already running.".
if we want multiple tasks to be executed parallely, we need to call the execOnExecutor passing Asynctask.THREAD_POOL_EXECUTOR (or maybe an user defined THREAD_POOL as the exec parameter.
You can read my discussion on Asynctask internals here.
Answer: AsyncTasks has a fixed size queue internally for storing delayed tasks. The queue size by default is 10. For example if you start 15 your tasks in a row, then first 5 will enter their doInBackground(), but the rest will wait in the queue for free worker thread. As one of the first 5 finishes, and thus releases the worker thread, a task from the queue will start execution. In this case at most 5 tasks will run together.
Yes, there is limit of how many tasks can be run run at a time. So AsyncTask uses thread pool executor with limited max number of the worker threads and the delayed tasks queue use fixed size 10. Max number of worker threads is 128. If you try to execute more than 138 custom tasks your application will throw the RejectedExecutionException.
Comment: Your answer is helpful. Thanks. I want to know one more thing. I am starting these 75000 tasks in for loop and don't get any exception (as you said). I am calling task.execute(param) in for loop and I don't see that exception. Then I am not understanding when this number 128 comes into picture? Means how can I reproduce your exception if I run more than 128 tasks?
Comment: Try using PARALLEL_EXECUTOR and executeOnExecutor(). Do not call default execute() method.
Comment: In Android 3.x the method `task.executeOnExecutor(Executor exec, Params... params)` allows to use custom thread pool executor and to configure the size of the delayed tasks queue.
Answer:
How many AsyncTask can be started at a time in an Android app?

AsyncTask is backed by a LinkedBlockingQueue with a capacity of 10 (in ICS and gingerbread). So it really depends on how many tasks you are trying to start & how long they take to finish - but it's definitely possible to exhaust the queue's capacity.

When I start 10 AsyncTask, will all tasks will run simultaneously or one by one?

Again, this depends on the platform. The maximum pool size is 128 in both gingerbread and ICS - but the *default behavior* changed between 2.3 and 4.0 - from parallel by default to serial. If you want to execute in parallel on ICS you need to call [executeOnExecutor][1] in conjunction with THREAD_POOL_EXECUTOR

Try switching to the parallel executor and spam it with 75 000 tasks - the serial impl. has an internal ArrayDeque that has no upper capacity bound (except OutOfMemoryExceptions ofc).
Comment: Thank you for your answer. I want to know one more thing. How can I get RejectExecutionException when I start more than 128 tasks in my app?
Comment: There's a hardcoded upper limit of 128 threads - the thread pool will not start more and the task will be put in a work queue (which has a maximum size of 10). If that queue in turn is full the task will be rejected and the default rejected execution policy is the `AbortPolicy` which throws a RejectedExecutionException.

How AsyncTask works in Android


Texmaker (WinXP) : How to open documents in separate windows?

Question: Is it me or is it actually not possible to open Latex files in separate Windows with Texmaker? Does anyone know of a way?
Answer: TeXmaker doesn't seem to allow multiple instances, but TeXstudio (a fork of TeXmaker) can be started multiple times using --start-always option.
Comment: I think I might just try that. The thing about TeXmaker is that there are some very good things about it, but there are other things that are extremely irritating. A better TeXmaker cannot be a bad thing. Thanks for the suggestion.
Answer: I created the following batch script to be able to run multiple instances of Texmaker. Once you opened a window, run this script and you will be able to open another instance. For this script you need the Sysinternals Handle utility to be in some folder of your <code>%path%</code> variable or in the same directory as the script.
<code>@echo off
rem Determine Process ID
for /F "tokens=1,2" %%x in ('tasklist /FI "imagename EQ texmaker.exe" ^| find "texmaker.exe"') do (
    rem Find Lock handle and kill it. Then delete the lockfile
    for /F "delims=: tokens=1,2,3*" %%i in ('handle -a -p texmaker.exe ^| find "lockfile"') do (
        echo Killing handle %%i of processid %%y
        echo y | handle -c %%i -p %%y>nul
        del "%%k">nul 2>&1
    )
)
pause
</code>
Comment: [Welcome to TeX.sx!](http://meta.tex.stackexchange.com/q/1436)


If a kind and generous Christian died 20 years ago, might their rebirth have resulted in there being a 19 year old person in the human realm now?

Question: If a kind and generous Christian died 20 years ago today, 21July2015, might their rebirth have resulted in there being a 19 year old person in the human realm now? I want to assume that the kindness and generosity perhaps trumped the ignorance about samsara, resulting in a new individual array of aggregates and store consciousness inclined toward learning about Buddhism and Nibana. I wonder this basis Theravada approach to rebirth.
Answer: I think it's difficult to talk about rebirth as if it were a sure thing: for example because kamma comes from a being's previous life, but it also comes from the life before that, etc.
"The [precise working out of the] results of kamma" is said to be unconjecturable.
Actually being able to directly see others' karmic destinations is one of the miraculous powers attributed to the Buddha:

Then, the records tell us, in the first watch of the night he directed his concentrated mind to the recollection of his previous lives. Gradually there unfolded before his inner vision his experiences in many past births, even during many cosmic aeons; in the middle watch of the night he developed the "divine eye" by which he could see beings passing away and taking rebirth in accordance with their karma, their deeds; and in the last watch of the night he penetrated the deepest truths of existence, the most basic laws of reality, and thereby removed from his mind the subtlest veils of ignorance. When dawn broke, the figure sitting beneath the tree was no longer a Bodhisatta, a seeker of enlightenment, but a Buddha, a Perfectly Enlightened One, one who had attained the Deathless in this very life itself.
Comment: Upvote, this is the best answer so far.
Answer: From the Tibetan Buddhism perspective this is a possible outcome. In philosophy David Lewis calls these possible worlds. It might even be true in the actual world. Which is the world we live in.
The bardo and rebirth would have to happen first of course.
Also human rebirth is considered to be rare,  rarer still is human rebirth that is interested in the dharma.
But it seems if the worldview is correct, that human rebirths would become more common.
But all that aside, the human realm is still Saṃsāra. Being reborn is rough, relearning everything takes time. Where they were born and to what families will determine what they learn and in what order.
I think that covers the first question.
As for the assumption after the first question, kindness and generosity is said to result in wealth, and being in safety respectively.
I do not understand what you mean in your last sentence though.
You mention one of the 3 vehicles.
Comment: Theravada Buddhism teaches an immediate rebirth after death, if I am correct, thus the age calculation.
Comment: Interesting. I did not know that.
Comment: @PaPa in that case wouldn't they have to be 20? Not 19?
Answer: The Buddha taught in many suttas the kind and generous go to heaven.
Answer: To be born as a human, you need to keep to the five precepts well. To be born to a Buddhist family, you need to dedicate your good deeds towards attaining Nibbana. If your friend died 20 years ago, he could be 20 human years old now, wherever he is born, unless he died again.
Comment: Thx for the reply. The individual in question probably kept the five precepts (as a "good" Christian). Assuming, per Theravadan death/rebirth approach, an immediate conception and a nine month human gestation, would not the resulting individual would be 19 years and 3 months old? I suppose that individual could be either male or female.
Comment: Usually Christians don't keep to the 5th precept, do they? He could be 20 years old, if he was born as a spontaneous being. Even for humans, conception is considered as the starting point in a life in Buddhism, even though conventionally, age is counted since the day you come out of the womb.
Comment: Thx, again, for the reply. If a "good and moral Christian" kept the five precepts in every detail EXCEPT killing animals for family meals (as in growing up on a family farm with chickens, etc.), does that preclude that individual from a rebirth in the human realm? I've not considered this before now! What realm would that individual's rebirth occur in? ... assuming no extenuating negative store consciousness "issues"...
Comment: Killing animals for meals can easily throw you in lower realms. Checkout the [states of deprivation](http://www.accesstoinsight.org/ptf/dhamma/sagga/loka.html)
Answer: Rebirth is using the conventional/conditioned truth of suffering, 'I, me, mine', to extrapolate, infer, into the unconditioned truth of the end of suffering, 'not I, not me, not mine'.
Majjhima Nikaya 27 does not allow for inference to grasp ultimate truth, only direct observation. In this respect, if you engage with rebirth, you are already one step away from the Tathagata's message of the cessation of suffering, as opposed to his message on suffering.
The issue is Dhamma is regular, without heights or extremes. Bardo naturally creates the extreme of existence and non-existence, as is therefore philosophically unacceptable. Spontaneous but memoryless continuation of the conditioned - that has NOT reached the unconditioned - is the only option that is not irregular.
That the conditioned randomly (not chaotically, that is still mathematically defined) wobbles around the mathematical approximate of emptiness that is '0', is irregular, so untenable for Dhamma.
Isms don't play any factor in the Newtonianesque law of kamma-vipaka. Intentional action and the result of intentional action, gives no concern for 'isms'.
Wisdom - the ideological space, and concentration - the supramundane path, are considered supreme within Dhamma, but that is different from the morality section.
There is no supremacy within Dhamma with respect to morality. Intentional action to non-harm, non ill-will, will bring benefit whether acted on by a murderer or by a saint.
A good Christian, within Dhamma - subject to accepting 'I' - will be reborn where they stand with respect to their morality.
Comment: The is no Dhamma where there is no condition. But well now here, maybe another time. May householder spend a blessed day, with a lot of gate, gate, paragate...
Comment: "if you engage with rebirth, you are already one step away from the Tathagata's message" Not so at all... [Apannaka Sutta: A Safe Bet](http://zugangzureinsicht.org/html/tipitaka/mn/mn.060.than_en.html). Right view is prerequisite. Maybe householder just expressed it not so skilful what was different thought.
Comment: changed to 'message of the cessation of suffering'.
Comment: I don't disagree there is a next world, but this is not nibbana as Ud 8.1 says. This is 'just the same world', of anicca. Engaging with this 'next world' of anicca, is no different to engaging with 'this world' of anicca.
Comment: As do philosophying and hold on as world on it, or? What was the first suggestion, quoting a sutta, that householder held up? So seems to be just an expression issue, right? Or a doing? Therefore: gate, gate, paragate...
Comment: I'm sorry, I don't understand your words exactly. I think grasping at a view of rebirth is philsophisising, and MN135 asks us to go against it (and I do engage in it!). The Buddha said, approximately, that trying to understand the exact workings of kamma is asking to fail. My first suggestion in the text was just that, if you accept 're'birth, you must naturally accept 'birth', which is accepting 'I', which is not the unconditioned side of Dhamma.
Comment: Certainly Dhamma is the best of the conditioned elements, may you have a blessed day too!
Comment: Sadhu, sadhu, .
Answer: It's hard to say where he is born (whether in bad places, human world or hevans) by referring to what he did in his normal life. Why I'm saying this is according to the Sotaapanno Sutta: The Sotaapanna ('Stream-winner') (quoted below) only stream-winner (or above) has the guarantee that he will not go to hells. That means only stream winner (or above) will not get unhappy births which are Thirisan (animal's world), Pretha, Asura or Narak. This also doesn't imply that only stream-winners (or above) can get happy births. It's only about the guarantee.
There's a story of a lady called Mallika in buddha's days. She did a big almsgiving (dāna) called asu disi dāna those days. But her immediate re-birth was an unhappy one. Why? it's because she was not a stream-winner. So nothing can be guaranteed. But the good work she did has gone to her karma and she will get those back in one day. (if my memory is correct and she spent only 7-days in unhappy birth and after 7 days she got a happy birth again).
This is why being a stream-entrant/winner is so much in our life. And that's what Buddha has advised to do as the first thing in our life.

"And when, monks, the Ariyan disciple understands as they really are
  the arising and the passing away, the attractiveness and the danger,
  and the deliverance from the five groups of clinging, he is called an
  Ariyan disciple who is a Stream-winner, not liable to states of
  woe, assured of final enlightenment."
Answer: If you attain stream entry in this lifetime, your next birth as a human being is assured. Stream enterants take next birth at most 7 times, as a human being, before they reach ultimate enlightenment.
To reach stream entery you must duplicate, understand, contemplate and realize the complete teachings of Buddha. Then you must make 100 days of concentrated effort of meditation. This process is described in the ancient Taoist scripture, Secret of the Golden Flower. I recommend reading The Secret of Secrets vol. 1&2 for complete understanding of this process.


A cop's situational handling

Question: I have this scene I'm writing were 10 16 year olds (three girls and seven boys) are loitering in a casino parking lot around a van, Cops have been called cause the kids haven't left. I'm trying to figure out how a cop(s) would talk or approach the situation.
So the question is would anyone know how a cop would approach this situation?
Would they be mean, professional, condescending, or casual what?
Comment: Yes. Professional and courteous is the usual approach, but ten kids and a van comes under the heading of community policing, which makes many cops extremely uncomfortable. A cop might cover his discomfort by pretending that he *knows* a crime is taking place.
Comment: Welcome to worldbuilding. This sounds like a question based on a story, rather than a question about a world. Please check the [help] and take the [tour] to understand how to improve it.
Comment: If you go on YouTube and search for "traffic stops" you will see all sorts of examples. Of course some are for serious criminals but others show how the police start politely and then things get more dramatic. Of course all the ones on YouTube will end up dramatic because otherwise no-one would bother to watch!
Comment: Thank you and sorry. this was the first sight I could think of to come to.
Comment: You might also be interested in the following site https://writing.stackexchange.com/
Answer: In America, usually calm and polite
I'm assuming (and hoping) this would be true in most places in the world, but I'm only familiar with American police.  The answer might vary based on country, region, size of town, et cetera. In general, police in America will approach calmly and politely inform the group they are not permitted to loiter in the area.  If the children become obstinate, disrespectful, or otherwise refuse to leave, then the police are trained to escalate as slowly as possible based on the actions of the children.  If things did escalate, backup would be called to both assist the situation and provide another set of police-trained eyes (and/or cameras) to cover the interaction.
Other factors which may influence this encounter:

crime rate in the area (an area with a high-level of violence will be treated differently than a small town with little to no violence)
previous events that day/week/month (a recent bomb threat involving a van of matching description, multiple police reports of "a group of teenagers" in nearby areas, etc, might make police more nervous or suspicious)
size of the town (small town cops have different training and resources than big city cops, and different psychology - you treat everyone differently when everyone knows everyone versus spending all day with new faces)
whether or not the teenagers have records, or are known by the department, dispatcher, or police officer who arrives at the scene
body language or other behavior of the children (are they smoking or drinking? Being loud? Playing music? Sitting down? Running around?)
political and cultural considerations (for example, a group of white and black children loitering on Native American casino grounds will be treated differently by tribal police, and if non-tribal police arrive, they will have a different protocol for enforcing laws on tribal land)
training the cop has (a seasoned officer has more experience and training than a rookie cop)
the officer's attitude (cops are human - if he's had a bunch of people screaming at him all day, and he's near the end of his shift, he might be more irritable or condescending than an officer who is on the first call of his shift, or is just having a good week in general)

A cop's situational handling


Extract part of a string in SQL

Question: In SQL Server 2008 R2 I need to extract from a string everything between the character 50 and 60.
I've searched for a similar function on the internet but didn't really find something which can address the problem.
I can do it on Excel with a formula like: MID(A2, 50, 10)
Comment: SubString() - http://technet.microsoft.com/en-us/library/ms187748.aspx
Answer: The sql function is called substring
  substring(fieldname,50,10)

Answer: If you need more flexibility, for instance if the separation does not occur in the same place each time you can use CHARINDEX. Here are a couple examples.
REVERSE(LEFT(REVERSE(fieldname),CHARINDEX(' ',REVERSE(fieldname))))

If your string is "AA 1235 A9" the output will be A9.
REPLACE(STUFF(fieldname,1,CHARINDEX('- ',fieldname),''),REVERSE(LEFT(REVERSE(fieldname),CHARINDEX(' ',REVERSE(fieldname)))),'')

Adding on to the first bit of code you can also pull the middle portion which will result in 1235. The ' ' is flexible and if there is a hyphen, asterisk or some other symbol/identifier you need to use then just use that instead of the space.

Extract part of a string in SQL


Unicode character folding with Swiper/Ivy

Question: Suppose I have bár written in a file, I would like to look for bar in swiper and find the occurrences of bár alongside the expected occurrences of bar, how I can do this? This would also apply to characters like e=é=è, o=ö, u=ü, a=å, o=ó=ö, c=ç, etc.
Answer: Having bar match bár is called character folding, and is supported by the built-in functions isearch and query-replace since Emacs 25; see (emacs) Lax Search and (emacs) Search Customizations for details.
swiper automatically performs character folding when search-default-mode is set to the function char-fold-to-regexp, which you can achieve by e.g. adding
(setq search-default-mode #'char-fold-to-regexp)

to your user-init-file. Voila:
Comment: @k76u4vkweek547v7 That is because the `char-fold-to-regexp` integration is specific to the [regexp builder](http://oremacs.com/swiper/#completion-styles) for `swiper`. Feel free to submit a feature request on the project's issue tracker: https://github.com/abo-abo/swiper/issues.
Comment: @k76u4vkweek547v7 Adding to that, one solution is to use that re-builder for `swiper`, i.e. `(setq ivy-re-builders-alist '((ivy-bibtex . swiper--re-builder)))`
Comment: That worked for swiper, but for some reason it did not work with ivy-bibtex.


How long to get "good" at gemara?

Question: Starting from 11th grade this year, I've been going to a dormitory yeshiva a few hours away from home and come back every off shabbos. Before that, I never been to Yeshiva, but since I was around 8 years old, I've always kept shabbos and kosher and learned halachah here and there. I've never really focused much on Gemara because my rebbe told me I had to focus on my middos and hashkafa. It's one of those yeshivas that are not schedule based, but each bochur decides what he needs to focus on with his rebbe. Now that I'm not in yeshiva, I've been learning gemara consistently for the past few days with my chavrusa and I'm already getting used to it. From those of you who have been in similar situations, how far would you say I will get in 2 months after learning for 45 minutes a day?
Comment: Hi, thanks for stopping by to ask. Unfortunately, this question is not really on-topic here, so it might be closed.
The short answer is - keep it up, and do what you can. Focus on getting a feel for the flow of the gemara, and finding a "rhythm" in your learning which suits you. Don't worry about how much you cover at this point.
Comment: Everybody is different. It depends on your aptitude probably.
Comment: Look at what I write on daf yomi in chat and see if you understand it. It will give you a good basis of how to learn rashi. @Nathan Shahkohi you dont tell us what you use.
Comment: @simyou Not that it's off-topic, but that it's opinion based. As Moshe answers very nicely, everybody is different and no two people have the same learning experience.
Comment: 10,000 hours.......
Answer: I understand your question, but at the same time I need you to understand that it's basically impossible to know since I don't know you. Though, it's always a  good idea to look at what Chazal (Sages) have to say about the matter...

Make sure that your Chavrusah (Teacher I presume) is a pure fellow - Moed Katan 17a
The use of Mnemonics is important, it always helps - Shabbos 104a
Remember that memory is divinely controlled, so don't get to frustrated - Megillah 6b
That once a mistake is learned, it is hard to un-learn it, so be careful - Pesachim 112a
That consuming too much torah at a fast paste can cause difficulty in learning - Nedarim 37b
Anger causes one to forget his learning, and to increase in stupidity - Nedarim 22b
Being in Israel sharpens one's mind, so if you're there good for you!
Temurah 29a
Torah can only be retained by one who kills himself for it. Work hard play hard - Shabbos 83b
Torah will not be found with a relaxed person: Eruvin 55a
Torah is like a fig tree, which has fruit at various stages of ripening; the longer one works at it, the more one finds: Eruvin 54a-b
Torah is like a nipple, producing for as long as one works at it - Eruvin 54b
The Torah is not in the heavens or across the sea, but were it there, we would have to go get it - Eruvin 55a

There's so much more...
Just keep Shteiging and NEVER give up.
Answer: I have been learning Gemara since i was in first grade, But my rebbeim always say once you master translating 10 daf you can basically translate all of shas. It depends on how quick of a learner you are and how uch you complete in that allotted time. Some people take longer some people learn quicker, but 10 daf is the average.
Comment: There is more to learning than just translating @Mordechai Schmerler.
Comment: I don't think this is true. 10 blatt, especially 10 blatt in the sections that are standardly studied by beginners, is nowhere near enough to be able to translate all of Shas. Sure, you'll get down standard terms like  הכא במאי עסקינן, קמ"ל, חסורי מיחסרא, etc. but you won't have any idea what גילוי דעתא בגיטא, פסיק רישא דלא ניחא ליה, כל הראוי לבילה, etc. are.
Comment: What yeshiva did you go to that they learned gemara in first grade,unless you learned at home?
Comment: No, I learned with my dad. I know how to learn.
           I first learned how to learn than I learned how to translate Gemara.
Answer: My tradition from more than one Yeshivah that I attended taught me that when you know 30 daf (double sided pages) of Gemara in a row anywhere in the Talmud with Rashi and Tosfos, then you will never have difficulty learning the rest of Shas.
Comment: How would 30 blatt of berachos help for zevachim?
Comment: by mastering the give and take and flow of the Gemara's speech and logic for 30 blatt with Rashi and Tosfos, one will be prepared for the flow of speech and logic anywhere in Shas. Actual subject matter and new words, would still need independent study and translation.
Comment: It's hard to say such a general rule,since some mashectos have different baalei tosfos and subject matter is an extremely important factor to mastering gemara ,since many subjects appear multiple times in shas. Since brachos is mainly aggadata the flow is very different than let's say zevachim which heavily a verse focused mashechta.  Anyhow, did any rebbi advise which 30 blatt to do?
Answer: My personal experience is that proficiency in learning Gemara comes in sudden leaps.  You work on it and work on it, and you may not notice much improvement.  Then, one day, something clicks.  Your start to 'get it'.
Now, what 'it' is, depends on where you are holding in learning Gemara.  'It' may be understanding the back and forth of the Geamra, how to ask a better Kasha (question) or how to read a Tosfos.
The key, though, to having those leaps and bounds is to focus on what you need to improve right now.  If you are having trouble with the basic meaning of the Gemara (i.e., peshat), then don't start working on Tosfos.  If you aren't clear what a line of the Gemara means, don't press forward so that you can 'cover ground'.
Rather, make sure that you understand each line, what it means and how it fits into the bigger picture.  If you do that (and don't fudge it), then you will improve.  When you will see that improvement, though, is not clear.  It may be in 2 months.  It may be less, it may be more.
Obviously, the more quality time that you are able to dedicate to learning, the sooner you will see improvement.  But the key is not so much 'when' as it is knowing that it will happen.  You just have to put in the hours and let the process naturally work out.
B'hatzlacha.

How long to get "good" at gemara?


Geometric Langlands as a partially defined topological field theory

Question: I have heard from several physicists that the Kapustin-Witten topological twist of $N=4$ 4-dimensional Yang-Mills theory ("the Geometric Langlands twist") is not expected to give
rise to fully defined topological field theory in the sense that, for example, its partition function on a 4-manifold (without boundary) is not expected to exist (but, for example, its
category of boundary conditions attached to a Riemann surface, does indeed exist). Is this really true? If yes, what is the physical argument for that (can you somehow see it from the path integral)? What makes it different from
the Vafa-Witten twist, which leads to Donaldson theory and its partition function is, as far as I understand, well defined on most 4-manifolds?
Comment: Alexander, you might want to look at Ben-Zvi and Nadler's paper (http://arxiv.org/abs/0904.1247). They construct the 2-1-0 part of the GL-twisted $N=4$ 4d SYM compactified on $S^1$. However, the 3d part is not well-defined precisely due to non-fully dualizability that Urs mentioned: the vector spaces you attach to 2d manifolds may be infinite-dimensional (see p. 15).
Comment: My question was really whether there exist physical arguments which  tell you "how defined" the theory will be.
Answer: In a "fully defined" TQFT the spaces of states are necessarily finite dimensional. This follows simply from the fact that the correlators assigned to the cap and the cup cobordism (the "2-point functions") equip the space of states with the structure of a dualizable object in the corresponding monoidal category of vector spaces, which are precisely the finite-dimensional objects.
Similarly, in a "fully defied" extended n-dimensional TQFT (a "fully local one") the "n-space of states" assigned to the point is a fully dualizable object.
But there are TQFTs with non-finite state spaces, and extended TQFT with not-fully dualizable $n$-space of states. In the case of d=2, these are (somewhat misleadingly) known as TCFTs . Famous examples are the A-model and the B-model. And the Kapustin-Witten 4d TQFT reduces to these in certain compactifications (see for instance Kapustin's review pages 17-18).
So how can this be? The answer is that a "TCFT" is a TQFT that, as a cobordism representation, is defined only on the subcategory of cobordisms that are called "non-compact" or "with positive boundary". Roughly speaking, this is simply the subcategory obtained by discarding the cup (or the cap) cobordism. This removes from the TQFT the requirement to have dualizable state spaces, but otherwise retails all the structure of a TQFT.
For an extended such TQFT (a "fully local one") the state 2-spaces (those assigned to the point) still have lots of nice structure, even without being fully dualizable. One says that they are Calabi-Yau objects.
A detailed discussion of all this is in is section 4.2 of Lurie's On the classification of TFTs
Comment: Thank you. I more or less know Lurie's paper. My question was to what extent this applies to the Kapustin-Witten theory and specifically why does it happen that for some other twist the theory is "more defined" (or am I mistaken here?)
Comment: Concerning to what extent this applies to KW theory: as I tried to indicate, at least we know that it has compactifications to 2d that in certain parts of the parameter space reproduce the A-model and the B-model. For these 2d TCFTs we know excactly what's going on (via Lurie's section 4.2). Since these are simple special cases induced from KW theory, it seems to follow that KW theory is "at least as non-fully defined" as this. Not sure if this helps, but this is the statement that I can see so far.
Comment: Thank you. In fact, orginally I wanted to know whether there is a way to predict how well defined given TQFT will be by looking at the path integral. But what you wrote is also very helpful!
Comment: When you say that the A and B model have non-finite (dimensional) space of states, do you mean in the case of a non-compact target?  I would naively think that the A model with a compact target $X$ has a finite-dimensional space of states (namely the quantum cohomology of $X$).
Comment: No, I mean the "2-space" of states, the A-oo algebra of string states assigned to the point. This is not a fully dualizable object for the A- and B-model.
Comment: Urs - in my understanding TCFT and noncompact (or positive boundary) field theory are two orthogonal issues.. In particular the A- or B-model for a smooth compact symplectic manifold/variety are full 2d TFTs in the derived sense (aka TCFT), and what they attach to a point is fully dualizable. In the example at hand the issue is (as Kevin  and Andy explained) the noncompactness of the target of the relevant B-model
Answer: From the path integral point of view, one can argue why the KW theory partition function won't be well defined as follows.
At the B-model point the KW theory dimensionally reduces to the B model for the derived stack $Loc_G(\Sigma')$ of $G$-local systems on $\Sigma'$.  The B-model for any target $X$ is expected to be given by the volume of a natural volume form on the derived mapping space from the de Rham stack of the source curve $\Sigma$ to $X$.
Putting this together, we see that the KW partition function on a complex surface $S$ is supposed to be the "volume" of the derived stack $Loc_G(S)$ (with respect to a volume form which comes from integrating out the massive modes).
Now we see the problem: the derived stack $Loc_G(S)$ has tangent complex at a a $G$-local system $P$ given by de Rham cohomology of $S$ with coefficients in the adjoint local system of Lie algebras, with a shift of one.   This is in cohomological degrees $-1,0,1,2,3$.
In other words: fields of the theory include things like $H^3(S, \mathfrak{g}_P)$ in cohomological degree $2$.   Because it's in cohomological degree $2$, we can think of it as being an even field -- and then it's some non-compact direction, so that we wouldn't expect any kind of integral to converge.
(By the way, I discuss this interpretation of the KW theory in my paper http://www.math.northwestern.edu/~costello/sullivan.pdf)
Comment: Of course, this is an example of the phenomenon Andy mentioned (non-compactness in field space). If you dimensionally reduce, you can see this from the categorical point of view Urs mentioned: the B-model for $Loc_G(\Sigma)$ is built from coherent sheaves on $Loc_G(\Sigma)$, but that's not a fully-dualizable category.  It seems (?) to be smooth but not compact. This means that the B model for $Loc_G(\Sigma)$ is only partially defined: operations are defined for surfaces with at least one outgoing boundary.
Comment: Thanks a lot - this is very interesting. I have two questions though: first, can you make a similar argument for Donaldson theory (to see why its partition function is well defined for most 4-manifolds). Second, I didn't understand the last point of your comment - it seems more or less possible to attach a category to $\Sigma$ without boundary (roughly speaking derived category of quasi-coherent sheaves on $Loc_G(\Sigma)$ - this has to be carefully defined, but in the context of geometric Langlands it is nowadays known what the right category is). why doesn't it contradict your argument?
Comment: For the second point: I'm very far from an expert on geometric Langlands, so I hope what I say is not wrong.  But, I would guess that the category you attach to Loc_G(Sigma) is not "proper".  Proper means $\operatorname{RHom}(E,F)$ is of finite total dimension, where $E,F$ are perfect complexes (i.e. compact objects of the category).  Proper is a necessary condition in order for the category to give a full TFT. There are more details on smooth + proper categories in Lurie's paper and Kontsevich-Soibelman's work.
Comment: The only thing I really understand about Donaldson theory is how to twist the $N=2$ gauge theory to get a holomorphic version of Donaldson theory, which should count holomorphic bundles. (Of course this should be equivalent.) There, you expect the partition function to be a kind of volume of $T[-1] Bun_G(S)$, where $S$ is a complex surface.  Since for stable bundles $Bun_G(S)$ has tangent complex in degrees $0,1$ (deformations + obstructions, no automorphisms) $T[-1] Bun_G(S)$ has tangent complex in degrees $-1,0,1$, which is fine.
Answer: I would think that Donaldson theory is also not a 4d TFT strictly speaking -- after all, there are some 4-manifolds for which it has metric dependence.  Isn't that enough to violate the letter of the law?
In that case it's usually said that the reason for the failure is some non-compactness in field space (one finds this claim e.g. near the bottom of page 5 of hep-th/9709193).  I suppose a similar problem could afflict the Kapustin-Witten twist of N=4 super Yang-Mills.
Comment: Thanks. I would actually be glad to understand the pattern: is there a way to predict how well defined given TQFT will be by looking at "the space of fields" and the path integral?
Comment: I am not fully competent to answer (and I am not at all sure that the issue I am mentioning is the same one your original interlocutors had in mind), but here is a partial comment.  Looking at hep-th/9709193 I get the sense that the non-compactness that is relevant is that of the space of supersymmetric (or "BRST invariant") field configurations.  If that's right, then there would be a potential issue whenever the equations that determine the supersymmetric field configurations define a non-compact moduli space.


File compilation order in VS2008

Question: I would like to explicitly set the order of compilation of C# files in VS2008. Order in the project file, file naming seems to have no effect.
Situation: I have dozens of partial classes, each split over two files: 1) The file containing the  auto-generated parts and 2) the file containing the manually written parts. The code generator is not very good, so I have to fix the constructors every time after regenerating the code.
My fool-proof solution is this:

fixing the constructor in the manual file
After regeneration, I get compiler errors about the duplicate constructors
Double click on the error which takes me to the duplicated constructor and delete it.

The problem: Visual Studio marks the later compiled constructor as duplicate. It always happens to be the manual, which causes me a hassle to find back the auto-generated one.
I want VS to pick the auto-generated constructor as a duplicate.
Current solution I use: After regeneration of the code, I copy the code from manual file (and comment out the original) and paste it on the TOP of the auto-generated file. Then VS picks the autogenerated constructors as duplicate. Should I paste it on the bottom of the auto-generated file, VS pick the manual constructors as duplicate.
Does anyone have a better idea how to overcome the issue?
Answer: I would tackle this in a different way: write a post-build tool to automatically remove the duplicate constructors. Presumably the ones you have to remove don't change, so you should be able to spot them fairly easily.

File compilation order in VS2008


How to use Bluetooth HIDL?

Question: I'm developping an AOSP app that have to send HCI command to Bluetooth chipset.
I found that i can use this interface : https://source.android.com/reference/hidl/android/hardware/bluetooth/1.0/IBluetoothHci
To use it, i tried to follow this page : https://source.android.com/devices/architecture/hidl-java/index.html
If i understand well, i have to create an Android.mk file and put
LOCAL_STATIC_JAVA_LIBRARIES +=  firstname.lastname@example.com

But i don't understand how ? I'm still new in AOSP developpement, how can i use this library ?
Answer: If you are an application developer:
The IBluetoothHci you found defines the interface of the Bluetooth Hardware Abstraction Layer (HAL). HAL interfaces are not accessible from apps directly but are used by framework services, which provide interfaces that can be used by apps.
I recommend checking out the Android SDK: https://developer.android.com/guide/topics/connectivity/bluetooth
If you are a platform developer:
In case you plan to write a service with more privileges (you build the AOSP yourself and flash the whole device) you are right.IBluetoothHci is the interface to use. You would probably want to switch from using Android.mk to Android.bp, because Android.mk files are deprecated.
In your code I would expect to see something like this:
import android.hardware.bluetooth.V1_0.IBluetoothHci;
...
// retry to wait until the service starts up if it is in the manifest
IBluetoothHci bluetooth = IBluetoothHci.getService(true /* retry */); // throws NoSuchElementException if not available
bluetooth.initialize();

Hints on how to use a HAL interface can also be found in the respective VTS tests (they are written in C++ though): https://android.googlesource.com/platform/hardware/interfaces/+/refs/heads/master/bluetooth/1.0/vts/functional/VtsHalBluetoothV1_0TargetTest.cpp

How to use Bluetooth HIDL?


Expectation of number of trials before success in an urn problem without replacement

Question: An urn contains b  blue balls ,r  red balls and y yellow balls. Balls (red and yellow) are removed at random without replacement until the first blue ball is drawn. What is the expectation of the total number of balls drawn?
Answer: Number the red and yellow balls from $1$ to $n$, where $n=r+y$. Don't bother to number the blue balls.
Let $X_i=1$ if the $i$-th numbered ball is drawn before any blue ball, and let $X_i=0$ otherwise.  Then the total number of draws is $1+(X_1+\cdots+X_n)$. It follows that the expected number of draws is
$$1+E(X_1+\cdots+X_n).$$
By the linearity of expectation, this is
$$1+E(X_1)+\cdots +E(X_n).$$
All the $X_i$ have the same expectation. so let us find $E(X_1)$.
The probability that ball with label $1$ comes before any blue ball is $\frac{1}{b+1}$. This is because all orderings of ball $1$ and the blue balls are equally likely. So $E(X_1)=\frac{1}{b+1}$.
It follows that the number of draws has mean
$$1+\frac{n}{b+1}.$$
Remark: We used the powerful tool of indicator random variables. One can alternately find the probability distribution of the number of draws, and then find the mean by using the standard formula, and a manipulation of binomial coefficients. That is substantially more messy. It is useful to know that in many cases the expectation of a random variable $Y$ can be computed without explicit knowledge of the probability distribution function of $Y$.
Comment: Thank you André. I would like to know if you meant n=r+y instead?
Comment: @user56962: Indeed I do! Thanks for noticing. Fixed.
Comment: A total of 6 objects hide the following values: 5 cents, 10 cents, 25 cents, 50 cents, 1 chain-reaction object to discover 2 objects and 1 chain-reaction object to discover ALL objects. The player is awarded 3 picks (or untill ALL object is selected, awarding all numeric values). The chain-reaction object randomly pick 2 additional objects. What is the expectation of the total number of objects picked?


Is it possible to deal with non-normal distribution in Black-Litterman model?

Question: Suppose that I know that the normality assumption about my data is unrealistic (as it is very frequently): is it possible to apply any distribution that I judge the right one to the Black-Litterman model? Does it make sense, or does it lose some of its usefulness?
Answer: Well there are two main things to consider here.
Many implementation of Black-Litterman use the market portfolio and the ex post volatility and correlation structure to back out implied returns to use as prior. As far as I know, there is no standard way to reverse-engineer the optimization problem in the presence of nonnormal markets. (the first guess is that it could be possible for elliptical markets)
So in that respect its probably at least hard (maybe not possible).
If you think of Black-Litterman as a Bayesian method, there is no reason why you shouldn't do it for nonnormal markets. You will lose the closed form posterior though. Check out Meucci's work on this. ("The Black Litterman approach: Original Model and Extensions" on SSRN or maybe you find something in his book., Also check out "Bayesian Portfolio Analysis" by Avramov and Zhou)
So the final question is: "Does it make sense?"
Well, it depends how you proceed from there. If you want to do an optimization there is the topic: either the market is normal (then, in theory, you have an optimal solution with mean-variance optimization) or you have a quadratic utility function (which says that higher moments dont matter for you as an investor). You have to think about the utility function you use.

Just to sum this up the things to think about are:

What are "implied returns" in a nonnormal market? (or do you want to use a different prior)
How to calculate the posterior distrubution (Bayes' formula)
If optimizing afterwards: What about the utility function? (Full-scale optimization)
Comment: @vanguard2k I've tried to do the reverse-engineer assuming that prices were multivariate log normal instead of multivariate normal. I wasn't that happy with it. I think it can't hurt to think about what is an appropriate prior, but once you start mixing in all kinds of different assets (equities, fixed income, options), the reverse engineer method of constructing the prior becomes a bit unwieldy I think.
Comment: @vanguard2k Suppose MVN(u, sigma) represents the multivariate normal distribution and MVLN(u*, sigma*) is what you get if you transform it to a multivariate log normal. Both u* and sigma* depend on u. This makes writing the optimization problem more challenging.
Comment: Thank you. Ok for the first two points (I'll use a different prior, probably, and plug it in the Bayes's formula, ending up with a very complicated new formula, i guess ;)). It is not clear to me the utility function/full-scale optimization argument you used.
Comment: @John I suppose so. But I guess this is one of the main points of the BL method. What was your problem with the lognormal version? For multi asset portfolios I dont even have an idea how to get the market cap (e.g. for commodities) even if there are no derivatvies though one could argue there are *some* sources.
Comment: @SimoneBortolato The output of the BL method is a vector of means. If you want to optimize based on that means, you have to either assume something about the investor or about the market.
Usually, its quadratic utility (only first two moments are relevant) or normality of the market. You say that your market is not normal and you care about it so you probably have to change these assumptions.
You could incorporate higher moments into your function or do a full-scale optimization for example.
Comment: @vanguard2k Well, suppose I want to optimize a portfolio based on some other measure (some risk measure for example): is it still B-L? Or is it just apply the Bayes' formula with some prior distribution (different from the normal one, probably) and do an optimization?
Answer: Yes there is a method to deal with non-normal market distributions in Black Litterman optimization. It is call Black Litterman Copula Opinion Pooling which uses copulas to model the market returns and therefore solve the non-normality problem. It was propose by Attilio Meucci and it can be implemented in R or Matlab. Never the less there is an other problem with the model and it is that it assumes that the priors are independent between them which in some cases it is not realistic.


How to make old bike's seatpost slide smoothly when it needs adjustment again?

Question: I've several old bikes with a common problem, the seatpost is a bit too much hard to slide just to adjust the height (sometimes the frame scratch the seatpost), which might be ok on a road bike, but not so much on MTB or folding bike, when you need to put the bike in a car for example. I thought about sanding both the seatpost and the inside of the bike, but is that a good idea? Of course I grease the seatpost to avoid galvanic corrosion and let it became stuck over time, but it slighty improve, doesn't solve the problem.
Answer: There are different things that can cause this problem and different tools and techniques that can be used to fix it, depending on the cause and severity.
Frame material cannot be left out of the conversation. Between steel, aluminum, and carbon, the problems and acceptable solutions are all different. Carbon should usually be approached with elbow grease and appropriate chemicals only, aluminum may need that plus material removal methods, and steel may need any of the above plus the full range of more metalworking/manipulation type fixes.
The problems that can potentially exist, from least to most severe, are:

A seatpost that's either out of tolerance or the wrong size.
Corrosion.
Burrs and other slight high spots in a frame that's never received basic honing.
More serious dimensional/distortion type problems left over from manufacture.
Acquired damage, the most common situations being seatpost binder "ears" that have become distorted inward from being forced to clamp an undersized post, or that have folded out and/or rippled from a post being ridden with inadequate insertion. (This one is mostly exclusive to steel frames. Aluminum will tend to either crack or break instead of taking a set if abused in this manner, and carbon will only crack or break.)

Of these maladies:

If it's the seatpost's fault, which isn't common but does happen, it should probably just be replaced if it can't be lived with. If you use a hone or reamer to enlarge the bore to accommodate a post with bad tolerances, you could run into problems getting any future replacements to hold right. That said, there are times when the advantages of doing the quick and dirty fix might win.

Corrosion needs to be cleaned up, ideally with something that won't mechanically enlarge the seattube bore, or at least not in any appreciable way. Doing it chemically or with light sanding is fine, as is appropriate for the frame material. (For the most part, don't sand or abrade a carbon seattube bore unless you really know what you're doing, and likely not even then.) Most shop mechanics would start with a flex-hone (see below) if confronted with a nasty looking bore on a steel or aluminum bike. They are capable of removing material, but won't instantly take a snug fit to a loose one.

Metal frames really should be hit with a flex hone at least once in their lives to take down any burrs or minor tight spots leftover from manufacture. Shops have these tools. Barring access to one, sanding around the seat cluster wherever your fingers can find potential sharp edges is acceptable as well. In some cases using tiny files to create a bevel at the slit/binder area is needed. Removing the binder bolt, sliding in sandpaper, and doing a shoe-shine motion can work too. The nice thing about flex hones is they deal with burrs and high spots it can be troublesome to otherwise see or get to, like from seatstay vent holes. Unless something weird happens, all of this is only necessary once in a bike's life, and will not be any kind of acquired problem. The default is to assume carbon frames need none of this, although in practice they sometimes do in a very targeted and judicious fashion, but only by hand.

A flex-hone

In some instances, on metal bikes only, a seat tube reamer can be an appropriate tool to solve this problem, but it's not the go-to and is a seldom-used tool even in most shops. Reamers are cutting tools that clear a seatpost-sized path down the bore. They remove material relatively freely and in contemporary times are primarily used by framebuilders to correct heat distortion, or in shops to address distortion left by the factory. Sometimes they have a role in fixing acquired damage to the bore, but usually this is limited to smoothing things out after other manipulation has been done to deal with most of the interference. Even then, assuming cold work only, the ideal goal of this kind of repair is to finesse the fit close enough that a hone does all the smoothing needed, i.e. you've gotten things so the post slides in but with some minor roughness. If you go to the reamer, you're making judgment calls about how much material the frame can lose safely and how much you'll be taking, neither of which you know. Reamers are not to be understood as a one-stop fix for reforming the bore after damage. Reamers are also capable of completely destroying the seatpost fit, and of getting damaged themselves if used inappropriately, so it's important to be judicious with them. Reamers can leave sharp edges, especially when their use was warranted in the first place, and should usually be followed by honing.

An adjustable seattube reamer

On steel bikes, sometimes the seattube bore acquires damage that can be fixed with metalworking tools and techniques. How to address this and what's appropriate for different frames/designs could be its own question. One would take different approaches on a thick-walled beater where you're willing to damage the paint and wind up with surface irregularities versus a vintage road bike restoration, for example. But, for the common situations of either crushed in or folded/rippled out openings on a utilitarian sort of bike, it usually comes down to working it back into shape with some combination of dummy posts, soft hammers, etc, with honing and possibly reaming as a final step. There are more framebuilder-level type approaches that would be appropriate on lightweight tubes or if you wanted a better looking result, and repainting would be involved then.

Before any kind of heavy cleaning, sanding, or cutting is done on the seattube bore, it's good to put a rag or paper towel down below where you'll be working to catch debris so it doesn't get in the bottom bracket area. Fish it out afterward with a spoke or small parts grabber.
Comment: At our shop we used to call seatposts that had been forced upwards with a burr scratching it "Charlie Browned" because it looked like the sweater he wears.
Answer: Check for burrs (a small 'lip' of material) on the inside edge of the top of the seat tube). That's probably what is scratching the seat post. If you stick you finger carefully in the top if the seat tube you should be able to feel any burrs, but be careful as they can be sharp. These can be gently filed or sanded away.
If the inside of the seat tube or outside of the seat post is roughened or corroded you could smooth them out with sandpaper, you'll want to be careful and use a fine grit so you don't take away too much material or cause additional scratches or roughness.
Comment: Thank you, the most easy and reasonable one! i will try it
Answer: A seatpost needs to be removed at least twice a year, cleaned and treated accordingly to its material and the frame material before re-insertion.
For the treatment: metal seatpost in metal frame, apply grease before re-inserting.
With a seatpost and/or a frame made from carbon, grease is TOTALLY out of question. Apply special carbon mounting paste which is available at a bike store or from an online seller. It avoids slippage and thus requires less torque on the clamping bolt.
Tighten with the recommended torque, with carbon components a torque wrench is recommended as it avoids over tightening and thus crushing the post or damaging the frame.
Apply a strip of tape on the seatpost where the post meets the seat-tube. It makes finding the previous position easier. On YouTube you find a number of videos that treat the subject of stuck seatposts.
Comment: Does anyome really do that twice a year?
Comment: Sadly I've discovered that this removal/reinsertion is even more important for quill handlebar stems.
Answer: The correct tool is a reamer and some good bikeshops and all framebuilders will have one.
Reaming will ensure the apeture is perfectly round (taking out misshaping from welding) and correctly sized. It will also remove burrs that scratch the seatpost.
This usually only needs doing once after manufacture.
Comment: A small hone might be better than a reamer.  A two or three block hone will self-align whereas a reamer requires the insertion angle to be correct.  Downside, it will have to be really small to fit inside a seat tube.  Only has to go 150mm / 6" down the seat tube and 100mm is probably enough for many frames.
Comment: I meant something more like https://www.aliexpress.com/i/4000459430327.html   That one should do a 19mm internal diameter, and it will self-align whereas the reamer is a fixed item to be inserted on the right angle only.    OP could also use a simple rectangle of sandpaper, centered in a slotted wooden dowel, and turned fast by a hand drill.  Definitely a case where less is more.
Comment: @Criggie https://www.mackadamfactors.co.uk/adjustable-reamer-260-295mm-2584-p.asp


What is the best way to distort the shape of this leaf image plane? (i'm a complete beginner)

Question:
I'm making a library of leaves. Once I have the image plane and all the maps assigned. What is the best method to distort the image plane to make the leaves look more natural/realistic shapes?
Comment: so all your meshes will be subdivided planes like the one we see on your picture? In that case just enable the Proportional Editing option, and move some vertices?
Answer: have not used Blender much in a long time either but hope this is what you're looking for.
I screen captured your leaf image used it as a reference to model a grid around it deleting the excess, made a material to map the image to the leaf, did a quick UV map , set displacement to bump, went into shading and changed settings to distort the surface.

rendered single leaf and it looked good , in object mode duplicated it nine times , took each leaf into edit mode > sculpt mode and used the elastic deform tool to distort the shape from flat plane.

I rendered both sides this is the bottom so it seemed to work good.
Comment: Thanks for your help, yes I looked into editing the plane in sculpt mode. I think that's the easiest way. Thanks


What does WPA2 traffic look like to a packet sniffer that is not connected to the network?

Question: Imagine there is a wireless network using WPA2, and an attacker has come along with his wireless card in monitor mode. What will the packets from that network look like? Is any information disclosed i.e Destination/Source IPs/Ports? Or is it all just encrypted? The attacker is not connected to the network.
Comment: Take a look in Wireshark: https://wiki.wireshark.org/SampleCaptures#Wifi_.2F_Wireless_LAN_captures_.2F_802.11
Answer: It doesn't ultimately matter whether the attacker is connected to the WPA2-PSK network or not: if an attacker can capture the 4-way handshake and knows the pre-shared key, it's possible to decrypt all traffic.
Theory
That said, if you just look at individual 802.11 datagrams without analysing their flow any further, they are link layer (L2) datagrams containing link layer information: four MAC addresses.

Those addresses represents the transmitter (TA), receiver (RA), source (SA) and destination (DA) addresses, but their meaning and placement in the header depends on the values of the To DS and the From DS bits in the header, as described in the document IEEE 802.11-05/0710r0:

Note that Address 1 always holds the receiver address of the intended
  receiver (or, in the case of multicast frames, receivers), and that
  Address 2 always holds the address of the station that is transmitting
  the frame.

WPA2 a.k.a. IEEE 802.11i-2004 doesn't encrypt these L2 headers, but all the information in the frame body, i.e. everything from network layer (L3) up to application layer (L7).
To address your concerns:

IP addresses are network layer (L3) information → encrypted.
TCP/UDP ports are transport layer (L4) information → encrypted.

On the other hand, there's still some information disclosed:

MAC addresses may reveal the manufacturer of the devices, which could be used for finding possibly vulnerable access points or e.g. IoT devices.
IPv4 & IPv6 multicast can be detected: although the IP headers including the multicast addresses are encrypted, the destination MAC addresses are standard and known (01-00-5E-, 33-33-).
The BSSID associates the datagram to a certain pair of AP & SSID, as an AP as a transmitter and as a receiver is distinguished solely based on the BSSID, and the SSID is typically broadcasted in the IEEE 802.11 Beacon frame. Therefore, the amount of data every device is transmitting can be associated to the network they are using, as well as the timing. E.g. if someone had an IoT washing machine we could identify, we could guess their laundry day.
Although the SSID might not be broadcasted with the beacon frames, devices connecting to the network will reveal the SSID in probe request frames. Also, computers trying to connect to saved networks will leak this information periodically even when they are not near any access points of this network. E.g. from my home I can guess several companies my neighbours are working for.

To be more exact, the data is encrypted and authenticated, but the 802.11 headers are only authenticated in both CCMP (AES) and TKIP. That is summarized in these two diagrams from 802.11i Overview, IEEE 802.11-04/0123r1:

CCMP MPDU Format

TKIP Design (1) – MPDU Format s

Practice
If you concretely want to see yourself how it looks like in detail, you could e.g.

Monitor a wireless adapter using airmon-ng. No need to connect to the network.
Capture some packets using airodump-ng.
Investigate the capture file (.cap) using Wireshark.

First without decrypting anything. (That's what you were asking.)

Configure Wireshark for decrypting the WPA2 and inspect the capture again.

Go to Preferences > Protocols > IEEE 802.11,
[x] Enable decryption
and add the PSK key for decryption.
Open the (.cap) again and let Wireshark decrypt it.
Comment: That's because you have been able to capture the 4-way handshake, too. The PSK can be used to decrypt the handshake and get the keys from it. That's explained in the first paragraph of my answer.
Comment: Really useful answer. I tried the practice you suggested and after adding the PSK to wireshark I'm able to see all the traffic, but @Pedro mentions that the packets are "encrypted with unique keys per client". How come I am able to see the traffic in plain text?
Answer:
What will the packets from that network look like?

Lots of 802.11 frames not specific to a particular network, since if you're not joined you cannot tell them apart per packet (you could filter by base station vs list of clients).

Is any information disclosed i.e Destination/Source IPs/Ports?

MAC addresses of the client and base stations, possibly SSIDs (can be forced to be revealed)... I don't think anything else is visible. Certainly no IP addresses or any identifiable network traffic in cleartext.

Or is it all just encrypted?

Yes. Encrypted with unique keys per client. Even if using WPA2-PSK.
Comment: It's possible to determine the network from the BSSID of the packet, although the possition of the BSSID on the IEEE 802.11 datagram isn't fixed (could be in address 1, 2 or 3), and bind the BSSID(s) to the correct SSID based on the IEEE 802.11 Beacon frame.
Comment: What you suggest would mean that every 802.11 datagram had information on the SSID, which it doesn't: the AP as a *transmitter* and as a *receiver* is distinguished solely based on the BSSID. Therefore: 1) every AP with the same SSID has a unique BSSID and 2) every SSID on a single AP has a unique BSSID, too. In other words, the BSSID associates the datagram both with the AP and with the SSID.
Comment: You can tell that a client is connected to a particular base station. But it could be associated to any of the SSIDs handled by that made station. Which eventually you can list given a bit of time a d a tiny bit of luck.
Comment: Also @esajokinen I would never claim I know the theory in such depth re 802.11 Vs my answer which reflects practical experience. Thanks for that primer.
Comment: Sorry I didn't mean that. SSIDs are uncovered on 4-way handshakes and on broadcasts if enabled. And of course you are right about the unique BSSIDs, I saw it earlier today and didn't notice then.
Comment: No problem – having this discussion gave me some ideas, and I was able to improve my answer with some examples. So actually: thank you, Pedro!