I am looking for a software or a way to list, extract, count, in short, analyze a book which is in epub or pdf format

Question: As someone who is interested in learning languages, I want to find a way to know the each word and their repetitions inside a book that I am going to read.
Comment: Welcome to the site! Can you clarify a little bit? Are you looking for software that you could run to generate a list of all the words in a book and how many times they are used? Also, are you looking to do this *before* you download or buy a book, or after?
Comment: Hi Ed,
As you mentioned, I want it to list the words and show how many times they are used. It is like the books which are printed for English Language learners and indicate the words that the reader will run across in the book. For now, I am interested in doing this for the books that are licensed as Creative Commons like in Gutenberg Project.
Comment: Interesting question -- I don't have an answer, but the clarification may help other users.
Comment: @Anthon: Please send me an EPUB that you would like to count the words in, I'll give it a shot. A shorter EPUB is better for testing purposes. I can't PM you so you'll have to PM me to ask for my email address. Thanks.
Comment: @Bulrush I think you are confusing me with the OP.
Answer: Here is one way to find the number of words in an epub file sorted by their frequency, with the words used the most at the top of the list.
This is done on a Mac laptop and will also work on Unix hosts.
The overview of the process:

Install Calibre
Use the ./ebook-convert command in Calibre to convert the epub file to text
Transform the entire text file to lowercase (so "Word" and "word" match)
Convert punctuation to whitespace (so "period." and "period " match)
Convert all whitespace to a new line.  This puts each word on its own line.
Exclude any blank lines from the list
Sort the list of words alphabetically
Pipe (send) that list of words through uniq -c You now have a count of how often each word appears.
Sort the result in numerical order.  If you use the sort command with the -r argument, the most frequent words are at the top.

Here's an example of steps (2) through (9).  The head command lists the top ten words in the final output.
$ ./ebook-convert ./book.epub ./book.txt
$ cat   ./book.txt | tr '[:upper:]' '[:lower:]' | tr """ " " | tr """ " " | tr "," " " | tr "." " " | tr " " "\n" | grep -v ^$ |  sort | uniq -c | sort -gr  | head
 5303 the
 1960 and
 1934 of
 1910 to
 1874 a
 1168 i
 1067 you
  844 in
  812 that
  703 it
 $

The result is pretty boring.  The word 'the" appears 5303 times, while the word 'it' appears 703 times.
I suspect in most books the most common words are the tiny conjunctions, articles, prepositions and pronouns.  Perhaps on something that is not a novel this might be more interesting.
Good luck!
Comment: Thank you for the detailed answer. As a windows user, I would like to know if all the steps you described are done in Calibre.
Comment: Step 1 is done at the OS level. Step 2 is done in Calibre from the command line.  The remaining steps are done at the Mac OS command line using  the  program named Terminal.  I know the Unix tools much better than I know the MS Windows tools. I wish you good luck!
Answer: We can achieve this using Windows PowerShell (with an example epub from
IDPF):
$book = [System.IO.File]::ReadAllBytes("$env:TEMP\cc-shared-culture.epub")
[System.Reflection.Assembly]::LoadWithPartialName('System.IO.Compression')
# The epub format is essentially a structured wrapper for zip archival
$zipStream = New-Object System.IO.Memorystream # Rather than expanding the archive to disk, let's do the vocab work in memory
$zipStream.Write($book, 0, $book.Count) | Out-Null
$zipFile = [System.IO.Compression.ZipArchive]::new($zipStream)
# The chapter files in this epub are like
$chapterFiles = $zipFile.Entries | Where-Object -Property FullName -Match "p\d+\.x?html"

# Strip remaining xml markup
function Remove-XmlMarkup {
    param (
        [System.__ComObject]$xml
    )
    # Check if $HTMLNode is HTMLFile
    if (-not (Test-IsHTMLFile $xml)) {
        Write-Verbose "Node '$xml' is not HTMLFile type; cannot continue"
        #throw exception wrong type
        throw "Get-InnerText: '$xml' is not an HTMLFile"
    }
    if($null -ne $xml -and $null -ne $xml.innerText) {
        return [string](($xml.innerText) -replace "<[^>]+>(?:[^<]+</[^>]+>)?",'')
    } else {
        return [string]::Empty
    }
}

# Create a function to recursively replace each node with its inner text
function Get-InnerText {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory)]
        [System.__ComObject] $HTMLNode
    )

    begin {
        # Check if $HTMLNode is HTMLFile
        if (-not (Test-IsHTMLFile $HTMLNode)) {
            Write-Verbose "Node '$HTMLNode' is not HTMLFile type; cannot continue"
            #throw exception wrong type
            throw "Get-InnerText: '$HTMLNode' is not an HTMLFile"
        }
        # Check recursion depthl

        if (($null -eq $recursionDepth) -or -not (Test-Path variable:\recursionDepth)) {
            New-Variable -Name recursionDepth -Value 0 -Description "Recursion depth" -Option AllScope -Visibility Public -Scope Local
        } else {
            Set-Variable -Name recursionDepth -Value $((Get-Variable -Name recursionDepth -ValueOnly) + 1) -Scope Local
        }
        Write-Verbose "Recursion depth is $recursionDepth"
    }

    process {
        #check if $HTMLNode has children
        if ($HTMLNode.hasChildNodes()) {
            Write-Verbose "Node has children to process"
            #Call this function recursively on each child
            foreach ($child in $HTMLNode.childNodes) {
                #try catch block to check for Type mismatch exception
                try {
                    #Call this function recursively on each child
                    Get-InnerText $child
                } catch {
                    Write-Verbose -Message "$($child.innerHTML) could not be splatted"
                }
            }
        } else {
            # If no children, get the inner text
            return (Remove-XmlMarkup $HTMLNode)
        }
    }
    end {
        #Reset recursion depth
        Set-Variable -Name recursionDepth -Value $((Get-Variable -Name recursionDepth -ValueOnly) -1) -Scope Local
    }
}
foreach ($chapterFile in $chapterFiles) {
    $chapter = $zipFile.GetEntry($chapterFile)
    $chapter.Open().Read(($chContentBytes = [byte[]]::new($chapter.Length)), 0, $($chapter.Length)) | Out-Null
    $chContentStr = [System.Text.Encoding]::Default.GetString($chContentBytes)
    $HTML = New-Object -Com "HTMLFile"
    $HTML.write([ref]$chContentStr) | Out-Null
    [System.Collections.Generic.Dictionary[string, int]] $wordCount = @{}
    [char[]] $delims = @(' ', "`r", "`n", '"', "'", ".", ',', "'")
    # Create an array of common English words to ignore
    $ignoreWords = @("i", "me", "my", "myself", "we", "our", "ours", "ourselves", "you", "your", "yours", "yourself", "yourselves", "he", "him", "his", "himself", "she", "her", "hers", "herself", "it", "its", "itself", "they", "them", "their", "theirs", "themselves", "what", "which", "who", "whom", "this", "that", "these", "those", "am", "is", "are", "was", "were", "be", "been", "being", "have", "has", "had", "having", "do", "does", "did", "doing", "a", "an", "the", "and", "but", "if", "or", "because", "as", "until", "while", "of", "at", "by", "for", "with", "about", "against", "between", "into", "through", "during", "before", "after", "above", "below", "to", "from", "up", "down", "in", "out", "on", "off", "over", "under", "again", "further", "then", "once", "here", "there", "when", "where", "why", "how", "all", "any", "both", "each", "few", "more", "most", "other", "some", "such", "no", "nor", "not", "only", "own", "same", "so", "than", "too", "very", "s", "t", "can", "will", "just", "don", "should", "now")
    Get-InnerText $HTML | ForEach-Object {
        $words = $_.Split($delims , @([System.StringSplitOptions]::RemoveEmptyEntries, [System.StringSplitOptions]::TrimEntries )) | Where-Object -FilterScript { ($_.Length -gt 1) -and ($_.ToLowerInvariant() -notmatch "(?:$($ignoreWords -join '|'))")}
        $words.ForEach( {
                $count = 0
                if ($wordCount.TryGetValue($_, [ref]$count)) {
                    $wordCount[$_] = ++$count
                }
                else {
                    $wordCount.Add($_, 0)
                }
            })
    }
}

The result is in $wordCount.
Answer: As English learner maybe you will be also interested in https://github.com/artemtomchuck/MyKnownWordsDictionary project.
This extracts unique words from your book (straight txt should be an input, but it is not a problem to transform your pdf or epub into txt).
Then it filters out the word you already know (based on your own dictionary).
Then it prints new words from book ordered by frequency (the most frequent on the top) with their translation to your native language (given that book is written in English or any other language which you are learning).
Notes:

in output csv file frequency of word is not printed, but if you understand the source code then you can include this frequency column into your output (currently it is just hidden).
this mechanism is very simplistic and does not recognize plural/singular forms, names or words with "-" symbol (e.g. single-minded is treated like two separate words: single and minded)

Full disclosure: this is my personal project and it may not fully suit your purposes or may be complex for using/setup by non-technical people (non-software engineers). But I tried to document setup as best as I can in github.
The reason why I am posting the answer here: initially I looked some solutions from answers in this question. I found the solution in https://ebooks.stackexchange.com/a/6557 , but it was quite a routine for regular usage and that's why I automated process a bit.


How to setup wifi on raspberry pi with username and password on Android Things using adb shell

Question: I have a raspberry pi running an older version of Android things that is capable of connecting to my wifi. I copied and inserted the wpa_supplicant file from that pi to another pi with an updated version of android things but the new pi is unable to connect to the wifi. Am I missing something here?
Answer: There is a guide on how to do this.
In the adb shell you can execute this command:
<code>am startservice \
-n com.google.wifisetup/.WifiSetupService \
-a WifiSetupService.Connect \
-e ssid my-ssid \
-e passphrase my-passphrase
</code>


Making KFC-like chicken

Question: I brine my chicken and use only salt, pepper and garlic powder in bleached flour to taste. I think this comes close to the Colonel's. Can anyone taste other spices in the original recipe?
Comment: We  have a rule where we support people trying to recreate a dish they had in a restaurant, even if they seem to be worded like recipe requests. I added the correct tag and reworded the title.
Comment: Try Google. Many many people have attempted this. Another ingredient is suggested to be MSG. Note: you may successfully duplicate the flavor, but duplicating the crust and texture will be difficult because the Colonel uses pressure fryers. Not many home cooks have that.
Answer: It's possible that the breading/spice mixture has been revealed, according to an article in the Chicago Tribune. They also provide a full recipe here, though as far as I can tell, the solid part is the spice mixture, and they're filling in the rest of the recipe.
They say it seems to be a good match... with the addition of MSG:

But more important, did it taste like the Colonel's secret blend of herbs and spices? It came very close, yet something was still missing. That's when a reporter grabbed a small container of the MSG flavor-enhancer Accent (how did that get in the test kitchen?) and sprinkled it on a piece of the fried chicken. That did the trick. Our chicken was virtually indistinguishable from the batch bought at KFC. (Does KFC add MSG? A KFC spokesperson confirms that it does use it in the Original Recipe chicken.)

The full story: a nephew of Colonel Sanders spoke with a reporter and showed him a recipe he claimed was the original recipe, then later downplayed it:

"That is the original 11 herbs and spices that were supposed to be so secretive," he says with conviction.
(In a subsequent phone interview with a Tribune editor, Ledington dialed back his certainty and expressed reluctance about sharing a recipe that — if it's legit — ranks among corporate America's most closely guarded secrets. "It could be; I don't know for sure," he said about the handwritten list of ingredients, adding that this was the first time he'd shown it to a reporter. "I've only had that album for four years, since my sister passed away.")

KFC's official responses were of a "neither confirm nor deny" flavor, so it's pretty hard to say if this is real or not, but it certainly could be. The clearest they said was:

"Lots of people through the years have claimed to discover or figure out the secret recipe, but no one's ever been right."

The handwritten recipe:

Transcribed:

11 Spices - Mix With 2 Cups White Fl.
1) 2/3 Ts salt
  2) 1/2 Ts Thyme
  3) 1/2 Ts Basil
  4) 1/3 Ts Origino
  5) 1 Ts Celery Salt
  6) 1 Ts Black Pepper
  7) 1 Ts Dried Mustard
  8) 4 Ts Paprika
  9) 2 Ts Garlic Salt
  10) 1 Ts Ground Ginger
  11) 3 Ts White Pepper
Answer: Most of the other spices are difficult to impossible to detect. I've gotten as far as paprika, onion powder and rosemary for sure. I think there is parsley powder involved as well. There is another couple that I'm close to identifying, but no luck yet.
I want to learn what to put together, too, because I love KFC but my body can't handle it, so I want to learn how to make the coating for my own foods.
Comment: Interesting that my answer was marked down, considering the question: "Can anyone taste other spices in the original recipe?" I would be interested in hearing the reasoning behind the downvote. Just curious. I'll happily delete if the explanation makes sense.
Answer: Here is a recipe, referenced by @Jefromi, as it was printed in the article by Joe Gray in The Chicago Tribune on August 23, 2016 (which I've not tested):
Makes: 4 servings
2 cups all-purpose flour
⅔ Tablespoon salt
½ Tablespoon dried thyme leaves
½ Tablespoon dried basil leaves
⅓ Tablespoon dried oregano leaves
1 Tablespoon celery salt
1 Tablespoon ground black pepper
1 Tablespoon dried mustard
4 Tablespoons paprika
2 Tablespoons garlic salt
1 Tablespoon ground ginger
3 Tablespoons ground white pepper
1 cup buttermilk
1 egg, beaten
1 chicken, cut up, the breast pieces cut in half for more even frying
Canola oil
Mix the flour in a bowl with all the herbs and spices; set aside. Mix the buttermilk and egg together in a separate bowl until combined. Soak the chicken in the buttermilk mixture at room temperature, 20-30 minutes. Remove chicken from the buttermilk, allowing excess to drip off. Dip the chicken pieces in the herb-spice-flour mixture to coat all sides, shaking off excess. Allow to sit on a rack over a baking sheet, 20 minutes.
Meanwhile, heat about 3 inches of the oil in a large Dutch oven (or similar heavy pot with high sides) over medium-high heat to 350 degrees. (Use a deep-frying thermometer to check the temperature.) When temperature is reached, lower the heat to medium to maintain it at 350. Fry 3 or 4 pieces at a time, being careful not to crowd the pot. Fry until medium golden brown, turning once, 15-18 minutes. Transfer chicken pieces to a baking sheet covered with paper towels. Allow the oil to return to temperature before adding more chicken. Repeat with remaining chicken.

Making KFC-like chicken


Approximation scheme for Multiple Choice Knapsack

Question: The paper _Fast Approximation Algorithms for Knapsack Problems (E. Lawler, 1979) gives an FPTAS (fully polynomial time approximation scheme) for the multiple choice knapsack problem (MKP). But MKP is strongly NP-hard, so it cannot have an FPTAS unless P=NP. Is the result of this paper wrong?
Answer: The paper you cite is not available to me. However, according to this paper: "Improved Fully Polynomial time Approximation Scheme for the 0-1
Multiple-choice Knapsack Problem",

0-1 MCKP is $\text{NP}$-hard [4] as it contains the 0-1 Knapsack problem as a special case [5, 13] but it can be solved in pseudo-polynomial time (emphasis added) through dynamic programming [7, 2].

Thus, 0-1 MCKP is not strongly $\text{NP}$-hard.
One of the citations is
[7] K. Dudzinski, S. Walukiewicz, "Exact Methods for the Knapsack Problem and its Generalizations", European Journal of Operational Research, (28), (1987): 3–21.


best licensing option for freest possible sharing of lyrics while retaining the right to get paid if someone else makes money from them

Question: I would like to share many song/poetry/lyric works.  I would like to share them as freely as possible, allowing modifications, use of small pieces or lines or concepts, etc..  I want to encourage commercial use of the material, but I would like to have the right to get a cut of any commercial proceeds.  I have perused the Creative Commons licenses and it seems the choice is between allowing commercial use with no compensation, or simply not allowing it.  Is there some sort of licensing choice that lies somewhere in between?
Comment: You can dual-license. For example, use CC-BY-NC for non-commercial use, and some other license for commercial use. The choice of which license(s) to offer is up to you as the copyright holder. Just be careful about derived works, as those would presumably be under the CC-BY-NC license offer from you, which can open a can of worms.
Comment: I'm of the opinion we allow professional questions for creators, and songwriters fall into our purview. IMHO, asking about licensing is on-topic and of interest to professions. (@rolfedh, et all - if this is something we aren't agreed on, I'd be happy to open a Meta topic - let me know what you think.)
Comment: @Standback I've withdrawn my close vote.
Comment: @Standback - I think this falls under the meta thread about legal questions, meaning it's currently on-topic.  Unless there's something newer that my searching missed. But this is an awfully broad policy and it's from over five years ago.
Answer: If you license something, you're giving people the freedom to use it under certain conditions (eg, for noncommercial purposes). That doesn't mean someone who wants to use it for commercial purposes can't do so—but they'll have to contact you first and negotiate a separate licence.
If you want, you could add an explicit commercial licence, but you'd have to state the prices and terms, and they should work for everyone, from the old lady who wants to use your poem in her home-printed poetry journal going to 90 subscribers, as well as for Disney who want to use your poem in their latest blockbuster.
Comment: You could simply say, "If you wish to licence this work for commercial use, contact..." and give your contact info. That way, the person wanting to use it commercially knows where to go.

Every case will be different. A common situation is that someone likes your poem and wants to use it on a web page which earns a small income in advertising. In cases like this, sharing income is usually more fuss than it's worth to administer and you're better off giving a free licence to get exposure.
Comment: So, perhaps the actual licensing language is limited in what it can convey, and one could include some sort of statement with their work - something like "although the license under which this material is published does not allow for commercial uses, I wish to encourage commercial uses, or attempts at commercial usage.  If income is generated from the use of the material, I would like to receive a fair share of revenues."
Answer: I hope this provides you with an idea of how this works.
My production company provides event photography which we currently shoot and provide digital photos under a Creative Commons License CC BY-NC-SA 4.0
We used to keep ALL RIGHTS RESERVED but that meant clients broke the license by sharing photos online. So we changed it rather than risk our rights by not enforcing them.
So our licensing page now states:
<code>Some event photography by Dreamspinner Media is licensed under Creative
Commons BY-NC-SA, your booking agreement will state if this is the case
with a section like this below.

Creative Commons License
Event photographs by Paul Zagoridis/Dreamspinner Media is licensed
under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0
International License.

Permissions beyond the scope of this license are available at
http://dreamspinnermedia.com/photography/licensing.

Contact us using the comment form below for additional licensing enquiries.
</code>
Editors feel free to obfuscate the url if it breaches the self-promotion guidelines of this site. It's in there to show that you MUST have contact details in your CC license.
So what that means is the photos can be used, changed and rereleased provided there is

attribution to the source (me)
the derived works are similarly licensed and attributed
Non-commercial

So if a photograph is used by a media outlet, they should obtain an appropriate license from me. If they don't they can't claim they just grabbed it from the public domain.
Answer: When you wrote the lyrics down, you established a copyright on those lyrics. Nobody can legally use them for commercial purposes without your permission. That is what copyright means. They have to get a license from you, you don't have to give one to them.
So the Creative Commons license that seems to you to be missing is not needed because that is the default license. That is the one that existed before Creative Commons. The one that everybody already has.
The Creative Commons licenses give you other options that not everybody needs. It sounds like you probably don't need them.
So just publish your work in such a way that it includes a copyright notice and also contact info so that the reader can contact you for a commercial use license.
Answer: You could license your lyrics for a percentage royalty (instead of say, for a flat fee).
That is to say, if the licensee gets nothing, you get nothing, but if some licensee "gets lucky," you get your percentage. That's only fair.
Comment: The problem with percentage royalty is that it gets complicated when the work gets incorporated as a minor part in a larger work. For example, I create an anthology collection with 200 works for 1% royalty each and release the whole as a combined work. Then I would pay twice as much in royalties than I have revenue, no matter how much I charge. Well, you could say "each work is just 0.5% of the whole, so each one gets 0.005%", but that doesn't represent that some works might be more important for the whole work than others.
Comment: @Philipp: That's not a perfect system. But better than nothing.
Answer: I have a practical problem with much of music licensing.
I sing with a Barbershop chorus, and with a couple of quartets.  All of this is "non-profit", but it is not "non-commercial".  We get paid for gigs, which give us a percentage of our yearly operating revenue.
I don't have a problem with buying music, or paying BMI when we have a show, or paying ASCAP an annual fee for using music -- actually we do all of there to have rights to sing almost anything.
If you have a song we want to arrange to sing, I need to know who to contact.   I need to know what your procedure is for granting a license to arrange.  When I have the arrangement, I need to know who to pay when I make copies of that arrangement for my singers.  I need to know if the performance fees are included in the fees we pay to BMI and ASCAP, or if we have an additional liability to you.
This can be a maze that is hard for a volunteer, non-profit, but not non-commercial organization to work through.
I hope this gives you a framework within which you can find an answer that will serve your need, and ultimately the needs of thousands of small performers everywhere.


How do I get Photoshop to support RAWs from FZ2000?

Question: I previously thought that RAW format is just a one format and any Ps version works with it. But now I know that different RAW formats from newer cameras are not supported by the Ps version that exists at the time the cameras is released.
Anyway, I found a table which what ACR and LR versions support this specific camera's RW2 files are mentioned. (It says ACR 9.8 & LR 6.8 support RW2s from the FZ2000, right?)
So, how can I know about what Photoshop version that supports FZ2000's RW2s? And, is the ACR (in this case, version 9.8) integrated in it?; do I need to get the ACR (9.8) separately?..
Answer: If your camera needs Adobe Camera Raw 9.8, then you need Photoshop CC [any version]
CS6 & earlier do not support ACR 9.8
Ref: Adobe KB: Legacy version update information
However, there is a workaround for earlier versions -
From the Adobe KB: Photoshop or Lightroom doesn't support my camera

What if I own an older version of Photoshop or Lightroom?
You do not have to buy a new version of Photoshop or Lightroom just to
  get the latest camera support.
Adobe provides backwards compatibility for the latest cameras for free
  in Photoshop CS, CS2, CS3, CS4, CS5, and CS6 as well as Lightroom 1,
  2, 3, 4, and 5, through the Adobe DNG Converter. With this tool,
  you can easily convert camera-specific raw files from supported
  cameras to a more universal DNG raw file.
Comment: Just be aware that there are a few caveats to converting image files in other raw formats to DNG. If all you ever use to edit images are Adobe products you'll likely never notice, but if you think you might ever want to use other editing tools you might want to preserve your raw files in the original format as well.
Answer: This link lists which versions of Adobe products work with which versions of Adobe Camera Raw. It is included in this answer to the question How do I tell if ACR will work with my camera and how do I get the correct version for Photoshop?
If you choose to use Adobe DNG Convertor to convert your raw files to a DNG version supported by an older version of Photoshop, be aware that the conversion may remove some of the functionality and flexibility you would have if you used a newer version of ACR/LR/PS to work with the raw files directly. This will probably be most noticeable with regard to color and noise profiles for newer cameras used with older versions of ACR/LR/PS. Again, it all depends on which camera (and more generally, which manufacturer) the raw files are coming from.


Where did the value of 0.1uF for bypass capacitors come from?

Question: Nearly everyone recommends 0.1uF for bypass capacitors. Why this value? I assume there is no harm to using larger values so is it merely a "sensible minimum"? And if so why do people go for the minimum rather than using higher values - it seems to me you can get higher values at no extra cost.
Comment: Even though as state higher value capacitors can be purchased at the same value, the frequency response of higher value capacitor is narrower than lower value capacitors, see http://electronics.stackexchange.com/questions/59325/whats-the-use-of-a-decoupling-capacitor-near-a-reservoir-capacitor/59329#59329
Answer: Higher value capacitors will not be as effective at dealing with high frequency current drawn by the chip. Above a certain frequency a capacitor will start to behave like an inductor. The value where its characteristic changes is the series self resonance of the device: -

Thus, you'll find that on microwave devices 100pF capacitors are also present as decoupling along with the bulk capacitors. Here's an example of three capacitors decoupling an FPGA: -

The black curve is the composite impedance of all three capacitors used. Taken from here.

Where did the value of 0.1uF for bypass capacitors come from?

It's a good compromise between bulk and high frequency capacitance BUT if you are designing radios your default decoupler may be 10nF or 1nF (UHF). If you are designing really high speed digital stuff you may also use 2 or 3 different values in parallel like in the FPGA picture above.
Comment: @Golaz - exactly - look at the 2nd graph in my answer - it shows a precise representation of three capacitors and bear in mind that a pcb track may have an inductance of 1nH per mm.
Comment: Note closely the antiresonant peaks in the black curve in Andy's graph -- they are why it's better to use several identical capacitors in parallel than to parallel different value capacitors.  (Of course, Ott explains it all quite well in Electromagnetic Compatibility Engineering...)
Comment: Could you explain why they would start to behave like inductors? Is it because at higher frequencies their impedance would lower untill the equivalent series inductance takes over?
Comment: I am sorry, but this answer and the diagrams in it is mostly based on wastly outdated information from the 90s.
The high frequency response of capacitors has nothing to do with the capacitor value, and everything to do with the capacitor package.

Today, you can get 10µF ceramics in 0603 or even 0402 package. It is completely pointless to connect 100nF caps in paralell with a 10µF cap of the same physical size.
See this for a much more up-to-date answer, including modern diagrams:
https://electronics.stackexchange.com/questions/327975/capacitance-vs-frequency-graph-of-ceramic-capacitors
Comment: In reality it's a nice round number, which is why it's so overhelmingly popular. Some people say you should match the resonant frequency with your IC fundamental frequency e.g. µCU clock speed. Other people say it makes the circuit switch faster and creates more high-frequency EMI. I believe the latter is wrong because sharp(er) edges are far beyond the fundamental frequency. Datasheet resonant frequency disregards vias and traces so in reality you'd have to experiment to get capacitance just right. Then there's the combination of the two, ~1µF "bulk" cap nearby and <100nF immediately close
Answer: Not everyone recommends 0.1uF as a decoupling capacitor, though it's a good starting point for 74HC and single gate logic. Kevegaro's answer here is a good one.
For example, for Xilinx FPGAs here is one recommendation for bypass capacitors:

They recommend 33 capacitors of three different values per device.
Comment: Odd recommendation with the three values -- having a reservoir cap per rail and then at least one 0.1uF per *power pin* would make more sense than trying to economize on caps at the expense of risking a failure due to antiresonant peaks, especially for a programmable device where the clocks may be well...anything!
Comment: Also this brings up another question I have: Why do they recommend to use multiple different values? Is it just because it is impossible to get the 100uF capacitors close enough to the device? Edit: Never mind, Andy's answer answers this.
Comment: Yes, Andy answers this quite thoroughly!
Answer: Andy's explanation is beautiful and in-depth. If you find it difficult to grasp, it may help you to visualize how decoupling works in simple terms. In your mind imagine a 3D view of your board, it has a load (ICs, etc) and a power source. The load may suddenly "request" more current from the power supply however it takes time for current from the supply to reach the load over the trace distance and trace resistance. Also the built-in resistance of the supply itself or time for a switching supply to detect the new current demand and adjust (supply bandwidth) is a factor. In short, a power supply does not supply current instantly, it takes time. So imagine the load sending a message to the power supply it needs more power and the supply then ramping up to more current and sending that current out along the delivery route (trace or power plane).
As the load is waiting for the current to arrive it has no choice but to pull the voltage down to compensate for the "missing" current. It has to obey the law V=IR, the load decreased it's resistance (R) to "indicate" it needs more power, there was no more current immediately available so I stays the same, so V has to decrease to compensate.
So how do we solve that? We put little capacitors close to the load. These capacitors are little "charge banks" that the load can quickly withdraw from during excess demand, quicker than waiting for the current to come out of the supply. Why is it faster? Because the distance between the capacitor and load is shorter, and because the built-in resistance of a capacitor is much smaller than a power supply.  If "I" is immediately available then "V" does not need to compensate - everyone is happy.
Although much faster than power supplies, capacitors also take time to "discharge" and provide power to the load in proportion to their internal resistance which increases with capacity (farads). So in short, larger capacitors take longer to supply the needed current. So you want to choose a bypass capacitor that is fast enough to respond to the load, but also holds enough charge to fill the demand while current from the power supply travels to the load.
<code>So where did the value of 0.1uF for bypass capacitors come from?
</code>
As previously mentioned, for common logic it was a good trade-off between response time and capacity requirements of the bypass caps to the load demands. You could get out the calculator and find out exactly what the best value is but there are also Bill of Materials costs to consider. If you tune each bypass capacitor to it's load you'll end up with many more line items on your BOM and it will get costly very quick! 0.1uF for most logic circuits or for high speed circuits 0.01uF is usually a good choice. Save money in your BOM where you can within the limits of the application.
For loads that frequently change current demand (high frequency loads) there are other ways of getting around the response time versus capacity problem of bypass capacitors. You can:

Use a better power regulator with higher bandwidth so it doesnt take so long to get power from source to load.
Put two capacitors in parallel. Two resistors in parallel decrease total resistance and it is no different with internal resistances of capacitors. Therefor the combined capacitors have increased capacity and increased response time!
You could use parallel caps of different capacity, big buddy and little buddy. So one might be 0.01uF and another 0.1uF. The first having quick response and the second lagging a bit in response but providing current for a longer duration.
You can also distribute capacitance in your circuit but not necessarily at the load point. This charge reservoir response is faster than the source supply so you can then use smaller bypass capacitors at the load knowing that your distributed charge reservoirs will pickup the slack in the supply.

This is a simplified view of everything. There are more factors especially in high-speed circuits. But if you can imagine the basic electric principles at play in your circuit as a dynamic system of supply and demand a lot of the "best practices" we read about become common sense. A simpler analogy might be Amazon's supply chain. Their goal: supply items as quick as possible anywhere in the US. Their solution, warehouses close to every city, less response time getting items out of the warehouse and in the truck. Next is drone delivery. It's a logistical battle of supply and demand and trade-off over response time and capacity versus size of each distribution node and costs!
A really good video from EEVBlog on factors for parallel capacitors:
https://www.youtube.com/watch?v=wwANKw36Mjw
Comment: Just to follow up briefly on your suggestions: 1. is helpful for cutting down on reservoir capacitance needs albeit not quick enough to deal with the spikes produced by most digital switching, 2. is a very good one, especially when scaled to 10 or 20 caps for a device instead of 2 or 3 (for large chips, my rule of thumb is 1 100nF cap per power pin), 3. isn't so great because of antiresonant spikes that can create noise peaks on your board (look closely at Andy's graph again!), and 4. is a surprisingly good suggestion (look up "Buried Capacitance" technology for an exceptional example)
Comment: Good simple answer, though from Andy's answer it seems like it is actually the inductance that is the limiting factor, not resistance.
Comment: Yes. I didn't touch on inductance in my analogy but it definitely matters. Really, I should be using the word impedance in place of resistance above since that is a factor of the inductance, resistance and frequency...resistance just sounds simpler to people. Resistance being 0Hz resistance and impedance being resistance when at a certain frequency.
Answer: The recommendation to use multiple values, such as 100nF + 10µF, is from the 90s and 80s when 100nF was the highest readily available ceramic capacitor with decent high frequency response.
The 10µF capacitor would be an electrolytic or tantalum capacitor with poor high frequency behavior.
That has changed completely today. Now you can easily buy 10µF ceramics in 0603 or even 0402 packages.
For ceramic capacitors, the high frequency response has nothing to do with the capacitor value, and everything to do with the package size of the capacitor.
With modern capacitors, it is usually pointless to connect a 100nF in parallel with a 10µF.
You can easily see in the diagram below that modern high value ceramic capacitors are just as good as low value capacitors for high frequencies, as long as the package size is the same.
(The small negative dips are the resonance frequencies. You do not want to rely on the resonance frequency for decoupling capacitors, so those dips should be ignored)

(Image source: Analog Dialogue Sep 2005 - A Practical Guide to High-Speed Printed-Circuit-Board Layout)


My static blocks are not displaying correctly anymore

Question: My static blocks are not displaying within my widget correctly anymore.  They use to display nicely from top to bottom, but now are displaying horizontally instead of vertically.  Basically, I have a widget named "Main Top", and I was displaying 4 static blocks within that widget.  Now if I only display one block within that widget all is fine, but as soon as I add more than one, boom its horizontal placement again.  Any help on this would be much appreciated!
Comment: ok I actually figured it out.  Some how the layout on large devices got messed up.  So i just scrolled each block over to fit 12, and now it is displaying correctly.
Answer: You just need to drag each block over until it reads "12" instead of "4"


Magento2 admin: Send Welcome Email From

Question: I've got a multisite installation: 2 websites: "Europe" and "Asia". Europe has 2 stores: "France" and "Germany". And "Asia" has only one store: "China".
"France" store as one storeview: "French". "Germany" has one storeview: "German". And "China" has 2 storviews: "Chinese" and "English".
The problem is when a new customer account is created in frontend in "Chinese" storeview, I've got in the backend site in "Account Information" tab "Send Welcome Email from "French" and if I change it to "Chinese" it does not change after saving the modification and the welcome email is sent to the customer in french language. Also "Associted to website" is "China" so it is ok. And in "Customer view" tab I've got "account created in chinese" which is ok. The same issue occures if the customer is created in "German storeview" or in "English". The system always displays in "Account Information" tab: "Send Welcome Email from "French".
Can someone help me please
Thanks
Answer: I noticed that when I change the default website to Europe then the email is sent in french even if the customer registers in Chinese storeview and needs to get the welcome email in chinese
Why is that?

Magento2 admin: Send Welcome Email From


What Makes a $W$-Boson a Gauge Boson?

Question: I understand that in beta decay, a quark changes its flavor and emits a $W$-Boson, and this $W$ (is the W Boson virtual, by the way?) quickly decays into an electron and an electron antineutrino or positron and neutrino. What makes the $W$-Boson a gauge boson? I don't see any fundamental force being exchanged, and the W seems to act as an intermediate particle.
Answer: Processes in elementary particle physics are described by Quantum Field Theory (QFT).  In this theory the fields (eletromagnetic force, weak force etc.) are quantized, so they are presented by particles. The attraction, but also repulsion between particles, even decays are represented by the exchange of a gauge boson.  Physicists use for this purpose the famous Feynman diagrams (they are very suggestive diagrams, but actually one has to be careful about the interpretation of these diagrams, each part of such a diagram corresponds to mostly complicated mathematical expressions). So the attraction respectively the repulsion of 2 charged particles is represented by the exchange of photons, and the beta decay is represented by the exchange of another boson, the W-boson.
They are called gauge-bosons since their corresponding force respects some local symmetry --- actually it is a freedom  (which is called gauge freedom)  in the choice of the corresponding potentials of the force -- in case of a photon it is called U(1), and in case of the W-boson it is SU(2).
But, I think the main message to keep here is that forces between elementary particles are described in QFT by the exchange of a particle, actually always bosons (excluding here SUSY). One can make a little computational exercise (you can look it up in the book of Zee, "QFT in a nutshell" in the first chapter)  and show that this approach (exchange of a particle)  leads to a decrease (increase in case of repulsion) of the energy of in the interaction participating particles, which is nothing else as a felt force.
Comment: I want to sincerely thank you for your answer. Usually people downvote my questions, knitpick my questions and give me answers that I cannot understand.
Comment: @Indigo2003 You are welcome.
Comment: Two questions though. First, is the W boson exchanged between two particles? I realize that the W is emitted by a quark in the nucleon, but is it received by another particle? In all other fundamental interaction, gauge bosons are exchanged between two particles, i.e. electromagnetic repulsion and gravitation. Also, What is gauge freedom?
Comment: In $\beta$-decay we have $n\rightarrow p + \overline\nu_e + e^-$,  in QFT (Feynman diagrams) the anti-neutrino is considered as an incoming (time backwards running, because it's an antiparticle) neutrino which upon absorption of the $W^-$ turns into an electron. Gauge freedom expresses the observation that potentials (electric magnetic e.g.) can be varied to some extent without changing the fields. If you have further questions, please create another post on SE.
Comment: @Indigo2003 Before Feynman diagrams, there were time-ordered diagrams, where one diagram has the $q$ emit a W that then decayed, while another had a W spawn a virtual $e\nu$ pair and then get absorbed by the quark. All this was possible because energy wasn't conserved at vertices (uncertainty principle). Feynman unified these into 1 diagram that conserved 4-momentum at all vertices, and here the qW vertex and We$\nu$ vertex have space like separation (the virtual W has negative invariant mass squared). The point: don't fixate on time ordering, it's just an "interaction".
Answer: Weak force is a force because it can also cause a push and a pull. It is just not the main part of the weak force. The main part of the weak force is the ability for quarks and lepton to change flavors. Force part of the weak force is for the most part enforced by the Z boson.

What Makes a $W$-Boson a Gauge Boson?


Convolution Theorem in Physics

Question: I'm getting ready for my classes to start next semester in Grad school, and I'm reading over Fourier Transforms and their applications. I came across the Convolution Theorem, namely, that if we have a convolution $$ \int_{-\infty}^\infty f(x)g(z-x)\, dx = h(z) ,$$ then the Fourier transform $${\hat f}[f(x)g(x)] = \left[{1 \over {\sqrt{2\pi}}}\right]{\hat f(k)}\ast{\hat g(k)}$$ of the product of $f(x)$ and $g(x)$ essentially equals the individual convolutions of the same functions in their "frequency" interpretation. The actual convolution theorem states $$\widehat {h(k)} = \sqrt{2\pi}{\hat f(k)}{\hat g(k)}$$ My question is:
 What's so important about these results? What's their significance when I am actually in a laboratory, taking measurements?
Comment: I think this question is *too broad* - you essentially ask for a list of cases where the convolution theorem is applied in physics.
Comment: If you look at Sergei Patiakin's answer, He didn't use any specific example, but answered my question exactly how I intended.
Comment: Actually, if you read the question, I am interested in the significance of the above-mentioned theorems. It has nothing to do with "broadness". Perhaps you should re-read the quedtion.
Comment: What is the "signficance" of a mathematical fact in physics if not the situations where it is applied?
Comment: @KnowledgeEnthusiast13, if that is the case, you should mark it as accepted.
Answer:
Fourier transforms occur very often in most fields of physics.
Products of functions occur very often in most fields of physics.

As a consequence of points 1 and 2, it is common to encounter Fourier transforms of products when manipulating an algebraic expression. To move forward algebraically, you would need to apply the convolution rule.
Comment: Yes, it is used in derivations just like other mathematical identities. I would compare it to the calculus identity for the derivative of products: `(fg)' = f'g + fg'`. When first learning this identity in a calculus class, it also seems of limited value. However, when you realise how often derivatives and products occur together in real-life algebraic manipulations, you see the value.
Comment: Ah, so it can be used as simply a way to advance the algebra of a problem?
Comment: @SergeiPatiakin Can you give a specific example in physics where convolution theorem can become useful?
Answer: A measurement typically involves the convolution of the thing being measured with the response function of the instrument. Now if the Fourier Transform of your response function has zeros in it, the convolution theorem tells you that information at the corresponding frequencies will be destroyed by the measurement process
There are many possible examples of this - but I will give just one. If you have an optical system (lens), the diffraction limit is often given as $\frac{1.22 \lambda}{d}$; this is actually the location of the first null in the response function, which is actually a Bessel function (Airy's disk). If your original image has a repeating pattern at exactly the frequency corresponding to that null, it will be destroyed. Higher and lower frequencies will be present (but attenuated to different extent).
So the convolution theorem helps you see how the frequency content of a measurement is affected by the measurement system. This can be quite useful in gaining a deeper understanding of your setup.
Answer: A process that is linear and shift invariant can be described by a convolution integral. As an example, consider the scalar diffraction of light, which can be computed with the Fresnel diffraction integral. Scalar diffraction is a linear shift invariant process. That's why the Fresnel diffraction integral is a convolution integral.
Answer: You ask about laboratory experiments involving convolution. A common example is measuring the time profile of some event when the process by which the event is triggered has a time profile similar to the duration of the event itself. For example measuring the decay of an exited species when excited by a laser pulse. A common experiment in biophysics and chemical physics is measuring fluorescence decays by time correlated photon counting.
The measured decay is distorted by the convolution of the exciting pulse with the true decay. The two can be untangled by re-convolution and so fitting by non-linear least squares to a model of the decay process with the measured excitation pulse profile.(The alternative, which is deconvolution, seldom works well in practice).  When using Fourier transforms to do the convolution it is important to have equal (say zero) signal at the start and end of the data set since the Fourier transform assumes a repeating signal and any discontinuity here distorts the data.
Another example is the distortion of spectral lines by the finite width of slits in a spectrograph.
(Note that although convolution is fast using digital fast fourier transform algorithms, convolution can still be performed numerically using the integral over the finite range of the experiment.)

Convolution Theorem in Physics


Schock absorbers misting

Question: My car has a schock absorbers misting report on a test
Audi a3 built 2009
Now my question  is could this case a bumpy ride I.e when the car rides of bumpy road,  the drive feels very rough
Best regards
Comment: Welcome to Motor Vehicle Maintenance & Repair! What do you mean by "misting"?
Answer: Yes this could well be the cause of your issues with the ride quality - if the shocks are "misting" that means they are losing oil (although not at a huge rate) and as the shock absorber's oil level goes down so does their damping performance.
It might not be the only factor causing it - there's likely to be wear to other suspension components as well but it's certainly a big factor.

Schock absorbers misting


Can wireless communciation be synchronous?

Question: I understand that in synchronous communication, the sender and receiver need a common clock. Is it possible that wireless communication be synchronous? Can some common clocking element be there for such purpose?
Answer: Yes. Kind of, at least.
As you're coming from a wired background, I'll build the analogy from there:
Where UART only works because  receiver and transmitter clocks are similar enough so that, for a short burst of bits, they don't drift apart, the same applies to wireless digital communications.
If your symbol rate is low enough, and the receiver knows exactly the symbol rate the transmitter uses, then the receiver can extract the symbols without first running logic to do clock recovery.
In high-rate, mobile and streaming systems, generally, this can't be assumed: no two oscillators in this universe are exactly the same, and hence, when you transmit many symbols, you must make very sure that the receiver has the same sample clock as the transmitter.
Now, if you tried to take the equivalent of SPI to the wireless domain:

Data signal
Square wave clock signal

you'll notice that the square wave clock signal has a really bad spectral shape – it has infinite bandwidth, and even if you accept some "rounding" on the edges, you'd still need about 5 to 7 times the actual data signal bandwidth to transport your square wave.
Hence, that is generally not done.
I'm sure earlier wireless comms had some sort of secondary carrier that was used to derive a symbol clock, but I haven't seen that in any modern standards.
You can either go what I'd call (and this is a term I just invented) the "synchronized asynchronous" route:

send a preamble of known signal, which allows the receiver to estimate its own rate relative to the rate of the transmitter, and just work from there for the duration of a burst

or the "continuous clock recovery control loop" way.
The second one is really done in a lot of different ways, depending on what system you're looking at, and how complex the designers could afford to make the receiver.
One very typical scheme is that you realize that all digital communications are in essence pulse-shaped.
Without having the time to go fully into that: You really can't send infinitely short pulses with amplitude +1, -1, -1, +1, -1, +1 ... over a channel of finite bandwidth.
Hence, you apply a pulse shape, which serves to smooth the transition between these; the idea is that still, at the exact symbol times, the values are exactly the symbols you wanted to send, but in between, there's a smooth, bandwidth-limited exchange.
You'll already recognize this if you've worked with wired buses: you know the eye diagram. The exact same diagram is used in wireless communications, though, typically, for a good short-run wired communication, you'd expect the eye to be nearly square, whereas the pulse shaping with a rounder shape is intentional (albeit necessary, too) from the beginning in wireless comms.

This, very geometrically, implies that at the exact right times, your signal "shape" has extrema, i.e. places where your derivative is 0.
You can now build a mechanism that looks at the slope of the signal at the times you assume your symbol times to be. If that slope is negative, oh, we're too late, better sample a little earlier, if it's positive, sample a little later. Notice that this isn't the case for all symbol transitions (same-symbol transitions usually do not have the maximum amplitude at the correct sampling time), but it's the case for most transitions, usually.
Do some minimal statistics, and you can make this adjustable to a (small) symbol rate error.
So, we wireless comms folks invest bandwidth that we could be using to transmit information (which is what we're getting paid for) into making the symbol rate synchronizable. It's not a direct equivalent to a "synchronous bus" in the wired world, because aside from a few especially strange systems that I'm sure exist (dear reader, if you know one, let me know in the comments), we'll be sure to avoid having a separate symbol clock carrier. But it's essentially the same idea: having a way of shoving the information about when the symbols should be sampled into the receiver.
Comment: Your "synchronized asynchronous" is clock recovery - ethernet and all sorts of wired protocols have that.
Comment: I think your answer needs to address Manchester encoding. That's how it is done; I have never heard of a second carrier for clock.
Comment: @Lundin I must admit that I can't think of a good reason to do Manchester encoding instead of properly pulse shaped communications; that just doubles the bandwidth. I don't know *any* modern standard that does Manchester, would you have a reference for me?
Comment: But to give you a rough calculation, @user2813274: Let's presume a ca 200 OFDM symbols WiFi frame. For OFDM like in Wifi, you get slightly more than \$\frac1\pi\$ times the relative rate error in ICI amplitude. That means that if you have a 64-carrier system (like 20 MHz WiFi) and have a sampling rate offset of say 1/256 of the sampling rate, you get \$\frac1{12}\$ of your signal's amplitude as interference amplitude from your neighbor. That'll not distort BPSK much, but will make 64-QAM unlikely and 256-QAM impossible to use.
Comment: Roughly how accurate of a sample clock would you need for a particular bandwidth? I.e. if both systems have a GPS & thus GPS-level time accuracy, would that be sufficient for wifi communications, or is it orders of magnitude too far apart?
Comment: Wifi comes with its own preambles and hence doesn't need GPS synchro – you'd never build a WiFi receiver that relies on GPS timing instead of the timing recovered from the signal itself.
Comment: If I recall rightly, Manchester Coding is/was used on tape recording of high value high-speed data, where the high speed led to tape sticktion between the tape and the tape heads, which produced a jerky motion  and thus lots of very fast and short-term timing jitter.
Comment: @analogsystemsrf ah cool! Not quite wireless, but I see where that is coming from.
Answer:
I understand that in synchronous communication, the sender and
  receiver need a common clock. Is it possible that wireless
  communication be synchronous? Can some common clocking element be
  there for such purpose?

In regular wired-up comms a common-clock can be achieved without resorting to applying a seperate clock wire. I'm thinking here of Manchester encoding: -

Data and clock are combined with an Exclusive-OR gate to produce a single signal that can be decoded without resorting to a seperate clock wire. It's a signal that carries both clock information and data together simultaneously.
Given that this is now a single (combined) signal, it makes it very suitable to be transmitted as a radio wave (with suitable modulation techniques).
Answer: GSM uses carefully-tweaked (tweaked in realtime, in each subscriber handset) 13MHz oscillators, to avoid drifting of the start and stop times of the GSM voice/data packets.
Thus GSM need not worry about packet collision and retry.
=======  regarding telemetry from rocket/missile testing
NASA, and its precursor organizations, developed various "coding" methods, with definitions standardized under the IRIG Inter Range Instrumentation Group. Some of these patterns have long runs of 111111s or 000000000s with no clocking information, and the ground-based phase-lock-loops recover the data just fine ---- without any parallel radio/wireless channel needed for clocks; there is very little timing jitter between a missile and the ground antenna. To handle hundreds of sensors on the missile, all multiplexed into a serial data stream, a special SYNCH_WORD pattern is inserted once a frame.
To function, such a downlink has this behavior
1) sweep the frequency span expected to cover the unavoidable Doppler shifts, while testing each RF carrier for identifying patterns (the expected bit rate)
2) once the proper bit rate is found, then pursue a phase-locking to the bit-transitions; this is slow in most cases because the PLL has NARROW bandwidth to avoid easy breaking phase-lock due to noise bursts; or the initial lock can be done broadband, and then the loop bandwidth severely tightened down, to where the Doppler shifts are just barely accommodated (this tracking of Doppler may require a higher-order control loop)
3) once we have a bit-lock, the telemetry system needs to find "start of frame", so the first sensor's data and the 2nd sensor's data, etc, can be correctly extracted from the serial bit stream; this may take a while, because the telemetry system MUST BE CERTAIN, and thus tests the bit stream for the expected SPECIAL bit-pattern over and over. Incorrect frame lock means all the data is useless.
Note the various "synchronous" approaches:
a) the telemetry system picks the correct RF channel
b) the telemetry system locks to, thus becoming synchronous with, the bit rate
c) the telemetry system locks to, thus becoming synchronous with, the start of Frame
As the PLUTO probe transmitted data to earth, after passing PLUTO and grabbing many photos and other sensor data, the downlink data rate was about 100 bits per second, with the RF carrier in the 8GHz range.
As the earth rotated, the 3 NASA DeepSpace 70 meter antennas each went thru this process of "acquisition" and then received that 100 bit datastream for the next 8 hours, all occurring synchronously.
The NASA systems were locked: RF, bit, frame.
============= history ================
Why was IRIG defined? because FM telemetry needs about 20--25 dB SignalNoiseRatio for clean data to plot on those chart-recorders.
Whereas digital data (even without error-correction) works well at 10dB (or 7dB, depending on how your bandwidth is defined) SNR. At about 0.1% error rate.
With finite transmitter RF power on a missile-under-test, the aerospace projects literally could not get telemetry from missiles that exited the atmostphere, unless just a few SLOW sensors were used. Unacceptable.
Dropping the SNR from 27dB to 7dB, a 20dB difference, and given the Range^2 effect of RF energy dispersion, the aerospace companies suddenly had 10X the range, even without error-detect-correct.
Importance of telemetry: the Soviets used 320,000 sensors on the final (still it exploded!) launch of the N1. Prior 3 launches only used 700 sensors.
Comment: @pjc50: A sudden doppler of a deep space probe is probably a catastrophic event.
Comment: That implies it could take a long time to lock, and would be vulnerable do unlocking in the event of sudden doppler - was it all done "live", or was some of this done "retroactively" by recording a stream and then re-parsing it until correct framing was achieved?
Answer: Yes, it is done by merging the clock and payload data signal into one (wireless) channel.
Examples are Manchester code or Pulse Position Modulation. In both cases (starting the) clock recovery at receiver side (e.g. by synchronizing a PLL) is often simplified by using a distinct preamble in the header of a data frame.
One application where wireless PPM is used for example is Secondary surveillance Radar (ADS-B etc.).
An oscillogram of a ADS-B frame is shown here.
Answer: Normally systems which recover the clock from a single channel are called "asynchronous", like UARTs, while "synchronous" systems require multiple channels. So I disagree with the claims that using Manchester encoding or similar is "synchronous".
In radio systems, even if you use multiple channels, it is difficult to ensure that the signals arrive at the same time, or even with a reliable skew, because there may be diffraction or multipath effects involved. Doppler effect may also skew your results.
GSM systems are time-slot based (TDMA), but as far as I understand it the central clock is just used to control which mobile equipment is allowed to transmit in any one timeslot - it does not determine bit boundaries.


mRNA Transcription from Nuclear DNA

Question: How does a cell "know" the coding strand vs. the non-coding strand of DNA during transcription of mRNA?
Answer: This is determined by the promoter -- the DNA sequence to which RNA polymerase binds. It's assymetric, and the polymerase binds in a specific orientation, that determines the strand and the direction it will go once elongation begins.
Answer: Strands in DNA double helix (and other nucleic acids) are anti-parallel: one strand in the helix is in 5'->3' direction while another is in 3'->5' direction. RNA polymerase can only synthesize RNA in 5'->3' direction, and so it uses the 3'->5' strand as the template (the second DNA strand is, I think, completely ignored by the polymerase during synthesis).
From one promoter transcription of both forward and reverse genomic strands can theoretically happen. However, the two possible nascent RNA strands are synthesized outward from the promoter and away from each other (i.e. they are not overlapping).
During evolution position of some promoters (but definitely not all of the promoters in the genome) have evolved to be in the place were at least one of the two theoretically possible RNA products can be translated. Transcription in the opposite direction (i.e. away from the gene) actually frequently happens in eukaryotes. These RNAs belong to a class of promoter associated RNA, but but they were not shown to be translated (their function is largely unknown).
UPDATE (2015/02/19): New publication in Molecular Cell by Duttke et al. is demonstrating that genic promoters are "intrinsically directional" and, just as @Leon Avery commented here before, the divergent (i.e. bidirectional) promoters are actually two unidirectional promoters side-by-side, in reverse orientation relative to each other. So my above statement "From one promoter transcription of both forward and reverse genomic strand can theoretically happen" is incorrect. However, the authors also say that approximately half of gene promoter regions in HeLa possess this bidirectional initiation property, so it is still an important feature to keep in mind when thinking of transcription of eukaryotic genes.
Comment: Sorry for not emphasizing this importnat point: a gene promoter typically leads to syntesis of mRNA transcripts in one direction from one strand only. However, other RNA transcripts, generally referred to as promoter associated RNA, were show to be syntesized at promoters in both directions. Promoter associated RNA was shown to be up to a few hunderd bases long, most frequently just about 18 bases long (I think the 2008 Science paper from [Lis lab](http://www.ncbi.nlm.nih.gov/pubmed/19056941) was one of the first papers on this phenomenon, but there was other important research into it since).
Comment: In short for me: You say that transcription runs into both directions and both strands are produced as mRNA? If not, please edit your question, at the moment this can be read into it.

mRNA Transcription from Nuclear DNA


Is it bad form to cold-email professors to do post-baccalaureate research?

Question: Suppose I am interested in doing more research in a particular area in order to strengthen PhD applications, but I have just finished my undergrad degree.
Is it considered bad form to cold-email professors who say on their websites that they are accepting undergrad researchers in their lab asking to do research this summer/next year, though I'm no longer an undergrad?
Comment: Its like the email version of 'cold-call' - meaning to initiate contact with someone who does not already know you.
Comment: What does "cold email" mean?
Comment: What is bacc? ----------
Answer: If you are not an undergrad, you cannot be hired as an undergrad. Most undergrads are unpaid (don't work for free!), and of those that are paid, many are paid using funds that can only go to enrolled students (the percentage varies by field). Besides, most school-year undergrad positions are on the order of 10 hours/week, although some summer positions can be full-time.
According to @J.., sometimes new grads can be hired for full-time summer internships in a transitory capacity. Usually, this would be people continuing to work in labs they have already started with, or professors hiring someone they're familiar with. I find it unlikely a professor would hire someone new for only a few months, and if the opportunities were available, they'd likely to be posted.
Now that you have your degree, apply for jobs.
Cold emailing isn't a good strategy for that (especially not in the current climate), mostly because budgets usually don't allow to hire as many people as you need, much less to have money laying around waiting for someone to ask for it. Apply for posted jobs and work your network.
Comment: @Chad Some immoral professors would be happy to let you labor for no compensation. Don't do it. You should absolutely not offer to work for free because it devalues the jobs you should be applying for (why pay $45k+benefits for an RA when I can get someone wealthy enough to work for free), and forces out undergraduates who should be taking those positions. In some places, it may be illegal.
Comment: @GrayLiterature I disagree, for the reasons I stated.
Comment: Great, thanks for this information. Supposing I did get a full time job elsewhere, though, would professors have a problem with me offering to do unpaid work (say 10 hr/wk) just for the experience/to get my name on a paper?
Comment: This isn't always true.  Lots of professors will pick up undergrads for four month (term) employment over summer months for research project/work-experience type positions.  It's not uncommon for these to be full-time and paid.
Comment: @AzorAhai True, but I've also known new grads to pick up such positions, either while finalizing grad school arangements (or as a prelude-to with same prof), or just for a few months of experience while they are job hunting.
Comment: I think that it is fine for OP to offer to volunteer for positions as long as they have made the free, informed, and consensual decision to do so. Unfortunately, the world is competitive in any market place. OP will need to compete against others who might have been in a more privileged position to do undergraduate research. Whether it's "immoral" for a prof to allow someone to get volunteer experience is, I don't believe, relevant to the OP getting the experience they want. Of course, if one does volunteer, they must set their own boundaries to avoid being taken advantage of.
Comment: @AzorAhai To be fair, in such cases it's also been my experience that the professors in question typically post these positions when they're available, so it's unlikely to be something that comes out of a cold-call.  If the professor has positions like this available they will typically be made public and OP will simply make a bad impression if they've not bothered to discover this for themselves.
Comment: @J... The OP has graduated, but it's a fair point about summer programs.
Comment: @J... Gotcha, guess I haven't heard of that practice, but I'll note it.
Answer: I actually think 'cold calling' is not so bad if you do it the right way. Actually, for most researchers it is vital. Let me explain by talking about some right and wrong ways to 'cold call' academics:
The wrong way

Write poorly worded or grammatically incorrect pleas for supervision/experience/paid work. EVERY academic and researcher gets these - and we all hate them;
Get their name/title/role/institution/department/gender wrong. Speaks for itself, doesn't it?;
Being too transactional (i.e. I need work/I need money/I need experience/I need supervision/gimme this/gimme that/ etc). It may sound harsh, but they have their own problems to deal with! They typically don't really care less, particularly if they don't know you from a bar of soap.

The right way
Some of these may sound obvious, but I'm pretty confident that if you manage to do at least more than one of the following you will get a positive response inviting you to follow-up every time:

Focus on your interests. For example: They work in a lab doing X. Hey guess what, you wrote a paper on X in second year and got a distinction and you have maintained an interest in X ever since. You are keen to build on... (etc etc). You'd really like to LEARN MORE about X, Y and Z and are considering working toward a PhD in that area;
Demonstrate your knowledge. For example: 'I see you are working with X proteins [or whatever], I'm really interested in how the Z process works with those...';
Make an effort. Show that you're willing to make an effort to develop 1 and 2 above. This may involve you travelling some distance for a tour of the lab or offering to meet/zoom/skype in a place/time/platform that suits their crazy schedule rather than yours.

[Bonus tips]

Flatter their ego.* Yea I know. You shouldn't have to. But these people have probably dedicated their lives to what they are doing. A little recognition from time to time wouldn't be so bad, right? Trust me - this one works. Young, old, male, female, intersex, etc; EVERY academic will read and respond to your email if you tell them that you have read and enjoyed their paper on Using Z Processes with X Proteins [or whatever].
[* Edit - note sensible comments re going over the top with flattery below]
Use conferences. This is tricky during a pandemic - but if you can get yourself to a conference or seminar of some kind where you know they will be there then strike when they are vulnerable! Time your move during breaks when they are heading for the pastries and NOT in the middle of a conversation where they are securing a deal for more funding for their lab. Give it a try... it works ;-)

Why do these things work?
Here's my take on at least a few reasons why these work:

They were in your situation once too, probably;
These days academics are always on the lookout for good students. More students means they can grow their lab and get more PhD completions over time, which counts as a performance metric for them;
Cheap labour. Let's face it, that's definitely part of the equation. If you look like you can be an asset for their research project and you won't cost a fortune in terms of time, funding, training or stress, then you are basically in.

Anyway, try these out and see how you go.  And good luck!
Comment: Some good advice: wrong/ and right way 1-3. Some bad one: 4! Don't flatter their ego. A smart prof can distinguish whether you are genuinely interested in their topic (good!) or just padding vacuous compliments to flatter them (bad!). 5. is so-so. Conferences are good to meet people - again interest in topics matters most. When the prof heads for the pastries, do not stand between them and the food, it's almost as bad as standing between them and their funding; profs do not eat as much as PhD students, so if they want to eat, they are hungry ;-)
Comment: Agree with most of this, but - be very careful with the degree of flattery on the first email. No more than would justify your reaching out to them, e.g. "I understand one of your research foci is XYZ" is good, "I understand you are one of the prominent researchers of XYZ" or is acceptable flattery, but don't go beyond this level (e.g. "supreme authority", "world's leading" etc.)
Comment: I agree with both of these comments - its actually worse if you overdo #4 above. Handle with care!
Answer: Some professors have money available to hire recent grads as research assistants, but this is a bit more difficult than hiring an enrolled undergrad (as other answers mention, hiring students at the university/college is cheaper and requires less administrative work).
Whether cold emailing is bad form is pretty subjective, and I think it is not. Keep your email short (professors are busy), polite, and clear about what your interests/timescales/constraints are (are you interested in working on research full time? are you flexible about being paid?). If they don't have something available for you, thank them for their consideration and move on.
Comment: -1 Do not be "flexible about being paid."


Why? My website is not showing "cache option" in search result

Question: Plz guide me. Why? my website is not showing "cache option" in search result.  when I check Ex- (Cache: http://example.com ) then it shows -
"404. That's an error."  URLs are indexed & no any issues in search console.   We dropped my ranking due to it. so plz
Comment: We've been getting several questions about this lately, so it seems you're not the only one. Please see the linked questions and answer below.
Answer: Google does not have a cached copy of the url (not website).
Google takes a snapshot of the url as of when Google last crawled it (source).
Not having one means either:

there is no cached version of the url; or
there is only one cached version (the current one) so a drop down is redundant.

Keep in mind that while you may have made various changes to the url, that does not mean that Google has cached all the changes - typical with 'younger' sites.
Hope that helps.


Were American, Australian, and New Zealand English dialects ever spoken in Britain before the colonization of these lands?

Question: Were American, Australian, and New Zealand English dialects ever spoken in Britain before the colonization of these lands?
Comment: The *current* American, Australian, and NZ English were never spoken *anywhere*, and neither was the current British English, for that matter. AmE and BrE have both diverged from their common ancestor. In different directions, but by the same amount. See [Where do accents and dialects come from?](http://english.stackexchange.com/questions/5815/) and the related questions linked from there.
Comment: One would naturally expect not. Dialects evolve within geographies by modifying upon what was brought in from elsewhere. That's just how I'd see it, though. See also, [linguistics.se]
Comment: Do you mean to ask if the colonial accents come from particular regional accents of the original British Isles, or some combination of them?
Comment: @Mitch, yes, but I was ignorant of the fact that dialects can change in such a short time.
Comment: Adipro, OK. But then to (only possibly) have your question reopened, you should probably reword without that assumption. As it stands now your question sounds like you're saying Was there an accent called 'Australian' in England, before the English colonized Australia, which is of course absurd. Reword you question to avoid this and maybe it is reopenable. Actually it might still be too broad then (too many accents to cover, needs a book to answer)
Comment: @Mitch, I am not sure if I understand the difference between the former question and the latter one, which you say is absurd. Why is it absurd? But I think the current answers have answered my question.
Answer: Languages change. Otherwise, we'd still be speaking like Chaucer. The British settlement of America started in the 17th century; there has been lots of time since then for several different American dialects to develop. The British settlement of Australia and New Zealand started over 100 years later, which is why these dialects are closer to those spoken in Britain.
There are aspects of American and Australian dialects which were brought over from England, but which have since been dropped from British English. And there are aspects of American and Australian dialects which developed in the colonies.
Comment: "brought over from England" would not be "American (or) Australian dialects" anyway. Some aspects may have survived in America/ Australia that did not in England.
Answer: The majority of the people who populated Australia and New Zealand (and English South Africa) didn't speak whatever was the RP English of its time anyway (Kentish perhaps), they spoke all different dialects of English including Irish, Scottish, Northern, Western and probably even Brummie.
Imagine all those people stuck together having to converse with each other on a daily basis, they will reach some sort of common dialect over time.
Here's a link to a large number of quite lengthy audio clips of many different British accents
http://www.bl.uk/learning/langlit/sounds/
There is some excellent information about* each accent.
*If you listen to the Lerwick Shetland track you can hear 'aboot' for about, Geordies also do that but not Canadians — they say 'aboat'.


Keyframing orthographic camera scale and reproducing it as an action?

Question: I have keyframed the scale of the camera in my scene so that it always does kind of a zoom in and zoom out to an object at the beginning of the scene.  Using the logic bricks I have established an "always" sensor with an "action" controller linking that specific action but it isn't working!  I have read this post and tried it but still is not working. Is it even posible to do it in the BGE? I am using v 2.72
Answer: Animating the orthographic scale the way you do it is not possible. You can however do this via python using bge.logic.getCurrentScene().objects["Your_Camera"].ortho_scale = your_value. Your_Camera has to be the name of the camera whose orthographic scale you want to adjust. your_value has to be the desired focal length. That can be a float between 0.01 and 4000.0. Following is an example of zooming in to a value of 5.0. Therefore the orthographic scale at the start needs to be above 5.0.
import bge

#getting the camera object called 'Camera'
cam = bge.logic.getCurrentScene().objects["Camera"]

#while the orthografic scale is above 5.0 the script reduces it by 0.5 everytime the script controller is activated
if cam.ortho_scale > 5.0:
    cam.ortho_scale -= 0.5

The script activated every logic tick (always sensor -> true level triggering) looks like this (it has a better frame rate of course). The starting orthographic scale is 23.0:

Now you can write a script that does what you want. Here's another example where we are zooming in first and when this has finished we are zooming out again.
import bge

#getting the camera object called 'Camera'
cam = bge.logic.getCurrentScene().objects["Camera"]

#while the orthografic scale is above 5.0 the script reduces it by 0.5 everytime the script controller is activated, if there's a game property with the name 'zoomed_in' it won't zoom in again
if cam.ortho_scale > 5.0 and not "zoomed_in" in cam:
    cam.ortho_scale -= 0.5
#if the cam is already zoomed in and the ortho_scale at a value of 5.0 the camera is zooming out again, adds the game property 'zoomed_in' to the camera to prevent it from zooming in again
elif cam.ortho_scale < 23.0:
    cam["zoomed_in"] = True
    cam.ortho_scale += 0.5

Here's the result:
Comment: thank you so much for taking time to provide such a complete answer!  May I ask how it was possible for you to came to this result? Is there a good reference on how to achieve stuff like this?
Comment: The reference for coding stuff I use the most is bgepython.tutorialsforblender3d.com there you have almost everything you need. For example knowing that I want to modify the camera's orthographic scale I select 'camera' and look for the fitting attribute. In this case it's 'ortho_scale' . Now you have to know what to do with the information you get there. And that's something you can only learn :D


How can I insert a hanger bolt into wood table leg?

Question: I am attaching table legs to table top with Waddell Metal Table Straight Top Plate. How can I insert a 5/16-18 X 3 in Hanger Bolt into wood table leg?
Comment: Welcome to DIYSE. The problem you're facing isn't clear. Hanger bolts have wood threads on one end. You drill and twist them in. Can you edit to clarify the question?
Answer:
How can I insert a ... Hanger Bolt into wood

Plan A
I would drill a pilot hole in the wood and drive the screw-threaded part of the hanger bolt into the wood by using a spanner/wrench/socket on a double nut on the bolt-threaded portion (the second nut tightened against the first nut using a pair of spanners/wrenches to thus lock both nuts in position on the bolt). Then unlock the nuts and remove them.
You can then screw the legs into the top-plate.
Plan B
I suspect you could also do this by attaching the top plate to the table top, turning the top upside down, inserting the bolts into the top plates and then screwing the pre-drilled legs onto the screw. Using lubrication/strap-wrench/clamp as needed.


Understanding the over sampling factor effects before a Matched Filter?

Question: I am trying to design a matched filter at the receiver side to a half sine pulse shaping function sent by the transmitter of an OQPSK transceiver. I am not sure I understand the effect of varying the over sampling factor of the ADC right before the matched filter (the minimum is twice the symbol rate). I think it may have something to do with the quality of the timing synchronization. I'm using an early-late gate synchronization algorithm, which needs at least 3 samples per symbol. I've also read in literature that the over sampling is typically a factor of 8 for QPSK systems. Does it have other effects, maybe the BER of the system?
Comment: Please see my answer to the question here: https://dsp.stackexchange.com/questions/40259/what-are-advantages-of-having-higher-sampling-rate-of-a-signal/40261#40261 I think that may help you understand some of the trade space with oversampling.
Answer: Perhaps the answer to a similar question may be helpful.
I've spent most of my career in SONAR, so I want to see a lot of oversampling After a matched filter.
A matched filter should be sampled at the exact time corresponding to it's delay, which is unknown but can be within a range of time that is often called a range gate. Any filter output that corresponds to a maximum deflection inside of that range gate is a possible return.  The nearer those samples are in time the more likely a match in delay will occur, and the nearer to the actual time, the greater the expected deflection. So, in this case matched filter miss match can be mitigated with a higher sample rate.
The hard part is calculating the performance because the range gate is moving in time so there are several chances to pick a maximum that is a signal and several chances to produce a false alarm and the tests are on overlapping data, so one's tests are not independent.  The detector also has memory because the probability of a threshold exceeding is different if the previous test exceeded the threshold or did't exceed a threshold.  It does not produce simple closed form expressions.
So oversampling should increase performance but showing by how much, is typically done by simulation, and results will depend on the details of the filter.  Doppler introduces more complication.
Hopefully, the answer to a question you didn't ask may help.


Can I bypass solenoid clutch switch by grounding the pin directly with a wire to the chassis/frame?

Question: My solenoid switch has 4 pin for an Honda motorbike. One of the primary pin to be energized for the solenoid to click is the clutch switch.
My question is: can I connect the pin with a wire that i'm going to connect directly to the ground (chassis/frame) of the motorbike?
This is the clutch diagram: https://pasteboard.co/HvLasSh.png
The clutch passes from a diode that i want to skip/bypass
Thanks, regards
Answer: It does depend how it is wired, it can be the clutch switch grounds or earths the pin or it feeds it with a supply, make sure you know which it does as you could blow a fuse or start a fire...
You need to work out what is happening - a multimeter can help or consult the wiring diagram.
Comment: Hi Mike, this is the diagram for the clutch switch: https://pasteboard.co/HvLasSh.png , it passes trough a diode which I want to completely bypass, and always show as a closed circuit.
Comment: Add relevant info, diagrams etc to the original question - people don't trawl through comments building your question...
Comment: Read the diagram, follow the wires and work out the function. The image you have posted is a small, barely legible, section of the complete diagram...
Comment: Hi Mike, sure, i will, this diagram is taken from a very bad quality black and white scanned old service manual of Honda XL Degree 250 found on the net, that's why i'm having trouble even to read the letter for the colors.
Comment: The go with the multi-meter, operate the switches and see what happens where...
Comment: Hi Mike, i tested with the multi-meter the solenoid and find out the configuration and which pin are the primary with longer resistance and the secondary one with shorter resistance, i also tested the solenoid "click" by connecting the clutch pin to one side of the battery and the "starter push button pin" to the other side and it works well as it clicks, now i would like to know how to replicate this with the wiring. I would like to connect the clutch pin with a wire (Perhaps with an inline fuse holder) directly to ground. Would that work in your opinion?


Is it illegal for an employer to know that their employees are lying to the companies customers for their own personal financial gain?

Question: An employer is aware that their employees are lying to customers of the company for their own personal financial gain. Is that against the law? If an employer is aware and continues to let the employees do so.
Comment: If you want a sensible answer, you will have to include the jurisdiction.
Comment: @TimLymington Fraud is bad regardless of the jurisdiction. The problem I see with the OP's question is the lack of context, since it is impossible to conjecture whether the employees' lies amount to fraud or akin torts whereby unlawful harm is inflicted on customers or third parties.
Answer: If the lies amount to fraud, and if the company also benefits from the fraud, the company might be held to be a party to the fraud. Even if the company does not benefit, the company, or more likely the responsible executives who had actual knowledge of the fraud, might be held to be accomplices in the fraud, if the fraud is treated as a criminal matter. Most fraud, of course, is treated as a tort, not a crime.
A lie would only constitute fraud if it was:

Intentional: the person telling the lie knew that it was false, intended it to be believed, and to affect the target's decisions;
Materiel: a reasonable person would be significantly influenced in whether to take an action by the content of the false statement;
Damaging: actual harm occurred as a result to the person defrauded relying on the false statement; and
Reasonable: it was reasonable for the target of the false statement to accept it and rely on it. If the statement was implausible, or if any reasonable person would check it, it is not reasonable to rely on it.

Only if all of those are proved in court will a lie be treated as a fraud.
Answer: So I'm a car salesman. Customer comes and says he's a Man United fan and would never buy a car from someone who isn't a fan as well. I lie by saying I'm a fan as well, I sell the car and make personal gain in form of a commission. Absolutely legal.
Comment: Dubiously legal (by the definition of fraud) and not really what OP was asking about; if your manager overhears you, should he (according to the law) stop you?


Coping with crash fear?

Question: This will be my first season racing collegiate this spring and I recently crashed on a group ride going about 22 mph around a corner and slammed my face into the ground. Crash was pretty bad and had road rash across my body and tons of stitches in my face. Before the crash, I had gotten really good at cornering at high speeds and was comfortable in doing so, however, now, I can't even take an easy corner at any speed without being way too wide and being fearful of falling. Any tips on overcoming my fear so I can get back into the crit-racing mindset?
Comment: Try mountain biking (I crash about once per ride)?
Comment: Do you understand why you crashed? Figuring out what happened, and what to look out for so you can anticipate, avoid or ameliorate it in future, is generally helpful.
Comment: Wear a mountain bike helmet
Answer: Some people never do.
In races, people push to the limit, and sometimes past it. And people make mistakes. So there will always be crashes.
It's part of race craft to learn to read the corners and your competitors, to know when to make sure you are ahead or inside the risky riders.
At this early stage of your career many of the riders you are riding against are enthusiastic learners. Some strong athletes can't cope with the risks and drop out of Crit racing or ride at the back to build condition for other kinds of race.
You don't say anything about how the crash happened. It might sound unkind, but you don't have to fall when someone bumps into you, or falls in front of you.
Keep at it. Try riding at the back for a while to watch and learn. And watch other higher grade races.
Comment: Great answer... I laughed at riding at the back of a crit to "avoid" risk.  So many do it, the irony is so thick you can cut it with a knife.
Comment: To support this answer. Crashing is a part of riding bikes. And there is no way around it. Right of passage in biking is when you have broken a collar bone once or multiple times - happens to everyone including pros. One comment I will make is this: every discipline of biking benefits the other. I race Downhill Mountain biking but riding road, XC, and even BMX benefits me greatly.  You can learn a lot about bike handling if you sample and learn a little from all forms of bike riding.
Answer: This seems like a question of how to get you mojo back.   As eloquently put by andy256, some never get over being spooked.
I know the feeling well.  Even a near misses can spook.  I can't tell you how frightened I was years back lining back up to restart a race, after seeing a number of people I trained with being carried away on a stretchers (the course was also well known for sending people to the hospital). I can still taste that fear today. In the end I was able to focus through the fear and perform.
I have also known people have come back from breaking their neck in a crit (I remember seeing it happen right in front of me) and somehow still retained their edge.
So in short, it can be done, but it can take work (like anything).  At some point you need to learn how develop a psychological switch, where when you flip the switch on you are in kill mode ready to kill or be killed. When you flip the switch off you are in normal mode.
Some things I found useful...
Each person is different but here is what worked for me.

Take some time and do some easy rides, no pressures, no club rides or pack rides. Just have fun.
Once you start to feel confident coming back reintroduce some pack riding then harder corners.  This should hopefully work at regaining your confidence.
In off time to do positive visualizations. Visualized yourself nailing that fast double apex turn.  Focus on all the components, from the feeling of the wind to the sound of the tires.
Learn how to become at peace with consequences (injury or even death). These are part of life whether or not you are on the bike.
Make an active decision that you want to win and are willing to risk consequences to do so.
When you hit 5 you are probably ready to come back in full form.

While you are spooked you may want to avoid crits as you you will either be pushed around and kicked out the back in no time (further deflating your confidence) or you will be a danger to yourself and others (depending on the level of competition).
What you are going through is normal.  If you didn't have concerns, then that would be concerning!  Focus and risk are tricky subjects for many.  Some I suspect perform well because the simply don't have a full cognitive understand the risks they are taking.
Comment: Excellent answer, in so many ways. The part about training partners going off from a notorious course is gold - it *can* be hard to balance known and unknown risks in the heat of the moment. But *a danger to yourself and others* deserves lots of emphasis. Coming back too early almost guarantees more trouble.
Answer: Crits are scary as hell. Road racing in general is scary, but crits are the scariest of the scary. I've seen some gnarly crashes and both as a racer and a spectator, and you've experienced one yourself, so kudos for wanting to get back on the saddle.

Do a postmortem: Sometimes crashes happen so fast that it seems like one moment everything is fine and then next instant you're on your back staring up at the sky. But try to think back to the instant before it happened. What caused the crash? Did you come into the corner too hot? Was the corner off-camber? Did you lap someone's wheel in front of you? Did you get your bars tangled up with someone else's? Did you panic and hit the front brake? Was your body positioning off when you entered the corner? Try to replay the moments leading up to the crash in slow motion again and again in your head and figure out how you ended up crashing in the first place. This will aid you greatly in avoiding the same mistake in the future.
Know your bike's limits: Understand what your bike is capable of, especially your tires. On a mountain bike, you can (and should!) flirt with the breaking point of traction on a regular basis, as it's typically easy to recover. This is not so on a road bike. When skinny wheel's cut, they cut fast and they rarely regain traction. Don't push your road bike tires that far. Road bike tires don't really drift or plow, so if you feel like you're pushing them too far, you probably are. In fact, if you feel like you are pushing them too far, you definitely are because your mind is already working against you, which leads me to the next point...
Know your own limits: Understand what you're capable of physically and mentally. Know your own skill level and don't jump straight into the deep end. Work your way up slowly, push your boundaries, but don't try to go smashing through them Leeroy Jenkins-style. An increase in your personal skill level will come naturally over time if you push your bounds within reason. Pushing too hard leads to bigger mistakes, which leads to crashes, which leads to big setbacks.
Baby steps: Take your time and keep at it. Find a loop that you're comfortable with or get to know one well. Ride it often and slowly push yourself to ride it more aggressively, working on your turns especially since that's what's got you spooked. Start out solo, then ride with a friend, then a couple of friends. When you feel comfortable to do so, start doing some of the organized B group rides. When that starts to feel good, do some A group rides. Go back to racing only when you're comfortable doing so. I've always found that I'm way more scared of the idea of a crit than I am when I'm actually racing in one, so keep that in mind as well.
Be persistent: It will come back if you keep at it. Don't give up. The crash was a learning experience. Find the silver lining in it and keep on riding.
Get out of Cat 5 if you're in it: They don't call it "Crash 5" for nothing. If you're racing as a Category 5 road racer, get 10 non-TT races under your belt and request an upgrade to Cat 4 through USA Cycling. You don't need to win or even do well, you just need to finish.
Answer: Wear something like this:
The MET Parachute is similar to the the Giro switchblade, which used to be the most popular helmet on the market in Western Canada for MTB, because it was a full face helmet that weighted as much as an open face helmet. The switchblade saved my face when I smashed in into the sidewalk after slipping off the top of a highway divider on a hill. It would look a bit goofy on a road bike, but you could wear it just until you felt confident to ride open face again.

Removable face guard:
Comment: I just remembered that I use a picture of my switchblade helmet as my avatar photo.
Answer: One drill that might be useful is to find a small hill with a relatively safe open view corner that you are currently braking while descending. Do repeats on descending the hill, each time braking later and later.
Ideally this will be a turn that confident riders can descend w/o braking. Almost every team/club has a test turn somewhere that is part of their drills.
Following a rider that you trust that is a better descender than you is also a good way to build confidence.
Answer: Here are my steps to mental recovery after a crash:

learn as much as you can about the crash
stop thinking about it
introduce (or re-introduce) the fun factor into your rides
done

Do not bother yourself with how much the transition from 3 to 4 will take. Thinking about it will make it longer. When you reach 4 you'll find yourself stronger, more confident and having more fun than before the crash.
Good luck.

Coping with crash fear?