Write data from a dstream to parquet

Question: After consuming data from kinesis using pyspark, I have a dstream with entries like this:
('filename_1', [{'name': 'test'}, {'name': 'more'}, {'name': 'other'}])
('filename_2', [{'age': 15}, {'age': 25}])

What I want to do now is to write the second part of the tuple to the location identified by the first part of the tuple.
At other places I did that by converting each list of dictionaries to a DataFrame using:
dataframe = sqlContext.createDataFrame(list_of_dicts)

and writing it with something like:
dataframe.write.parquet('filename')

My problem is now how to turn every line in the dstream into a DataFrame. My intuition was to use map to get every single line and do the conversion. This would need a sqlContext which you can actually not pass to a map function, as it fails with this error:
Exception: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transforamtion. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063

I'm not absolutely tied to parquet, but I need some kind of schema (hence the detour to DataFrame). Is there a way to do that with spark?
Answer: You can create new instance of SqlContext inside foreach method.
words.foreachRDD(
  new Function2<JavaRDD<String>, Time, Void>() {
    @Override
    public Void call(JavaRDD<String> rdd, Time time) {
      SQLContext sqlContext = JavaSQLContextSingleton.getInstance(rdd.context());

For more detail you can go through this link.
Comment: Ok so try this, First you get all the rdds from dstream using `foreachRdd` function and then collect all lines as a list from each rdds using `collect function`. now you have all the lines, convert each line to rdd and then create dataframe using that rdd with your custom schema.
Comment: As far as I understand, each RDD might still consist of multiple lines (at least that's why I think there is the map a bit later in the example you linked to), so I still cannot put a complete RDD into one dataframe as each line should be its own dataframe.
Comment: you can't use sqlContext inside the map. it only works inside foreachRdd function. Have you go through that link?
Comment: Yes, that is exactly the problem. Each RDD may still contain more than one row, but as each row is supposed to become one DataFrame I end up with a construct like this:
`for name in names:`

`        filtered = rdd.filter(lambda name_record: filter_by_name(name, name_record))`

`        df = sqlc.createDataFrame(records)`

`        df.write.parquet(OUT_PATH + name, mode='append')`

Which is less than optimal.
Comment: Sorry for the malformated code. I'm not allowed to edit this post anymore, here is the code in a more readable format:
http://codetidy.com/6893/

Write data from a dstream to parquet


T8 LED Burn-in Process

Question: Direct Wire 4ft T8 were purchased and installed.  Is there a rule of thumb or recommended process to burn-in the bulbs?
Is there a good test or process to weed out 'infant mortality?'
Answer: Most modern lamps do not need a burn in period. You can just install them. As far as "infant mortality" I know of no field test other than install them and if you get a bad one it will usually burn out in 30 days or less. After that they shouldn't go bad for quite a while.
Comment: I agree, I have installed hundreds of direct wire lamps both t8 and t5. I have a few fail but the ones that did fail did so immediately or within a few minutes, I used to burn them in for an hour but now I just verify that all the lamps light I have had more problems with tombstones on older fixtures than anything else..
Comment: The takeaway is that burn-in is unnecessary and that infant mortality is within 30 days?

T8 LED Burn-in Process


Why I'm not getting the params in activate method?

Question: Working more on my previous example, I've got three classes. First the parent:
import { Router, RouterConfiguration } from 'aurelia-router';

export class MainPage
{
    router: Router;

    configureRouter(config: RouterConfiguration, router: Router)
    {
        config.map([
            { route: '', redirect: 'entities' },
            { route: 'entities', name: 'entities', moduleId: './entities/entities', nav: true, title: 'Entities' },
            { route: 'structures', name: 'structures', moduleId: './structures/structures', nav: true, title: 'Data Structures' },
        ]);

        this.router = router;
    }
}

Then the bigger brother:
import { Router, RouterConfiguration } from 'aurelia-router';

export class Entities
{
    private router: Router;

    configureRouter(config: RouterConfiguration, router: Router)
    {
        config.map([
            { route: '', name: 'entities-list', moduleId: './list', nav: true, title: 'Entities' },
            { route: ':name/events', name: 'entity-events', moduleId: './events', nav: true, title: 'Events' },
        ]);

        this.router = router;
    }
}

And finally the little sister:
import { inject, computedFrom } from 'aurelia-framework';
import { Services } from './services';

@inject(Services)
export class Event
{
    private events = [];

    constructor(private services : Services) {}

    async attached(params, routeConfig)
    {
        debugger;
        this.events = <any> await this.services.getEvents(params.name);
    }
}

And I use the following method call to navigate to the little sister:
this.router.navigateToRoute('entity-events', { name: "Some Name" });

But when I get to the debugger breakpoint, there's no params, it's undefined. According to the documentation, there supposed to be an object passed to the activate method, containing the parameters of the route. Where did go wrong?
Answer: Use the activate method.
The params argument is passed to activate, not attached as you've written above.
async activate(params, routeConfig)
{
    debugger;
    this.events = <any> await this.services.getEvents(params.name);
}

Comment: although you mention `activate` in the question, so I imagine this is not a real question and just a typo
Comment: It seems like I've been too tired last night!
Comment: I stumbled across this trying to work out why my `activate` (yes, really, `activate`) did not work :). Turns out that it does work if I drop `async` but the example here uses `async` is `async` supported with params?


lldb read memory pointer

Question: It is possible to easily read a memory at a location pointed to by a another address.
For example, $r0 = 0x15942600 at this memory address there is AC B8 EC 14
Now to read the memory at 0x14ecb8ac, I will have to do:
mem read $r0
mem read 0x14ecb8ac

Any way to easily do a mem read ($r0) such that I can read the memory easily?
I have tried this:
(lldb) mem re $r1
0x007db594: 7c d5 e7 15 00 00 00 00 e4 b5 7d 00 70 f6 9a 36

(lldb) mem read 0x15e7d57c
0x15e7d57c: 74 69 63 72 63 74 37 35 00 00 63 61 00 00 49 39

(lldb) mem read '*(char **)$r1'
error: invalid start address expression.
error: address expression "*(char **)$r1" evaluation failed

Comment: I don't have an architecture with an r0 register to test this, but `mem read  '*(char **)$r0'` ought to work.
Comment: Hey @MarkPlotnick I've tried it, doesn't work.
Comment: Interesting. What architecture is the target, and what version of lldb?
Comment: Does `p *(int **)$r1` give an error?
Comment: Hmm. So *(char **) gives an error but *(int **) works?
Comment: yep. Oh yea, target arch is ARM.
Answer: Thanks to @MarkPlotnick this works,
mem read '*(int **)$r1'

If you need to read a mem address at a certain offset, the following can be done,
mem read '*(int **)($r1+4)'

Tried and tested against Xcode debugging against ARMv7 and ARM64
Comment: Isn't it more correct to write `mem read '*(int*)$r1'`? In this case, you read the memory of the address given by dereferencing `$r1`.

lldb read memory pointer


How to disable the future date selection from DatePicker in SwiftUI

Question: I am working on a Custom DatePicker in SwiftUI, for that I have written below code which is woking great -
<code>struct DatePickerView: View {
    @State var searchDate = Date()
    let title = "Select a date"
    let dateChanged: (_ date: Date) -> Void

    var body: some View {

            VStack {
                DatePicker(selection: Binding(get: {
                    self.searchDate
                }, set: { newVal in
                    self.searchDate = newVal
                    dateChanged(searchDate)
                }), displayedComponents: .date) {
                    Text(title)
                        .font(.headline)
                }
                .background(.bar)
            }
    }
}
</code>
Now I want to disable the future date selection, can someone help me to do that?
Answer: It can be done with allowed range, like
<code>   DatePicker(selection: Binding(get: {
            self.searchDate
        }, set: { newVal in
            self.searchDate = newVal
            dateChanged(searchDate)
        }), in: ...Date(),          // << here !!
            displayedComponents: .date) {
            Text(title)
                .font(.headline)
        }
</code>


What gauge speaker wire do I need?

Question: I am looking to run some speaker cable through my unfinished basement.
I will need 35 ft long cables for a couple of Technics SB-A10 speakers.  What gauge of cable will I need?  How is the gauge calculated?  How much noise should I expect with a 35 ft long cable?
<code>Impedance: 8 ohm

Input power: 200 W

Crossover frequency: 3000 Hz–6000 Hz

Output sound pressure level: 91 dB/W (at/a 1 m)

Woofer cone type: 25 cm (10")

Midrange cone type, ferrofluid: 10 cm (4")

Tweeter, cone type, ferrofluid: 5.5 cm (2 - 1/8")

Dimensions (W x H x D): 360 x 733 x 265
</code>
Answer: The most important aspect of speaker wire is resistance. There is a ton of junk-science and stupid audiophile marketing claims that will try to confuse you and make you spend absurd amounts of money, but don't be fooled.
Wikipedia actually has a good write-up on it:

Resistance is by far the most important specification of speaker wire. Low-resistance speaker wire allows more of the amplifier's power to energize the loudspeaker's voice coil. The shorter the cable and the greater the conductor's cross-sectional area, the lower its resistance. Depending on the hearing ability of the listener, this resistance begins to have an audible effect when the resistance exceeds 5% of the speaker's impedance.

They also have a chart: for 35' at 8Ω, you're good with 18 or 16AWG.

All you need is basic stranded cable. Don't pay extra for fancy brands, or "oxygen-free" or "hand woven by the dalai lama" or whatever (unless you have the money to burn and really way to, I guess) as there's really no difference -- you can use lamp cord, if you want.
In proper tests, the best audiophiles can't tell the difference between $500/ft "top of the line" speaker wire and coat hangers -- and if you can, James Randi will give you $1 million (sorry, a bit OT from your question, but people that believe in this nonsense really piss me off).

For in-wall applications, you should get wire rated for in-walls - which basically boils down to fumes that the wire gives off in a fire. There's a good article on Understanding In-wall Speaker, Video and Audio Cable Ratings on Audioholics. In short, in a 1-2 family dwelling, any cable with the following ratings is suitable for in-wall: CM, CMP, CMR, CMG, CL2*, CL3*.
Comment: solid wire isn't good in situations where you need flexibility or where the cable is periodically moved (e.g. when the speaker isn't built-in or you need to get to the back of the amplifier).  But for the in-wall portions of the install, solid wire would be good.
Comment: + several million if I could - especially for "you can use lamp cord, if you want". Actually lamp cord is probably better than some speaker cables as it has a greater cross section and therefore lower resistance.
Comment: +1 for the good summery and answer for the specific size in this case.  I would add that you should use wire that is rated for in-wall use.  It should not be hard to find, monoprice or crutchfield should have plenty of options.
Comment: Great advise, 16/2 SPT lamp cord works great. Expensive speaker cable is a joke, over-rated, just part of the snob factor. I have been using 16/2 zip cord on a Denon 70watt RMS analog amp with four Boston A100's for years. Works great.
Comment: If resistance is so important, wouldn't using solid wire be better than stranded?
Comment: In addition to the distance and resistance, you should also consider the amount of power going to the speakers.  With most home systems, and runs of 35', you should be good with 16 gauge.
Comment: @Tester101: but it would be much harder to use solid wire... Despite that, if you can, solid wire would be much better. Just make sure that the connection in your amp and in your speakers. Stranded wire is softer, if it's connected through nuts and bolts, for example, it has a better connection than a solid wire. Example: [with this connector](http://www.goldmine-elec-products.com/prodinfo.asp?number=G13930), stranded wire will connected better.
Answer: Crutchfield has a somewhat informative article: http://www.crutchfield.com/S-nBs26ogHK9Y/learn/learningcenter/home/speakers_wire.html

    Distance from speaker to amplifier  Gauge
    Less than 80 feet           16
    80 to 200 feet              14
    More than 200 feet          12

So you're fine with 16ga.
Remember that your total run from speaker to amplifier includes the leads from the amp to the wall at one end, and from the wall to the speaker at the other end. Still, with a 35' behind-the-wall run, you're still fine with 16ga.
That said, you could easily upsize to 14ga, as it's not very expensive. Lamp cord is great. Hardware stores often sell cheap speaker wire, too. Either works fine.
At the ends of the run, terminate in 5-way-binding posts:

While you're at it, pull an ethernet cable. Pulling an extra wire is only slightly more work, and you can use ethernet for a lot of different signals.
Comment: @Kevin: when I wired my house, I pulled 3 ethernet + coax, so very similar to you. However, two coax is good for satellite HD signals. What about security cameras, audio/video, home automation? What if one of the wires you pull is damaged? Decide what you want, then double it. :-)
Comment: Why run one ethernet when you can do two at twice the price! :)  Seriously, one for networking, one for phone wiring, and they can use the same modular plates as the connectors in the above pic.
Comment: I'm also running 2 ethernet cables, and a coax cable :)
Comment: Every house I buy, I end up running conduits and generally 2 ethernet cables as standard, with a string left in situ for whatever else I want to add. For some rooms I also add speaker cables, and/or telephone cables. I have no idea why builders don't put these in as standard - it has to be easier/cheaper at build time than as an afterthought.

What gauge speaker wire do I need?


Accessing invoice data with a subscription in Recurly PHP API

Question: I am working with the Recurly PHP API and am trying to figure out a way to find out if a subscription has overdue invoices hence the subscription is overdue (even when there are overdue invoices the subscription state does not change and still shows as active).
To access the subscription I am using:
try{
    $subscriptions = Recurly_SubscriptionList::getForAccount('1');
    foreach ($subscriptions as $subscription) {
        print $subscription;
    }
}

This returns:
activated_at="2014-10-08 13:18:27 +00:00",
collection_method="manual",
currency="USD",
current_period_ends_at="2014-11-08 14:00:00 +00:00",
current_period_started_at="2014-10-08 14:00:00 +00:00",
invoice=,
net_terms=0,
plan="",
quantity=120,
state="active",
subscription_add_ons=[],
trial_ends_at="2014-10-08 14:00:00 +00:00",
trial_started_at="2014-10-08 13:18:27 +00:00",
unit_amount_in_cents=500,
uuid="VALUE"

Within $subscription is invoice but it is blank, however, if I use:
print_r($subscription->invoice);

it returns:
Recurly_Stub Object ( [objectType] => invoice [_href:protected] => https://subdomain.recurly.com/v2/invoices/1004 [_client:protected] => [_links:protected] => Array ( ) )

Invoice 1004 is an overdue invoice so I want to be able to show that the subscription is overdue but I cannot access the _href:protected value and cannot see any other way of determining if a subscription contains overdue invoices?
Does anyone know an alternate way to tell if a subscription contains overdue invoices or, failing that, how can I access _href:protected so I can at least then run the get invoice API against the invoice to check?
Comment: Dear bhttoan, kindly update if you have worked it out. I'm also trying the same.
Comment: @sariDon give my answer a look. guessing you've figured it out by now though.
Answer: The short version is you should call $subscription->invoice->get(). $subscription->invoice returns a Recurly_Stub object and you'd need to call its get() method to force it to load the full object.
We document this but clearly we don't have it in the right spot.
Stepping back though, you might find it's faster to just get all the past due invoices Recurly_InvoiceList::getPastDue() and look for their subscriptions.


Refresh jsp page from Spring Controller

Question: I'm having problem with refreshing my jsp page after calling Spring controller.
In my index page, I have login, and after the user enter username and password, the form gets submitted to the controller and then validate and create new user session. After all this, my page is refreshed and everything is working fine. The problem occur when I implement logic for changing page language. I'm doing the same but this time, the page is not refreshing!
Here is what I have tried so far:
JAVA CODE:
    @RequestMapping(value="/index", method=RequestMethod.GET)
public String initializeIndex(ModelMap model) {
    // IMPLEMENT SOME BUSINESS LOGIC HERE
    String l = (String) model.get("language");

    if (l == null) {
        l = "English";  //Default
    }

    model.addAttribute("currentLanguage", l);
    return "index";
}

/**
 * When the user login.
 *
 * @param model
 * @return
 */
@RequestMapping(value="/welcome", method=RequestMethod.POST)
public String login(ModelMap model) {

    // IMPLEMENT SOME BUSINESS LOGIC HERE

    return "redirect:/index";
}

/**
 * When the user change website language.
 *
 * @param model
 * @return
 */
@RequestMapping(value="/changeLanguage", method=RequestMethod.POST)
public String changeLanguage(@RequestParam String language, ModelMap model) {

    //CHANGE SITE LANGUAGE...

    model.addAttribute("language", language);

    return "redirect:/index";
}

EDIT
Here is how I'm calling /changeLanguage method. This function is called after a button click event.
JS
function changeLanguage(language) {
$.ajax({
    type: "POST",
    url: "changeLanguage",
    data: "language=" + language,
    success: function(){
      alert($('#languageHolder').val());
    }
    }
});

HTML
<input type="hidden" id="languageHolder" value='<c:out value="${currentLanguage}" />'/>

Now when I debug the app, I see that everything is working perfect. The only difference is that when call /welcome method and the redirect to /index method, after that the index.jsp is refreshed and when call /changeLanguage method and then to /index method, the index.jsp is not refreshed.
One other difference is that on login I user form and submit the form. In change language I use ajax to submit the data to the server. Could this be the problem ?
Any help will be appreciated.
EDIT
I had just made some small experiments: I add one holder in my index.jsp
and then in the controller I'm setting "language" variable to this holder. Then I
alert this value in js and observe the result.
I just edit the code above.
What I observe is that the JAVA code is working perfect - e.g. when there is no language set in the model, it set the default "English" and set it to the model. Also when there is such attribute, I uses it and set the new value to the model. BUT in the js, alert is always print "English" ???
Comment: How are you calling the /changeLanguage controller? Are you making a  POST request or are you making a GET request.
Are you using Jquery ajax or any other similar method to submit form data ?You will need to post the code which you are using to call changeLanguage
Comment: I'm using POST request. It's called by clicking a button.
Comment: Did you add some debug line to see if the controller is getting called.Also can you post the code you are using to call the controller
Comment: Yes. In debug I see that /changeLanguage method is calling /index method.
Comment: is your changeLanguage() located in a RESTController? Considering its also a async call why do you have it redirecting to index, all it will do is return a the whole index.jsp page to the ajax call.
Comment: I'm not sure I understand what is RESTController. The reason for redirecting to index page is because after call /changeLanguage method, I need to refresh the index page. If there is a better approach, I will be more then happy to see it.
Answer: I have solved my problem. Not the way I wished but still...
The way I solved it is by create form in my jsp. In the controller I create one
wrapper "LanguageWrapper" and add it like model attribute.
Last, when a button is clicked, I submit the form " $('#formName').submit();" and this
work perfectly for me.
I still don't know my when submit with ajax the page is not refreshed and when submit the form like this it refresh. I will spend some time researching this...
If someone knows more about this, please fill free to add more info.
I hope my answer can help others with similar problems.

Refresh jsp page from Spring Controller


Make Javascript Load Before Content

Question: I have this site set up so that if you visit from Switzerland (CH), two of the links at the top disappear.
However, they're disappearing too slowly and you can see the script happen.
I was wondering if there is a way to make the script run and have the links disappear before anyone can see?
Here is the script:

Comment: Why not just hide them with css in the first place?
Comment: Sorry - I didn't explain. The content should be available to users from all other countries. However, some things need to be hidden from users from CH.
Answer: Invert the logic.  Instead of conditionally hiding the content, hide it by default and then conditionally show it.
So instead of this:
element.style.display = "none";

Style the element(s) with CSS to already use display: none; and then do this:
element.style.display = "block"; // or inline, or whatever value it should be

You'd just need to invert the querySelectorAll selector as well, so it selects elements which don't match the country code.
For example:

const url = (() => {
    const protocol = "https";
    const domain = "freeipapi.com";
    const endpoint = "api/json/";
    return `${protocol}://${domain}/${endpoint}`;
})();

const fetchLocationData = async () => {
    try {
        const response = await fetch(url);
        if (!response.ok) {
            throw new Error('Network response was not ok');
        }
        const data = await response.json();

        const countryCode = data.countryCode;

        // Select all elements to show
        const elements = [...document.querySelectorAll(`[data-hide-by-country]:not([data-hide-by-country="${countryCode}"])`)];
        for (let i = 0; i < elements.length; i++) {
            const element = elements[i];
            await new Promise(resolve => setTimeout(resolve, 100));

            // show the element
            element.style.display = "block";
        }

        return data;
    } catch (error) {
        console.error('There was a problem with the fetch operation:', error);
        return null;
    }
}

(async () => {
    await fetchLocationData();
})();
div {
  display: none;
}

Hide for US.

Hide for CA.

The difference here is that all potentially-hidden elements are initially hidden by CSS, and then the JavaScript code iterates over the non-matching elements and shows those ones.
Comment: Thanks for the advice - That was very easy to plug in and working well.

However, now for users in other countries, the hidden elements are loading with a little bit of delay.

This is much better than how it was previous, but I wonder if there is a way to hide the javascript load delay.
Comment: @Dnilly99: The delay exists either way.  Content is either shown by default and then later hidden, or it's hidden by default and then later shown.  If you have some other way of determining what content to show in server-side code before returning the page to the user, that could allow for an initial page state which meets the need.  But if the only way to know the information you need about the user is to make an AJAX request then that's always going to involve *some* delay.  You could potentially improve the UX with loading indicators, skeleton elements, etc.

Make Javascript Load Before Content


Copy and Merge files to another S3 bucket

Question: I have a source bucket where small 5KB JSON files will be inserted every second.
I want to use AWS Athena to query the files by using an AWS Glue Datasource and crawler.
For better query performance AWS Athena recommends larger file sizes.
So I want to copy the files from the source bucket to bucket2 and merge them.
I am planning to use S3 events to put a message in AWS SQS for each file created, then a lambda will be invoked with a batch of x sqs messages, read the data in those files, combine and save them to the destination bucket. bucket2 then will be the source of the AWS Glue crawler.
Will this be the best approach or am I missing something?
Answer: Instead of receiving 5KB JSON file every second in Amazon S3, the best situation would be to receive this data via Amazon Kinesis Data Firehose, which can automatically combine data based on either size or time period. It would output fewer, larger files.
You could also achieve this with a slight change to your current setup:

When a file is uploaded to S3, trigger an AWS Lambda function
The Lambda function reads the file and send it to Amazon Kinesis Data Firehose
Kinesis Firehose then batches the data by size or time

Alternatively, you could use Amazon Athena to read data from multiple S3 objects and output them into a new table that uses Snappy-compressed Parquet files. This file format is very efficient for querying. However, your issue is that the files are arriving every second so it is difficult to query the incoming files in batches (so you know which files have been loaded and which ones have not been loaded). A kludge could be a script that does the following:

Create an external table in Athena that points to a batching directory (eg batch/)
Create an external table in Athena that points to the final data (eg final/)
Have incoming files come into incoming/
At regular intervals, trigger a Lambda function that will list the objects in incoming/, copy them to batch/ and delete those source objects from incoming/ (any objects that arrive during this copy process will be left for the next batch)
In Athena, run INSERT INTO final SELECT * FROM batch
Delete the contents of the batch/ directory

This will append the data into the final table in Athena, in a format that is good for querying.
However, the Kinesis Firehose option is simpler, even if you need to trigger Lambda to send the files to the Firehose.
Comment: Thanks. I did have a look at AWS Kinesis Firehose. I just have to figure out the best approach to passing my data (StepFunction Output) to Kinesis FIrehose. I can create a lambda to post to Kinesis Firehose directly using its API or EventBridge -> Kinesis Datastream -> Kinesis Firehose
Answer: You can probably achive that using glue itself. Have a look here https://github.com/aws-samples/aws-glue-samples/blob/master/examples/join_and_relationalize.md
Comment: Thanks for this. This will require me to keep track of all the newly received files.
Answer: This is what I think will be more simpler

Have input folder input/ let 5kb/ 1kb files land here; /data we will use this to have Json files with max size of 200MB.
Have a lambda that runs every 1minute which reads a set of files from input/ and appends to the last file in the folder /data using golang/ java.
The lambda (with max concurrency as 1) copies a set of 5kb files from input/ and the XMB files from data/ folder into its /tmp folder; and merge them and then upload the merged file to /data and also delte the files from input/ folder
When ever the file size crosses 200MB create a new file into data/ folder

The advantage here is at any instant if somebody wants data its the union of input/ and data/ folder or in other words
With little tweeks here and there you can expose a view on top of input and data folders which can expose final de-duplicated snapshot of the final data.


Invalid hook call for useReducer in context

Question: I am getting an Invariant Violation for an invalid Hook call.
The hook is getting called inside of a function component.
There is only one version of React on the page at a time.
I do not believe I am Breaking the rules of hooks.
The hook call is supposedly violated when calling useReducer in the below file.
import React, { createContext, useReducer } from 'react';
import { node } from 'prop-types';
import toggleReducer from '../../reducers/ToggleReducer';
import defaultToggles from '../../lib/config/toggles';

const ToggleContext = createContext();

const ToggleContextProvider = ({ children }) => {
  const [toggleState, toggleDispatch] = useReducer(
    toggleReducer,
    defaultToggles,
  );

  return (
    <ToggleContext.Provider value={{ toggleState, toggleDispatch }}>
      {children}
    </ToggleContext.Provider>
  );
};

const ToggleContextConsumer = ToggleContext.Consumer;

ToggleContextProvider.propTypes = {
  children: node.isRequired,
};

export { ToggleContext, ToggleContextProvider, ToggleContextConsumer };

Edit* I am only getting this error when running the test suite
Comment: Are either toggleReducer or defaultToggles undefined?
Comment: Nope, defaultToggles is just pulling in a small JS object
Comment: It must be something in the test environment, then. Try putting together a tiny reducer and initialState in the same file and see if the test will run.
Comment: Looks like it is an issue with my test suite, nit 100% sure what yet, but this has pointed me in the right direction thanks.
Answer: The error is being caused my some tests.


Degree Angle and Vecorization with Python

Question: Every time I run the script on a new dataset the angle lines move around. I want to have two lines angled at 345 (green line) and 330 (red line) degrees respectively each time a new dataset is read in.
What is wrong with my code?
Results should look something like this:

1st Data Set
data = np.array([10.79,10.87,10.94,10.95,11,11.5,10.89,11.45,11.94,12.17,12.45,12.09,13.65,13.5,13.25,13.18,13.28,
             13.45,13.81,13.8,14.08,14.09,14.48,14.5,14.08,14.54,14.6,15.48,16,17.049999,17.219999,16.99,
             17.23,17.200001,17.110001,18.190001,23.540001,22.25,21.15,22.09,22.85,21.4,21.41,20.780001,18.84,
             18.389999,18.09,18.280001,17.959999,21.969999,21.120001,20.25,19.879999,21.309999,21.84])

2nd Data Set
data = np.array([12.44,12.02,12.58,12.09,12,11.98,12.19,11.75,11.44,11.4,10.68,10.46,10.95,10.6,11.44,
             10.6,10.41,10.3,11.45,12.5,12.65,11.62,11.45,11.16,10.8,12.5,12.23,13.99,12.49,13.49,12.69,12.72,
             12.81,13.1,12.89,13.50,13.35])

Script
import matplotlib.pyplot as plt
import numpy as np
import math
from scipy import signal

for number in data:

signal_max = (data > np.roll(data,1)) & (data > np.roll(data,-1))
signal_min = (data < np.roll(data,1)) & (data < np.roll(data,-1))

xm = np.argmax(data)
ym = np.amax(data)

angle1 = 345 # green
angle2 = 330 # red

x1, y1 = xm + len(data)-xm, ym + math.tan(angle1 * math.pi/180) * len(data)-xm
x2, y2 = xm + len(data)-xm, ym + math.tan(angle2 * math.pi/180) * len(data)-xm

plt.plot( [ xm, x1 ], [ym, y1 ], '-', color='g')
plt.plot( [ xm, x2 ], [ym, y2 ], '-', color='r')
plt.plot(data)
plt.show()

Answer: One thing that you should be careful of when visualising angles on your plot is the aspect ratio of the plot. If the aspect ratio of the plot is not equal, then the angles visualised will not look like you expect them to!
You can tidy up the plotting of the angles as:
angle1 = np.deg2rad(345) # green
angle2 = np.deg2rad(330) # red

x1, y1 = np.cos(angle1), np.sin(angle1)
x2, y2 = np.cos(angle2), np.sin(angle2)

plt.plot( [ xm, xm + x1 ], [ym, ym + y1 ], '-', color='g')
plt.plot( [ xm, xm + x2 ], [ym, ym + y2 ], '-', color='r')

To ensure an equal aspect ratio you can use plt.gca().set_aspect('equal')
If you'd like to alter the length of the lines you can introduce a scale variable as:
scale = 2
x1, y1 = scale * np.cos(angle1), scale * np.sin(angle1)
x2, y2 = scale * np.cos(angle2), scale * np.sin(angle2)

So, if you'd like the lines to reach to the end of the data you can set
scale = data.size - xm
Comment: Hi. I'm glad this worked for you :) Please see my edit for how you can alter the length of the line as you desire
Comment: That worked like a miracle thank you...however the two lines become very short and doesnt extend all the way to the latest data point as it used to
Comment: you are a rockstar =D
Comment: If you insist :) :) !


Ubuntu touch - remote access to device

Question: in our company we developed an android app installed on nexus 7 tablets and on tablet we release to several customers. This app stays in foreground active 24/7 but we have problems in monitoring  the devices due to the restrictive access policies of unrooted Android devices.
We can't for example reboot the device remotely.
We are evaluating the possibility of installing Ubuntu touch to replace android on this tablets.
Does Ubuntu touch offer more freedom for the remote access of the device? May I do remotely everything I could do normally with the tablet?
Moreover, are the apps developed with ubuntu sdk stable enough to run 24/7?
Thanks
Answer: To allow communication between your Nexus 7 and your host system you'll need to enable the developer mode on your Ubuntu-touch device.
Go to System Settings -> About this phone -> Developer mode:

With the developer mode activated, you have several options to start a ssh connection to your device:

From the Ubuntu SDK (you can even reboot from there):

Using the phablet-tools package (sudo apt-get install phablet-tools):
Start the connection with phablet-shell and once connected type sudo reboot as you would normally do on a desktop system:
$ phablet-shell
/home/sylvain/.ssh/known_hosts updated.
Original contents retained as /home/sylvain/.ssh/known_hosts.old
Warning: Permanently added '[localhost]:2222' (RSA) to the list of known hosts.
Welcome to Ubuntu Utopic Unicorn (development branch) (GNU/Linux 3.4.0-3-flo armv7l)

 * Documentation:  https://help.ubuntu.com/
Last login: Fri Oct 17 09:50:34 2014 from localhost.localdomain
phablet@ubuntu-phablet:~$ sudo reboot
[sudo] password for phablet:

Comment: Thanks Sylvain, do you think enabling this developer mode on devices released to customers is a good practise?
Moreover, is there the possibility that Ubuntu in the future will remove or limit this freedom of access?
Thanks again
Comment: @frankieta Enabling the developer mode to test your app is ok, but no, delivering a product with this mode enabled by default is a bad idea / a security hole. Unfortunately I can't tell you how this feature will evolve. To follow the ubuntu-touch development you can subscribe/post to the mailing list (https://launchpad.net/~ubuntu-phone). You may also find this [post](https://lists.launchpad.net/ubuntu-phone/msg09773.html) interesting.
Comment: Do you have a link to some detailed technical info like this on Ubuntu Phone??? (or should I ask a question?)  ;-)
Comment: @Fabby It depends on what sort of info you'd like to find. You may have better luck on the phone mailing list for technical details.
Answer: On Ubuntu Touch 15.04 r4 I was able to enable SSH following this elegant method using good old netcat.
Now it is possible to reboot the device remotely with just ssh phablet@PHONEIP sudo reboot

Ubuntu touch - remote access to device


Application Insights - How to set Custom Operation Id

Question: In our current on-prem setup we have 20+ .net core 3.1 API apps (separate ASP.NET Core API Apps). We have started migrating 2 APi app to Azure App Service tagged with a single Application Insights instance.
In On-Prem, we use some other log framework which the rest of the 18 Apps. All these API apps talk to each other and all the logs are tied to some unique_id in on-prem.
Now, for the apis which is in Azure, we need to leverage the same unique_Id and co-relate everything.
In order to achieve it, I started exploring the functionality of setting a same Operation Id for the 2 apps which are hosted in azure.
Created TelemetrInitializer in both the APIs. and if i set Operational Id as shown below in both the APIs, it's works. All the logs are tied to Single Operation Id "12345"
      telemetry.Context.Operation.Id = "12345";

However, as it is obvious to make the Operation Id to be dynamic, I have changed it to the below in my First API
  telemetry.Context.Operation.Id = "CR" + Guid.NewGuid().ToString();

So, the next challenge is, I need to tie this new Operation Id in my second API's TelemetryInitiializer. In order to achieve that I tried to grab the Request-Id Header in the TelemetryInitializer of 2nd API. It's always NULL.
Is there a way to achieve this?
Thanks,
Praveen Sreeram.
Comment: Can you take a look at what other headers are being set? In particular, I'm wondering if traceparent and tracestate headers are being set. App Insights is transitioning to W3C Trace-Context and that replaces the Request-Id header. If you are getting the new headers and still need the request-id header for any reason, you can still enable it. https://learn.microsoft.com/en-us/azure/azure-monitor/app/correlation#enable-w3c-distributed-tracing-support-for-aspnet-core-apps
Comment: Yeah, i even checked on that path. i don't see those traceparent and tracestate headers too..
Comment: I attempted to repro this and did some additional digging. The next question is how you are calling the other API. If you are using a .NET Core HttpClient, correlation headers should be set by the HttpClient and App Insights just consumes that information automatically- no initializer needed.
Comment: Yes, I'm using .NEt Core HttpClient to make the call from 1st API to the 2nd APIs. If I'm changing the Operation Id in the first API then it is not being used by the second API. If i'm not changing, yes, it's working as expected.
Comment: it would be great if you can share the code that you were talking about. I can have a look
Comment: That makes more sense now. Your original question implied you weren't getting the header at all so that's what I've focused on. Activity Ids are read only and are already unique- you don't need to manually set it up that way. The reason why setting it in your initializer isn't working is because App Insights isn't setting the header- it's consuming the one already set by the HttpClient. If you want to set it manually, you could theoretically disable the automatic tracking and set your own headers and send your own dependency telemetry to App Insights, but I don't recommend it.
Comment: Actually, Yes, I thought, the way you link two different API is via Headers. That's the reason I focused on that point my question. In fact, i'm able to set Operation Id in my first API. And, I can see that in App Insights timeline too. All I need is to use the same Operation Id in the 2nd API as well. Is that possible?
Comment: If you do nothing the operation id in the 1st and 2nd API should match and the value is passed via the header. The reason why they don't match for you is because you are manually changing the operation id value in the 1st API after the request has already been sent.
Comment: Exactly. in my case, I already use some other tool which has a uniqueId in all my on-prem apps. As we are moving to azure incrementally, we are migrating the very first few apps into azure. So, we need to maintain the co-relation b/w azure apps and with on-prem apps too. So, instead of using the default operation id, can we change it?
Comment: It did require quite a bit of customization, but I have gotten it working- see my answer below.
Answer: tldr: This is possible by disabling the built in dependency tracking in .NET Core and App Insights and handling it on your own. In most cases, the best thing to do is let .NET Core and App Insights do the tracking.
I uploaded a simple WebAPI app with the code I'm going to go over to Github: https://github.com/SamaraSoucy-MSFT/customoperationid
There are two things that need to be overridden to get both the headers and App Insights to get the custom operation Id. The first is the Activity the wraps the HttpClient as that controls the correlation headers. The second is the dependency tracing in App Insights.
It is possible to disable Actions completely in your HttpClients, but to minimize side effects, you can just remove the one in the client by setting Activity.Current = null
var operationId = "CR" + Guid.NewGuid().ToString();
var url = "https://www.microsoft.com";
using (var client = new HttpClient())
{
    using (var requestMessage =
        new HttpRequestMessage(HttpMethod.Get, url))
    {
        //Makes the headers configurable
        Activity.Current = null;

        //set correlation header manually
        requestMessage.Headers.Add("Request-Id", operationId);
        await client.SendAsync(requestMessage);
    }
}

The next step is to remove the App Insights default tracking for this request. Again, you can disable dependency tracking completely, or you can filter out the default telemetry for this request. Processors are registered inside the Startup class just like initializers.
services.AddApplicationInsightsTelemetryProcessor<CustomFilter>();

public class CustomFilter : ITelemetryProcessor
{
    private ITelemetryProcessor Next { get; set; }

    // next will point to the next TelemetryProcessor in the chain.
    public CustomFilter(ITelemetryProcessor next)
    {
        this.Next = next;
    }

    public void Process(ITelemetry item)
    {
        // To filter out an item, return without calling the next processor.
        if (!OKtoSend(item)) { return; }

        this.Next.Process(item);
    }

    // Example: replace with your own criteria.
    private bool OKtoSend(ITelemetry item)
    {
        var dependency = item as DependencyTelemetry;

        if (dependency == null) return true;

        if (dependency.Type == "Http"
            && dependency.Data.Contains("microsoft.com")
            //This key is just there to help identify the custom tracking
            && !dependency.Context.GlobalProperties.ContainsKey("keep"))
        {
            return false;
        }
        return true;
    }
}

Finally, in the method that makes the remote call, you need to inject a telemetry client and call TelemetryClient.TrackDependency()
var operationId = "CR" + Guid.NewGuid().ToString();

//setup telemetry client
telemetry.Context.Operation.Id = operationId;
if (!telemetry.Context.GlobalProperties.ContainsKey("keep"))
{
    telemetry.Context.GlobalProperties.Add("keep", "true");
}
var startTime = DateTime.UtcNow;
var timer = System.Diagnostics.Stopwatch.StartNew();
//continue setting up context if needed

var url = "https:microsoft.com";
using (var client = new HttpClient())
{
    //Makes the headers configurable
    Activity.Current = null;

    using (var requestMessage =
        new HttpRequestMessage(HttpMethod.Get, url))
    {
        //Makes the headers configurable
        Activity.Current = null;

        //set header manually
        requestMessage.Headers.Add("Request-Id", operationId);
        await client.SendAsync(requestMessage);
    }
}

//send custom telemetry
telemetry.TrackDependency("Http", url, "myCall", startTime, timer.Elapsed, true);


Assigning a zero to all array elements in C

Question: In the program I'm working on, this particular operation is definitely not going to be the bottleneck, but it did get me thinking. From the answers to questions such as this one and this one I've learned two ways to easily (efficiently) set all the elements of an array to zero in C:
double myArray[3];
static const double zeroes[3] = {0};
memcpy(myArray, zeroes, sizeof(zeroes));

and
double myArray[3];
memset(myArray, 0, numberOfElementsInMyArray * sizeof(myArray[0]));

Before I move onto my real question: I'm not entirely sure but based on the information I've read, I assume this method would, at least in principle, fill the array with int zeroes (well, unsigned char's but these seem to be fairly equivalent). Is that correct? If so, is an explicit conversion of the int zeroes to double zeroes necessary or is it done implicitly if myArray is declared as an array of double's?
Anyway, my real question is this: if the array isn't very big at all (like the myArray I've declared above), is either of these methods still preferred over a little loop? What if you have a few arrays of the same small size that all need to be assigned zeroes? If commented properly, do you think readability is a factor in the decision and favours a particular solution?
Just to be entirely clear: I am not looking to initialize an array to zeroes.
Comment: If you are concerned about readability, memset is the way code. Or, if you are working on the Windows platform using the native API, ZeroMemory function is used
Answer: If it's just a small array (like three elements), it probably won't make much difference whether you use mem* functions, or a loop, or three distinct assignments. In fact, that latter case may even be faster as you're not suffering the cost of a function call:
myArry[0] = myArray[1] = myArray[2] = 0;

But, even if one is faster, the difference would probably not be worth worrying about. I tend to optimise for readability first then, if needed, optimise for space/storage later.
If it was a choice between memcpy and memset, I'd choose the latter (assuming, as seems to be the case, that the all-zero bit pattern actually represented 0.0 in your implementation) for two reasons:

it doesn't require storage of a zeroed array; and
the former will get you into trouble if you change the size of one array and forget the other.

And, for what it's worth, your memset solution doesn't need to have the multiplication. Since you can get the size of the entire array, you can just do:
memset (myArray, 0, sizeof (myArray));

Comment: Of course that assumes that the floating point format on the specific hardware is such, that all zero bytes actually result in a value of 0.0.
Comment: @Rudy, yes. But that _is_ the case for the OP since they mentioned the use of memset. You'd be hard-pressed trying to find a C implementation nowadays that doesn't use IEEE754 for floating point :-) But I'll update for completeness, thanks.
Comment: Thanks, I actually didn't know you could do a triple assignment on one line like that in C!
Comment: Wouter, keep in mind it's probably no more efficient than three assignment statements, it's more just syntactic sugar. Well, it _is_ more efficient if what you're trying to reduce is space taken up by your source code rather than running time :-)
Comment: If the scope of double myArray[3]; is within the file. I will use static. The static will take care of assigning it to zero. All the static variables are assigned to zero by default. Remember?
Comment: @dexterous_stranger, it's storage duration rather than scope that decides that but, in any case, that only covers the initialisation at program startup. OP specifically stated they were interested in subsequent assignment rather than initialisation.
Answer: i think the first method of setting without using a loop is better for performance
what happen is that the merroy of array is bitwised by 0  (& 0) so it faster than using a loop for each element in the array.
Comment: It is not *bitwise and-ed*, it is simply *overwritten* with all zeroes, no matter what the existing contents were.


proof that Ackermannfunction is uniquely defined and finding algorithm without recursions to calculate its values

Question: my question is involving the Ackermannfunction.
Let's call a function $a: \mathbb{N} \times \mathbb{N} \rightarrow \mathbb{N}$ "Ackermannfunktion", if for all $x,y \in \mathbb{N}$ the following conditions are fulfilled:
1) a(0,y) = y+1
2) a(x+1,0) = a(x,1)
3) a(x+1, y+1) = a(x, a(x+1, y))
Now I have to proof that there exists
a) at least one and
b) at most one
such function.
c) write a programme (or describe an algorithm) without recursive calls that for every input (x,y) calculates a(x,y).
d) Then calculate or describe a(4,2018).
I am not sure, what in a) is to do. Do you know what is meant?
For b) I tried with functions A and B, that fulfill all three requirements, to prove that it's (A-B)(x,y) = 0 for every input (x,y), but I didn't manage to do so (I only managed for the input (0,y)).
In c) I have no clue how to approach it.
d) I found on the internet how a(4,y) looks like, so I could write down the solution, but I don't know how you get to the expression of a(4,y).
I'd appreciate your help on this and am looking forward to your replies.
Comment: By recurrence on the first parameter, you can show there is only one such function (You already did for x=0). To make a program without recursion, you'll need a stack and a while loop.
Answer: Proving uniqueness is pretty straightforward via two layers of induction. To give you some pointers:

$a(0,y)$ is uniquely defined.
If $a(X,y)$ is uniquely defined for some fixed $X$ and all $y$, then:

$a(X+1,0)$ is uniquely defined.

If $a(X+1,Y)$ is uniquely defined, then $a(X+1,Y+1)$ is uniquely defined

$\implies a(X+1,y)$ is uniquely defined for that $X$ and all $y$.

To avoid recursive calls, notice how this function expands according to your rules. Take $A(6,3)$ for example:
$$\begin{align}A(6,3)&=A(5,A(6,2))\\&=A(5,A(5,A(6,1)))\\&=A(5,A(5,A(5,A(6,0))))\\&=\underbrace{A(5,A(5,A(5,A(5,}_{3+1}1))))\end{align}$$
In general,
$$A(x+1,y)=\underbrace{A(x,\dots A(x,}_{y+1}1)\dots)$$
which allows you to avoid a lot of recursion via loops. You can keep track of which $x$'s you have separately as well, to avoid ever calling the function again, and basically repeatedly expand based on the "most recent" $x$ value.
For example, if you wanted to calculate $A(6,3)$, you'd start by making $4$ copies of $5$ evaluated at $1$. Then you'd have $3$ copies of $5$ followed by $2$ copies of $4$ evaluated at $1$. Then you'd have $3$ copies of $5$ followed by $1$ copy of $4$ followed by $2$ copies of $3$ evaluated at $1$. etc. Visually:
$$\begin{align}A(6,3)&=\underbrace{A(5,A(5,A(5,A(5,}_41))))\\&=\underbrace{A(5,A(5,A(5,A(5,}_3\underbrace{A(4,A(4,}_21))))))))\\&=\underbrace{A(5,A(5,A(5,A(5,}_3\underbrace{A(4,}_1\underbrace{A(3,A(3,}_21))))))))\\&=\dots\end{align}$$
Answer: If I recall correctly, the "closed form" of the Ackermann function I knew was
$$A(x,y)=2\uparrow^{y-3}x+3$$ where the uparrow is Knuth's and the power is the number of uparrows.  I am not sure calculating this is without recursive calls.  It doesn't call the routine to compute any other value, but unpacking the uparrows is a big job.  There are several different Ackermann functions out there, so your closed form may differ in detail.  I was able to prove mine by induction.


JsonProvider<...>.Root does not have null as proper value

Question: I am trying to query results of the parsed Json and if I cannot find I want to do something else.
[
  {
    "orderId": 136,
    "quantity": 5,
    "price": 3.75
  },
  {
    "orderId": 129,
    "quantity": 9,
    "price": 3.55
  },
  {
    "orderId": 113,
    "quantity": 11,
    "price": 3.75
  }
]

My code is like:
type OrdersProvider = JsonProvider<"Orders.json">
let orders = OrdersProvider.GetSamples()

let test id =
    let res = query{
                    for i in orders do
                        where (i.OrderId = id)
                        select i
                        headOrDefault
                }

    if isNull(res)
        then NOT_FOUND("")
        else OK(res.JsonValue.ToString())
    )

However I am getting compiler error "JsonProvider<...>.Root does not have null as proper value". Which is kinda makes sense except I still want to catch the case when id is not in the file. I guess I could change headOrDefault to head and trap the exception but wonder if there is something better.
Update #1:
Following one of the links in comments I was able to get away with
    if obj.ReferenceEquals(res,null)
        then NOT_FOUND("")
        else OK(res.JsonValue.ToString())
    )

Update #2:
While mentioned code works but still feels unnatural for the language. Accepted answer looks more natural.
Comment: See this answer: https://stackoverflow.com/a/11696947/679898, it's a more general question, but the solution should apply.
Comment: Possible duplicate of [Type does not have null as a proper value](https://stackoverflow.com/questions/11696484/type-does-not-have-null-as-a-proper-value)
Comment: @scrwtp it's not a duplicate of the https://stackoverflow.com/questions/11696484/type-does-not-have-null-as-a-proper-value unless there is a way to apply attribute AllowNullLiteral to OrdersProvider.Root. However another link helped. I was able to get away with obj.ReferenceEquals
Comment: That's why I linked to that answer ;)
Answer: I think the headOrDefault operation was designed for compatibility with LINQ to SQL, which is why it returns null in the default case - this is not something you'd normally want in well behaved F# code, so using it in the way your query does is not a good idea.
Fortunately, headOrDefault will work with F# option type - if you return Some from your select clause then headOrDefault returns None when the value is not available:
let res =
  query {
    for i in orders do
    where (i.OrderId = id)
    select (Some i)
    headOrDefault }

Now you can handle the missing case with pattern matching:
match res with
| None -> NOT_FOUND("")
| Some order -> OK(order.JsonValue.ToString())


Rendering HTML files in Grails

Question: I've looked around but could not find a way of simply including or rendering *.html files in Grails. My application needs to g.render or <g:render> templates which are delivered as html files. For this, as we know, html files have to be converted to _foo.gsp files in order to get rendered. I am totally surprised as to why isn't there a direct support for html or is there one??
Thanks!
Answer: One obvious option is to simply rename your HTML files from foo.html to _foo.gsp and then use <render template="foo">. However this is so obvious that I'm sure you've already thought of it.
If you simply want to render a HTML file from within a controller you can use the text parameter of the render controller method
def htmlContent = new File('/bar/foo.html').text
render text: htmlContent, contentType:"text/html", encoding:"UTF-8"

If you want to do the same thing from within a .gsp, you could write a tag. Something like the following (untested) should work:
import org.springframework.web.context.request.RequestContextHolder

class HtmlTagLib {

  static namespace = 'html'

  def render = {attrs ->

    def filePath = attrs.file

    if (!file) {
      throwTagError("'file' attribute must be provided")
    }

    def htmlContent = new File(filePath).text
    out << htmlContent
  }
}

You can call this tag from a GSP using

Comment: @Ray If you put the HTML files under web-app, you should be able to retrieve them using `g.resource`
Comment: Thanks Don! This looks like it (the second part ie.). I'll give this a try and update my results here.
Comment: Don - don't know if you'll see this, hope so.  I tried the code you posted, but it seems like I have to specify a file relative to my computers root directory, e.g. "C:/grails2.0/..."  How do I make it work relative to my webroot directory, like the render view currently works, looking for files under the appropriate view folder?
Answer: What is it you are trying to accomplish?

Render html from a controller?
In that case, all you should have to do is redirect the user to file from your control.
redirect(uri:"/html/my.html")
Use html-files instead of gsp template-files?
Thing is, Grails is a "Convention over Configuration"-platform and that means you will have to do some things "the Grails way". The files needs the _ and the .gsp but the name can be whatever you like even if it's easier when you use the same name as the controller. What you gain from doing that is the knowledge that every developer that knows grails and comes into your project will understand how things are tied together and that will help them get started quickly.
Comment: In a real world scenario, might I say, things don't work according to rigid Conventions. A Framework should at least support working with its most fundamental elements - HTML in this case.

Answering your question, we have a single GSP page wherein multiple Gauges are rendered. Each of these Gauges are pure _GSP_ templates BUT every once a while they need to be directly rendering HTML files which are dropped to a particular folder /some_folder/*.html
In the TAGLIB which I've created we need to toggle between rendering from GSP templates or HTML files.

Apologies for not clearing this first.
Comment: Thanks for clearing that up. Don's answer seems like a good fit for you and honestly your need is specific enough to warrant it's own custom taglib.
Answer: Little bit fixed the Don's example, works fine for me
import org.apache.commons.io.IOUtils
class HtmlTagLib {

    static namespace = 'html'

    def render = {attrs ->

        def filePath = attrs.file

        if (!filePath) {
            throwTagError("'file' attribute must be provided")
        }
        IOUtils.copy(request.servletContext.getResourceAsStream(filePath), out);
    }
}

Answer: I wanted to write static html/ajax pages hosted in grails app (v2.4.4), but use the controller for the url rewrite. I was able to accomplish this by moving the file to web-app/ (for ease of reference), and simply use the render() method with 'file' and 'contentType' params, such as:
// My controller action
def tmp() {
    render file: 'web-app/tmp.html', contentType: 'text/html'
}

Note: I only tried this using run-app, and haven't packaged a war and deployed to tomcat, yet.
Doc: http://grails.github.io/grails-doc/2.4.4/ref/Controllers/render.html
Answer:   Closure include = { attr ->
        out << Holders.getServletContext().getResource(attr.file).getContent();
    }

It is relative to the web-app folder of your grails main application
Comment: i modified your approach to allow inclusion of external files for grails rendering plugin, where network paths would get botched in higher environments - mine ended up like:

`def includeFile = { attr ->
        URL resource = Holders.getServletContext().getResource(attr.file)
        InputStream content = resource.getContent()
        String text = content?.text
        log.info "includeFile($attr) - resource: $resource, content.text.size(): ${text?.size()}"
        out << text
    }`
Answer: I was able to use @momo's approach to allow inclusion of external files for grails rendering plugin, where network paths would get botched in higher environments - mine ended up like:
def includeFile = { attr ->
    URL resource = Holders.getServletContext().getResource(attr.file)
    InputStream content = resource.getContent()
    String text = content?.text
    log.info "includeFile($attr) - resource: $resource, content.text.size(): ${text?.size()}"
    out << text
}

Rendering HTML files in Grails


Does a virtual environment clone the folders under Home?

Question: I'm trying to wrap my head around how to effectively use virtualenv. I made a test environment called "quandl". In this environment I imported the Quandl package and am playing around with some examples from their site.
While in my virtual environment, when I type ls I see my home directory and all its sub-folders. Does the test environment create its own copies of each of those folders? If that's the case, then with every virtual environment I create from here on out I will see different folders every time I load a new environment. Not necessarily a bad thing, I just want to understand how this all works.
Answer: No, a virtual environment does not clone the directories in $HOME. A virtual environment is an isolated project-specific environment for invoking the Python interpreter and installing Python packages. This allows you to isolate the Python dependencies of multiple projects from each other.
Comment: You can do it by giving the all virtual environments the same name (common ones I see are `venv` and `env`) and then search for these names. You can also check out `virtualenvwrapper` which provides a way to manage multiple virtual environments.
Comment: But then if I install a certain version of a package that I need for a specific project, where does this package get installed to? And how would another virtual environment know to ignore this package?
Comment: That is what activation (and deactivation) of a virtual environment does: to specify the virtual environment that you want to use (activate) and installation will then be specific to that activated virtual environment.
Comment: Is there any way I can see the different virtual environments that I am creating? Like, do they get stored somewhere?


How to install legacy nvidia drivers in 14.04?

Question: I have an older desktop with an NVIDIA GeForce FX 5500, which means only the 173.xx versions of nvidia's drivers work.
In the "hardware drivers" thing, I selected the "proprietary drivers" option, and clicked "apply now", but then it switched right back to the "use open source driver", without any visible error message or anything.
So I tried to install it with apt:
        Reading package lists...
        Building dependency tree...
        Reading state information...
        Some packages could not be installed. This may mean that you have
        requested an impossible situation or if you are using the unstable
        distribution that some required packages have not yet been created
        or been moved out of Incoming.
        The following information may help to resolve the situation:
    # apt-get install nvidia-173
    The following packages have unmet dependencies:
      nvidia-173 : Depends: xorg-video-abi-11 but it is not installable or
                   xorg-video-abi-12 but it is not installable or
                   xorg-video-abi-13 but it is not installable or
                   xorg-video-abi-14 but it is not installable or
                   xorg-video-abi-15
    E: Unable to correct problems, you have held broken packages.

Forcing did not help.
So I tried installing the proprietary drivers from here.
I blacklisted nouveau, stopped X, and updated the ramdisk.
The installation failed with a bunch of errors, as you can see here.
I can't figure out what I should do to make it compile. I installed build-essentials, and my gcc version is 4.8.4. My kernel version is kernel 3.19.0-25-generic.
I searched online, and all I could find was info for newer drivers.
The drivers worked fine in openSUSE 12.3, and nvidia had an official repo for prebuilt ones, but it seems to be defunct now, although there's bound to be an archive somewhere. Should I try to somehow install those binaries?
Or is there something else I should try?
Thanks in advance.
Comment: Since the kernel version is 3.19, you must have installed 14.04.3 or the enablement stack. The default kernel version for 14.04 is 3.13, and the default xorg version is different as well. To try and solve it, you might want to install the original 14.04: http://old-releases.ubuntu.com/releases/14.04.1/.
Comment: You can use nouveau drivers. They should work OK.
Comment: @Pilot6 they've never worked right for me. The annoyances vary with each OS (and probably the different versions of nouveau, I don't jump around from OS to OS very often), but this time it was that the screen flickers randomly.
Answer: Unfortunately, there is no way to make 173.xx work on anything past Ubuntu 10.04 (that's when I last used the nvidia drivers on Ubuntu).
My issue was that my computer only has the "old" PCI - that is, no PCI Express slots. As such, I had to find a graphics card that fits in there.
Luckily, I found a GeForce 6200 card (with PCI!) on eBay, which uses the nvidia-304 drivers. Now it works fine on 12.04.
This fix -- if you want to call it that -- will cost you about US$40, or whatever people are selling them for.
Good luck to anyone else who has this issue.
Comment: nvidia-173 works on 12.04.
Comment: @Pilot6 not really. I tried, and got something about a symbol missing and that it couldn't load the driver. At that point, I was fed up with the whole thing, and basically "paid to make it go away". It may be a defeatist attitude, but I don't have all day to deal with this.
Answer: There is a very easy solution for this issue. You just need to install Ubuntu 14.04.1.
I did it on ancient machine with GeForce FX 5200 using lubuntu-14.04.1-desktop-i386.iso.
I think you shouldn't download any updates during install (or better unplug network) and after first boot just use Ubuntu's drivers tool to install NVIDIA 173.14.39. After reboot you can fully update your system to 14.04.4 and it won't destroy NVIDIA's dependencies.

Switching to this driver gave me a little performance increase. Some old games works better (via PlayOnLinux) and it even solved an issue with one game. Videos also plays slightly better (including HTML5). SMTube works considerably good now.
Just don't upgrade your distribution above 14.04 :)
Here's the link where I got my ISO: http://cdimage.ubuntu.com/lubuntu/releases/14.04/release/
Answer: Problem: Can not install the driver 173.14.39 for Graphics Card Model NVIDIA GeForce FX 5200.
Solution: I have searched and read for a solution for this issue for many days. nvidia-173 (173.14.39) is the driver package for NVIDIA GeForce FX 5200 graphics card. I have followed these sources to Install nVidia GeForce FX 5200 driver in Lubuntu 14.04.1 and it WORKED. Games(AssultCube) and (Warzone 2100) runs smoothly now. Here is the pure solution with technical details:
nVidia site says: "Support for X.Org xserver version 1.15 was added to the 173.14.* legacy driver series with version 173.14.39. No further releases from the 173.14.* series are planned."
Look here: https://nvidia.custhelp.com/app/answers/detail/a_id/3142/~/support-timeframes-for-unix-legacy-gpu-releases
Supported cards for 173.14.39 driver: https://www.nvidia.com/object/IO_32667.html
And here: https://www.nvidia.com/Download/driverResults.aspx/71302/en-us
On Ubuntu forums site: The latest version of Ubuntu that will support the 173.xx driver is Ubuntu 14.04.1.
Look here: https://ubuntuforums.org/showthread.php?t=2293379
If you are doing a new Ubuntu installation, do not install any version above 14.04.1, and do not download any updates during installation.
First you must connect to internet and check that by browsing web pages for example.
To install from desktop:

Go to Synaptic package manager then update package info.
Go to Software & Updates -> Additional Drivers-> select nvidia
driver 173.14.39 -> Apply.
Once the driver is downloaded and installed, reboot your computer.

To Install from terminal:

sudo apt-get update
sudo apt-get install nvidia-173
sudo reboot

You can fully update your system to 14.04.6 and it won't destroy nVidia's dependencies. it'll still be using the 14.04.1 kernel series and graphics stack. Don't upgrade your distribution above 14.04.
To remove nVidia driver from the system: sudo apt-get remove --purge nvidia-*
 You can also install (Linux Mint 17.1). I think it has X.Org xserver version 1.15 and will work too.
I solved this issue


Second coat of Quikrete High Performance Self-Leveling Resurfacer

Question: I've just poured a first coat of Quikrete High Performance Self-Leveling Resurfacer and have irregularities due to not pouring each bucket fast enough for the area I'm resurfacing (250 sf) and, i think, that the water:concrete ratio was too thick to flow. Perhaps due to the relatively low humidity where i am?). I need to level the first coat.
I submitted a support request on Quickrete's site but no response yet.
Can i pour a second skim coat?
How long can i wait? Or must i wait?
Is any additional surface prep required?
Thanks
Answer: The OP added this in a comment:

I never heard back from Quikrete from the email support request so
  called the customer support number and the guy was very helpful.
Answer: Yes. Multiple coats can be applied. Restrictions - wait 24
  hours. Pour additional applications up to 1".
I was envisioning the stuff freely flowing all over the floor like
  water, but it's cement. Mistakes I made:

Not having maximum amount pre-mixed and ready to pour (literally) at once.
Underestimating amount of material I needed.
Trying to smooth it with a floor squeegee. Don't touch it!
A bit too thick


Cannot Generate PDFs using Tree.io (Django and Apache)

Question: I have installed Tree.io using this tutorial:
Tree.io is installed in /home/treeio/treeio
The Folder belongs to www-data:www-data with chmod rwxrwxr-x
Whenever I try to generate a PDF, I get this error (Some URLs Blanked out for Security Reasons):
<code>Environment:

Request Method: GET
Request URL: http://office.mydomain.de/contacts/contact/view/75.pdf

Django Version: 1.5.1
Python Version: 2.7.3
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.admin',
 'django.contrib.markup',
 'django_websocket',
 'django.contrib.messages',
 'treeio.account',
 'treeio.core',
 'treeio.core.api',
 'treeio.core.search',
 'treeio.documents',
 'treeio.events',
 'treeio.finance',
 'treeio.identities',
 'treeio.infrastructure',
 'treeio.knowledge',
 'treeio.messaging',
 'treeio.news',
 'treeio.projects',
 'treeio.reports',
 'treeio.sales',
 'treeio.services',
 'dajaxice',
 'dajax',
 'coffin',
 'captcha',
 'south')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'treeio.core.middleware.user.AuthMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'treeio.core.middleware.user.LanguageMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.common.CommonMiddleware',
 'johnny.middleware.LocalStoreClearMiddleware',
 'johnny.middleware.QueryCacheMiddleware',
 'django.middleware.gzip.GZipMiddleware',
 'treeio.core.middleware.user.SSLMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'treeio.core.middleware.chat.ChatAjaxMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'treeio.core.middleware.modules.ModuleDetect',
 'minidetector.Middleware',
 'treeio.core.middleware.user.CommonMiddleware',
 'treeio.core.middleware.user.PopupMiddleware')

Traceback:
File "/home/treeio/treeio/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  121.                             response = middleware_method(request, e)
File "/home/treeio/treeio/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  115.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/treeio/treeio/core/decorators.py" in wrap
  124.                 response = f(request, *args, **kwargs)
File "/home/treeio/treeio/core/decorators.py" in wrap
  48.                         return f(request, *args, **kwargs)
File "/home/treeio/treeio/identities/views.py" in contact_view
  403.                               context_instance=RequestContext(request), response_format=response_format)
File "/home/treeio/treeio/core/rendering.py" in render_to_response
  181.         f = open(output)

Exception Type: IOError at /contacts/contact/view/75.pdf
Exception Value: [Errno 2] No such file or directory: u'/home/treeio/treeio/static/media/pdfs/174ce9c2e1ac29733db566275c5eb76b'
</code>
Answer: I have exactly the same issue - been scratching my head for hours on this and still cannot fathom it out.
I can however submit the suggestion to sudo service apache2 reload after every change.
E


On Ubuntu Server 18.04, can't get my domain to resolve unless www is used

Question: I can't get pangeawerks.com to resolve but www.pangeawerks.com works great
I pinged the domain and here are the results:
karmacomposer@PANGEAVERSE:~$ ping pangeawerks.com
ping: pangeawerks.com: Temporary failure in name resolution

karmacomposer@PANGEAVERSE:~$ ping www.pangeawerks.com
PING www.pangeawerks.com(::1 (::1)) 56 data bytes
64 bytes from ::1 (::1): icmp_seq=1 ttl=64 time=0.039 ms
64 bytes from ::1 (::1): icmp_seq=2 ttl=64 time=0.048 ms
64 bytes from ::1 (::1): icmp_seq=3 ttl=64 time=0.053 ms
64 bytes from ::1 (::1): icmp_seq=4 ttl=64 time=0.047 ms
64 bytes from ::1 (::1): icmp_seq=5 ttl=64 time=0.052 ms

--- www.pangeawerks.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4094ms
rtt min/avg/max/mdev = 0.039/0.047/0.053/0.010 ms

I know it has to do with my DNS server (it's our own server here at my office, so not a host).
If anyone could help me get this working, it's causing me a lot of grief.
Thank you.
Mike
Comment: Question is not `Ubuntu` related but you have my answer below :)
Answer: Problem is with DNS resolution.
dig pangeawerks.com

; <<>> DiG 9.11.3-RedHat-9.11.3-12.fc28 <<>> pangeawerks.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 39391
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;pangeawerks.com.       IN  A

;; Query time: 462 msec
;; SERVER: 192.168.0.1#53(192.168.0.1)
;; WHEN: Wed Jul 18 21:02:31 CEST 2018
;; MSG SIZE  rcvd: 44

and
dig www.pangeawerks.com

; <<>> DiG 9.11.3-RedHat-9.11.3-12.fc28 <<>> www.pangeawerks.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36304
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;www.pangeawerks.com.       IN  A

;; ANSWER SECTION:
www.pangeawerks.com.    604424  IN  A   188.8.131.52

;; Query time: 7 msec
;; SERVER: 192.168.0.1#53(192.168.0.1)
;; WHEN: Wed Jul 18 21:02:49 CEST 2018
;; MSG SIZE  rcvd: 53

Create fallowing records. First A record to point to you ip
pangeawerks.com     A     18.104.22.168

After that make CNAME record to connect www with hostname
www.pangeawerks.com   CNAME    pangeawerks.com

This records must be make on Authority DNS server.
Here are the current results of dig pangeawerks.com:
; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> pangeawerks.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 57836
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 79cc3057e00d2854be8b8e965b4f92ce402e52c17af6f46e (good)
;; QUESTION SECTION:
;pangeawerks.com.               IN      A

;; Query time: 405 msec
;; SERVER: 220.127.116.11#53(18.104.22.168)
;; WHEN: Wed Jul 18 15:19:42 EDT 2018
;; MSG SIZE  rcvd: 72

Here is the contents of my zone file for pangeawerks.com.db:
; BIND data file for local loopback interface
;
$TTL    604800
@       IN      SOA     pangeawerks.com. PANGEAVERSE.pangeawerks.com. (
                         33         ; Serial
                     604800         ; Refresh
                      86400         ; Retry
                    2419200         ; Expire
                     604800 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       188.8.131.52
@       IN      AAAA    ::1

;Name Server Information
@       IN      NS      PANGEAVERSE.pangeawerks.com.

;IP address of Your Domain Name Server(DNS)
pangeawerks.com  A      126.96.36.199

;Mail Server MX (Mail exchanger) Record
pangeawerks.com. IN  MX  10  mail.pangeawerks.com.

;A Record for Host names
www     IN       A       126.96.36.199
mail    IN       A       188.8.131.52

;CNAME Record
ftp     IN      CNAME    www.pangeawerks.com.
www.pangeawerks.com     CNAME   pangeawerks.com.

And here is the reverse lookup file db.72:
;
; BIND reverse data file for local loopback interface
;
$TTL    604800
@       IN      SOA     pangeawerks.com. PANGEAVERSE.pangeawerks.com. (
                          6         ; Serial
                     604800         ; Refresh
                      86400         ; Retry
                    2419200         ; Expire
                     604800 )       ; Negative Cache TTL
;
@          IN      NS      pangeawerks.com.
26.51.31   IN      PTR     pangeawerks.com.

;Your Name Server Info
@               IN      NS      PANGEAVERSE.pangeawerks.com.
PANGEAVERSE     IN      A       220.127.116.11

;Reverse Lookup for your DNS Server
26              IN      PTR     PANGEAVERSE.pangeawerks.com.

;PTR Record IP address to HostName
26              IN      PTR     www.pangeawerks.com.
30              IN      PTR     mail.pangeawerks.com.

PANGEAVERSE is the hostname of the server - the FQDN is PANGEAVERSE.pangeawerks.com
Here is the contents of the /etc/hosts file (lots of commenting out of wrong stuff - did not want to get rid of them in case I needed them again or did something wrong):
#127.0.0.1 localhost localhost.pangeawerks.com PANGEAVERSE
#127.0.0.1      localhost PANGEAVERSE
184.108.40.206     PANGEAVERSE.pangeawerks.com     PANGEAVERSE

# The following lines are desirable for IPv6 capable hosts
#::1     localhost ip6-localhost ip6-loopback
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
#18.104.22.168  pangeawerks.com
#220.127.116.11  www.pangeawerks.com

Please suggest proper formatting of any of these files, as it's a bit of a hodge podge of other templates.  I am SURE I did something wrong.
Thanks.
Comment: "This records must be make on Authority DNS server."  - what does that mean?
Comment: I made those changes and restarted bind9. Now it can ping, but on my mobile phone, the site does not come up if I type pangeawerks.com
Comment: Do I need to reboot the server?
Comment: Nope. DNS must resolve logical name to ip. Give me output from command `dig pangeawerks.com`
Comment: Note the SERVFAIL - how do I resolve this?
Comment: I added what you requested above with proper formatting (i'm brand new at stack exchange - forgive my lack of etiquette).
Comment: @karmacomposer who runs the server your site sits on?  Do you?  If you do, where do you define DNS records through?  A server you maintain or your domain registrar?  Do you have an `/etc/hosts` entry for `www.pangeawerks.com` and not `pangeawerks.com`, or do you use actual DNS?  (I.E. where do you set "this domain points to 18.104.22.168")
Comment: Our own server - right in my office. DNS is through the server.
Comment: We have static IPs and have that set.  I'll post the /etc/hosts above and make corrections if need be
Comment: So should I uncomment out the www.pangeawerks.com and pangeawerks.com entries? Should I comment out the one towards the top?  I am just not sure what's right and what's wrong.  I am soooo rusty at linux.
Comment: @karmacomposer I don't see an /etc/hosts anywhere.  Where'd you put that info?
Comment: I added it to the top - but it's at the bottom of the original edit. Perhaps it's not yet approved - every edit I make says it has to be approved.
Comment: I posted a bunch of stuff - zone files and just about everything else I thought one could use to help me - are you seeing any of it?
Comment: With the help of STEEL DRIVER, I believe we solved the SERVFAIL problem, but I still cannot omit the www to get to our domain's website.  I appreciate your help.
Comment: Last comment for today - this is NOT SOLVED. Anyone's help would be immensely appreciated.


Processor factory output is full and factories have nowhere to ship freight. How do I fix that?

Question: My processor factory output is full and factories have nowhere to ship freight. How do I fix that?
I have 1 small trade depot which has 1 coal storage lot, 1 ore storage lot and 1 freight storage lot. I cannot add any more storage lots to my trade depot.
I have 1 big trade pot which only has 1 freight storage lot, and I cannot add more storage because it says "NOT APPROVED".
Also, my whole city has High TECH level, university running at full and 1 Electronics HQ, which earlier used to say NEED ALLOY Import from global, but somehow I fixed it with Recycling center.
Electronics HQ now has produced too much and its OUTPUT STORAGE is full. Same problem now arising to my factories "Closing because of no places to ship freight."
How can I fix my FREIGHT SHIPPING problem? I have no place for Municipal airport. How do I allow my factories to ship freight and my processor factory to sell processors?
I have 10,000,000$+ money and Income is 13500$ per/hr. Population is 100,000.
Answer: For the freight, you need to build more freight storage lots.
For the processors, you need to build a processor storage lot, and set the trade depot/port to export.
Below is how to get more storage lots.

Build more trade depots. There is a limit of 4 storage lots per depot, but you can build as many trade depots as you need. Also in the edit tool, you can destroy any storage lots you no longer use.
There are two ways to unlock storage lots for the trade port.

Plop the commerce division for the Electronics HQ.
Plop the electronics division for the Trade HQ.
Comment: i have one big trade depot, which has only 1 Storage lot. It wont let me add any storage lot other than fright storage. Says "NOT APPROVED."
Comment: You can build another trade depot (the small one) and add more storage lots to your new trade depot.
Answer: Factories want to ship freight to commercial buildings. Only if you don't have enough of those do they resort to the freight lots of your Trade Depots/Ports. Try adding more commercial zones.
The processor factory requires your Trade Depots/Ports to have open processor storage lots; according to your question you have none. Add a Processor Lot to your Depot (you say you have 3 lots currently, so you have room for a 4th unless you've built yourself into a corner) and start exporting them. As long as your traffic is decent your backlog should get out the door and you'll earn mad cash; do this at the start of a day just after midnight and you should be able to upgrade your Electronics HQ. Then give it a Commerce Division so you can add a Processor Lot to your Port. (Alternatively, add an Electronics Division to your Trade HQ, depending which one you can upgrade first.)
Comment: yes i have 3 storages. but it wont let me added 4rth. says "NOT APPROVED" I have all TOWN HALL unlocked buildings.
Comment: If you have 3 storage lots, it won't say "not approved". Maybe instead of *having* 3, you *added* 3 to the 1 freight lot that comes free with the main building for a total of 4?
Answer: I think this is a bug. A similar bug happens with recycling centers. When the storage lots get full, it shuts the facility down, and it won't seem to restart no matter what you do. I added 4 delivery trucks. I bulldozed a module to put a processor storage lot in the trade center, and I set it to export. I have a rail depot. And I've built a lot of 'expensive' buildings around this microprocessor thing, and I can't get it to export the product...


Sum of reciprocal binomial coefficients

Question: I am aware that
$$\sum_{n=0}^\infty \binom{2n}{n}^{-1} = \frac{4}{3} + \frac{2\pi\sqrt{3}}{27}$$
though I do not know why it is true.  More generally, I'm interested in the value of the series
$$S_k = \sum_{n=k}^\infty \binom{2n}{n-k}^{-1}$$
where $k$ is a fixed positive integer.  The series converges by the ratio test.  Does anybody know how to evaluate these sums, or have a reference where they are evaluated?
Answer: Generally similar sums can be evaluated using the Beta function:
$$
B(x+1,y+1)=\int_0^1 t^{x}(1-t)^{y}dt=\frac{\Gamma(x+1)\Gamma(y+1)}{\Gamma(x+y+2)}=
\frac{x!y!}{(x+y+1)!}=\frac{1}{x+y+1}\binom{x+y}x^{-1}.
$$
Applying this in your case ($x=n-k,y=n+k$) one has:
$$
\binom{2n}{n-k}^{-1}=(2n+1)\int_0^1 t^{n-k}(1-t)^{n+k}dt
$$
or
$$\begin{align}
\sum_{n=k}^\infty\binom{2n}{n-k}^{-1}
&=\sum_{n=k}^\infty(2n+1)\int_0^1 t^{n-k}(1-t)^{n+k}dt\\
&=\int_0^1 dt \sum_{n=k}^\infty(2n+1)t^{n-k}(1-t)^{n+k}\\
&=\int_0^1 \frac{(1-t)^{2k}[2+(2k-1)(1-t+t^2)]}{(1-t+t^2)^2}dt\tag1
\end{align}$$
It can be shown that the integral $(1)$ is a sum of a rational number and a multiple of $\dfrac\pi{9\sqrt3}$.
Indeed:
$$
\frac{(1-t)^{2k}[2+(2k-1)(1-t+t^2)]}{(1-t+t^2)^2}=Q_k(t)+\frac{A_k^0+A_k^1t+A_k^2t^2+A_k^3t^3}{(1-t+t^2)^2},\tag2
$$
where both coefficients of the polynomial $Q(t)$ and $A^0,A^1,A^2,A^3$ are integer numbers. The integral of $Q(t)$ is obviously a rational number and
$I_r= \int_0^1\frac{t^r\,dt}{(1-t+t^2)^2}$ can be evaluated as:
$$
I_0=\frac23+\frac49\dfrac\pi{\sqrt3};\quad
I_1=\frac13+\frac29\dfrac\pi{\sqrt3};\quad
I_2=-\frac13+\frac49\dfrac\pi{\sqrt3};\quad
I_3=-\frac23+\frac59\dfrac\pi{\sqrt3}.\quad
$$
Thus the irrational term can be written as:
$$
C_k\frac\pi{9\sqrt3}\quad\text{with}\quad C_k=4A_k^0+2A_k^1+4A_k^2+5A_k^3.
$$
Moreover the term can be evaluated explicitly using the following table:
$$
\begin{array}{c|c|c|c|c|c}
k\mod 3& A_k^0&A_k^1&A_k^2&A_k^3&C_k\\
\hline
0&+1-2x&+1+6x&-1-6x&+0+4x&2\\
1&+2+4x&-5-6x&+3+6x&-1-2x&\hphantom{-1}5+18x\\
2&-3-2x&2&0&-1-2x&-13-18x\\
\hline
\end{array},\tag3
$$
with $x=\left\lfloor\dfrac k3\right\rfloor$, so that $C_k=2,5,-13,2,23,-31,2,41,-49,2,\dots$ for $k=0,1,2,3,4,5,6,7,8,9,\dots$.

The expression $(3)$ can be proved in the following way:
Let
$$
P_k(t)=(1-t)^{2k}[2+(2k-1)(1-t+t^2)];\quad R_k(t)=A_k^0+A_k^1t+A_k^2t^2+A_k^3t^3.\\
$$
Then we have from (2):
$$R_k(t_\pm)=P_k(t_\pm);\quad R'_k(t_\pm)=P'_k(t_\pm),\tag4$$
where
$$
t_\pm=e^{\pm\frac{i\pi}3}
$$
are the roots of the polynomial $t^2-t+1$.
Explicitly (4) amounts to the system of four linear equations:
$$\begin{align}
A_k^0+A_k^1t_\pm+A_k^2t_\pm^2+A_k^3t_\pm^3&=2t_\pm^{-2k}\\
A_k^1+2A_k^2t_\pm+3A_k^3t_\pm^2&=(1-2k-2t_\pm)t_\pm^{-2k}\\
\end{align},
$$
which solutions are given by (3).
Answer: Maple writes your sum as a hypergeometric function:
$$ S_k = {\mbox{$_3$F$_2$}(1,1,1+2\,k;\,k+1,1/2+k;\,1/4)}$$
The values for $k=0$ to $3$ are
$$ \eqalign{k = 0: &{\frac{4}{3} + \frac {2\,\sqrt {3}\pi}{27}}
\cr
k = 1: &\frac13+{\frac {5\,\sqrt {3}\pi}{27}}
\cr
k = 2: &{\frac{23}{6}}-{\frac {13\,\sqrt {3}\pi}{27}}
\cr
k = 3: &\frac34+{\frac {2\,\sqrt {3}\pi}{27}}
\cr
}$$
Maple doesn't give a closed form for $k=4$ and up.  But I can get $k=4$ this way.
With $n = k + m$, write the summand as
$$ \frac{(m+2k)!\; m!}{(2m+2k)!} = \frac{m!^2}{(2m)!} \prod_{j=1}^{2k} \frac{m+j}{2m+j}  = 2^{-k}  \frac{m!^2}{(2m)!} \frac{\prod_{i=k+1}^{2k} (m+i)}{\prod_{i=0}^{k-1} (2m + 2i+1)}$$
Expand the quotient of products in partial fractions as a constant plus a sum of coefficients over $2m+j$.  Then sum individually.
That gives me
$$ S_4 = -\frac{211}{60} + \frac{23 \sqrt{3}\pi}{27}$$
but no farther since Maple won't give a closed form for
$$ \sum_{m=0}^\infty \frac{m!^2}{(2m)!\; (2m+9)}$$
However, I think it should be possible to get closed forms for these: stay tuned....
EDIT: OK, it seems that
$$ F(z) = \sum_{m=0}^\infty \frac{m!^2}{(2m)!} z^{2m} = \frac{4}{4-z^2} + \frac{4z}{(4-z^2)^{3/2}} \arcsin(z/2) \ \text{for} |z|<2$$
so that
$$ \sum_{m=0}^\infty \frac{m!^2}{(2m)!(2m+j)} = \int_0^1 F(z) z^{j-1}\; dz $$
and these can be done in closed form.  So this gives me, for example,
$$S_5 = \frac{6169}{840} - \frac{31 \sqrt{3} \pi}{27} $$
and
$$ S_6 = \frac{1709}{2520} + \frac{2 \sqrt{3} \pi}{27} $$
Answer: Just to add more terms in the list.
$$\left(
\begin{array}{cc}
 0 & \frac{4}{3}+\frac{2 \pi }{9 \sqrt{3}} \\
 1 & \frac{1}{3}+\frac{5 \pi }{9 \sqrt{3}} \\
 2 & \frac{23}{6}-\frac{13 \pi }{9 \sqrt{3}} \\
 3 & \frac{3}{4}+\frac{2 \pi }{9 \sqrt{3}} \\
 4 & -\frac{211}{60}+\frac{23 \pi }{9 \sqrt{3}} \\
 5 & \frac{6169}{840}-\frac{31 \pi }{9 \sqrt{3}} \\
 6 & \frac{1709}{2520}+\frac{2 \pi }{9 \sqrt{3}} \\
 7 & -\frac{24923}{3465}+\frac{41 \pi }{9 \sqrt{3}} \\
 8 & \frac{3941153}{360360}-\frac{49 \pi }{9 \sqrt{3}} \\
 9 & \frac{7457}{11440}+\frac{2 \pi }{9 \sqrt{3}} \\
 10 & -\frac{26565167}{2450448}+\frac{59 \pi }{9 \sqrt{3}} \\
 11 & \frac{338662421}{23279256}-\frac{67 \pi }{9 \sqrt{3}} \\
 12 & \frac{29719175}{46558512}+\frac{2 \pi }{9 \sqrt{3}} \\
 13 & -\frac{5168552017}{356948592}+\frac{77 \pi }{9 \sqrt{3}} \\
 14 & \frac{40526745521}{2230928700}-\frac{85 \pi }{9 \sqrt{3}} \\
 15 & \frac{50607208969}{80313433200}+\frac{2 \pi }{9 \sqrt{3}} \\
 16 & -\frac{42190362918239}{2329089562800}+\frac{95 \pi }{9 \sqrt{3}} \\
 17 & \frac{3146154503067509}{144403552893600}-\frac{103 \pi }{9 \sqrt{3}} \\
 18 & \frac{2312776975921}{3702655202400}+\frac{2 \pi }{9 \sqrt{3}} \\
 19 & -\frac{1570173112141273}{72201776446800}+\frac{113 \pi }{9 \sqrt{3}} \\
 20 & \frac{27153272350852367}{1068586291412640}-\frac{121 \pi }{9 \sqrt{3}} \\
 21 & \frac{473757364639811}{763275922437600}+\frac{2 \pi }{9 \sqrt{3}} \\
 22 & -\frac{132365433369215539}{5215718803323600}+\frac{131 \pi }{9 \sqrt{3}} \\
 23 & \frac{1183965646415001041}{40777437916893600}-\frac{139 \pi }{9 \sqrt{3}} \\
 24 & \frac{63942535017037643}{103511957789037600}+\frac{2 \pi }{9 \sqrt{3}} \\
 25 & -\frac{20848468248125325229}{718702343285249700}+\frac{149 \pi }{9 \sqrt{3}}
   \\
 26 & \frac{101214560276297183097311}{3099044504245996706400}-\frac{157 \pi }{9
   \sqrt{3}} \\
 27 & \frac{3717806717925956611}{6041022425430792800}+\frac{2 \pi }{9 \sqrt{3}} \\
 28 & -\frac{1340201691907238426586179}{41062339681259456359800}+\frac{167 \pi }{9
   \sqrt{3}} \\
 29 & \frac{26967769215388607878493}{743209767986596495200}-\frac{175 \pi }{9
   \sqrt{3}} \\
 30 & \frac{100780402282112197484981}{164249358725037825439200}+\frac{2 \pi }{9
   \sqrt{3}}
\end{array}
\right)$$

Sum of reciprocal binomial coefficients


Why the ArrayDeque class use bitwise operation in the pollFirst method?

Question: I look through java source code try to learn the implementation of collection.
Found a interesting thing in the ArrayDeque class.
public E pollFirst() {
    int h = head;
    @SuppressWarnings("unchecked")
    E result = (E) elements[h];
    // Element is null if deque empty
    if (result == null)
        return null;
    elements[h] = null;     // Must null out slot
    head = (h + 1) & (elements.length - 1);
    return result;
}

public E pollLast() {
    int t = (tail - 1) & (elements.length - 1);
    @SuppressWarnings("unchecked")
    E result = (E) elements[t];
    if (result == null)
        return null;
    elements[t] = null;
    tail = t;
    return result;
}

What does the following 2 lines mean?
Is it a bitwise operation? Why they use it and what's the purpose here?
    head = (h + 1) & (elements.length - 1);
    int t = (tail - 1) & (elements.length - 1);

I know one scenario to use the bitwise is to pack 2 values in 1 variable. But it seems this is not the case.
Answer: Take a look at the initialization code - the Deque is represented as an array whose size is always a power of 2:
195    public ArrayDeque(int numElements) {
196        allocateElements(numElements);
197    }

124    private void allocateElements(int numElements) {
125        int initialCapacity = MIN_INITIAL_CAPACITY;
126        // Find the best power of two to hold elements.
127        // Tests "<=" because arrays aren't kept full.
128        if (numElements >= initialCapacity) {
129            initialCapacity = numElements;
130            initialCapacity |= (initialCapacity >>>  1);
131            initialCapacity |= (initialCapacity >>>  2);
132            initialCapacity |= (initialCapacity >>>  4);
133            initialCapacity |= (initialCapacity >>>  8);
134            initialCapacity |= (initialCapacity >>> 16);
135            initialCapacity++;
136
137            if (initialCapacity < 0)   // Too many elements, must back off
138                initialCapacity >>>= 1;// Good luck allocating 2 ^ 30 elements
139        }
140        elements = (E[]) new Object[initialCapacity];
141    }

so elements. Length - 1 in binary is basically a series of 1 bits before the size of the array is exceeded.
For instance, if elements is initialized to an array of size 16, then elements.length - 1 is 15, meaning 0..001111 (truncated leading zeros).
So when the head element is reset in pollFirst method (advanced by one), the bitwise & operator is used in order to make the Deque cyclic. Again, if elements is of size 16 and currently head is 15, then head + 1 would be 16, and so:
10000
01111
-----
00000

Meaning, the head is reset to index 0. This allows you to reuse the space you've already allocated, using the array and its O(1) efficiency in inserting and retrieval, without needing to allocate new space.
The same is true for pollLast where you reset the tail variable, i.e. if tail is 0 and elements is of size 16, then:
tail      00000
tail-1    11111   (-1 in two's complement)
          01111
          -----
          01111

Meaning tail is decremented one value, but moves from 0 to elements.length - 1.
You could have achieved the same with a more "complex" if statement (or by using the trinary operator), however this is a fairly common and acceptable way of implementing a cyclic array.
Answer: It's a more efficient way of calculating (head+1) % elements.length, which we can do because elements.length is a power of 2. It is more efficient because the mod operator is expensive compared to bitwise and.
On the other end, only using mod for the tail would not work because in Java, -1 % N == -1.


Help following instructions (stuck at editing a file)

Question: I'm following these instructions. This is what I've done so far:

Installed JDK
Downloaded NDK & SDK, extracted them and MOVED them to usr/src
Configured my $PATH
Ran android and now I'm installing API 15

Now I am on this step:

Edit build-scripts/androidbuild.sh, find the $ANDROID update project
  line, and add --target 'android-XX' to it (replace XX with your
  installed API number above)

Where is build-scripts/androidbuild.sh?
Ran find /usr/src/ -name build-scripts/androidbuild.sh, no results. What should I do.
I just want to run my C + SDL program on android. I don't know anything about linux stuff. Any way I could accomplish this via lubuntu software center?
Comment: take a look at this http://stackoverflow.com/questions/18838635/compiling-sdl2-for-android
Answer: The file ./build-scripts/androidbuild.sh is in the source code folder of libSDL.
Look here and download the source code archive
E.G.:
wget https://libsdl.org/release/SDL2-2.0.3.tar.gz
tar xf SDL2-2.0.3.tar.gz
cd SDL2-2.0.3

After that you can see the folder build-scripts
./build-scripts/androidbuild.sh


Remove space between border-image and linear background

Question: Edit : added Codepen
I have a small issue with my css, there us a weird space between border-image and linear background on the top and the left of a button. Could you help me to remove it please? Thank you for your help.
Here is the codepen. The problem is on the button "text". I seems like the problem appears only on certain levels of zoom on Chrome : https://codepen.io/zamehan/pen/ZMXWeg

Here is the associated css, the button has the class .special-button :
<code>.special-button{
    background: linear-gradient(to right, #ececec 0%,#ececec 50%, #ececec 50%,#f1d0c1 50%,#f1d0c1 100%) no-repeat ;
    color:#616060;
    border: 1px solid transparent;
    border-image:  linear-gradient(to right, #ececec 0%,#ececec 50%, #ececec 50%,#f1d0c1 50%,#f1d0c1 100%) 5 !important;
}

.color-button {
    font-family: "Noxa";
    flex: 1 100%;
    margin: 6px;
    font-weight: 700;
    letter-spacing: 0.8px;
}

button {
    color:white;
    border: none;
    text-align: center;
    text-decoration: none;
    padding: 6px 11px;
    font-size: 12px;
    margin: 4px 5px;
    background-position: center;
    cursor: pointer;
    &[data-color="dark"] {
        $color: #616060;
        color: $color !important;
        &[data-selected="true"] {
            color: lighten($color, 10%) !important;
        }
    }
    border: 1px solid transparent;
}
</code>
Comment: share the html code .. as a side note, the boder image is useless since you are using the same gradient
Comment: share `html` code too
Comment: You should give us html code as well or put this in some codepen and give link here
Comment: What browser are you using as I do not get the additional line in chrome: http://jsfiddle.net/vjnfcmtu/
Comment: Try removing `  border: 1px solid transparent;`
Comment: hey, try to click inspect on button and look styles:) maybe it will help.
Answer: I ran into this problem too, and found the following article:
https://css-tricks.com/the-backgound-clip-property-and-use-cases/
I set the background-clip property of the element with the linear-gradient to "padding-box" and the line/space went away.


What are the security reasons for disabling plugin.jar /JSObject

Question: I am developing an applet and wonder what are the exact security reasons why 'plugin.jar' is not accessible like the rest of the Java libraries and why JSObject also is disabled for security reasons.
The only statement I can find, is that it is disabled for security reasons but I can't find any further information.
http://docs.oracle.com/javase/7/docs/technotes/guides/plugin/developer_guide/java_js.html
Hope someone here knows some details.
Answer: My take on it is that as soon as another language is involved that is not immediately subject to the Java security sand-box, the JVM effectively cannot be held responsible for the total effect of applet/other thing and notes that is therefore insecure.
See the other side of the effect with the AccessController do privileged methods.  They add trust back into an applet method that might be called by a 3rd party.
Comment: Yes, seems to be reasonable. Nevertheless unfortunate that there is no official statement