Find the probability of double drawing

Question: Two urns contain, respectively, $20$ white and $10$ black balls and $10$ white and $50$ black balls. $5$ balls are transferred from the first urn to the second urn, and then $1$ ball is drawn from the second urn and is found to be white.  What is the probability that this white ball comes form first urn?
I can find the probability of 'the ball drawn from the 2nd urn is found to be white', I also can find the probability of 'the five balls transferred are all white when it is known that we found the ball from the 2nd urn is white'... but this particular Question is very confusing

How do I find the probability of drawing a white ball from the second urn?

let
$W =$ White, $B= $ Black
$X \equiv$ Event , founding a white ball from 2nd urn.
Following events are considered when transferring the balls from 1st urn to 2nd urn :
$A \equiv $ Event, $(1W,4B)$
$B \equiv $ Event, $(2W,3B)$
$C \equiv $ Event, $(3W,2B)$
$D \equiv $ Event, $(4W,1B)$
$E \equiv $ Event, $(5W,0B)$
$P(X) = P(X|A)P(A)+ P(X|B)P(B)+P(X|C)P(C)+P(X|D)P(D)+P(X|E)P(E)$
Comment: Consider cases:  When the five balls are transferred from the first urn to the second, the number of white balls that are transferred can vary from $0$ to $5$.
Comment: Please do not use all-caps in your question. This is basically the most universal rule on the internet so you should've figured it's not appropriate on maths.SE either.
Comment: What do you mean by : "and then 1 ball is drawn form the second urn and is found to be white"
Comment: after transferring 5 balls arbitrarily from 1st urn to 2nd urn, one ball is drawn from the 2nd urn and found to be white..
Comment: When you pose a question here, it is expected that you include your own attempt to solve the problem and indicate where you are stuck so that you receive responses that address the specific difficulties you are encountering.
Comment: I tried this problem using Bayes' theorem... but failed to do that.
Comment: I can find the probability of 'the ball drawn from the 2nd urn is found to be white', I also can find the probability of 'the five balls transferred are all white when it is known that we found the ball from the 2nd urn is white'... but this particular Question is very confusing
Comment: The edited question is a big improvement. You might want to edit again to show how you found the probability that the ball drawn from the second urn is white. That would give a clue to other facts you have deduced about the problem, which people could use to give you better hints; it might also be the case that you made an error already, and someone will spot it and help set you on the right track.
Comment: Your method for finding $P(X)$ is correct and will give the right answer if you calculate all the probabilities on the right-hand side correctly. There is an easier method that relies on finding the _expected_ number of white balls in the second urn just before a ball is drawn from the urn, but you have to understand expected values well in order to be sure you can use that method correctly.
Answer: Hint
Say that the white balls of the first box are in fact blue. So, what is the probability to draw on a blue ball ?
Comment: @DEEP If you only draw one ball from the first urn, probability that _that_ ball is blue is then $20/30$. But that tells you little about drawing five balls from the first urn or one ball from the second urn.
Comment: probability of finding blue ball is 20 /30
Comment: still confused.. :-(
Answer: A good first step in trying to apply Bayes' theorem is to write it down.
There are a few different ways people do this, but the formula could look like this:
$$
P(B \mid A) = \frac{P(A \cap B)}{P(A)} = \frac{P(A \mid B)\,P(B)}{P(A)}.
$$
In order to apply the theorem, you need $A$ and $B$ in the formula to match two events you can describe in your problem.
Since you want to know the probability that the ball drawn from the second urn came originally from the first urn, given that the ball drawn from the second urn is white, and since $P(B \mid A)$ can be read "probability of $B$ given $A$," the obvious way to apply the theorem is to set
\begin{align}
A &= \text{the ball drawn from the second urn is white}, \\
B &= \text{the ball drawn from the second urn came originally from the first urn}.
\end{align}
You say you found $P(A)$ already, so you next might want to try to find
$P(A\cap B)$ or to find $P(B)$ and $P(A \mid B)$.
Comment: I get the same answer in the same way.
Comment: Yes, i also do in this way ... but event $A\cap B$ and event $B$ is not cleared to me
Comment: Of all the probabilities you might need, $P(B)$ should be the easiest to find: just before you draw the ball from the second urn, how many balls were in that urn and how many of those came from the first urn? Then you only need $P(A\mid B)$, which also is not hard if you think about what it really means.
Comment: okay before drawing the ball from the 2nd urn there are total 65 balls in the 2nd urn. and 5 of them are from 1st urn
Comment: i think $P(B)= {5 \over 65}$ ... isn't it??
Comment: now to find $P(A|B)$ we have to consider the $(A|B)$ =  F and $P(F)$ =$  P(F|A1)P(A1)+ P(F|A2)P(A2)+P(F|A3)P(A3)+P(F|A4)P(A4)+P(F|A5)P(A5)$ where F= finding one white,A[i]= finding i white balls at transferring i.e       (i W,(5-i)B) ... isn't it ??
Comment: I wouldn't do separate cases like that. For $P(A\mid B)$ the "given" event $B$ says the single ball you draw at the very end came from the first urn. Consider one of the balls originally in that urn before you do any drawing. Does that ball have a better chance or a worse chance to be in your hand on the last draw, compared to any other ball the first urn?
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/29610/discussion-between-deep-and-david-k).

Find the probability of double drawing


getter-setter error #python

Question: Gostaria de saber onde está o erro nesse código supostamente simples
class Nome:
    def __init__(self,nome,sobrenome):
        self.__nome = nome # variável privada
        self.__sobrenome = sobrenome # variável privada
        self.nome_completo = nome + ' ' + sobrenome # variável pública
    def get_nome(self):
        return self.__nome
    def get_sobrenome(self):
        return self.__sobrenome
    def set_nome(self,novo_nome):
        self.__nome = novo_nome
    def set_nome(self,novo_sobrenome):
        self.__sobrenome = novo_sobrenome

a = Nome('Alan','Aragao')
a.set_nome('ovo')
a.get_nome()

quando eu executo o código, o nome 'ovo' não é retornado, e sim o nome 'Alan'
Answer: Acontece que você definiu dois métodos set_nome, um está trocando o valor da propriedade __nome e o outro da propriedade __sobrenome:
def set_nome(self,novo_nome):
    self.__nome = novo_nome
def set_nome(self,novo_sobrenome):
    self.__sobrenome = novo_sobrenome

Com isso ao chamar o método set_nome:
a.set_nome('ovo')

Na verdade quem está sendo atualizada é a propriedade __sobrenome

Você pode corrigir o nome do método set_nome que atualiza a propriedade __sobrenome para set_sobrenome, corrigindo assim a situação:
def set_sobrenome(self,novo_sobrenome):
    self.__sobrenome = novo_sobrenome

Veja o exemplo com o seu código completo:
class Nome:
    def __init__(self,nome,sobrenome):
        self.__nome = nome # variável privada
        self.__sobrenome = sobrenome # variável privada
        self.nome_completo = nome + ' ' + sobrenome # variável pública

    def get_nome(self):
        return self.__nome

    def get_sobrenome(self):
        return self.__sobrenome

    def set_nome(self,novo_nome):
        self.__nome = novo_nome

    def set_sobrenome(self,novo_sobrenome):
        self.__sobrenome = novo_sobrenome

a = Nome('Alan','Aragao')
a.set_nome('ovo')

print("Nome:", a.get_nome())
print("Sobrenome:", a.get_sobrenome())

Veja online: https://repl.it/repls/DirtyLimegreenThings
Comment: vish! passou despercebido heheh. Obrigado, amigo, boa sorte com seus projetos.

getter-setter error #python


Spark 2.4 to Elasticsearch : prevent data loss during Dataproc nodes decommissioning?

Question: My technical task is to synchronize data from GCS (Google Cloud Storage) to our Elasticsearch cluster.
We use Apache Spark 2.4 with the Elastic Hadoop connector on a Google Dataproc cluster (autoscaling enabled).
During the execution, if the Dataproc cluster downscaled, all tasks on the decommissioned node are lost and the processed data on this node are never pushed to elastic.
This problem does not exist when I save to GCS or HDFS for example.
How to make resilient this task even when nodes are decommissioned?
An extract of the stacktrace:

Lost task 50.0 in stage 2.3 (TID 427, xxxxxxx-sw-vrb7.c.xxxxxxx, executor 43): FetchFailed(BlockManagerId(30, xxxxxxx-w-23.c.xxxxxxx, 7337, None), shuffleId=0, mapId=26, reduceId=170, message=org.apache.spark.shuffle.FetchFailedException: Failed to connect to xxxxxxx-w-23.c.xxxxxxx:7337
Caused by: java.net.UnknownHostException: xxxxxxx-w-23.c.xxxxxxx
Task 50.0 in stage 2.3 (TID 427) failed, but the task will not be re-executed (either because the task failed with a shuffle data fetch failure, so the previous stage needs to be re-run, or because a different copy of the task has already succeeded).

Thanks.
Fred
Answer: I'll go through a bit of background on the "downscaling problem" and how to mitigate it. Note that this information applies to both manual downscaling as well as preemptible VMs getting preempted.
Background
Autoscaling removes nodes based on the amount of "available" YARN memory in the cluster. It does not take into account shuffle data on the cluster. Here's an illustration from a recent presentation we gave.
In a MapReduce-style job (Spark jobs are a set of MapReduce-style shuffles between stages), data from all mappers must get to all reducers. Mappers write their shuffle data to local disk, and then reducers fetch data from each mapper. There is a server on every node dedicated to serving shuffle data, and it runs outside of YARN. Therefore, a node can appear idle in YARN even though it needs to stay around to serve its shuffle data.

When a single node gets removed, pretty much all reducers will fail, since they all need to fetch data from every node. Reducers will specifically fail with FetchFailedException (as you saw), indicating they were unable to get shuffle data from a particular node. The driver will eventually re-run necessary mappers, and then re-run the reduce stage. Spark is a bit inefficient (https://issues.apache.org/jira/browse/SPARK-20178), but it works.

Note that you can lose nodes in one of three scenarios:

Intentionally removing nodes (autoscaling or manual downscaling)
Preemptible VMs
getting preempted. Preemptible VMs get preempted at least every 24 hours.
(Relatively rare) a standard GCE VM is
ungracefully terminated by GCE, and restarted. Usually, standard VMs
are transparently live migrated.

When you create an autoscaling cluster, Dataproc adds several properties to improve
job resiliency in the face of losing nodes:
yarn:yarn.resourcemanager.am.max-attempts=10
mapred:mapreduce.map.maxattempts=10
mapred:mapreduce.reduce.maxattempts=10
spark:spark.task.maxFailures=10
spark:spark.stage.maxConsecutiveAttempts=10
spark:spark.yarn.am.attemptFailuresValidityInterval=1h
spark:spark.yarn.executor.failuresValidityInterval=1h

Note that if you enable autoscaling on an existing cluster, it will not have these properties set. (But you can set them manually when creating a cluster).
Mitigations
1) Use graceful decommissioning
Dataproc integrates with YARN's Graceful Decommissioning, and can be set on Autoscaling Policies or manual downscale operations.
When gracefully decommissioning a node, YARN keeps it around until applications that ran containers on the node finish, but does not let it run new containers. That gives nodes an opportunity to serve their shuffle data before being removed.
You will need to ensure that your graceful decommission timeout is long enough to encompass your longest jobs. The autoscaling docs suggest 1h as a starting point.
Note that graceful decommissioning only really makes sense only long-running clusters that process lots of short jobs.
On ephemeral clusters, you would be better off "right-sizing" the cluster from the start, or disabling downscaling unless the cluster is completely idle (set scaleDownMinWorkerFraction=1.0).
2) Avoid preemptible VMs
Even when using graceful decommissioning, preemptible VMs will be periodically terminated through "preemptions". GCE guarantees preemptible VMs will get preempted within 24 hours, and preemptions on large clusters are very spread out.
If you are using graceful decommissioning, and the FetchFailedException error messages include -sw-, you are likely seeing fetch failures due to nodes being preempted.
You have two options to avoid using preemptible VMs:
 1. In your autoscaling policy, you can set secondaryWorkerConfig to have 0 min and max instances, and instead put all workers in the primary group.
 2. Alternatively, you can keep using "secondary" workers, but set --properties dataproc:secondary-workers.is-preemptible.override=false. That will make your secondary workers be standard VMs.
3) Long term: Enhanced Flexibility Mode
Dataproc's Enhanced Flexibility Mode is the long term answer to the shuffle problem.
The downscaling problem is caused by shuffle data getting stored on local disk. EFM will include new shuffle implementations that allow placing shuffle data on a fixed set of nodes (e.g. just primary workers), or on storage outside of the cluster.
That will make secondary workers stateless, which means they can be removed at any time. This makes autoscaling far more compelling.
At the moment, EFM is still in Alpha, and does not scale to real-world workloads, but look out for a production-ready Beta by the summer.


django-restframework serialize relations as dictionaries rather then arrays

Question: I'm trying to serialize foreign keys as dictionaries instead of arrays.
Right now the json looks the following:
{
"slug": "en",
"children": [{
        "slug": "pants",
        "children": [{
                "slug": "products/:level1",
                "children": [{
                    "slug": ":level2/:level3",
                    "children": []
                }]
            },
            {
                "slug": ":productSlug",
                "children": []
            }
        ]
    },
    {
        "slug": "pullovers",
        "children": []
    }
   ]
}

But I'd like it to use the slugs as keys:
{
"en": {
    "children": {
        "pants": {
            "children": {
                "products/:level1": {
                    "children": {
                        ":level2/:level3": {
                            "children": {}
                        }
                    }
                }
            },
            ":productSlug": {
                "children": {}
            }
        ]
    }
 }
}

Is it possible to do that directly in the serializer or do I have to convert it in an additional step?
Answer: It is possible by overriding the list serializers and setting list_serializer_class for each serializer that requires it, as I suggested in this answer.
Of course, you will need to adapt it slightly:
class <YourClass>ListSerializer(serializers.ListSerializer):
    def to_representation(self, data):
        r = super().to_representation(data)

        return { item['<key_field>']: item for item in r }

Comment: Nice answer. I had to tweak it a little, so I was able to use it for the top level as well. Please look at my answer.
Answer: @Michael Rigonis answer (https://stackoverflow.com/a/45238191/270265) was the key to success. I had to tweak it a little, so I was able to use it for the top level as well.:
class DictSerializer(serializers.ListSerializer):
    key = None

    def __init__(self, *args, **kwargs):
        self.key = kwargs.pop('key', self.key)
        super().__init__(*args, **kwargs)

    def to_representation(self, data):
        r = super().to_representation(data)
        return {item[self.key]: item for item in r}

    @property
    def data(self):
        # This is a bit nasty, because the only "Many-Serializer" is a ListSerializer we inherit of it,
        # but when converting it to json we call the BaseSerializer directly, because we want a Dictionary rather then a list
        ret = super(serializers.ListSerializer, self).data
        return ReturnDict(ret, serializer=self)

Comment: Looking at DRF code, it indeed seems necessary to use `ReturnDict` in `data`.


test of two proportions

Question: Apologies for an elementary question - I am trying to understand the solution to the following problem:

"Suppose interviews were conducted at two different shopping centers. This two sample t-test between percents could be used to determine if
  the responses from the two shopping centers were different. The
  research question is: Is there a significant difference between
  shopping centers A and B with respect to the percent that say they
  would buy product X? The null hypothesis is: There is no significant
  difference between shopping centers A and B with respect to the
  percent of people that say they would buy product X. A two-tailed
  probability will be used because the hypothesis does not state the
  direction of the difference. The results of the survey were: 89 people
  were interviewed as shopping center A and 57 of them (64.0%) said they
  would buy product X. 92 people were interviewed as shopping center B
  and 51 of them (55.4%) said they would buy product X."

This is an example given at https://www.statpac.com/statistics-calculator/percents.htm
And they go on to give an answer:

"t-value = 1.179 Degrees of freedom = 179 Two-tailed probability = .240"

Could someone help me understand how the answers are obtained?
Answer: I don't know what you have done in the Online Calculator, but it seems to me
that this is a test to see if proportions of buyers in the two shopping
centers are the same. I don't see how it would be appropriate to do a
two-sample t test here. (However, a P-value around 0.24 does seem roughly correct.)
It seems as if this question may need a 'self-study' tag, so to get you
started on a useful track, I will show results from a couple of statistical
software packages, and a formula from a standard textbook. Methods for
this test differ slightly among various texts and software packages, a
textbook you have may give a slightly different answer.
In Minitab 17 the test looks like this:
Test and CI for Two Proportions

Sample   X   N  Sample p
1       57  89  0.640449
2       51  92  0.554348

Difference = p (1) - p (2)
Estimate for difference:  0.0861016
95% CI for difference:  (-0.0562171, 0.228420)
Test for difference = 0 (vs ≠ 0):  Z = 1.19  P-Value = 0.236

Also, here is the output from R statistical software (without continuity correction):
prop.test(c(57,51), c(89,92), corr=F)

        2-sample test for equality of proportions without continuity correction

data:  c(57, 51) out of c(89, 92)
X-squared = 1.3936, df = 1, p-value = 0.2378
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.05621706  0.22842029
sample estimates:
   prop 1    prop 2
0.6404494 0.5543478

A formula for the test statistic in Ott & Longnecker, 5e, Chapter 10
is as follows:
With estimates $\hat p_a = 0.640, \hat p_b = 0.554,$ the normally
distributed test statistic is
$ Z = \frac{\hat p_a - \hat p_b}{\text{SE}},$ where
$\text{SE}^2 = \frac{\hat p_a(1-\hat p_a)}{89} + \frac{\hat p_b(1-\hat p_b)}{92}.$
The first term in $\text{SE}^2$ is an estimate of $Var(\hat p_a)$
and the second term is an estimate of $Var(\hat p_b).$
The approach used here involves a normal approximation to the difference
of two binomial proportions. Sample sizes are large enough and estimates
are far enough from 0 and 1 that the normal approximation should work quite well.
If you need further help with this, please edit some of your work based
on what I have shown into your question, and explain your difficulty.
Then perhaps someone will provide a clarification.
Comment: Thank you @BruceET. Being a newbie, my upvote isn't counted. The answers I quoted were provided by the site - not something I calculated. I take from you that the site is incorrect in applying t-test in this case. If it were the right test to apply, how would they have got the answers? Given that they got the test incorrect, perhaps the answers too are?
Comment: To an extent (certainly not invariably), the menu systems in established software such as Minitab, Stata, SPSS may offer a useful guide what procedure to use. Even so, best to know what you're doing before engaging software. // Sites that pop up on the Internet are considerably less reliable. // The correct analysis uses an approx of the normal distribution to functions of binomial random variables. I don't know what proc the online your 'Calculator' used. Note that the t dist'n with 179 DF is nearly normal, so hardly surprising that _some_ calc with your data produced close to right answer.
Comment: Welcome to the site. If you are going to ask more questions here it is probably worthwhile to 'take the tour' of the site. No worries about up-voting for now. That will come soon. I think already you can click to 'Accept' and answer when you feel you've gotten what you need from the various answers and comments. (That will take your post off our list of unanswered questions.)
Comment: Thanks @BruceET. Could I oblige you with another question, please. I have some data related to exams: same course, two successive years. I have the counts of students who got A, B, C, and F in both years. Population size is different. The second exam was done in a somewhat different environment. What statistical test should I consider using if my null hypothesis is to say that the new environment made no difference? As you probably guessed, I am true newbie and scoring through several sources to learn some of the very basics.
Comment: I think I get the idea. Provided students are different in the two years, you can make a $4 \times 2$ table of counts: Rows (A, B, C, F), Columns (Yr1, Yr2). Do a chi-squared test for homogeneity of grade proportions across years. Same procedure as test of independence. Look at [this Q&A](https://stats.stackexchange.com/questions/364605/test-for-significant-for-more-than-2x2-consistency-table/364611#364611) for some R code and hints for interpretation. Post table as new question if you have trouble with interpretation, and someone will probably help with that.

test of two proportions


How to store total no. of lines in a variable in a shell script?

Question: I'm recieving this in the error -
syntax error near unexpected token `total_lines=$(wc -l < $extracted_log_path_value)'

$extracted_log_path_value stores the path of the file which is correctly defined. I'm using simple shell script. No plugin is installed. I have also tried following commands -  `
syntax error near unexpected token `total_lines="$(wc -l "$extracted_log_path_value" | cut -d' ' -f1)"'.

This is the code that is written in shell script. I'm pasting the actual line as well -
total_lines="$(wc -l "$extracted_log_path_value" | cut -d' ' -f1)"

`
Comment: @SMA: no, that's not the problem, this is just how bash outputs error messages. Try it out with "$ for foo iiiin bar", you will get something like "bash: syntax error near unexpected token `iiiin' " with the problematic token enclosed in a backtick and a single quote.
Comment: If you quote the variable name, your example is fine. Without the quotes the example is buggy. The problem is somewhere else in your script. The error message says "syntax error near ...", which means the syntax error is before the line you have written in the question.
Comment: What are those two backticks in your plain text, right after "commands -" and in the very end of your question? Do you have any backticks randomly flying around in your code? The problem seems to be somewhere outside of the line that you've given us.
Comment: back ticks and single quote?
Answer: wc does not expect the file to be fed into STDIN, it wants the file name as a simple argument.
total_lines="$(wc -l "$extracted_log_path_value" | cut -d' ' -f1)"

should do the trick. The cut part drops the filename and retains only the number of lines.
---
Correction: if you don't specify any files as arguments, wc does read from STDIN, and prints out the number of lines. If you do it like this, you don't need to invoke an additional cut:
total_lines="$(wc -l < "$extracted_log_path_value")"

Comment: That's correct. However, I strongly prefer to use "$( ... )" instead of backticks, and to wrap everything into double quotes. As long as it is enclosed in double quotes, I don't have to stop and look at it and ask myself "Can I really leave the double quotes out in this case?". I just prefer this more defensive style.
Comment: `wc` reads from stdin if no file name is given in the command line arguments.
Comment: you may remove the wrapper double quotes for the assignment.
Comment: I agree to above as a best practice. `:-)` . Was just making a point regarding the assignment. Being defensive with this is indeed better and safe !
Comment: Still facing issue - syntax error near unexpected token `total_lines="$(wc -l < "$extracted_log_path_value")"'
Comment: tried your way as well - syntax error near unexpected token `total_lines="$(wc -l "$extracted_log_path_value" | cut -d' ' -f1)"'
Comment: @basari66 open a terminal, type in this command, substituting "$extracted_..._value" by a name of some existing file. My version of bash saves the number of lines to the `total_lines` variable. What does your bash version do, if you type only this command, and nothing else? If it breaks, maybe it's because of the context around the command, not because of the command itself.
Comment: @ceving oh, indeed, you're right. If you feed the file to stdin, you don't even need the `cut` in the end. Then the problem definitely isn't in the command itself.
Comment: I got it mates. My bad, i forgot to write 'do' after for loop before the mentioned command and thought there's a problem in wc -l command


Conditions for Simultaneous move game & Sequential move games to lead to same equilibrium

Question: Consider a game with two players with profit functions : $ \pi_1 (p_1; p_2) $ and $ \pi_2 (p_2; p_1) $, where $p_1 $ and $p_2 $ are the players' actions (say prices).
Is there any theorem that establishes conditions, when the pure strategy equilibrium will lead to the same prices, in the following two cases? :

If the prices are decided simultaneously
If the player 1 sets the price first and then player 2 observes the price of player 1 and then sets their own price
Answer: A different way to phrase your question is to ask for sufficient conditions under which the Nash equilibrium for a price competition game yields the same payoffs under simultaneous versus sequential choices.
A partial negative result is in: E. Gal-Or, "First Mover and Second Mover Advantages", International Economic Review, Vol. 26, No. 3 (Oct., 1985), pp. 649-653.
Suppose that the game is symmetric and has a unique symmetric equilibrium under simultaneous choices: then the players' payoffs are the same. This paper shows that, under sequential choices, this is never the case when the best replies are increasing (the Leader earns less than the Follower) or decreasing (the Follower earns less than the Leader). So in this simple context, you would need to have non-monotonic best replies to achieve the desired result.
On the side, let me suggest that you add more context to your question. It seems you are implicitly assuming unique equilibria (how are we supposed to compare multiple equilibria?) and it is unclear whether $\pi_1$ and $\pi_2$ should be supposed to be symmetric.
Comment: Thanks for your answer and the reference. To clarify on the points you asked, the game that is of interest to me is not symmetric ($\pi_1$ and $\pi_2$ are different), but has a unique equilibrium (in both simultaneous move and sequential move).
Comment: The literature has results more general than Gal-Or about non-symmetric games. Do a reverse bibliographic search. Most of these results are going to be negative, I'm afraid.


How to read env in Dockerfile?

Question: I have docker-compose.yml in the root of my project, service frontend and its directory in the root. Frontend has its own .env file. And i need to read that .env inside Dockerfile of frontend. What I found so far is passing env vars in docker-compose.yml but there variables are not being grabbed from /frontend. I also have tried defining env_file in config but does not seem to be working. Is there any workaround or I have to create separate docker-compose for frontend ?
Answer: .env files are for docker-compose. You need to use the docker-compose cli instead of just docker.
You may see this documentation.

How to read env in Dockerfile?


Homotopy and chain homotopy determine each other

Question: In continuation of this previous question, I'm having problems with the following proposition from Kamps and Porter's Abstract Homotopy and Simple Homotopy Theory:

Proposition (3.7). [page 210]

If $H:X\otimes I \rightarrow Y$ is a homotopy between $f$ and $g$, then defining $h(z)=H(0,0,z)$ gives a chain homotopy between $f$ and $g$.
If $h$ is a chain homotopy between $f$ and $g$ then $$H(x,y,z)=f(x)+g(y)+h(z)$$ gives a homotopy, $H:X\otimes I \rightarrow Y$ between $f$ and $g$.

The proof is left to the reader. I tried to work it out but failed. Here are the details:
As in the previous question, $(X\otimes I)_n=X_n \oplus X_n \oplus X_{n-1}$. First of all, we have the cylinder object $(X\otimes I,e_0, e_1, \sigma)$ defined by $e_0:X\rightarrow X\otimes I,\;x\mapsto (x,0,0)$, $e_1:X\rightarrow X\otimes I,\;y\mapsto (0,y,0)$, and $\sigma :X\otimes I\rightarrow X,\;(x,y,z)\mapsto x+y$.
Define $i(z):X_{n-1}\rightarrow X_n \oplus X_n \oplus X_{n-1}$ by $i:z\mapsto (0,0,z)$. Now $h_{n-1}=H_n\circ i$. For the first part we want to prove the relation $$\partial _n ^Y h_{n-1}+h_{n-2}\partial ^X_{n-1}=g_{n-1}-f_{n-1}$$ where $h_n:X_n \rightarrow Y_{n+1}$. I tried calculating separate terms. Since $H$ is a homotopy $g-f=He_1-He_0=H(e_1-e_0)$ so $(g-f)(x)=H(-x,x,0)$. Since $H$ is in particular an arrow in $\mathsf{Ch}_\bullet$, it commutes with boundaries so $\partial ^Y _n H_n =H_{n-1}\partial ^\prime _n$ where $\partial ^\prime$ is the boundary of $X\otimes I$ as in the previous question. Hence $$\partial _n ^Y h_{n-1}=\partial ^Y _n H_n\circ i =H_{n-1}\circ \partial ^\prime _n \circ i$$ which means $\partial _n ^Y h_{n-1}(z)=H_n(0,0,-\partial ^X z)=-h_{n-2}\partial ^X _{n-1}(z)$ and the LHS of the chain homotopy relation is identically zero..
For the second part, I need to verify for instance that $He_0=f$. By definition $$(He_0)(x)=H(x,0,0)=f(x)+g(0)+h(0)$$ but if $f$ and $h$ are homomorphisms, wouldn't $g(0)=h(0)=0$? In that case I get $He_0=f$ without even using the fact $h$ is a chain homotopy..

I would like explanations of my mistakes as well as an actual correct proof of the proposition.
Added: Here is a related question.
Answer: For part 1: The part where you (between lines) write $H_{n-1} \circ \partial '_n \circ i_n = -h_{n-2}\circ \partial^X_{n-1}$ is the error. Remember that $\partial '(0,0,z)=(-z,z,-\partial^X z)$. Then you should get your homotopy.
For part 2: To begin with you only know that $H$ is a degreewise homomorphism. The property of being a chain homotopy comes into play when you want to show that $H$ is actually a chain homomorphism.


Student's t-test with a covariate?

Question: I am testing the different between some variables $X$ and $Y$ using the Student's t-test. I suspect that there might be a latent variable $Z$ that has an effect on both $X$ and $Y$. How could I control $Z$?
I am using Python for the data analysis.
Comment: Do you have any data (directly or indirectly) related to Z? You say you suspect there is a "latent" variable Z, but if you don't have any data, there isn't anything you can do.
Comment: I have measured $Z$ too. It correlates with both $X$ and $Y$, so I think I should try to control its effect.
Answer: One common means of controlling for some other covariate would be via regression. Put the X and Y values into the response (DV), and a Y-group indicator (0 if in X, 1 if in Y) as a DV, along with your covariate (or some suitable proxy for it if the variable can't be measured directly) as another DV.
You regress on your covariate and the group-indicator. If there's a difference between X and Y based on the covariate, this will be "adjusted" for by the regession, and significance of the coefficient of the group-indicator will then be a test of a mean-shift between the two groups after accounting for the covariate.

This is sometimes called ANCOVA (unless Z is a factor, in which case it would usually be called ANOVA; you can still do it using regression).
Some people would formally test parallelism of the two group-lines (by including an interaction between covariate and group-indicator). I think that's wrong-headed (do we believe the hypothesis of no-interaction is exactly true? I don't -- in which case the hypothesis test is a noisy answer to a question we already know the answer to -- surely they're never going to be exactly parallel) ... but don't especially care about. The better question here is "are they close enough to parallel that it doesn't badly impact the properties of the inferences we wish to make?'. Answering that is nearer to measuring an effect size, so a residual display - e.g. residuals vs covariate that distinguishes the groups with symbols or colors - might come closer to addressing that.

However, depending on what you mean by 'latent' in your question, it's possible that you may actually be after something more like an instrumental variable. (There are numerous questions on site on the topic.)
Answer: By means of t-test you are assessing whether there is a significant difference between two sets of data --- e.g. the realizations of two random variables $X$ and $Y$. When using t-test you are doing a hypothesis test, and you can't control for any variable. To be more specific, when doing hypothesis tests you are not establishing any causal relationship between random variables.
If you want to investigate the effect of a latent variable $Z$ on both $X$ and $Y$, you should rely on a regression analysis. For a deeper analysis on the role and effects of some variables (which can cause a bias in your analysis), you can apply some methods of graphical models.
Here you find a nice IPython Notebook on regression.

Student's t-test with a covariate?


Factor analysis using stata "predict" command and get negative value for non-negative variable?

Question: I am running factor analysis on stata to reduce a few variables to a single explanatory variable which means "experience" of a manager (should be non-negative value), however, after using "predict" command I check the range of the new variable and found that there are many negative values, how do I avoid this?
The original variables are 1) number of projects done by a manager 2) number of people interacted for each project 3) number of unfinished projects. Since these 3 variables are highly correlated so I want to use factor analysis to reduce them to one single variable.
If there is any other method that can also tackle this problem, please let me know! Thank you guys!!
Comment: How are latent variable parameters estimated by default in Stata? Often, it's assumed that the mean=0 (with some estimated variance), which would explain negative values.
Comment: @D L Dahly: See my answer of March 22. Factors have zero mean in Stata.
Answer: This is expecting much more of factor analysis than it will give, at least by default. My own view is that this is somewhere between non-standard and downright weird as an application of factor analysis, but there is considerable variation among statistically-minded people on the merits of factor analysis and how it might be well used, so conflicting advice is highly likely.
Factor analysis works on either the correlation or the covariance matrix. Necessarily information on means is discarded and new factors have zero means, so negative values are inevitable. You have missed that in your reading! I suppose that using zero means could be regarded as a convention, not an absolute requirement, but it seems to be a universal convention.
You could, I suppose, rescale a factor using what you think are an appropriate mean and standard deviation, but there is no real gain in that, especially as you want a predictor, not a proxy for your response variable. That rescaling  wouldn't guarantee all values being positive. What would guarantee all positive values would be to factor analyse the logarithms of the variables, predict a factor and then exponentiate, but whether that would work well depends on many things. There would be still a units of measurements issue: in effect units are discarded and you would have to work to put them back. I would expect all these variables you name to be skewed, so transformation might be a good idea on other grounds; however, if there are zeros then logarithmic transformation is not possible, unless you add a constant to make all values positive, in turn in my view not a good idea, but views differ on that.
All that said, I think your whole strategy would strike most statistical people as dubious at best, both statistically and scientifically. You have three predictors. It is easiest and probably most illuminating to keep them separate and to use them all in some regression-type model, possibly a generalised linear model with logarithmic link given that your unnamed (!) response variable is also likely to be non-negative. The correlation between predictors is clearly something you have to think about, but throwing away the detail on those predictors is more likely to make your model difficult to interpret and discuss.
I see no Stata-specific issues here.
Comment: I don't see this as an unusual application of factor analysis -- I've seen tons of these (and by the way, putting factor scores into regression as predictors leads to [measurement error attenuation bias](http://ideas.repec.org/a/spr/psycho/v66y2001i4p563-575.html)), but as Maarten Buis pointed out in another thread, these variables look more like a formative/causal than reflective/effect indicators of the manager experience.
Comment: Stas: It is clearly common that people use factor analysis as preparation for regression (often with little substantive gain, but that's a different point). My suggestion is that you don't need it with three predictors and that expecting results to have values similar to the response is a misunderstanding of what factor analysis does. I don't object to applications to bunches of predictors (in my experience that's more common).
Comment: In addition, note that no single predictor, even a composite from factor analysis, could possibly have as much predictive power in practice as your three predictors combined. In principle, there is an exception: if your predictors are perfectly correlated, then each is as good as any other as a predictor, but you would have noticed that, and you would not seek a way of getting a composite.
Comment: I wrote this 10 years ago. Since then my position on the disadvantages of log (x + 1) or more generally log (x + c) has softened a bit, but that is a side issue and I will stop with this flag.
Comment: Stata got the appropriate `sem` / `gsem` commands... although those would have been available in 2013. There is no need for the factor analysis voodoo, you can just do the appropriate SEM
Answer: The three variables you want to combine sound to me like resources that result in a manager having more experience rather than characteristics of managers that are influenced by his/her experience. The difference between the two is that I think that it is the number of projects done by a manager, number of people interacted for each project, and number of unfinished projects should influence experience rather than experience influencing those three observed variables. If you also think that this is the way that the three observed variables relate to the latent experience variable then you have a problem in that factor analysis assumes that the latent variable influences the observed variables rather than the other way around. (Bollen 1984; Bollen and Lennox 1991) In that case you will probably want to look at something like a sheaf coefficient (Heise 1972) rather than factor analysis. In Stata this is implemented in the sheafcoef package.
Bollen, Kenneth A. 1984. "Multiple Indicators: Internal Consistency or No Necessary Relationship" Quality and Quantity 18(4): 377-385.
Bollen, Kenneth A. and Richard Lennox. 1991. "Conventional Wisdom on Measurement: A Structural Equation Perspective" Psychological Bulletin 110(2): 305-314.
Heise, David R. 1972. "Employing nominal variables, induced variables, and block variables in path analysis." Sociological Methods & Research 1(2): 147-173.
Comment: We had a follow up to the 1991 paper here: http://www.ncbi.nlm.nih.gov/pubmed/19266502
Answer: FA is for unconstrained variables.  First transform the data to make it this way.  Use any inverse CDF (I use the quantile function for the normal distribution, qnorm in R) for variables that start out between 0 and 1; use any log function for positive variables (if zero can occur, add 1 first.)  Then do factor analysis.  The results will be for the transformed variables, use their inverses to recover the original ones, if needed.


Filling a two dimensional array of structs with information from a one dimensional array of structs

Question: I wrote a function that successfully reads a CSV file with the following format for each record:
name_of_class,name_of_class_member,double_associated_with_class,double_associated_with_class_member,another_double_associated_with_class_member

My function reads the CSV file and creates a one-dimensional array of structs, where each record corresponds to an element in the one-dimensional array of structs.
My question is this: how can I copy the information for each record into a two-dimensional array of structures, with the indices data[class][member_of_class] instead of data[record]?
It's tricky, because different classes have different numbers of members. I initialized a 2-dimensional array with the dimensions classes * max_num_members and set the values of the 2D array to zero (or, in the case of char arrays, I used spaces that would later be written over by other characters). So, obviously, there will be many values (equal to zero for doubles or white space for char arrays) in the 2D array that have not been changed by copying from the 1D array. I can skip these in my main algorithm using a sentinel.
The goal is to use my nested for loops later in the code to loop over members of the class as the nested loop and loop over classes as the outer loop. That part of the code is finished and working correctly, but I need the data to be read into the array of structs that the algorithm uses. Anyone know how to do this?
The alternative is to read my records from the csv file into a 2D array, but that really just transfers from the problem from one location to another. Class names have no more than ten characters and member names have no more than thirteen characters.
Here is my code:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <math.h>

struct csv
{
    char class_name[12];
    char member[18];
    double class_value;
    double member_value1;
    double member_value2;
    double calc_output1;  // Program I'm writing adds values here
    double calc_output2; // And here
};

void readFile(char *csvFile, struct csv *csv_rec)
{
    /* code for reading file here - this works as planned.
       Reads csv file and makes 1D array of structs with num_records
       elements */
    FILE *dataFile = NULL;
//  size_t idx_i = 0;       // struct array index i (increments after every record -> resets to zero when a new ligand is found)
//  size_t idx_j = 0;       // struct array index j (only increments when a new ligand is found)
    size_t idx = 0;         // okay - this lie_read struct array will be one-dimensional - one index per record
    size_t fldidx = 0;      // struct field index
    char fileLine[255] = {0};
    char *p = NULL;         // pointer to line

    dataFile = fopen(myFile, "r");

    // Open the file set by the command line and read it into the file object
    dataFile = fopen(myFile, "r");
    if (dataFile == NULL){
        perror("Error opening data file.\n");
        exit(EXIT_FAILURE);     // Exit program if data file cannot be read by runtime service
    }

    while ((fgets (fileLine, 254, dataFile)) != NULL){
        fldidx = 0;     // reset field index for each line
        // Use strtok to separate each line into tokens on ','
        for (p = strtok(fileLine, ","); p != NULL; p = strtok(NULL, ",")){
            // Use fldidx to separate data into struct
            switch (fldidx){

                case 0:
                    strncpy(lie_rec[idx].class_name, p, strlen(p));
                    break;

                case 1:
                    strncpy(lie_rec[idx].member, p, strlen(p));
                    break;

                case 2:
                    lie_rec[idx].class_value = atof(p);
                    break;

                case 3:
                    lie_rec[idx].member_value1 = atof(p);
                    break;

                case 4:
                    lie_rec[idx].member_value2 = atof(p);
                    break;
            }
            // Increment field index
            fldidx++;
        }
        // Increment record index
        idx++;
    }
    // Close dataFile object
    fclose(dataFile);
}

int main(int argc, char *argv[])
{
    int num_records = 75; // Later a function will count these
    struct csv csv_data[m_members][n_classes]; // Declare 2D array
    struct csv csv_read[num_records]; // 1D array of structs
    int i; // index for printing out 2D array
    int j; // index for printing out 2D array
    int members; // members counter for algorithm I wrote
    int classes; // classes counter
    int records; // index of records
    int m_members = 25; // Bigger than it needs to be for any file
    int n_classes = 25; // Bigger than needs to be for any file

    // Initialize csv_data 2D array:
    for (i = 0; i < m; i += 1){
        for (j = 0; j < n; j += 1){
            strcpy(csv_data[i][j].class_name, "          ");
            strcpy(csv_data[i][j].member, "                ");
            csv_data[i][j].class_value = 0.0;
            csv_data[i][j].member_value1 = 0.0;
            csv_data[i][j].member_value2 = 0.0;
            csv_data[i][j].calc_output1 = 0.0;
            csv_data[i][j].calc_output2 = 0.0;
        }
   }
// Initialize csv_read struct array
    for (records = 0; records < num_records; records += 1){
        strcpy(csv_read[records].class_name, "          ");
        strcpy(csv_read[records].member, "                ");
        csv_read[records].class_value = 0.0;
        csv_read[records].member_value1 = 0.0;
        csv_read[records].member_value2 = 0.0;
        csv_read[records].calc_output1 = 0.0;
        csv_read[records].calc_output2 = 0.0;
    }

    if (argc != 2){
        printf("Usage: ./optimization datafile.csv\n");
        return 1;
    }

    readFile (argv[1], csv_read);

    // Here is what I have so far. It doesn't work:
    // First, set member and class indices to zero
    members = 0;
    classes = 0;

   // Next, copy values held in first record to 0,0 index of 2D array
       strcpy(csv_data[classes][members].class_name, csv_read[0].class_name);
       strcpy(csv_data[classes][members].member, csv_read[0].member);
       csv_data[classes][members].class_value = csv_read[0].class_value;
       csv_data[classes][members].member_value1 = csv_read[0].member_value1;
       csv_data[classes][members].member_value2 = csv_read[0].member_value2;

       // This is the part I'm not doing correctly:
      for (records = 1; records < num_records; records += 1){
          if (csv_read[records].class_name == csv_read[records -1].class_name){
              members += 1;
              strcpy(csv_data[classes][members].class_name, csv_read[records].class_name);
              strcpy(csv_data[classes][members].member, csv_read[records].member);
              csv_data[classes][members].class_value = csv_read[records].class_value;
              csv_data[classes][members].member_value1 = csv_read[records].member_value1;
              csv_data[classes][members].member_value2 = csv_read[records].member_value2;
          }
          else{
              members = 0;
              classes += 1;
              strcpy(csv_data[classes][members].class_name, csv_read[records].class_name);
              strcpy(csv_data[classes][members].member, csv_read[records].member);
              csv_data[classes][members].class_value = csv_read[records].class_value;
              csv_data[classes][members].member_value1 = csv_read[records].member_value1;
              csv_data[classes][members].member_value2 = csv_read[records].member_value2;
          }
      }
  for (i = 0; i < n_classes; i += 1){
      for (j = 0; j < m_members; j += 1){
            printf("Class: %s\tMember: %s\tClass val: %.2f\tMem val 1: %.2f\tMem val 2: %.2f\tClass index i: %d\tMember index j: %d\n", \
            csv_data[i][j].class_name, csv_data[i][j].member, csv_data[i][j].class_value, \
            csv_data[i][j].member_value1, csv_data[i][j].member_value2, i, j);

            }
        }
    return 0;
}

EDIT: Consider the following data:
RTI-31,nma-10-325,-9.189617432,-1.695719104,-17.76402898
RTI-31,nma-04-176,-9.189617432,-0.708200772,-17.29511154
RTI-31,nma-04-176r,-9.189617432,-1.776907169,-18.88678457
RTI-31,md2-02-280,-9.189617432,-6.716602886,-17.30751521
RTI-31,nma-10-346,-9.189617432,-11.25116643,-15.84743342
RTI-31,nma-04-263,-9.189617432,-11.7115346,-15.04227866
RTI-31,nma-08-195,-9.189617432,-4.283013542,-14.8644087
RTI-31,nma-04-009,-9.189617432,-9.957288662,-15.3833286
RTI-32,nma-10-088,-8.122152987,3.400737344,-13.82064615
RTI-32,nma-04-394,-8.122152987,5.883495783,-19.00254755
RTI-32,nma-08-230,-8.122152987,0.413386417,-17.50831526
RTI-32,md2-01-186,-8.122152987,2.302141507,-18.42063173
RTI-32,md2-02-029,-8.122152987,-0.853363293,-19.87712031
RTI-32,md2-03-113,-8.122152987,-2.126838817,-17.89035308
RTI-32,md2-03-408,-8.122152987,-2.378811839,-19.10012534
RTI-32,nma-08-230r,-8.122152987,-3.518076683,-15.81202945
RTI-32,md2-02-440,-8.122152987,-1.188031285,-19.53715485
RTI-32,nma-10-450,-8.122152987,-8.146774789,-16.31970845
RTI-55,nma-10-001,-10.81805764,-5.533121698,-21.43344143
RTI-55,mod-15-206,-10.81805764,-7.885292487,-15.77724091
RTI-55,mod-15-315,-10.81805764,-8.856570799,-14.75351417
RTI-55,md2-02-326,-10.81805764,-5.707098223,-17.26686476
RTI-55,md2-02-385-2ns,-10.81805764,-4.121861538,-19.3591163
RTI-55,md2-02-385,-10.81805764,-4.947616016,-18.97695889
RTI-55,mod-20-399,-10.81805764,-10.22068836,-16.32632494
RTI-55,nma-04-221,-10.81805764,-10.71500752,-16.91671567
RTI-55,nma-10-386,-10.81805764,-1.85786946,-16.19619261
RTI-83,nma-10-442,-8.776665815,-2.051376338,-15.3196037
RTI-83,nma-07-088,-8.776665815,-0.539301859,-15.72769426
RTI-83,md2-02-373,-8.776665815,-1.906149626,-16.11155781
RTI-83,nma-08-131,-8.776665815,0.313343619,-19.40078361
RTI-83,nma-08-496,-8.776665815,1.261873671,-19.855084
RTI-83,nma-08-496r,-8.776665815,-3.209539767,-19.77852301
RTI-83,nma-10-083,-8.776665815,-1.389051189,-16.46193737
RTI-112,nma-04-356,-10.01552067,-7.703118545,-18.74684866
RTI-112,nma-12-269,-10.01552067,-2.041048837,-17.43378148
RTI-112,md2-02-251,-10.01552067,-4.164610755,-18.10792907
RTI-112,md2-03-020,-10.01552067,-5.530673965,-18.65868875
RTI-112,md2-03-112,-10.01552067,-4.616037982,-17.8977875
RTI-112,md2-05-001r,-10.01552067,10.61243604,-18.16726877
RTI-112,md2-02-424,-10.01552067,-6.748664848,-14.45256624
RTI-112,md2-04-136,-10.01552067,-1.418702078,-17.52802883
RTI-112,md2-04-432,-10.01552067,5.711770139,-19.62800474
RTI-112,nma-12-269r,-10.01552067,6.293115989,-21.18562611
RTI-112,md2-02-366,-10.01552067,-7.398526621,-15.27947209
RTI-112,nma-10-060,-10.01552067,0.203776856,-16.07514793
RTI-112,nma-10-009,-10.01552067,-8.496101809,-15.4687674
RTI-121,nma-04-080,-8.23077347,-13.26577889,-19.92387515
RTI-121,md2-02-121,-8.23077347,-6.908416206,-21.19849859
RTI-121,md2-01-352,-8.23077347,-3.905511834,-20.05735835
RTI-121,md2-02-008,-8.23077347,-7.466873501,-20.41795091
RTI-121,md2-03-176,-8.23077347,-10.52147618,-20.31241965
RTI-121,md2-03-414,-8.23077347,-8.16490714,-21.51488985
RTI-121,md2-02-161,-8.23077347,-10.90084,-18.37585263
RTI-121,md2-02-385,-8.23077347,0.522082608,-16.89841543
RTI-121,nma-09-401,-8.23077347,-9.558324522,-18.10430351
RTI-121,nma-10-395,-8.23077347,-2.742911754,-17.7883527
RTI-311,nma-10-120,-9.189617432,-13.42937635,-14.86997145
RTI-311,md2-02-009,-9.189617432,-6.894222241,-15.63990294
RTI-311,mod-22-300,-9.189617432,-12.76870958,-16.02814399
CPT,mod-08-107,-7.817821853,4.236459518,-19.75666039
CPT,md2-02-359,-7.817821853,-3.738716789,-16.07115237
CPT,mod-22-290,-7.817821853,6.6227657,-16.84444512
CPT,nma-10-155,-7.817821853,-1.792262141,-13.302963
CPT,nma-10-405,-7.817821853,-2.6011009,-15.99324198
CFT,mod-08-220,-8.451839575,-3.405297094,-18.91334067
CFT,md2-02-111,-8.451839575,-6.573973681,-14.8400486

Produces these strange results (notice the last class - CFT -has not been printed):
]]RTI-31,nma-10-325,-9.189617432,-1.695719104,-17.76402898
]]RTI-31,nma-04-176,-9.189617432,-0.708200772,-17.29511154
]]RTI-31,nma-04-176r,-9.189617432,-1.776907169,-18.88678457
]]RTI-31,md2-02-280,-9.189617432,-6.716602886,-17.30751521
]]RTI-31,nma-10-346,-9.189617432,-11.25116643,-15.84743342
]]RTI-31,nma-04-263,-9.189617432,-11.7115346,-15.04227866
]]RTI-31,nma-08-195,-9.189617432,-4.283013542,-14.8644087
]]RTI-31,nma-04-009,-9.189617432,-9.957288662,-15.3833286
]]RTI-32,nma-10-088,-8.122152987,3.400737344,-13.82064615
]]RTI-32,nma-04-394,-8.122152987,5.883495783,-19.00254755
]]RTI-32,nma-08-230,-8.122152987,0.413386417,-17.50831526
]]RTI-32,md2-01-186,-8.122152987,2.302141507,-18.42063173
]]RTI-32,md2-02-029,-8.122152987,-0.853363293,-19.87712031
]]RTI-32,md2-03-113,-8.122152987,-2.126838817,-17.89035308
]]RTI-32,md2-03-408,-8.122152987,-2.378811839,-19.10012534
]]RTI-32,nma-08-230r,-8.122152987,-3.518076683,-15.81202945
]]RTI-32,md2-02-440,-8.122152987,-1.188031285,-19.53715485
]]RTI-32,nma-10-450,-8.122152987,-8.146774789,-16.31970845
]]RTI-55,nma-10-001,-10.81805764,-5.533121698,-21.43344143
]]RTI-55,mod-15-206,-10.81805764,-7.885292487,-15.77724091
]]RTI-55,mod-15-315,-10.81805764,-8.856570799,-14.75351417
]]RTI-55,md2-02-326,-10.81805764,-5.707098223,-17.26686476
]]RTI-55,md2-02-385-2ns,-10.81805764,-4.121861538,-19.3591163
]]RTI-55,md2-02-385,-10.81805764,-4.947616016,-18.97695889
]]RTI-55,mod-20-399,-10.81805764,-10.22068836,-16.32632494
]]RTI-55,nma-04-221,-10.81805764,-10.71500752,-16.91671567
]]RTI-55,nma-10-386,-10.81805764,-1.85786946,-16.19619261
]]RTI-83,nma-10-442,-8.776665815,-2.051376338,-15.3196037
]]RTI-83,nma-07-088,-8.776665815,-0.539301859,-15.72769426
]]RTI-83,md2-02-373,-8.776665815,-1.906149626,-16.11155781
]]RTI-83,nma-08-131,-8.776665815,0.313343619,-19.40078361
]]RTI-83,nma-08-496,-8.776665815,1.261873671,-19.855084
]]RTI-83,nma-08-496r,-8.776665815,-3.209539767,-19.77852301
]]RTI-83,nma-10-083,-8.776665815,-1.389051189,-16.46193737
]]RTI-112,nma-04-356,-10.01552067,-7.703118545,-18.74684866
]]RTI-112,nma-12-269,-10.01552067,-2.041048837,-17.43378148
]]RTI-112,md2-02-251,-10.01552067,-4.164610755,-18.10792907
]]RTI-112,md2-03-020,-10.01552067,-5.530673965,-18.65868875
]]RTI-112,md2-03-112,-10.01552067,-4.616037982,-17.8977875
]]RTI-112,md2-05-001r,-10.01552067,10.61243604,-18.16726877
]]RTI-112,md2-02-424,-10.01552067,-6.748664848,-14.45256624
]]RTI-112,md2-04-136,-10.01552067,-1.418702078,-17.52802883
]]RTI-112,md2-04-432,-10.01552067,5.711770139,-19.62800474
]]RTI-112,nma-12-269r,-10.01552067,6.293115989,-21.18562611
]]RTI-112,md2-02-366,-10.01552067,-7.398526621,-15.27947209
]]RTI-112,nma-10-060,-10.01552067,0.203776856,-16.07514793
]]RTI-112,nma-10-009,-10.01552067,-8.496101809,-15.4687674
]]RTI-121,nma-04-080,-8.23077347,-13.26577889,-19.92387515
]]RTI-121,md2-02-121,-8.23077347,-6.908416206,-21.19849859
]]RTI-121,md2-01-352,-8.23077347,-3.905511834,-20.05735835
]]RTI-121,md2-02-008,-8.23077347,-7.466873501,-20.41795091
]]RTI-121,md2-03-176,-8.23077347,-10.52147618,-20.31241965
]]RTI-121,md2-03-414,-8.23077347,-8.16490714,-21.51488985
]]RTI-121,md2-02-161,-8.23077347,-10.90084,-18.37585263
]]RTI-121,md2-02-385,-8.23077347,0.522082608,-16.89841543
]]RTI-121,nma-09-401,-8.23077347,-9.558324522,-18.10430351
]]RTI-121,nma-10-395,-8.23077347,-2.742911754,-17.7883527
]]RTI-311,nma-10-120,-9.189617432,-13.42937635,-14.86997145
]]RTI-311,md2-02-009,-9.189617432,-6.894222241,-15.63990294
]]RTI-311,mod-22-300,-9.189617432,-12.76870958,-16.02814399
]]CPT,mod-08-107,-7.817821853,4.236459518,-19.75666039
]]CPT,md2-02-359,-7.817821853,-3.738716789,-16.07115237
]]CPT,mod-22-290,-7.817821853,6.6227657,-16.84444512
]]CPT,nma-10-155,-7.817821853,-1.792262141,-13.302963
]]CPT,nma-10-405,-7.817821853,-2.6011009,-15.99324198
]]CFT,mod-08-220,-8.451839575,-3.405297094,-18.91334067
]]CFT,md2-02-111,-8.451839575,-6.573973681,-14.8400486
[0,1] - A
[0,2] - A
[0,3] - A
[0,4] - A
[0,5] - A
[0,6] - A
[0,7] - A
[1,0] - B
[1,1] - A
[1,2] - A
[1,3] - A
[1,4] - A
[1,5] - A
[1,6] - A
[1,7] - A
[1,8] - A
[1,9] - A
[2,0] - B
[2,1] - A
[2,2] - A
[2,3] - A
[2,4] - A
[2,5] - A
[2,6] - A
[2,7] - A
[2,8] - A
[3,0] - B
[3,1] - A
[3,2] - A
[3,3] - A
[3,4] - A
[3,5] - A
[3,6] - A
[4,0] - B
[4,1] - A
[4,2] - A
[4,3] - A
[4,4] - A
[4,5] - A
[4,6] - A
[4,7] - A
[4,8] - A
[4,9] - A
[4,10] - A
[4,11] - A
[4,12] - A
[5,0] - B
[5,1] - A
[5,2] - A
[5,3] - A
[5,4] - A
[5,5] - A
[5,6] - A
[5,7] - A
[5,8] - A
[5,9] - A
[6,0] - B
[6,1] - A
[6,2] - A
[7,0] - B
[7,1] - A
[7,2] - A
[7,3] - A
[7,4] - A
[8,0] - B
[8,1] - A
Class         Member                C-Value  M-Value 1  M-Value 2   Class  Member
RTI-31        nma-10-325              -9.19      -1.70     -17.76       0       0
RTI-31        nma-04-176              -9.19      -0.71     -17.30       0       1
RTI-31        nma-04-176r             -9.19      -1.78     -18.89       0       2
RTI-31        md2-02-280              -9.19      -6.72     -17.31       0       3
RTI-31        nma-10-346              -9.19     -11.25     -15.85       0       4
RTI-31        nma-04-263              -9.19     -11.71     -15.04       0       5
RTI-31        nma-08-195              -9.19      -4.28     -14.86       0       6
RTI-31        nma-04-009              -9.19      -9.96     -15.38       0       7
                                       0.00       0.00       0.00       0       8
                                       0.00       0.00       0.00       0       9
                                       0.00       0.00       0.00       0      10
                                       0.00       0.00       0.00       0      11
RTI-32        nma-10-088              -8.12       3.40     -13.82       1       0
RTI-32        nma-04-394              -8.12       5.88     -19.00       1       1
RTI-32        nma-08-230              -8.12       0.41     -17.51       1       2
RTI-32        md2-01-186              -8.12       2.30     -18.42       1       3
RTI-32        md2-02-029              -8.12      -0.85     -19.88       1       4
RTI-32        md2-03-113              -8.12      -2.13     -17.89       1       5
RTI-32        md2-03-408              -8.12      -2.38     -19.10       1       6
RTI-32        nma-08-230r             -8.12      -3.52     -15.81       1       7
RTI-32        md2-02-440              -8.12      -1.19     -19.54       1       8
RTI-32        nma-10-450              -8.12      -8.15     -16.32       1       9
                                       0.00       0.00       0.00       1      10
                                       0.00       0.00       0.00       1      11
RTI-55        nma-10-001             -10.82      -5.53     -21.43       2       0
RTI-55        mod-15-206             -10.82      -7.89     -15.78       2       1
RTI-55        mod-15-315             -10.82      -8.86     -14.75       2       2
RTI-55        md2-02-326             -10.82      -5.71     -17.27       2       3
RTI-55        md2-02-385-2ns         -10.82      -4.12     -19.36       2       4
RTI-55        md2-02-385             -10.82      -4.95     -18.98       2       5
RTI-55        mod-20-399             -10.82     -10.22     -16.33       2       6
RTI-55        nma-04-221             -10.82     -10.72     -16.92       2       7
RTI-55        nma-10-386             -10.82      -1.86     -16.20       2       8
                                       0.00       0.00       0.00       2       9
                                       0.00       0.00       0.00       2      10
                                       0.00       0.00       0.00       2      11
RTI-83        nma-10-442              -8.78      -2.05     -15.32       3       0
RTI-83        nma-07-088              -8.78      -0.54     -15.73       3       1
RTI-83        md2-02-373              -8.78      -1.91     -16.11       3       2
RTI-83        nma-08-131              -8.78       0.31     -19.40       3       3
RTI-83        nma-08-496              -8.78       1.26     -19.86       3       4
RTI-83        nma-08-496r             -8.78      -3.21     -19.78       3       5
RTI-83        nma-10-083              -8.78      -1.39     -16.46       3       6
                                       0.00       0.00       0.00       3       7
                                       0.00       0.00       0.00       3       8
                                       0.00       0.00       0.00       3       9
                                       0.00       0.00       0.00       3      10
                                       0.00       0.00       0.00       3      11
RTI-112       nma-04-356             -10.02      -7.70     -18.75       4       0
RTI-112       nma-12-269             -10.02      -2.04     -17.43       4       1
RTI-112       md2-02-251             -10.02      -4.16     -18.11       4       2
RTI-112       md2-03-020             -10.02      -5.53     -18.66       4       3
RTI-112       md2-03-112             -10.02      -4.62     -17.90       4       4
RTI-112       md2-05-001r            -10.02      10.61     -18.17       4       5
RTI-112       md2-02-424             -10.02      -6.75     -14.45       4       6
RTI-112       md2-04-136             -10.02      -1.42     -17.53       4       7
RTI-112       md2-04-432             -10.02       5.71     -19.63       4       8
RTI-112       nma-12-269r            -10.02       6.29     -21.19       4       9
RTI-112       md2-02-366             -10.02      -7.40     -15.28       4      10
RTI-112       nma-10-060             -10.02       0.20     -16.08       4      11
RTI-121       nma-04-080              -8.23     -13.27     -19.92       5       0
RTI-121       md2-02-121              -8.23      -6.91     -21.20       5       1
RTI-121       md2-01-352              -8.23      -3.91     -20.06       5       2
RTI-121       md2-02-008              -8.23      -7.47     -20.42       5       3
RTI-121       md2-03-176              -8.23     -10.52     -20.31       5       4
RTI-121       md2-03-414              -8.23      -8.16     -21.51       5       5
RTI-121       md2-02-161              -8.23     -10.90     -18.38       5       6
RTI-121       md2-02-385              -8.23       0.52     -16.90       5       7
RTI-121       nma-09-401              -8.23      -9.56     -18.10       5       8
RTI-121       nma-10-395              -8.23      -2.74     -17.79       5       9
                                       0.00       0.00       0.00       5      10
                                       0.00       0.00       0.00       5      11
RTI-311       nma-10-120              -9.19     -13.43     -14.87       6       0
RTI-311       md2-02-009              -9.19      -6.89     -15.64       6       1
RTI-311       mod-22-300              -9.19     -12.77     -16.03       6       2
                                       0.00       0.00       0.00       6       3
                                       0.00       0.00       0.00       6       4
                                       0.00       0.00       0.00       6       5
                                       0.00       0.00       0.00       6       6
                                       0.00       0.00       0.00       6       7
                                       0.00       0.00       0.00       6       8
                                       0.00       0.00       0.00       6       9
                                       0.00       0.00       0.00       6      10
                                       0.00       0.00       0.00       6      11
CPT           mod-08-107              -7.82       4.24     -19.76       7       0
CPT           md2-02-359              -7.82      -3.74     -16.07       7       1
CPT           mod-22-290              -7.82       6.62     -16.84       7       2
CPT           nma-10-155              -7.82      -1.79     -13.30       7       3
CPT           nma-10-405              -7.82      -2.60     -15.99       7       4
                                       0.00       0.00       0.00       7       5
                                       0.00       0.00       0.00       7       6
                                       0.00       0.00       0.00       7       7
                                       0.00       0.00       0.00       7       8
                                       0.00       0.00       0.00       7       9
                                       0.00       0.00       0.00       7      10
                                       0.00       0.00       0.00       7      11

Any idea what is wrong here? Is it the data file format?
Comment: How do you know how many records were read by `readFile()`?  You have no way to tell it how much space it can use, nor can it tell you how much space it used.  You have a design problem there.
Comment: You should read up on structure assignment.  It would radically simplify the code.  Also, string comparison requires `strcmp()`; this code `if (csv_read[records].class_name == csv_read[records -1].class_name){` is comparing two pointers which are guaranteed to be unequal, even if the strings that they contain are equal.
Comment: Hi Jonathan, I was going to write a function that reads the number of records in the input file. For now, I just want it to work for a single file I have, and I know how many records there are. There will never be a need for this script to read more than 100 records.
Comment: Thank you for that tip - this could explain my problem. This is the first C code I've ever written. Definitely still learning.
Comment: Generally, it will be better — simpler, easier to understand, more performant because the file is only read once (which really matters with big files, though not so much with 100-row files) — if you tell the input function how much space it has in the array, and it returns the number of entries it filled (0 or -1 can indicate errors, possibly different errors).  Don't assume that you'll never have a file that's too big — nor that you'll be able to open every file specified on the command line, nor that the files specified will always be in the correct format.
Comment: I will edit the code shown above to show my readFile function.
Answer: Here's some mildly fixed code.  There were problems with myFile vs csvFile and lie_rec vs csv_rec in the readFile() function.
Otherwise, I simplified record initialization with empty and structure assignment; I used simpler/different loop indexes in places; and added some diagnostic printing.
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

struct csv
{
    char class_name[12];
    char member[18];
    double class_value;
    double member_value1;
    double member_value2;
    double calc_output1;  // Program I'm writing adds values here
    double calc_output2; // And here
};

static
int readFile(char *csvFile, struct csv *csv_rec, size_t max_records)
{
    /* code for reading file here - this works as planned.
       Reads csv file and makes 1D array of structs with up to max_records
       elements */
    FILE *dataFile = NULL;
    size_t idx = 0;
    size_t fldidx = 0;      // struct field index
    char fileLine[255] = {0};
    char *p = NULL;         // pointer to line

    // dataFile = fopen(csvFile, "r"); // Rename myFile -> csvFile; delete duplicate

    // Open the file set by the command line and read it into the file object
    dataFile = fopen(csvFile, "r");
    if (dataFile == NULL)
    {
        perror("Error opening data file.\n");
        exit(EXIT_FAILURE);     // Exit program if data file cannot be read by runtime service
    }

    while ((fgets(fileLine, sizeof(fileLine), dataFile)) != NULL)
    {
        size_t len = strlen(fileLine);
        if (len == 0)
            continue;
        fileLine[len - 1] = '\0';
        if (idx >= max_records)
            break;
        printf("[[%s]]\n", fileLine);
        fldidx = 0;     // reset field index for each line
        // Use strtok to separate each line into tokens on ','
        for (p = strtok(fileLine, ","); p != NULL; p = strtok(NULL, ","))
        {
            // Use fldidx to separate data into struct
            switch (fldidx)
            {
            case 0:
                // JL: lie_rec --> csv_rec
                strncpy(csv_rec[idx].class_name, p, strlen(p));
                break;
            case 1:
                strncpy(csv_rec[idx].member, p, strlen(p));
                break;
            case 2:
                csv_rec[idx].class_value = atof(p);
                break;
            case 3:
                csv_rec[idx].member_value1 = atof(p);
                break;
            case 4:
                csv_rec[idx].member_value2 = atof(p);
                break;
            }
            // Increment field index
            fldidx++;
        }
        // Increment record index
        idx++;
    }
    // Close dataFile object
    fclose(dataFile);
    return idx;
}

int main(int argc, char *argv[])
{
    int m_members = 25; // Bigger than it needs to be for any file
    int n_classes = 25; // Bigger than needs to be for any file
    int n_records = 75; // Later a function will count these
    struct csv csv_data[m_members][n_classes]; // Declare 2D array
    struct csv csv_read[n_records]; // 1D array of structs
    struct csv empty = { "           ", "                 ", 0.0, 0.0, 0.0, 0.0, 0.0 };

    // Initialize csv_data 2D array:
    for (int i = 0; i < m_members; i++)
    {
        for (int j = 0; j < n_classes; j++)
            csv_data[i][j] = empty;
    }

    // Initialize csv_read struct array
    for (int i = 0; i < n_records; i++)
        csv_read[i] = empty;

    if (argc != 2)
    {
        printf("Usage: ./optimization datafile.csv\n");
        return 1;
    }

    int n_recs = readFile(argv[1], csv_read, n_records);

    // Here is what I have so far. It doesn't work:
    // First, set member and class indices to zero
    int members = 0;
    int classes = 0;
    int max_members = 0;

    // Next, copy values held in first record to 0,0 index of 2D array
    csv_data[classes][members] = csv_read[0];

    // This is the part I'm not doing correctly:
    for (int i = 1; i < n_recs; i++)
    {
        assert(members < m_members && classes < n_classes);
        if (strcmp(csv_read[i].class_name, csv_read[i - 1].class_name) == 0)
        {
            members += 1;
            csv_data[classes][members] = csv_read[i];
            printf("[%d,%d] - A\n", classes, members);
        }
        else
        {
            if (members > max_members)
                max_members = members;
            members = 0;
            classes += 1;
            csv_data[classes][members] = csv_read[i];
            printf("[%d,%d] - B\n", classes, members);
        }
    }

    printf("%-12s  %-18s  %9s  %9s  %9s  %6s  %6s\n",
           "Class", "Member", "C-Value", "M-Value 1", "M-Value 2", "Class", "Member");
    for (int i = 0; i < classes; i += 1)
    {
        for (int j = 0; j < max_members; j += 1)
        {
            printf("%-12s  %-18s  %9.2f  %9.2f  %9.2f  %6d  %6d\n",
                   csv_data[i][j].class_name, csv_data[i][j].member,
                   csv_data[i][j].class_value, csv_data[i][j].member_value1,
                   csv_data[i][j].member_value2, i, j);
        }
    }

    return 0;
}

For this data file (partially randomly generated, partially systematically edited):
Eeikehkt,Juxeizdiybnc,20.07,985.12,8816.17
Eeikehkt,Juxeizdiybnc,40.07,385.12,5816.17
Eeikehkt,Juxeizdiybnc,60.07,485.12,2816.17
Iamtppjo,Duxacewxhh,61.79,385.93,2477.32
Iamtppjo,Duxacewxhh,41.79,385.93,2477.32
Iamtppjo,Duxacewxhh,91.79,385.93,2477.32
Iamtppjo,Duxacewxhh,21.79,385.93,2477.32
Harbegghs,Javqaqetlmf,66.53,103.14,6106.43
Harbegghs,Javqaqetlmf,56.53,103.14,7106.43
Harbegghs,Javqaqetlmf,46.53,103.14,8106.43
Harbegghs,Javqaqetlmf,36.53,103.14,9106.43
Harbegghs,Javqaqetlmf,26.53,103.14,1106.43
Harbegghs,Javqaqetlmf,16.53,103.14,2106.43
Zumnyqmiq,Luoyjcvvdtwh,36.89,435.08,5337.35
Zumnyqmiq,Luoyjcvvdtwh,56.89,435.08,5337.35
Zumnyqmiq,Luoyjcvvdtwh,76.89,435.08,5337.35
Zagahwfn,Hanhmrcdgryn,84.58,598.20,8854.05
Keivsvmcb,Maaqjraizoaz,36.60,316.95,4320.89
Hoawiopqby,Puqezsvrscs,44.77,131.01,4315.89
Zifummkfm,Uitgzpnziz,94.29,342.45,2152.53
Fikwzidv,Iamjbgihhrb,59.63,915.11,9885.05
Yutihgddt,Augpvalibavo,44.64,472.51,3092.39

I get the following diagnostic output:
[[Eeikehkt,Juxeizdiybnc,20.07,985.12,8816.17]]
[[Eeikehkt,Juxeizdiybnc,40.07,385.12,5816.17]]
[[Eeikehkt,Juxeizdiybnc,60.07,485.12,2816.17]]
[[Iamtppjo,Duxacewxhh,61.79,385.93,2477.32]]
[[Iamtppjo,Duxacewxhh,41.79,385.93,2477.32]]
[[Iamtppjo,Duxacewxhh,91.79,385.93,2477.32]]
[[Iamtppjo,Duxacewxhh,21.79,385.93,2477.32]]
[[Harbegghs,Javqaqetlmf,66.53,103.14,6106.43]]
[[Harbegghs,Javqaqetlmf,56.53,103.14,7106.43]]
[[Harbegghs,Javqaqetlmf,46.53,103.14,8106.43]]
[[Harbegghs,Javqaqetlmf,36.53,103.14,9106.43]]
[[Harbegghs,Javqaqetlmf,26.53,103.14,1106.43]]
[[Harbegghs,Javqaqetlmf,16.53,103.14,2106.43]]
[[Zumnyqmiq,Luoyjcvvdtwh,36.89,435.08,5337.35]]
[[Zumnyqmiq,Luoyjcvvdtwh,56.89,435.08,5337.35]]
[[Zumnyqmiq,Luoyjcvvdtwh,76.89,435.08,5337.35]]
[[Zagahwfn,Hanhmrcdgryn,84.58,598.20,8854.05]]
[[Keivsvmcb,Maaqjraizoaz,36.60,316.95,4320.89]]
[[Hoawiopqby,Puqezsvrscs,44.77,131.01,4315.89]]
[[Zifummkfm,Uitgzpnziz,94.29,342.45,2152.53]]
[[Fikwzidv,Iamjbgihhrb,59.63,915.11,9885.05]]
[[Yutihgddt,Augpvalibavo,44.64,472.51,3092.39]]
[0,1] - A
[0,2] - A
[1,0] - B
[1,1] - A
[1,2] - A
[1,3] - A
[2,0] - B
[2,1] - A
[2,2] - A
[2,3] - A
[2,4] - A
[2,5] - A
[3,0] - B
[3,1] - A
[3,2] - A
[4,0] - B
[5,0] - B
[6,0] - B
[7,0] - B
[8,0] - B
[9,0] - B

followed by the following main output:
Class         Member                C-Value  M-Value 1  M-Value 2   Class  Member
Eeikehkt      Juxeizdiybnc            20.07     985.12    8816.17       0       0
Eeikehkt      Juxeizdiybnc            40.07     385.12    5816.17       0       1
Eeikehkt      Juxeizdiybnc            60.07     485.12    2816.17       0       2
                                       0.00       0.00       0.00       0       3
                                       0.00       0.00       0.00       0       4
Iamtppjo      Duxacewxhh              61.79     385.93    2477.32       1       0
Iamtppjo      Duxacewxhh              41.79     385.93    2477.32       1       1
Iamtppjo      Duxacewxhh              91.79     385.93    2477.32       1       2
Iamtppjo      Duxacewxhh              21.79     385.93    2477.32       1       3
                                       0.00       0.00       0.00       1       4
Harbegghs     Javqaqetlmf             66.53     103.14    6106.43       2       0
Harbegghs     Javqaqetlmf             56.53     103.14    7106.43       2       1
Harbegghs     Javqaqetlmf             46.53     103.14    8106.43       2       2
Harbegghs     Javqaqetlmf             36.53     103.14    9106.43       2       3
Harbegghs     Javqaqetlmf             26.53     103.14    1106.43       2       4
Zumnyqmiq     Luoyjcvvdtwh            36.89     435.08    5337.35       3       0
Zumnyqmiq     Luoyjcvvdtwh            56.89     435.08    5337.35       3       1
Zumnyqmiq     Luoyjcvvdtwh            76.89     435.08    5337.35       3       2
                                       0.00       0.00       0.00       3       3
                                       0.00       0.00       0.00       3       4
Zagahwfn      Hanhmrcdgryn            84.58     598.20    8854.05       4       0
                                       0.00       0.00       0.00       4       1
                                       0.00       0.00       0.00       4       2
                                       0.00       0.00       0.00       4       3
                                       0.00       0.00       0.00       4       4
Keivsvmcb     Maaqjraizoaz            36.60     316.95    4320.89       5       0
                                       0.00       0.00       0.00       5       1
                                       0.00       0.00       0.00       5       2
                                       0.00       0.00       0.00       5       3
                                       0.00       0.00       0.00       5       4
Hoawiopqby    Puqezsvrscs             44.77     131.01    4315.89       6       0
                                       0.00       0.00       0.00       6       1
                                       0.00       0.00       0.00       6       2
                                       0.00       0.00       0.00       6       3
                                       0.00       0.00       0.00       6       4
Zifummkfm     Uitgzpnziz              94.29     342.45    2152.53       7       0
                                       0.00       0.00       0.00       7       1
                                       0.00       0.00       0.00       7       2
                                       0.00       0.00       0.00       7       3
                                       0.00       0.00       0.00       7       4
Fikwzidv      Iamjbgihhrb             59.63     915.11    9885.05       8       0
                                       0.00       0.00       0.00       8       1
                                       0.00       0.00       0.00       8       2
                                       0.00       0.00       0.00       8       3
                                       0.00       0.00       0.00       8       4

If that's not what you're expecting, show some sample data and the desired output.

Continued debugging
The code above omits printing the last class processed, as discussed in the commentary. I've done some more variable renaming and avoided the initialization loops.  The use of class_idx (for current class index in array) and num_classes for number of classes should make the code easier to understand.  I also added a class_size array to keep a record of how many entries are present in each class.  The code also strips carriage returns from the end of the line, carefully.  It still skips empty data lines (ones with only a line ending).  I removed the two 'unused' structure members, calc_output1 and calc_output2, since they are not relevant to this code.  I've made the debugging output conditional on the value of the variable debug, which is set to 0 in this code.  You could arrange to be able to set it to one by a command line argument or an environment variable.
I've also cleaned up the use of strncpy() in the readFile() function.  It was not doing what you wanted — it was providing no overflow protection because you specified that the length available was the length of the source string, not the length of the target string. In fact, it was also ensuring that strncpy() was not null terminating the copied string, leaving it blank padded (because of the initialization loops — which were removed in this revision) — something that's quite hard to spot given the tabular output. The safer_copy() function encapsulates the operation correctly, I believe.  It takes the view that overlong names are fatal errors; you can choose to truncate instead if you prefer.  I'd then use memmove() (or perhaps memcpy()) to copy the data and would manually null-terminate the target string).  As with most things, there's more than one way to do it (the Perl motto — aka TMTOWTDI) and you need to choose for yourself which is best for your requirements.
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

struct csv
{
    char class_name[12];
    char member[18];
    double class_value;
    double member_value1;
    double member_value2;
};

static int debug = 0;

static
void safer_copy(const char *p, char *dst, size_t dst_size, const char *tag)
{
    if (strlen(p) >= dst_size)
    {
        fprintf(stderr, "Overlong %s [%s] (%zu vs %zu maximum)\n",
                tag, p, strlen(p), dst_size - 1);
        exit(EXIT_FAILURE);
    }
    strcpy(dst, p);
}

static
int readFile(char *csvFile, struct csv *csv_rec, size_t max_records)
{
    FILE *dataFile = NULL;
    size_t idx = 0;
    char fileLine[255] = {0};

    dataFile = fopen(csvFile, "r");
    if (dataFile == NULL)
    {
        fprintf(stderr, "Failed to open file %s for reading\n", csvFile);
        exit(EXIT_FAILURE);
    }

    for (idx = 0; fgets(fileLine, sizeof(fileLine), dataFile) != NULL; idx++)
    {
        size_t len = strlen(fileLine);
        if (len == 0)                   // Should never occur
            continue;
        assert(fileLine[len - 1] == '\n');
        fileLine[--len] = '\0';
        if (len == 0)                   // Could occur for empty line
            continue;
        if (fileLine[len - 1] == '\r')  // Detect CRLF line endings too
            fileLine[--len] = '\0';
        if (len == 0)                   // Could occur for empty line
            continue;
        if (idx >= max_records)
            break;
        if (debug) printf("[[%s]]\n", fileLine);
        char *data = fileLine;
        char *p;
        for (size_t fldidx = 0; (p = strtok(data, ",")) != NULL; data = NULL, fldidx++)
        {
            switch (fldidx)
            {
            case 0:
                safer_copy(p, csv_rec[idx].class_name,
                           sizeof(csv_rec[idx].class_name), "class name");
                break;
            case 1:
                safer_copy(p, csv_rec[idx].member,
                           sizeof(csv_rec[idx].member), "member");
                break;
            case 2:
                csv_rec[idx].class_value = atof(p);
                break;
            case 3:
                csv_rec[idx].member_value1 = atof(p);
                break;
            case 4:
                csv_rec[idx].member_value2 = atof(p);
                break;
            }
        }
    }
    fclose(dataFile);
    return idx;
}

int main(int argc, char *argv[])
{
    int max_members = 25;
    int max_classes = 25;
    int max_records = 75;
    struct csv csv_data[max_members][max_classes];
    struct csv csv_read[max_records];
    int class_size[max_members];

    if (argc != 2)
    {
        fprintf(stderr, "Usage: %s datafile.csv\n", argv[0]);
        return 1;
    }

    int num_records = readFile(argv[1], csv_read, max_records);
    if (num_records == 0)
    {
        fprintf(stderr, "%s: empty data file %s\n", argv[0], argv[1]);
        return 1;
    }

    int member_idx = 0;
    int class_idx = 0;
    int max_group = 0;

    csv_data[class_idx][member_idx] = csv_read[0];

    for (int i = 1; i < num_records; i++)
    {
        assert(member_idx < max_members && class_idx < max_classes);
        if (strcmp(csv_read[i].class_name, csv_read[i - 1].class_name) == 0)
            member_idx++;
        else
        {
            if (member_idx > max_group)
                max_group = member_idx;
            class_size[class_idx++] = member_idx + 1;
            member_idx = 0;
        }
        csv_data[class_idx][member_idx] = csv_read[i];
        if (debug)
            printf("[%d,%d] - %s\n", class_idx, member_idx, csv_read[i].class_name);
    }
    class_size[class_idx] = member_idx + 1;
    int num_classes = class_idx + 1;
    if (debug)
        printf("Classes = %d, max members = %d\n", num_classes, max_group);

    printf("%-12s  %-18s  %9s  %9s  %9s  %6s  %6s\n",
           "Class", "Member", "C-Value", "M-Value 1", "M-Value 2", "Class", "Member");
    for (int i = 0; i < num_classes; i++)
    {
        for (int j = 0; j < class_size[i]; j++)
        {
            printf("%-12s  %-18s  %9.2f  %9.2f  %9.2f  %6d  %6d\n",
                   csv_data[i][j].class_name, csv_data[i][j].member,
                   csv_data[i][j].class_value, csv_data[i][j].member_value1,
                   csv_data[i][j].member_value2, i, j);
        }
    }

    return 0;
}

One thing this shows is that you can almost always polish a program.  I'd not claim this is perfect by any stretch of my imagination.  For example, if a line is missing some fields, you get arbitrary garbage stored in the records; the code doesn't trim leading or trailing blanks from the name fields.
On the sample data from the question, the output is:
Class         Member                C-Value  M-Value 1  M-Value 2   Class  Member
RTI-31        nma-10-325              -9.19      -1.70     -17.76       0       0
RTI-31        nma-04-176              -9.19      -0.71     -17.30       0       1
RTI-31        nma-04-176r             -9.19      -1.78     -18.89       0       2
RTI-31        md2-02-280              -9.19      -6.72     -17.31       0       3
RTI-31        nma-10-346              -9.19     -11.25     -15.85       0       4
RTI-31        nma-04-263              -9.19     -11.71     -15.04       0       5
RTI-31        nma-08-195              -9.19      -4.28     -14.86       0       6
RTI-31        nma-04-009              -9.19      -9.96     -15.38       0       7
RTI-32        nma-10-088              -8.12       3.40     -13.82       1       0
RTI-32        nma-04-394              -8.12       5.88     -19.00       1       1
RTI-32        nma-08-230              -8.12       0.41     -17.51       1       2
RTI-32        md2-01-186              -8.12       2.30     -18.42       1       3
RTI-32        md2-02-029              -8.12      -0.85     -19.88       1       4
RTI-32        md2-03-113              -8.12      -2.13     -17.89       1       5
RTI-32        md2-03-408              -8.12      -2.38     -19.10       1       6
RTI-32        nma-08-230r             -8.12      -3.52     -15.81       1       7
RTI-32        md2-02-440              -8.12      -1.19     -19.54       1       8
RTI-32        nma-10-450              -8.12      -8.15     -16.32       1       9
RTI-55        nma-10-001             -10.82      -5.53     -21.43       2       0
RTI-55        mod-15-206             -10.82      -7.89     -15.78       2       1
RTI-55        mod-15-315             -10.82      -8.86     -14.75       2       2
RTI-55        md2-02-326             -10.82      -5.71     -17.27       2       3
RTI-55        md2-02-385-2ns         -10.82      -4.12     -19.36       2       4
RTI-55        md2-02-385             -10.82      -4.95     -18.98       2       5
RTI-55        mod-20-399             -10.82     -10.22     -16.33       2       6
RTI-55        nma-04-221             -10.82     -10.72     -16.92       2       7
RTI-55        nma-10-386             -10.82      -1.86     -16.20       2       8
RTI-83        nma-10-442              -8.78      -2.05     -15.32       3       0
RTI-83        nma-07-088              -8.78      -0.54     -15.73       3       1
RTI-83        md2-02-373              -8.78      -1.91     -16.11       3       2
RTI-83        nma-08-131              -8.78       0.31     -19.40       3       3
RTI-83        nma-08-496              -8.78       1.26     -19.86       3       4
RTI-83        nma-08-496r             -8.78      -3.21     -19.78       3       5
RTI-83        nma-10-083              -8.78      -1.39     -16.46       3       6
RTI-112       nma-04-356             -10.02      -7.70     -18.75       4       0
RTI-112       nma-12-269             -10.02      -2.04     -17.43       4       1
RTI-112       md2-02-251             -10.02      -4.16     -18.11       4       2
RTI-112       md2-03-020             -10.02      -5.53     -18.66       4       3
RTI-112       md2-03-112             -10.02      -4.62     -17.90       4       4
RTI-112       md2-05-001r            -10.02      10.61     -18.17       4       5
RTI-112       md2-02-424             -10.02      -6.75     -14.45       4       6
RTI-112       md2-04-136             -10.02      -1.42     -17.53       4       7
RTI-112       md2-04-432             -10.02       5.71     -19.63       4       8
RTI-112       nma-12-269r            -10.02       6.29     -21.19       4       9
RTI-112       md2-02-366             -10.02      -7.40     -15.28       4      10
RTI-112       nma-10-060             -10.02       0.20     -16.08       4      11
RTI-112       nma-10-009             -10.02      -8.50     -15.47       4      12
RTI-121       nma-04-080              -8.23     -13.27     -19.92       5       0
RTI-121       md2-02-121              -8.23      -6.91     -21.20       5       1
RTI-121       md2-01-352              -8.23      -3.91     -20.06       5       2
RTI-121       md2-02-008              -8.23      -7.47     -20.42       5       3
RTI-121       md2-03-176              -8.23     -10.52     -20.31       5       4
RTI-121       md2-03-414              -8.23      -8.16     -21.51       5       5
RTI-121       md2-02-161              -8.23     -10.90     -18.38       5       6
RTI-121       md2-02-385              -8.23       0.52     -16.90       5       7
RTI-121       nma-09-401              -8.23      -9.56     -18.10       5       8
RTI-121       nma-10-395              -8.23      -2.74     -17.79       5       9
RTI-311       nma-10-120              -9.19     -13.43     -14.87       6       0
RTI-311       md2-02-009              -9.19      -6.89     -15.64       6       1
RTI-311       mod-22-300              -9.19     -12.77     -16.03       6       2
CPT           mod-08-107              -7.82       4.24     -19.76       7       0
CPT           md2-02-359              -7.82      -3.74     -16.07       7       1
CPT           mod-22-290              -7.82       6.62     -16.84       7       2
CPT           nma-10-155              -7.82      -1.79     -13.30       7       3
CPT           nma-10-405              -7.82      -2.60     -15.99       7       4
CFT           mod-08-220              -8.45      -3.41     -18.91       8       0
CFT           md2-02-111              -8.45      -6.57     -14.84       8       1

Comment: This is exactly what I was trying to do. Thank you! As I mentioned, I'm still learning, and this problem was an excellent learning opportunity. My apologies for the typos in the code; I had a more specific bit of code but tried to make it more general with the hopes that it would be more useful to Stack Overflow users.
Comment: Hi Jonathan; last night when I ran this with your sample csv it worked fine - and since mine is formatted the same way, I thought it would also work the same. However, when I ran the code using some of my own sample data, there seemed to be errors. For example, the following data file (see original pose above for my data file and output). I used C99, as the compiler demanded - could it be a related problem?
Comment: Does your data have `CRLF` line endings (was it transferred via a Windows machine at some point)?  If so, that accounts for the odd positioning of the `]]` markers in the diagnostic output — the CR moved the print position back to the start of the line before the `]]` was added.  I'm looking into the absence of the CFT records.
Comment: @RoryClague: If you observe more carefully than I did last night, running my program on my sample data skips the last entry (it prints the entry for `Fikwzidv Iamjbgihhrb` but not the one for `Yutihgddt Augpvalibavo`).  The last loop needs to be `for (int i = 0; i <= classes; i++)` with `<=` rather than `<`.  Or `classes` needs to be incremented once more in the processing loop so that the `<` condition remains correct.
Comment: Ah, yes - this particular csv was transferred from Windows. Thank you for that insight.
Comment: Yes - easy fix, that. The data structure is being loaded - it just wasn't printing quite as expected. I hoped that was the case. Thank you again for all of your help, Jonathan.


Como saber se comando do script foi executado com sucesso?

Question: Preciso que o script identifique se o arquivo tar.gz foi gerado com sucesso em um if. Segue um exemplo não funcional:
VAR=tar -zcf teste.tar.gz teste/testeDir
if [VAR]
then
echo "Sucesso"
else
echo "Erro"
fi

Bem simples eu acho mas não consegui fazer! Aguardo respostas...
Answer: use a váriável $?
ela retorna "0" se o ultimo comando foi executado com sucesso, e retorna "1" se houve erro neste comando.
Abaixo segue uma tabela com essas variáveis especiais que estão disponíveis no shell:
Variável    Descrição

$0    Parâmetro número 0 (nome do comando ou função)
$1    Parâmetro número 1 (da linha de comando ou função)
...   Parâmetro número N da linha de comando ou função)
$9    Parâmetro número 9 (da linha de comando ou função)
${10} Parâmetro número 10 (da linha de comando ou função)
$#    Número total de parâmetros da linha de comando ou função
$*    Todos os parâmetros, como uma string única
$@    Todos os parâmetros, como várias strings protegidas
$$    PID do processo atual (do próprio script)
$!    PID do último processo em segundo plano
$_    Último argumento do último comando executado
$?    Valor de retorno do último comando executado

veja este exemplo do comando cat em meu terminal:
felix@asgard:[~]: cat escrever.py
arq = open('arquivo.txt','w')
arq.write("Pyhton é legal")
arq.close()
felix@asgard:[~]: echo $?
0
felix@asgard:[~]: cat escreve.py
cat: escreve.py: No such file or directory
felix@asgard:[~]: echo $?
1

No exemplo abaixo só coloquei o resultado de "sucesso" porque em meus testes, quando tentei compactar um arquivo inexistente ou mesmo colocando argumentos errados para compactar, simplesmente o comando não era executado, 'quebrando' assim o script:
#!/bin/bash

tar -zcf escrever.tar.gz escrever.py

if [ $? -eq 0 ]
then
    echo "sucesso"
fi

Comment: muito bom, resolveu meu problema, vlw!
Comment: +1 Sua resposta tá legal, mas se puser um exemplo tipo `tar -zcf teste.tar.gz teste/testeDir` `if [ $? -eq 0 ]` etc, aplicado ao comando da pergunta, para que outros usuários aprendam como fica na pratica, fica mais legal ainda