I'm going to describe the procedure for gradient descent using the language of English:1) For all of the files (e.g. documents) in our training corpus.--> commence, now we are considering our first document2) For all the features in the feature vector associated with that training example (document)3) Take the dot product of the vector of weight values * and the vector of features specific to the document under consideration* vector of weight values is applicable to all features, though all features will not be present in all documents, indeed only some features will be present in each individual document4) Take the value resulting from step 3 and apply this to our loss function. I'm using the Hinge Loss function, i.e. the formulaIs that equation saying that the hinge loss objective function should be the sum of the misclassification error for every file in our training set?So is that equivalent to the java code:Math.max(0, 1 - y * value_from_step_three)With y = {-1, 1}^ THIS IS MY FIRST POINT OF CONFUSIONI guess that is inaccurate but I'm not sure, according to my mental model it should be right. 5) For every weight, update it as:new weight valuej = old weight valuej - learning rate * value from step 4 * value of featurej.I'm a bit confused about what value of featurej should be, since I guess this would correspond to a vector, that has values for every weight under consideration, i.e.Example:Document 1 = [I, am, awesome]Document 2 = [I, am, great, great]Dictionary is:[I, am, awesome, great]So the documents as a vector would look like:Document 1 = [1, 1, 1, 0]Document 2 = [1, 1, 0, 2]There will be weights:[WeightI, Weightam, Weightawesome, Weightgreat]
procedure for gradient descent
gradient descent