This application is based on and claims the benefit of priority from Japanese Patent Application No. 2019-076851, filed on 15 Apr. 2019, the content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a machine learning device which performs reinforcement learning for optimizing a coefficient of at least one filter that attenuates at least one specific frequency component provided in a servo control device for controlling a motor, to a control device which includes such a machine learning device and to a machine learning method.

Related Art

Devices which adjust the characteristic of a filter are disclosed in, for example, Patent Documents 1 to 4. Patent Document 1 discloses a vibration reduction device of a robot which includes: a robot axis drive source that is provided in each axis of the robot and that drives the robot axis according to an input control signal; and a signal processing means that removes, from the control signal, a frequency component corresponding to the natural frequency of the robot axis, and which applies, to the robot axis drive source, the control signal subjected to signal processing in the signal processing means so as to reduce vibrations generated in the robot axis. In the vibration reduction device of the robot, a neural network is provided which inputs the current position of each axis of the robot so as to operate and output the natural frequency of each axis of the robot and which applies this to the signal processing means, and the frequency component corresponding to the natural frequency of the robot axis output from the neural network is removed from the control signal. Patent Document 1 also discloses that the signal processing means is a notch filter and that a notch frequency is varied according to the natural frequency of the robot axis output from the neutral network.

Patent Document 2 discloses a control device of an XY stage in which movable guides that intersect each other vertically and laterally are arranged on a table and in which moving stages are arranged in the intersections thereof, and the control device of the XY stage includes: a variable notch filter which can variably set a notch frequency for absorbing the gain of the resonance motion of the moving stages; and a switching means which inputs the position information of the moving stages on the table and which outputs a switching signal for switching the notch frequency of the notch filter.

Patent Document 3 discloses a servo control device according to an embodiment which includes: command sampling that controls, based on a torque command or a current command, a servo amplifier for driving a motion member which performs a rotation motion or a reciprocation motion and that samples the torque command or the current command for the servo amplifier when a velocity control gain is adjusted; and an operation processing unit that converts, when the velocity control gain is adjusted, the sampling value of the torque command or the current command into the strength of the torque of the motion member at a frequency and that performs an oscillation band determination for determining that a frequency band where the strength of the torque of the motion member peaks is an oscillation band and a filter setting for setting a band stop filter so as to attenuate the strength of the torque of the motion member in the oscillation band when the velocity control gain is adjusted.

Patent Document 4 discloses a servo control device which includes: a velocity command calculation unit; a torque command calculation unit; a velocity detection unit; a velocity control loop; a velocity control loop gain setting unit; at least one filter that removes a specific band of torque command values; a sinusoidal disturbance input unit that performs a sinusoidal sweep on the velocity control loop; a frequency characteristic calculation unit that estimates the gain and the phase of a velocity control loop input/output signal; a resonant frequency detection unit; a filter adjustment unit that adjusts a filter according to a resonant frequency; a gain adjustment unit; a sequence control unit that automatically performs online the detection of the resonant frequency, the adjustment of a velocity control loop gain and the adjustment of the filter; and an adjustment state display unit, and in which the adjustment state display unit displays the adjustment stage and the progress of the sequence control unit.Patent Document 1: Japanese Unexamined Patent Application, Publication No. H07-261853Patent Document 2: Japanese Unexamined Patent Application, Publication No. S62-126402Patent Document 3: Japanese Unexamined Patent Application, Publication No. 2013-126266Patent Document 3: Japanese Unexamined Patent Application, Publication No. 2017-022855

SUMMARY OF THE INVENTION

In a case where when the characteristic of a filter such as a notch filter in a servo control device of one axis is determined, a machine characteristic is affected by the position of another axis or the velocity gain of a servo control device of the other axis, even if the characteristic of the filter is optimized with a certain position of the other axis or a certain velocity gain, oscillation may occur in the other position or the velocity gain. Even in a case where the machine characteristic is not affected by the position of the other axis, oscillation may occur depending on the position of the present axis. Hence, it is desired to perform the optimum adjustment of a filter characteristic even when a machine characteristic is changed by the position of the present axis or is affected by another axis.

(1) An aspect of the present disclosure is a machine learning device that performs reinforcement learning in which a servo control device for controlling a motor is driven under a plurality of conditions and which optimizes a coefficient of at least one filter for attenuating at least one specific frequency component provided in the servo control device, and the machine learning device includes: a state information acquisition unit which acquires state information that includes the result of calculation of a frequency characteristic calculation device for calculating at least one of an input/output gain of the servo control device and a phase delay of an input and an output, the coefficient of the filter and the conditions; an action information output unit which outputs, to the filter, action information including adjustment information of the coefficient included in the state information; a reward output unit which individually determines evaluation values under the conditions based on the result of the calculation so as to output, as a reward, the value of a sum of the evaluation values; and a value function updating unit which updates an action value function based on the value of the reward output by the reward output unit, the state information and the action information.

(2) Another aspect of the present disclosure is a control device which includes: the machine learning device of (1) described above; the servo control device which includes at least one filter for attenuating at least one specific frequency component and which controls the motor; and the frequency characteristic calculation device which calculates at least one of the input/output gains of the servo control device and the phase delay of the input and the output in the servo control device.

(3) Yet another aspect of the present disclosure is a machine learning method of a machine learning device that performs reinforcement learning in which a servo control device for controlling a motor is driven under a plurality of conditions and which optimizes a coefficient of at least one filter for attenuating at least one specific frequency component provided in the servo control device, and the machine learning method includes: acquiring state information that includes the result of calculation for calculating at least one of an input/output gains of the servo control device and a phase delay of an input and an output, the coefficient of the filter (130) and the conditions; outputting, to the filter, action information including adjustment information of the coefficient included in the state information; individually determining evaluation values under the conditions based on the result of the calculation so as to determine, as a reward, the value of a sum of the evaluation values; and updating an action value function based on the value of the reward determined, the state information and the action information.

In each of the aspects of the present disclosure, even when the machine characteristic of a machine tool, a robot, an industrial machine or the like is changed depending on conditions, for example, even when the machine characteristic is changed depending on the position of one axis or the machine characteristic is affected by another axis, it is possible to perform the optimum adjustment of a filter characteristic.

