CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 USC 119 of German Application No. DE 102018121031.0 filed on Aug. 29, 2018, the disclosure of which is herein incorporated by reference.

BACKGROUND OF THE INVENTION

The invention is directed to an agricultural work machine and to a method for operating an agricultural work machine.

Agricultural work machines which include, in particular, self-propelled harvesters such as combine harvesters and chopper-type forage harvesters generally have, as a header, a crop collection arrangement by which crops in the field stand can be cut and collected for further processing in the combine harvester. A crop collection arrangement in itself generally has a plurality of individual components which can be operated with different parameters. Examples of such components in a combine harvester are a crop pickup device in the form of a reel, a crop cutting device in the form of a cutting unit table with a knife bar, a crop conveying device in the form of a conveyor screw, in particular a cross auger, and a crop intake device in the form of an inclined conveyor. In a combine harvester, the placement (cut height, position) and rate of rotation of the reel and the speed of the cross auger and of the inclined conveyor, for example, can be adjusted. The crop stream inside the crop collection arrangement can be influenced in this way. The driving speed of the harvester further influences the crop stream inside of the crop collection arrangement because the amount of collected crop which is processed by the crop collection arrangement is influenced in this way.

Despite a number of automatic adjustments of the crop collection arrangement, the driver of the agricultural work machine must constantly monitor the crop stream inside of the crop collection arrangement as well as inside of the combine harvester. For example, the driver must ensure that the produce flow velocity inside of the header and in the rest of the interior of the combine harvester is as uniform as possible because when the produce flow is irregular the speed of the vehicle and, therefore, collection of crops may have to be reduced in order to avoid a crop jam. If a crop jam occurs, the harvesting process must be interrupted and a time-consuming clearing of the crop jam must be performed in the header and, further, in the combine harvester. The time required for this significantly impairs the efficiency of the harvesting operation.

Therefore, it is desired that an open loop control and/or closed loop control of the process sequences in the agricultural work machine, for example, in the combine harvester, is automated as far as possible so as to relieve the burden from the driver. This requires an analysis of the process sequences which is as differentiated as possible beginning with the process sequences inside of the header. For an optimal analysis, it is necessary to monitor the crop stream in an optimal manner and to detect changes in the crop stream, particularly the produce velocity, as promptly as possible and to carry out corresponding control steps.

To this end, it is known, for example, from EP 3 300 019 A1, to provide a control device with a sensor unit and an image processing unit via which the crop stream is monitored using the method of optical flow. The sensor device has, for example, two cameras which are directed from the driver's cab onto the crop collection arrangement and which encompass individual portions of the components and the crop stream. The sensor device generates an image sequence, i.e., a plurality of consecutive images. According to the prior art, every two consecutive images are grouped to form an image pair, and positional displacements of intensity patterns, that is, of pixels or groups of pixels, between the images of the respective image pair are determined. The intensity patterns are not necessarily formed by objects or particular crop features but generally by pixels or groups of pixels in the image whose position changes from one image to the next image. When the time interval between the images of the image pair is known, the velocity at which the intensity patterns travel can be inferred from these positional displacements of the intensity patterns. The velocities of a plurality of intensity patterns can be combined for each image pair and each image sequence comprising a plurality of image pairs to form velocity characteristic maps. A velocity characteristic map of this kind relates to optical flow. With regard to the method of optical flow and its application for monitoring the crop stream inside of a header, reference is made to EP 3 300 019 A1 which originated with the Applicant and the disclosure of which is incorporated by reference into this application.

One challenge consists in considering and analyzing the crop stream in the most differentiated manner possible in order to respond to an incipient crop jam in a timely manner.

The problem upon which the invention is based is to configure and further develop an agricultural work machine in such a way that the monitoring of a header is further improved with respect to accuracy, reliability and timeliness.

SUMMARY OF THE INVENTION

The above-stated problem is solved in an agricultural work machine such as a harvester, having a header for performing agricultural work and having a control device which has at least one sensor unit for detecting a crop stream in and/or around the header and an image processing unit for processing images which are generated by the sensor unit based on the crop stream detected via sensor. The control device is configured to detect regions of like characteristics, components of the header and properties of the crop stream and is configured to use that which has been detected for open loop control and/or closed loop control of process sequences in the agricultural work machine.

A key basic consideration consists in detecting the header, in particular a crop collection arrangement of a combine harvester or chopper-type forage harvester, using sensors such that individual areas in the header and around the header can be detected, i.e., identified. The areas are, for one, regions of like characteristics. A region of like characteristics is, for example, the unharvested field stand in front of the header in driving direction. A further region of like characteristics is, for example, the harvested field stand (stubble field) behind the header.

Regions of like characteristics may also include produce flow regions of like characteristics. For example, a region of like characteristics is formed by the crop stream within the crop pickup device, in particular the reel, and another region of like characteristics is formed by the crop stream vertically above the crop cutting device, particularly vertically above the cutter unit table, in particular in the area between the crop pickup device or reel and the crop conveying device, in particular the conveyer screw.

Apart from these areas which are characterized by the presence of mainly biomass (crop, field stand) within the area, further areas are components of the header having a surface portion of a machine part of the component, which surface portion can be detected by the sensor unit. Corresponding components are, for example, the crop pickup device, particularly the reel, the crop cutting device, particularly the cutter unit table, the crop conveying device, particularly the conveyer screw, and/or the crop intake device, particularly the inclined conveyor, of a crop collection arrangement, for example, of a combine harvester. Accordingly, each of these areas is defined by the presence of a surface portion of a machine part of a component of the header.

In addition to these areas, it is suggested that a detection of properties of the crop stream, for example, produce flow velocities of the crop stream or portions thereof and/or a crop jam, is carried out.

On the basis of the information gathered in an analysis process of this type, process sequences in the agricultural work machine are then controlled according to the suggestion in a subsequent or simultaneously running open loop control process and/or closed loop control process. As a result of dividing into regions and components, the behavior of a header and of the crop stream guided along therein can be considered and analyzed in a particularly differentiated manner. Correspondingly, the monitoring of a header, in particular a crop collection arrangement, is further improved with respect to accuracy, reliability and timeliness.

It is provided according to the suggestion that the control device is configured to detect regions of like characteristics, components of the header and properties of the crop stream and is configured to use that which has been detected, i.e., the detected regions of like characteristics, the detected components of the header and/or the detected properties of the crop stream, for open loop control and/or closed loop control of process sequences in the agricultural work machine.

“Detection” of the above-mentioned regions and components preferably does not simply mean a determination of areas but also an unambiguous identification of the respective area, that is, an association of the area with a function and/or property. Accordingly, for example, detection of the region with the field stand in front of the header means that an area is first detected or located by sensor detection and image processing and this area is then identified as a specific area with a particular function and/or property, for example, specifically as field stand in front of the header. Therefore, the individual areas can be distinguished from one another not only based on their position but also based on their function and/or property. This makes it possible to differentiate determined motions.

In one embodiment, an identification is possible based on stored identification data characterizing the respective area (region or component). For example, determined dimensions or positions of a region or component inside of the header or outside of the header can be stored in a data storage as identification data, and the region or component can then be identified as a determined region or component on the basis of these identification data.

In order to allow the determination of regions and/or components and/or produce stream properties, the image processing unit can use various methods based on the images generated by the sensor unit. For example, the image processing unit can use the method of velocity characteristic map determination, also known as optical flow method, which will be explained in more detail later. Additionally or alternatively, it can use the method of line detection which will also be explained later. Additionally or alternatively, it can also use at least one color filter. In this way, an object detection in which, in particular, biomass is distinguished from mechanical objects can also be carried out.

A velocity characteristic map determination, or optical flow method, is an image processing method, know per se, and is based on the determination and analysis of intensity patterns (pixels or pixel groups) in consecutive images of an image sequence. Accordingly, an image sequence, i.e., a plurality of consecutive images, is generated particularly via one or more cameras. Every two consecutive images, in particular two directly successive images, of the image sequence are grouped to form an image pair, and positional displacements of intensity patterns, i.e., of pixels or groups of pixels, between the images of the respective image pair are determined. When the time interval between the images of the image pair is known, the velocity at which the intensity patterns travel can be inferred from these positional displacements of the intensity patterns. The direction in which the intensity patterns travel can also be determined from the positional displacements. The optical flow method accordingly comprises the pixel-based generation of information on velocities of intensity patterns on the one hand and on the movement directions of these intensity patterns on the other hand. The velocities of a plurality of intensity patterns can be combined for each image pair and each image sequence comprising a plurality of image pairs to form velocity characteristic maps, which can also be referred to broadly as optical flow.

Since the biomass on the one hand and the surface portions of the mechanical components of the header on the other hand move, respectively, in a characteristic manner, the image processing unit can also distinguish the individual areas from one another.

A method of line detection can be used additionally or alternatively. This is an image processing method in which an edge detection is initially carried out for the respective image and an edge image is generated therefrom. In an edge detection, two-dimensional areas in a digital image are separated from one another when they differ from one another sufficiently with respect to color value or grayscale value, brightness or texture along straight or curved lines. An edge detection of this kind and generation of an edge image are carried out in particular by means of the Canny algorithm, known per se. In the method of line detection, according to the suggestion, straight lines are subsequently determined and displayed or highlighted in the edge image by means of a straight line detection. A straight line detection of this kind is preferably carried out by means of the Hough transform, known per se. Contours of components or of machine parts thereof can be detected by means of the line detection, which likewise allows individual areas (regions or components) to be distinguished from one another and possibly also identified. In this way, in a digital image showing, for example, a crop collection arrangement of a combine harvester, the reel in particular can be emphasized and accordingly identified because it is shown by many straight lines in a two-dimensional image. Also, apart from the detection of the respective component, it is possible to determine geometric parameters of the respective component, for example, the inclination and/or height of the component.

When image pairs of consecutive images of an image sequence generated by the sensor unit are processed using line detection, image line patterns, namely, patterns of the straight line or lines determined by means of straight line detection, result in every image. In a corresponding image pair, positional displacements of line patterns, that is, of lines or line groups, between the images of the respective image pair can then be determined similarly as in the method of velocity characteristic map determination. When the time interval between the images of the image pair is known, the velocity at which the line patterns travel can also be inferred from these positional displacements of line patterns. In principle, the direction in which the line patterns move can also be determined. That is, the line detection comprises line-based generation of information on velocities of line patterns and possibly also on movement directions of the latter. Therefore, in addition to the detection of the areas, it is also possible to determine the velocity at which lines or contours of components move between two consecutive images.

The method of the invention also includes determining reference regions or reference surface portions with which another region to be analyzed currently can be compared. A comparison of this type between two regions, one of which serves as basis (reference), makes it possible to infer properties, particularly the velocity, in the region to be currently analyzed. This holds true not only for regions but also for components and surface portions thereof. In a particularly preferred manner, there is the possibility of determining a reference velocity for a reference region. The reference velocity can be determined, for example, by relating the driving speed of the agricultural work machine, which is known or can be accurately detected by appropriate sensors, to a velocity of the field stand of a region, which velocity is calculated from the displacement amounts. The reference velocity is a velocity corresponding to the driving speed in a determined ratio. This is also correspondingly possible for velocities of surface portions of machine parts of the components. In this case, for example, the rate of rotation of a component, for example, the reel, which is known or can be accurately detected by appropriate sensors, can be correlated to the velocity of the surface portion calculated from the displacement amounts. The reference velocity is then a velocity corresponding to the rate of rotation in a determined ratio. Velocities in other regions or of other surface portions of components can then also be inferred based on reference velocities of this kind.

According to one embodiment, the header is a crop collection arrangement for cutting and picking up crop from a field stand which has, as components, a crop pickup device, a crop cutting device, a crop conveying device and a crop intake device. In particular, it is a crop collection arrangement of a combine harvester or chopper-type forage harvester.

In the further configuration, the velocity of a surface portion of a machine part of the respective component can also be determined on the basis of a marker which then forms the surface portion. Either a characteristic location, for example, edge or surface area, on the respective machine part or a separately applied marking serves as marker. The marker then shows the surface portion that is detected by the sensor unit for determining the velocity.

In a preferred configuration, different sensors for detecting the crop stream and/or header can be provided, in particular at least one optical sensor, for example, a camera or a plurality of cameras. An optical sensor of this kind is useful particularly when the method of velocity characteristic map determination (optical flow method) is to be used. In principle, the sensor unit can also have at least one Lidar sensor, at least one radar sensor and/or at least one ultrasound sensor. All of the sensors mentioned above are also suitable for use with the method of line detection.

According to a further teaching a method is claimed for the operation of an agricultural work machine, in particular an agricultural work machine as suggested above, in which regions of like characteristics, components of the header and properties of the crop stream are detected, and process sequences in the agricultural work machine which are based on the latter are controlled by open loop controlling and/or closed loop controlling.

