Parallel rule-based data transmission method and apparatus
This application is a continuation of application Ser. No. 697,006 filed May 8, 1991, now abandoned. 
    
    
     FIELD OF THE INVENTION 
     This invention relates to data transmission systems and more specifically to a multiple telephone line/multiple modem rule based parallel data transmission systems. 
     The novelty of the invention lies in the integration of existing devices, products and networks along with software and firmware which makes the process of data transmission much more efficient, less costly and less time consuming. By utilizing low cost dial-up telephone lines and data compression techniques inherent in the hardware and software, in concert with rule-based file distribution and segmentation techniques, the cost of each data character transmitted and received is significantly reduced. Furthermore, the speed at which data is transferred is increased by an order of magnitude over that normally available for dial-up service. 
     In effect, the economic viability of leased line or other dedicated data circuits is greatly diminished while increasing the economic viability of other low cost data communications channels such as dial-up telephone circuits as a cost effective alternative. 
     BACKGROUND OF THE INVENTION 
     Multi channel data transfer has been described in patents issued to Giorgio (U.S. Pat. Nos. 4,862,456 and 4,864,567) and to Nash (U.S. Pat. No. 4,577,312). However, these patents either use a central office switch or similar system to obtain simultaneous transmission which requires additional equipment and expense or do not comprise compression of data or multi tasking capabilities thereby limiting the over all throughput of data. Further none of these systems use a rule-based approach to the file management associated with transmission of large amounts of data. 
     In contrast, the present invention does not suffer from any of these restrictions. The present invention does not require any additional equipment beyond a normal computer with communication channels and modems. Further the present invention employs compression algorithms to further speed the transmission of data and executes in a multi-tasking environment to give further throughput of information. These factors, together with the use of a rule-based system of file management and channel selection renders the current invention extremely fast and easy to use, with a minimum of operator interaction. 
     Data files are read from the directory of a computer and analyzed as to their data content, format and category (i.e. binary, ASCII text, image format, etc.). Individual files or parts of individual files are then directed to any number of attached modems and their respective dial-up circuits. Data files are received at the remote end, and if segmented, re-appended (reconstructed) and stored on the remote end disk system component of a computer which is also equipped with multiple modems and multiple dial-up circuits. Individual line speeds exceed 60 kbps (6,000 characters per second) while aggregate baud rates (data throughput rates) are only limited by the number of lines and modems available for transmission. 
     U.S. Pat. No. 4,311,877 to Kahn discloses a method and means for improving the reliability of systems that transmit relatively wideband signals over two or more relatively narrowband transmission circuits. This invention divides the transmission line into separate lines, each carrying a different frequency range component of the overall frequency range carried on the original transmission line. The system also segments input signals according to frequency range, splitting the bandwidth of the original signal into frequency range components for transmission on the transmission lines. All input signals are segmented in the same manner so that they are passed by the matching line, thereby conserving bandwidth for the transmission. The present invention segments data files according to file size, not frequency range, in order to increase the overall throughput of the transmission. The segmentation performed by the present invention is based on rules for efficient transmission, not on a fixed set of frequency ranges. 
     Failures in a transmission circuit are detected by the Kahn invention. When a failure is found, a switching circuit is activated and the signal on that line is switched to a new line having the same bandwidth. However, only complete failures in the transmission circuit are detected by the Kahn invention. The present invention checks the signal to noise ratio of each transmission channel, switching channels if line degradation occurs. 
     In the Integrated Services Digital Network (ISDN) embodiment, utilizing 64 KBps or faster channels, individual data rates exceed 300 KBPS while aggregate data throughput is only limited by the number of ISDN circuits (and associated ISDN interfaces) available at both ends of the transmission system. 
     The system utilizes a novel file naming convention which enables the computer hardware and software to optimize data compression and/or file segmentation in order to achieve maximum data throughput. Destination address (telephone number) data are automatically computed based upon unique data file suffix interpretation. 
     A user friendly software front-end system is provided to automatically configure the system to the individual requirements of the user. 
     Data throughput rates are achieved which were hitherto only accomplished utilizing expensive leased data circuits operating at data rates of from 4800 to 56 KBPS or higher or other non-switched dedicated services. The system also facilitates switching of data which is not normally possible when leased lines (i.e., point to point) are utilized. 
     It is therefore an objective of the present invention to employ an expert system/rule-based approach to data transmission to minimize operator interaction in such data transmission, yet maximize the speed of transmission of data files of any type. 
     It is another objective of the present invention to provide a cost effective system of data transmission that is similar in performance to more expensive leased lines or dedicated higher capacity data transmission lines but which relies upon the use of low-cost communications channels such as dial-up service. 
     It is another objective of the present invention to further reduce communications costs by conducting as much file related manipulation as possible off line. 
     It is yet another objective of the present invention to provide a low cost replacement for more sophisticated and expensive data transmission controllers such as the IBM 3725 or 3705, or similar communications front end systems. 
     Further it is an objective of the present invention to provide rapid data communication in both foreign and domestic ISDN standard environments. 
     SUMMARY OF THE INVENTION 
     The Parallel Rule-Based Data Transmission method and apparatus comprises several major components: 
     A. Multiple computer systems equipped with communications ports to send and receive data, 
     B. Multiple modems attached to multiple communications ports on each computer system, 
     C. Multiple telephone company dial-up lines or other data transmission media attached to each computer (both sending and receiving units), 
     D. Rule-based data communications software programs providing multi-tasking and multiprogramming capabilities to divide or segment files to facilitate simultaneous transmission and reception of data based upon the number of telephone lines or data transmission channels and modems utilized. 
     The computer system is a low cost conventional 80286, 386 or 486 microprocessor based PC or other computer or data processor (such as those based on Motorola 68000 or other CPU) having the ability to run the rule-based software that is used for the data transmission management and having an amount of memory sufficient to store the data files to be transmitted. The data processor used must of course also have enough random access memory to store and run the software. It will be readily apparent to those skilled in the art that other computers capable of running rule-based systems which are not based on these same chips are still viable processors for the invention with only slight changes to the software disclosed. The computer further comprises multiple communication ports to facilitate the multi channel simultaneous transmission of data. Each computer system can transmit data to or receive data from any other computer system similarly equipped without regard to distance or individual line conditions. Provisions are made for ISDN compatibility through the automatic distribution of data files or their components to each ISDN B channel connected. 
     Multiple modems on a single computer are employed to serve as the outgoing telecommunication equipment for simultaneous transmission of segmented (where appropriate) data files. The modems also employ data compression/decompression means to further speed the parallel transmission of data, and to decompress incoming data &#34;on-the-fly&#34; thus further reducing communication time. The invention uses standard run length encoding or Huffman encoding as its data compression scheme for ASCII data and a standard commercially available data compression algorithm known as CommPressor available from Adaptive computer Technologies, Santa Clara, Calif. for binary data. Data are compressed via hardware and software techniques to levels up to or in excess of 6:1 thus elevating data transmission rates on standard telephone lines to in excess of 6,000 characters per second per line utilized. Thus through simultaneous use of multiple data channels the aggregate data transmission rate is only limited by the number of communications channels available. 
     Multiple dial-up lines are used as the basic transmission medium over which the present invention sends its data. These lines are much less expensive than dedicated leased lines thereby yielding a further financial advantage of users of the present invention. It is important to note however, and it will be readily apparent to those skilled in the art of telecommunications that other transmission media also exist for which the present invention is equally applicable. For example, radio frequency links, satellite data communication, laser communication, fiber optic links and others are all candidate transmission media for use with the present invention. Collectively, these are referred to as transmission media. The transmission channels refer to the transmission media together with the send and receive ports and modems. 
     The system incorporates several rule-based computer software programs which facilitate data file segmentation, data compression, and reconstruction, as well as error detection and correction and automatic transmission speed control responsive to the condition of each individual transmission channel. As is well known in the art, rule-based systems are run according to &#34;rules&#34; that are loaded into the system software. These rules govern the functions of the system in any given situation. This is accomplished by sensing certain control parameters within the system. The status of these parameters characterizes the status of the system at any point in time and also provides an address to retrieve from the system memory the rule corresponding to that system status. The rule is then utilized by the system software to effect a change in the system based on the current system status. Further rules relating to Least Cost Routing are also employed to further minimize costs. In addition, software provides automatic dialing directories based upon a novel file naming convention, selective adaptive compression based upon data file contents and automatic retry if a line is dropped or intentionally interrupted. If one channel is inordinately noisy, the invention senses the problem automatically, automatically reallocates the data to be transmitted to less noisy channels and all other communications channels will adjust to accept the increased load automatically until the noise level decreases. The communications software currently in use is the Relay Gold Communications software package available from Microcom, Inc. Other such communications packages can also be used as a substitute for Relay Gold. 
     Table lookup software programs are provided to automatically dial the correct destination telephone number or other equipment address to which data files will be transmitted based upon the naming conventions utilized in each file group to be transmitted. Each group of files may be associated with and targeted for any number of individual telephone numbers or addresses thus providing multiple simultaneous transmission/reception circuits only limited by the number of lines, ports and modems/ISDN circuits available. Each destination site or equipment, as previously stated, has a computer(s) each with multiple modems with separate addresses or telephone numbers. These modems receive the parallel transmission of multiple files or segmented files simultaneously. 
     File segmentation is the process by which the present invention examines the number of transmission channels available, based upon the number of sending and receiving modems available, and divides the file or files to be sent, more or less evenly among the available channels. Rules exist for the orderly segmentation of files as explained below. The actual segmentation occurs at points in the file where a division of that file naturally occurs (such as a carriage return entry). Alternatively to the extent that addresses on a network are designated in a specific pattern an algorithmic determination of the address can be substituted for the table look-up procedure. 
     All file segmentation (file division) and file reconstruction processes take place off-line, while the computers are not communicating, thus reducing &#34;connect time&#34; expense. For example, a file of 2,000,000 characters is automatically divided by the number of transmission channels available and each segment is directed to a different channel. Segment prefixes and suffixes are added to the file to denote how the segment relates to one another. At the destination, the data file segments are reassembled to recreate the individual file as it was originally input to the transmission equipment. Reassembly is the act of reading the file segment identifier information (segment prefixes and suffixes) appended to the segments and contained in a special file (known as MAP.DAT) file and using that information together with segment identification information to &#34;piece&#34; the file back together into its original form. 
     The present invention allows for data to be transmitted in a highly flexible way. Single files may be simultaneously transmitted to multiple locations. Alternatively multiple files may also be transmitted to single or multiple locations simultaneously. This multi-tasking occurs from a single copy of the software. There is no requirement for multiple copies of the software to be running for each transmission session. Data transmission sessions run from a single RAM resident copy of the program. This has the obvious advantage of preserving RAM space for other applications or to allow the transmission process to occur in a speedy fashion. 
     Once the system has been configured and files loaded, the rule-based software operates automatically to perform the necessary file management and segmentation functions. All such functions are performed without any need for operator intervention. After transmissions are complete, the system automatically terminates its data transmission session and readies itself for the next series of data transfers. 
     Transmission channel allocation optimization is accomplished by the software system via a series of rules which determine file segmentation. The segmentation is characteristically based on the transmission characteristics of the data file, namely, the number of files to be transmitted, their respective sizes and contents, their compression attributes, and the number of available communication equipment (modems) at both the transmission and destination locations. Optimization software then allocates the optimal number and type of files to each data transmission channel for subsequent transmission according to these data filed characteristics. If one or more of the destination ports is busy or otherwise inoperable, the system redistributes targeted data files automatically to the remaining operable channels (ports) for transmission on a first-in-first out (FIFO) basis. Further, the present invention automatically monitors the transmission channel itself to determine that the channel is operating properly and is not inordinately noisy. If the transmission channel malfunctions, the invention senses this failure, and the software reallocates the data file or segments to be sent to the transmission channels that are functioning properly. 
     The resident software also provides full reporting capability to the user including file(s) transmitted, destination(s), throughput achieved and associated error detection and correction statistics. 
     The present invention relies upon several novel file and file suffix naming and interpretation conventions in order to achieve full automation without user intervention. User file naming rules are integrated into a table lookup facility inherent in the software which determines file collating sequences when files are segmented and reassembled. The software sorts files by type, size and structure, looks up and determines, based upon three character (or more) file suffix (alphanumeric) destination telephone numbers or addresses for each group of files to be transmitted. This information is then passed to the operating system software and actual telephone numbers are then loaded into memory and passed to the modem(s) for off hook dialing. 
     The rule-based software then determines the sizes of files and counts number of files to be sent. If number of available ports exceeds number of files or there is only one file to be transmitted, file segmentation is invoked above a certain threshold size length. File(s) are divided based upon number of complete transmission channels available and segments are marked for reassembly. File segments are then sub-labeled with new prefixes indicating their component serial level and prepared for transmission. 
     Data compression occurs based upon the type of data being sent (ASCII or binary). The type of data is determined by the system and the data compression peculiar to that type of data is applied by the modem and software as the data is transmitted (as discussed above). Similarly, the destination modem decompresses the data as it is received, first determining via a transmitted identifier, the type of data compression used. 
     At the destination, file segmentation detected via the presence of a special file known as a MAP.DAT file, segment serial numbers are sorted and files are reassembled subsequent to disk storage at which time file naming conventions are reinvoked. It is important to note that file segmentation and reassembly functions take place off line (on hook) so as to minimize telephone line connect time and its attendant expense. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1. The Preliminary Data Transmission Flow 
     FIG. 2. The Preliminary Data Transmission Flow-Continued 
     FIG. 3. The Data Transmission and Monitoring Flow 
     FIG. 4. The Data Transmission and Monitoring Flow-Continued 
     FIG. 5. Data Receiving Flow 
     FIG. 6. Incoming Data file or File Segment Processing. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     Referring to FIG. 1, the transmission process is described. 
     The invention first ascertains the user identity and whether the user will be transmitting or receiving data [10]. If transmitting data, the invention assigns file names to the data to be sent including suffixes which uniquely identify the destination telephone numbers or addresses [11]. 
     The invention next reads the data file presented for transmission. The file is configured for subsequent operations [12]. After configuration the Volume Table of Contents (VTOC) is read [13]. A directory is opened for the storage of the data file(s) to be transmitted [14] and the configured file is stored in the designated directory [15]. Simultaneously with the storage, the data file characteristics are determined. These include the size and contents of the file, and compression attributes, as well as the number of modems available at the transmission and destination locations. 
     The files so stored are next scanned and formats are determined for the transmission of the files [16]. In addition, the suffixes to any files are interpreted to determine the destination of the file [16]. 
     Based upon the files scanned and the interpretation of the suffixes, a table lookup file is addressed to determine the telephone number or address of the destination [18]. These telephone numbers are then loaded for the dial-up calls to be made subsequently by the multiple modems [33]. In preparation for the transmission of files, the data files are collated [17] and the total number of bytes to be transmitted are determined [19]. The optimal segmentation of the file for transmission is next calculated [20] (if appropriate) based upon certain segmentation rules. Finally, the file is segmented for subsequent transmission [21]. 
     The file segmentation rules implemented by the rule-based software apply the following criteria: 
     1. If the number of available complete transmission channels (i.e. a transmission modem with corresponding receiving modem) exceeds the number of files to be sent, segment the files, provided that the files exceed a certain threshold. (In the case of the preferred embodiment the threshold is set at 2 KB. Other thresholds can also be set). 
     2. If the number of available complete transmission channels does not exceed the number files to be sent and the files are more or less equal (as further explained below) allocate the files evenly over the available transmission channels without segmentation. 
     3. If the number of available channels is equal to or less than the number of files but the file sizes are disproportionate to one another, then segment to achieve optimum throughput. &#34;Disproportionate&#34; size is determined by examining the Volume Table of Contents to determine the various file sizes. If the largest file is more than twice the size of the next largest file, the largest file will be segmented and allocated to more than one transmission channel. 
     The invention also incorporates a rule based subsystem, comprising a subset of the segmentation rules to optimize the cost associated with data transfers based upon the types of communications channels available to the user and their respective relative costs per character transmitted. This function is invoked via a least cost routing table, which is the volume of system memory storing the least cost routing rules, which takes into account: 
     Time of day/day of week tariff charges and discounts. 
     Distance sensitive (i.e. NPA/NXX/VH) tables which articulate the cost per minute of connection based upon the distance of the individual call or calls. 
     Anticipated call duration (which is computed based upon total byte count divided by the number of connections available at both ends of the connection.) sensitivity. 
     The cost optimization algorithm predicts the lowest possible connection cost (i.e. selecting the least expensive lines for each respective communications session) based upon the types and categories of channels available to the user which might include: 
     standard dial up telephone lines. 
     PBX dial access trunks. 
     Tie (dedicated circuit) lines. 
     Flat cost per minute dial up lines. 
     Microwave Central Office bypass circuits. 
     Other Central Office bypass circuits. 
     Volume discount dial up circuits. 
     Time of day/day of week discount dial up circuits. 
     Any combination of these and other channels available to the user at both ends of the communication link can be used for purposes of these calculations. 
     Provision is made for the updating of all tariff tables in order to assure that current cost data is maintained and properly applied to each communications based file transfer. 
     Referring to FIG. 3 the transmission process is further described. The invention next determines if binary files are present (vs ASCII files) [30]. If binary files are present, the invention invokes a binary data compression algorithm [31]. If binary files are not present an ASCII file compression algorithm is invoked [32]. Once the file segmentation is accomplished, the multiple modems of the invention are dialed [33] based upon the telephone numbers or channel addresses loaded from the lookup table [18]. 
     The VTOC of the receiving computer is tested [34] to determine if appropriate space is available to receive all of the files about to be transmitted. If insufficient space is available the transmission session is terminated and an error message is displayed at the transmitting system. If sufficient space is available, the files are subsequently transmitted over the multiple modems [35]. Transmission channel quality and status are continuously monitored during the course of transmission [35]. If line quality remains adequate transmission continues [37]. If line quality falls below minimum standards of signal-to-noise ratio the remaining untransmitted segments are reallocated to those transmission channels that are functioning properly [36] and transmitted on a FIFO basis. At the end of the transmission the system detects the end of transmission (ETX) [38] signal, and transmits a hang up tone and terminates the transmission over the various telephone lines [39]. At this point the invention is off-line. The invention next writes reports [40] that include segment size, transmission time, errors encountered and aggregate throughput and the system returns to the start point ready for the next transmission. 
     Referring to FIG. 5, the receive and decompression function is described. The receiving system is first configured to accept files that will be transmitted to it [50]. A directory is opened to receive the incoming data [51] and the multiple modems are set and Data Terminal Ready (DTR) is established [52]. The answer mode is set on the modems and the system awaits the transmission of data [53]. When transmission begins the incoming ring is detected and the modems answer the incoming calls [54]. Based upon incoming information, the compression scheme of the incoming data is determined [55]. The data is decompressed [56] in accordance with the appropriate decompression algorithm. 
     Referring to FIG. 6 the receive and decompression process is further described. After the data is decompressed the invention determines if file segmentation has occurred. This is accomplished by determining if segment file entitled MAP.DAT exists [57]. The MAP.DAT file is a file that comprises instructions for how a file has been segmented and is the primary input to the file reassembly subroutine to allow the segmented file to be reassembled. If the MAP.DAT file exists, the reassembly subroutine is called and loaded into RAM [59] together with the data from the MAP.DAT file. At the same time that segmentation is being detected, the decompressed data is stored for subsequent operations [58]. The invention next continuously monitors incoming data for the &#34;end of transmission&#34; (ETX) signal [60]. Once this signal is sensed, the system goes off-line [61] and processing continues. For those files that were segmented for data transmission, the invention reads the file collating information in the MAP.DAT file and reassembles the segments and stores them [58] into the original file format [62]. 
     If the MAP.DAT file is not detected [57] the invention detects the end of transmission signal [60] and goes off line [61]. Non-segmented data can then be recalled from storage [58] for subsequent display [64] storage [63], printing [65] and/or report writing [66]. The system is thereafter returned to the starting point for receipt of the next data file. 
     An important aspect of the invention is the file naming convention mentioned above. A three (or more) character file suffix is utilized which is equivalent to the &#34;target&#34; destination address (i.e., telephone number) to which the file is to be transmitted. Valid entries are any combination of alphanumerics (i.e., a-z, 000-999), which are then related to a table which specifies from one to any number of target telephone numbers to be utilized for file transfers. 
     File name identifiers (prefixes) are composed of eight or more alphanumeric characters which correspond to the individual user&#39;s internal file naming conventions. The latter forms the basis for the resulting file collating sequence. An example might be a series of image files for a medical diagnostic procedure where the first five characters are the x-ray number of the patient, followed by the image number where: 
     R12345001 is the file name. 
     R12345 is the x-ray number. 
     001 is the image number for that patient. 
     Thus, files conforming to this format would be directed as follows: 
     
         R12345001.xxx, R12345002.xxx, R12345003.xxx 
    
     All of the above file names contain the suffix xxx. The table entry identifies that suffix as telephone numbers 1-n and those numbers are loaded for subsequent dialing. Files are sorted by prefix and loaded for transmission and are received at the remote location in the collated order. In the event that the user requires file transfers between minicomputers and mainframes, the invention also provides terminal emulation capability for the IBM 3270 and 3101, the DEC VT100, 220 and 240 and Telex and TTY modes as well as other terminal emulations and transmission protocols. Terminal emulation programs are well known and widely commercially available. They are basically translation programs which allow data that are entered on a particular terminal or computer to be adapted for communication with and use by another type of terminal or computer, or to allow data communication over a communication channel that is not normally adapted for use with the sending and/or receiving terminal or computer. Thus, communication among the listed terminals, personal computers (PCs), and mainframes, as well as others, is possible using the software resident in the data processor utilized by the system of the present invention. 
     While the embodiment just described may employ single byte (such as the 8250 chip) communications buffers another embodiment employs multiple byte communications buffering (such as the 16550AH FIFO interrupt driven buffer.) Yet another embodiment eliminates the serial port/UART (universal asynchronous receiver/transmitter) combination completely and substitutes the parallel computer port as the communications interface medium. The latter embodiment provides data transfers at rates exceeding four times that of a standard serial port modem connection. 
     While the source code to be executed may reside on the computer&#39;s disk drive for loading into its random access memory, it may also reside in EPROMS as an electronic disk. The latter embodiment provides greater speed of execution, enhanced security from tampering and greater ease of installation. 
     On the following pages, the software of the preferred embodiment is presented. It will be apparent from certain of the annotations that this software is adapted to a medical application, specifically that of transmitting image files between locations. However, it will be apparent to those skilled in the art that the present invention can be utilized to transmit all manner of data files and is not limited by the application presented. Thus it will be apparent to those skilled in the art that new applications for the invention may be devised without departure from the spirit and scope of the invention as claimed. ##SPC1##