FIELD OF THE DISCLOSURE

The present disclosure relates to systems and methods for using natural language processing (NLP) to automatically generate three-dimensional objects in a virtual space.

BACKGROUND

Online gaming platforms using three-dimensional virtual spaces are known. Voxels are known, and may be used to form three-dimensional objects. User-generated three-dimensional content is known.

SUMMARY

One aspect of the present disclosure relates to a system configured to use natural language processing (NLP) to automatically generate three-dimensional objects in a virtual space. The system may include electronic storage, one or more hardware processors, and/or other components. The electronic storage may be configured to electronically store information. The stored information may include a set of three-dimensional objects. The set may include an annotated object that includes a textual description of the annotated object and an unannotated object that lacks a description. The annotated object may use a first three-dimensional format. The unannotated object may use a second three-dimensional format. The one or more hardware processors may be configured by machine-readable instructions. The one or more hardware processors may obtain the set of three-dimensional objects. The one or more hardware processors may convert one or more three-dimensional objects in the set into one or more objects using a three-dimensional voxelized format that natively supports voxels, responsive to using three-dimensional formats that fail to support voxels, such that the set includes voxelized three-dimensional objects. The voxelized three-dimensional objects may include a first voxelized object based on the annotated object. The one or more hardware processors may encode, using a variational autoencoder that includes an encoder and a decoder, the voxelized three-dimensional objects into pairs of multi-dimensional vectors. Individual pairs may include a vector of means and a vector of standard deviations. The one or more hardware processors may create sampled multi-dimensional vectors having a particular dimensionality by sampling from the individual pairs of multi-dimensional vectors such that a first sampled multi-dimensional vector is created based on an encoding of the first voxelized object. The one or more hardware processors may decode the sampled multi-dimensional vectors into decoded voxelized three-dimensional objects. The one or more hardware processors may determine loss information for the decoded voxelized three-dimensional objects by comparing the decoded voxelized three-dimensional objects to corresponding individual voxelized three-dimensional objects. The one or more hardware processors may use the loss information to train the variational autoencoder. The one or more hardware processors may fine-tune a pretrained text-based system to generate multi-dimensional vectors having the particular dimensionality from textual descriptions included in annotated objects in the set of three-dimensional objects, wherein one or more prompts for the pretrained text-based system are based on a combination of the textual description of the annotated object and the first sampled multi-dimensional vector. The one or more hardware processors may receive, from a user, a user input, wherein the user input includes user-provided text describing a three-dimensional object. The one or more hardware processors may generate, using the fine-tuned pretrained text-based system, a user-provided multi-dimensional vector from the user-provided text in the user input. The one or more hardware processors may decode, using the decoder, the user-provided multi-dimensional vector into a user-provided voxelized three-dimensional object. The one or more hardware processors may present, to the user, the user-provided voxelized three-dimensional object.

Another aspect of the present disclosure relates to a method to use natural language processing (NLP) to automatically generate three-dimensional objects in a virtual space. The method may be implemented in a computer system including electronic storage, one or more hardware processors, and/or other components. The method may include obtaining a set of three-dimensional objects. The set may include an annotated object that includes a textual description of the annotated object and an unannotated object that lacks a description. The annotated object may use a first three-dimensional format. The unannotated object may use a second three-dimensional format. The method may include converting one or more three-dimensional objects in the set into one or more objects using a three-dimensional voxelized format that natively supports voxels, responsive to using three-dimensional formats that fail to support voxels, such that the set includes voxelized three-dimensional objects, wherein the voxelized three-dimensional objects include a first voxelized object based on the annotated object. The method may include encoding, using a variational autoencoder that includes an encoder and a decoder, the voxelized three-dimensional objects into pairs of multi-dimensional vectors, wherein individual pairs include a vector of means and a vector of standard deviations. The method may include creating sampled multi-dimensional vectors having a particular dimensionality by sampling from the individual pairs of multi-dimensional vectors such that a first sampled multi-dimensional vector is created based on an encoding of the first voxelized object. The method may include decoding, using the decoder, the sampled multi-dimensional vectors into decoded voxelized three-dimensional objects. The method may include determining loss information by comparing the decoded voxelized three-dimensional objects to corresponding individual voxelized three-dimensional objects. The method may include using the loss information to train the variational autoencoder and the decoder. The method may include fine-tuning a pretrained text-based system to generate multi-dimensional vectors having the particular dimensionality from textual descriptions included in annotated objects in the set of three-dimensional objects, wherein one or more prompts for the pretrained text-based system are based on a combination of the textual description of the annotated object and the first sampled multi-dimensional vector. The method may include receiving, from a user, a user input, wherein the user input includes user-provided text describing a three-dimensional object. The method may include generating, using the fine-tuned pretrained text-based system, a user-provided multi-dimensional vector from the user-provided text in the user input. The method may include decoding, using the decoder, the user-provided multi-dimensional vector into a user-provided voxelized three-dimensional object. The method may include presenting, to the user, the user-provided voxelized three-dimensional object.

As used herein, any association (or relation, or reflection, or indication, or correspondency) involving servers, processors, client computing platforms, user interfaces, three-dimensional objects, neural networks, databases, formats, voxels, encoders, decoders, generators, discriminators, vectors, (loss) information, pretrained text-based systems, instructions, operations, determinations, presentations, and/or another entity or object that interacts with any part of the system and/or plays a part in the operation of the system, may be a one-to-one association, a one-to-many association, a many-to-one association, and/or a many-to-many association or N-to-M association (note that N and M may be different numbers greater than 1).

As used herein, the term “obtain” (and derivatives thereof) may include active and/or passive retrieval, determination, derivation, transfer, upload, download, submission, and/or exchange of information, and/or any combination thereof. As used herein, the term “effectuate” (and derivatives thereof) may include active and/or passive causation of any effect, both local and remote. As used herein, the term “determine” (and derivatives thereof) may include measure, calculate, compute, estimate, approximate, generate, and/or otherwise derive, and/or any combination thereof.

These and other features, and characteristics of the present technology, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the invention. As used in the specification and in the claims, the singular form of “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise.

