[
    {
        "abstract": "Write the 'Introduction' section of the article titled 'Adobe Firefly'.",
        "related_work": "Adobe Firefly is a generative machine learning model included as part of Adobe Creative Cloud. It is currently being tested in an open beta phase @cite_1 @cite_2 @cite_3. Adobe Firefly is developed using Adobe's Sensei platform. Firefly is trained with images from Creative Commons, Wikimedia and Flickr Commons as well as 300 million images and videos in Adobe Stock and the public domain @cite_4 @cite_5 @cite_6. It uses image data sets to generate various designs @cite_7.",
        "ref_abstract": 
        {
            "@cite_1": "Adobe brings Firefly to the enterprise. Adobe today announced that it is bringing its Firefly generative image generator to its enterprise customers and allowing them to customize the model with their own branded assets. In conjunction with this, the company is also bringing its Adobe Express design app (which you remember under its previous name of Adobe Spark) to enterprise users, who will be able to access Firefly from there as well. “Enterprise leaders expect content demands will increase by five-fold over the next two years, making it imperative for them to drive efficiencies internally,” said David Wadhwani, president, Digital Media Business at Adobe. “This new enterprise offering empowers users of any skill level to instantly turn ideas into content with Firefly, while tapping into the power of Express and Creative Cloud to quickly modify assets and deliver standout designs.” Today’s announcement comes only two weeks after Adobe also integrated Firefly into Photoshop (where it has now been used more than 150 million times). Like all of the major tech companies, Adobe is moving quickly to integrate these new capabilities across its product portfolio. The major advantage that Adobe has been banking on since the launch of Firefly is that it produces commercially safe images. It’s training the model on images from its stock imagery marketplace (in addition to openly licensed images and public domain content), which means it has the rights to all of these images and doesn’t have to scrape the web to gather them, with all of the copyright issues that entails. In return, though, the model is a bit more limited in the kind of images it can produce. In the enterprise context, though, being commercially safe will likely trump flexibility. Adobe is willing to bet on this and will indemnify businesses that use Firefly-generated images. With this move, Firefly is now available in the standalone Firefly web app, Adobe Express and Creative Cloud. In addition to these Firefly announcements, Adobe also today launched a number of other generative AI-powered services as part of its Sensei GenAI platform. While Firefly focuses on images, Sensei GenAI is Adobe’s branding for text and data-centric models that leverage multiple large language models, including OpenAI through Microsoft Azure and the Google-incubated FLAN-T5 model. Maybe the most interesting of these use cases here is that Adobe Experience Manager and Adobe Journey Optimized now get a generative AI-based marketing copy generator (currently in beta). This will allow brands to edit, rephrase and summarize their marketing copy by selecting their preferred tone of voice, for example. Marketing copy — and SEO content generation — are one of the lowest-hanging fruits for generative text generators (maybe because readers’ expectations here are already low?). One interesting twist here is that brands can tune the model with their own data to ensure that the automatically generated content remains on-brand. Users of Customer Journey Analytics will now also be able to use natural language queries to analyze their data and the service can now automatically caption charts and graphs. Meanwhile, a new chat tool will provide brands with “an automated way to interact with prospects who engage online, addressing questions on products while assisting sales teams with custom responses and summarized interactions” that can be used inside of Marketo Engage. Adobe says it’s already working with “hundreds of brands,” including Mattel, IBM and Dentsu, to help them adopt these AI-powered tools.",
            "@cite_2": "Adobe Brings Its Generative AI Tool Firefly To Businesses. Firefly saw the “highest number of users” engage in any of Adobe’s beta products. Now, the company plans to introduce its newest AI capabilities to enterprise customers. Users have created about 200 million AI-generated images using Adobe’s in-house text-to-image model Firefly since it launched in March. After seeing broad consumer adoption, the creative design software giant announced Thursday that it plans to launch the features for its 12,000 enterprise customers. Firefly is trained on more than 100 million images including Adobe’s stock images, licensed images and public images whose copyrights have expired. The company relies on its treasure trove of high-quality stock images sourced from contributors who typically get 33% of royalties when their images are sold or used. Adobe Stock consists of more than 330 million assets ranging from photos and illustrations to videos and music tracks. A recently added category of assets is AI-generated images submitted by contributors, which are accepted only if they have the rights to use them. Multiple Adobe Stock contributors have expressed concerns over the use of AI-generated images to train their consumer-facing AI tool, Firefly, and that contributors can’t opt out of their work being used to train and create tools like Firefly. Fu confirmed that they are not able to opt out because contributors have signed licensing agreements stating that their images may be used for AI training purposes. But the company says it plans to compensate them in the future when Firefly comes out of beta (Fu declined to say how much it plans to pay the company’s contributors). As some text-to-image AI models have faced legal ramifications for using copyrighted content to train their systems, enterprises want a guarantee that they will not get sued or face backlash for using generative AI tools to create images or content, Meredith Cooper, senior director of digital media business at Adobe, told Forbes in an interview. “Firefly is designed to be commercially safe and backed by Adobe via indemnification,” Cooper says. Adobe, which counts Coca Cola, Walgreens, Home Depot, General Motors and U.S. Bank among its enterprise customers, recorded $17.6 billion in revenue in 2022. Its generative AI tools (still in beta) were rolled out to its Creative Cloud users through photo editing software Adobe Photoshop in late May. One user used Adobe’s Generative Fill tool to edit popular memes by expanding the images and showing what could have been beyond the original frame of the meme. There is also potential for misuse. In response to people using its tools to create deepfakes and spread misinformation, Adobe in 2019 launched the Content Authenticity Initiative to appropriately label images or content produced by AI and tell if an image has been tampered with using AI. Any content created with Firefly comes with content credentials including metadata about the creation of the image such as name, data and any edits made to the image. The company also announced in early May that it also plans to bring its prompt-based text-to-image tools to Google’s conversational chatbot Bard, where users will be able to create synthetic images within Bard using Firefly. In addition to offering AI tools to create text, photo or video marketing copy for the web, enterprises will also be able to access language models from Microsoft Azure OpenAI service, Google’s language model Flan-T5 and others. Through its enterprise-focused product, Adobe Sensei, enterprise users can automate tasks like analyzing customer information, querying data and adjusting advertising budgets. Adobe did not disclose pricing for any of its enterprise generative AI services nor when it plans to launch them.",
            "@cite_3": "Adobe is adding AI image generator Firefly to Photoshop. Adobe Photoshop is getting a new generative AI tool that allows users to quickly extend images and add or remove objects using text prompts. The feature is called Generative Fill, and is one of the first Creative Cloud applications to use Adobe’s AI image generator Firefly, which was released as a web-only beta in March. Generative Fill is launching today in beta, but Adobe says it will see a full release in Photoshop later this year. As a regular Photoshop tool, Generative Fill works within individual layers in a Photoshop image file. If you use it to expand the borders of an image (also known as outpainting) or generate new objects, it’ll provide you with three options to choose from. When used for outpainting, users can leave the prompt blank and the system will try to expand the image on its own, but it works better if you give it some direction. Think of it as similar to Photoshop’s existing Content-Aware Fill feature, but offering more control to the user. I haven’t been able to try Generative Fill myself, but did get a live demonstration. It’s simultaneously impressive and far from perfect. Some of the objects it generated like cars and puddles didn’t look like they were a natural part of the image, but I was surprised how well it handled backgrounds and filling in blank spaces. In some examples, it even managed to carry over features from the photograph being edited, such as mimicking light sources and ‘reflecting’ existing parts of an image in generated water. Such feats won’t be a huge surprise for creators familiar with AI image generation tools, but, as ever, it’s the integration of this technology into mainstream apps like Photoshop that bring them to a much wider audience. Apart from the functionality, another important element of Firefly is its training data. Adobe claims that the model is only trained on content that the company has the right to use — such as Adobe Stock images, openly licensed content, and content without any copyright restrictions. In theory, this means anything created using Generative Fill feature should be safe for commercial use compared to AI models that are less transparent about their training data. This will likely be a consolation to creatives and agencies who have been wary about using AI tools for fear of potential legal repercussions. Generative Fill also supports Content Credentials, a “nutrition label” system that attaches attribution data to images before sharing them online, informing viewers if the content was created or edited using AI. You can check the Content Credentials of an image by inspecting it via verify.contentauthenticity.org, where you’ll find an overview of information. By integrating Firefly directly into workflows as a creative co-pilot, Adobe is accelerating ideation, exploration and production for all of our customers,” said Ashley Still, senior vice president, Digital Media at Adobe. “Generative Fill combines the speed and ease of generative AI with the power and precision of Photoshop, empowering customers to bring their visions to life at the speed of their imaginations. Generative Fill isn’t available in the full release of Photoshop just yet, but you can try it out today by downloading the desktop beta app or as a module within the Firefly beta app. Adobe says we can expect to see a full release onto the public Photoshop app in “the second half of 2023.” Adobe has been injecting AI-powered tools into its products for some time now. At Adobe Max last year the company rolled out some new Photoshop features like higher-quality object selections that are powered by Sensei, another of Adobe’s AI models. Firefly is already being used in Adobe Illustrator to recolor vector-based images, and Adobe also said that it plans to integrate Firefly with Adobe Express, a cloud-based design platform rivaling services like Canva, though there’s still no confirmation on when that will be released.",
            "@cite_4": "What Are the Limitations of AI Image Generation in Adobe Firefly? There’s an abundance of artificial intelligence image generation software that lets you create almost anything. But what are the limitations you might stumble upon? If you’re interested in using Adobe Firefly, in particular, you’ll be surprised to know there are a handful of limitations to using this generative AI software. Find out what Adobe Firefly is and what the limitations of using it are. What Is Adobe Firefly? Adobe Firefly is Adobe’s generative AI software tool. Adobe announced Firefly in March 2023 as a beta model tool with plans to integrate its features across various Adobe Creative Cloud software. Since it’s in beta mode, many things might change before the features are rolled out publicly. With Firefly, you can recolor vector graphics, which is a big deal for designers, generate a variety of 3D text designs, and use text-to-image prompts to generate images. Adobe also released an AI tool called Generative Fill, which removes and replaces parts of an image. There are plans for new features to arrive. These include a 3D model-to-image generation for use with Adobe Substance 3D software, and an extend-image software to easily change the ratio of images in one click. There are many more exciting features to be released, but for now, let’s look at the limitations of Adobe Firefly’s AI image generation technology. 1. You Can Only Create Still Images Menu MUO logo Sign In Now Close TRENDING SUBMENU PC & MOBILE SUBMENU INTERNET SUBMENU PRODUCTIVITY SUBMENU LIFESTYLE SUBMENU TECH EXPLAINED SUBMENU REVIEWS BUYING GUIDES MORE SUBMENU Sign In Newsletter What Are the Limitations of AI Image Generation in Adobe Firefly? 4 By Ruby Helyer Published May 27, 2023 Follow Share Link copied to clipboard Sign In To Your MUO Account Firefly-on-mac There’s an abundance of artificial intelligence image generation software that lets you create almost anything. But what are the limitations you might stumble upon? If you’re interested in using Adobe Firefly, in particular, you’ll be surprised to know there are a handful of limitations to using this generative AI software. Find out what Adobe Firefly is and what the limitations of using it are. What Is Adobe Firefly? Adobe Firefly is Adobe’s generative AI software tool. Adobe announced Firefly in March 2023 as a beta model tool with plans to integrate its features across various Adobe Creative Cloud software. Since it’s in beta mode, many things might change before the features are rolled out publicly. With Firefly, you can recolor vector graphics, which is a big deal for designers, generate a variety of 3D text designs, and use text-to-image prompts to generate images. Adobe also released an AI tool called Generative Fill, which removes and replaces parts of an image. There are plans for new features to arrive. These include a 3D model-to-image generation for use with Adobe Substance 3D software, and an extend-image software to easily change the ratio of images in one click. There are many more exciting features to be released, but for now, let’s look at the limitations of Adobe Firefly’s AI image generation technology. 1. You Can Only Create Still Images Adobe Firefly Text to Image AI prompt Although it is expected that Adobe Firefly features will be integrated into Adobe’s animation and video software—After Effects and Premier Pro—at the time of writing, you are limited to creating still images. While it’s not quite the same as Firefly, Adobe previously integrated Adobe Sensei AI features into most Adobe software. You can still benefit from Adobe’s AI tools for video or animation, although Sensei features are integrated within the software rather than being standalone tools. You can add fun textures to your text, recolor any vector graphics, or use the text-to-image generator. All results are static and there is no way to animate them. And none of the current Firefly features are specifically for video or animation projects. 2. Results Are for Personal Use Only Anything you create using Adobe Firefly is not available for commercial use while Firefly is still in beta mode. This means you cannot sell any designs you’ve made with Firefly or make money from them for the time being. You should use Adobe Firefly for personal projects and experimentation. While the software is in beta mode, it’s the best time to experiment and explore what you can do with it. When the software becomes widely available, you should be able to design for commercial use, and by then you’ll have experimented through the learning curve and can jump straight into the deep end. 3. You’re Limited to the Provided Artwork This point isn’t strictly true for the entire of Firefly’s offerings, but it is prominent to note for its AI text generation tool. There isn’t a way to upload your own art or designs to use as inspiration or to edit using Firefly technology. The 3D text effects feature only creates artwork from a text prompt. This is also the same for the text-to-image feature (although the clue is in the name for that one). While future features may allow you to use AI technology with your own images or creations, at the time of writing, most of the features only allow you to generate from scratch. However, all is not to be lost; the vector recoloring tool can be used with your own vectors, so long as they are saved in SVG format. It’s easy to save SVGs, they can be used for many purposes including using SVGs for editable elements in Canva. 4. The AI Is Trained With Adobe Stock Files If you’ve ever wondered if Adobe uses your content to train its AI, you can be rest assured that it does not. Adobe says it only uses existing files in Adobe Stock to train its Adobe Firefly AI systems. While this is great for copyright reasons and IP standards, it does potentially limit how the AI can be trained, since it only has access to Adobe Stock files for training. Adobe Stock is vast, but it can never be as extensive as the Internet of Things as a whole. You may find other generative AI systems are trained better due to broader access to training materials—even if they are sought unethically. Better training equals better results, but maybe Adobe will surprise us. 5. Only Adobe Users Get Invited to the Beta Many other AI tools allow anyone to access them, even if they need to sign up to do so. Most AI tools are free, although they may have a waiting list to join. Adobe Firefly is limited to Adobe subscribers—of any plan—who sign up for the Firefly beta group. Not only that, but you must actively request an invitation to the Firefly beta and painstakingly wait for the email invite from Adobe. If you’ve managed to get on the beta list, you won’t have any further limitations to accessing Firefly, other than the limitations we’ve listed here for its use. 6. Limited File Formats and Save Options As mentioned earlier, the vector recoloring feature can only use SVG files. While it’s not difficult to save a vector as an SVG, it does add an extra step to use the tool. The 3D text effects generation is limited in its save options for your results. You can copy it to your clipboard to open it directly in Photoshop or Illustrator, or you can save it. Text effect results are saved in PNG format with a transparent background and an Adobe watermark. Text-to-image results are saved as a JPG file. Quite the limited choice of file formats and not the one most digital artists would choose to use. Once integrated, we expect our save options to reflect those found within all the Creative Cloud programs. 7. Watermarked Images After generating an AI image in Adobe Firefly, you have options to copy your results to the clipboard—for pasting into Photoshop or Illustrator—or saving it. Both options are limiting because your results will have a garish red and purple Adobe Firefly watermark in the lefthand corner. The watermark comes with a label reminding you that your generated art is not for commercial use. This is the same for the majority of Firefly’s generated image results, and it looks like we’ll have to wait for the full roll-out of software before it offers watermark-free results. The vector recoloring tool does not watermark your vectors—and allows you to save in high-quality SVG format. This tool is fairly versatile without the limit of watermarking. 8. Your Results Are Uneditable Outside of Firefly Along with the watermarks, limited save formats, and the fact that none of Firefly’s tools are yet fully integrated into the Creative Cloud, it’s probably no wonder that the results of your efforts are also uneditable. Technically, you could edit them globally with adjustments or effects like warping or blur. But the downloaded Firefly images are flattened, so it’s not possible for more detailed or isolated editing. While we won’t know until the tools reach the Creative Cloud, we can imagine that this won’t be the case forever. Once integrated into the software, you’ll probably—hopefully—be able to edit in layers and move, remove, or add elements to your new AI-generated work as easily as with typical Adobe projects. Know Your Limits With Adobe Firefly Adobe has brought the future of digital design to your computer screen. It’s exciting to see the newest AI offerings, but you shouldn’t go into using them without understanding that just because AI seems limitless, doesn’t mean it is. Adobe hasn’t specifically stated whether these limits will be lifted in the future, but given Firefly is only in beta mode, you should expect that these limitations are only temporary.",
            "@cite_5": "Is Adobe Using Your Files to Train Its AI? How to Opt Out if So. Adobe is one of the biggest creative software companies. It provides leading technology to help you design your greatest work. With its cloud-based subscription model, you are always ahead of the curve and provided access to its latest tools and features. But could this be a negative? Is there a chance that Adobe is accessing your data to train its AI? Is Adobe Using Your Data to Train Its AI? Since Adobe became a largely cloud-based product in 2013, we’ve seen the benefits in the quick release of new design tools, the option to back up your work to the cloud, and the easy integration of saving and opening work across different Adobe products. But your files in the cloud might be used to train Adobe’s AI. Adobe Sensei is Adobe’s AI technology integrated into its software to help improve your workflow. How does it improve your workflow? It uses your data for content analysis to track common work patterns. Adobe’s content analysis FAQ page claims it may analyze your content to develop and improve its products and services. It isn’t explicit what this means or how it works, but if you’re precious about your work, it may be an issue. How to Opt Out of Content Analysis The good news is you can opt out of content analysis. Log in to your Adobe account and go to the Privacy and personal data page. Switch off the Content analysis toggle. You can also toggle off Desktop app usage to opt out of any tracking. Another way to ensure Adobe cannot access your work for any reason is by not using cloud storage via Adobe Creative Cloud and saving your work locally instead. Then your work will be safe from Adobe’s prying eyes. Adobe Creative Cloud works in clever ways to help keep your local storage unused and offers multiple programs via the cloud, but by analyzing your data and workflow for other purposes, it could also be to your detriment. Stay in Control of Your Files It’s sneaky of Adobe to not make it obvious that it wants to follow your workflow and track your data, but it’s easy to opt out. This is a small reminder to read the terms and conditions when signing up for products and to routinely check privacy information for cloud-based services like Adobe Creative Cloud. Of course, Adobe doesn’t make it obvious what it means by analyzing your content, so agreeing to let it use your information for product improvements is a personal choice.",
            "@cite_6": "Adobe’s Firefly Image Generator Is Going to Make Photoshop Much Easier to Use. Soon, even your grandparents could be Photoshop experts. Adobe’s Photoshop has grown to become an incredibly capable and powerful image editing tool, but its ever-increasing complexity also makes it harder for new users to learn how to use it. That could change later this year, however, as Photoshop will soon introduce some new AI-powered tools that allow users to perform complex edits by simply asking the app to do it for them. In March, Adobe revealed its AI image generation tool, called Firefly, which stood out from similar tools with a promise that it wouldn’t infringe upon the existing work of artists and photographers, as the AI was only trained on images from Adobe’s own stock image site, public domain content, and openly licensed work. It didn’t take long for Adobe to reveal its grander plans for Firefly, as just a few weeks later, the company revealed the tool would be incorporated into some of its video and image editing apps. Coming as a surprise to no one, today Adobe also revealed that Firefly would be incorporated into Adobe Photoshop, the company’s flagship image editing tool, through a new tool called Generative Fill. The tool can either expand the borders of an image by generating additional out-of-frame content that matches the lighting, style, and perspective of the original photo, or completely replace parts of an image based on simple text prompts from the user. One of the examples Adobe has provided that demonstrates the benefits of Generative Fill is changing the size or aspect ratio of an image. Often times, a photo is shared on several different platforms, including social media consumed on phones, or in a browser on a laptop. Cropping an image to a smaller size is easy enough, but expanding the borders of a photo to make it taller or wider often involves very complex editing by a skilled Photoshop artist. Generative Fill promises to fill in the missing areas of an expanded photo automatically in a matter of seconds. Generative Fill is useful beyond just recreating out-of-frame areas of a photo. Adobe also demonstrates how the tool can be used to intelligently replace or edit parts of a photo the user has highlighted using Photoshop’s selection tools using a simple text prompt. In this instance, the middle of the road was highlighted and Generative Fill was asked to add yellow lines to make it more obvious this cyclist was riding on an empty road, and not a random section of pavement. The added lines not only matched the perspective of the photo, but also the level of wear already on the road. As with many of Photoshop’s tools, the automated Generative Fill edits are non-destructive, and are added to a document as additional layers that can be turned on and off, or further manually tweaked by an artist using other filters. And in some cases, Generative Fill can also suggest several different versions of an edit, letting the artist have the final say in which one is actually used. Users with access to beta releases of the desktop version of Adobe Photoshop will have access to Generative Fill starting today, while wider availability of the tool in release versions of Photoshop is expected sometime in “the second half of 2023.”",
            "@cite_7": "Generative Fill in Photoshop (Beta) Hands-On. It takes something special to make jaded photographers exclaim in genuine surprise when editing photos. The exclamations were rampant after Adobe recently released the latest public beta of Adobe Photoshop with a new Generative Fill feature that can create photorealistic objects, backgrounds, and scene extensions in existing photos using generative AI technology. We’ve seen Adobe’s implementation of generative AI in Adobe Firefly, the web-based tool for creating entire synthetic images from text prompts. Generative Fill uses Firefly technology to edit existing images in a more targeted way, bringing generative AI to Photoshop as a standard feature soon. (A few third-party Photoshop plug-ins that tie into other generative AI systems have been available for a while, such as Alpaca and Stability AI.) How the image-making machine works A generative AI system like Firefly creates entirely original images based on what’s described in a text prompt. It doesn’t lift sections of existing images and throw them together to create a new composition. Instead, using what it has learned from ingesting millions of photos, the system invents scenes and objects to match what it understands the text to mean. A prompt such as 'Antique car on a rainy street at night' assembles an image from a random mass of pixels to match what the system understands is a 'car,' 'rain,' 'street,' and 'night.' The systems usually provide multiple variations on your theme. What if you already have a photo of a rainy street at night, and you want to add an antique car to the composition? Using Photoshop’s Generative Fill feature, you can select an area where you want the car to appear and type 'Antique car' to generate one (this is also known as 'inpainting'). Or you can select objects you want to remove from an image and use Generative Fill without a specific text prompt to let the tool determine how to fill the missing area. Adobe is making this process more user friendly than other generative AI systems. It has the home-field advantage of building the tool directly into Photoshop, where Generative Fill sports a clean and direct interface. In comparison, the popular service Midjourney requires that you join a Discord server, subscribe to the service, enter a chat room set up to receive text prompts, and then type what you want to generate using commands such as 'Imagine antique car on a rainy street at night.' Your results appear in a scrolling discussion along with images generated by others who are in the same chat room. Photoshop's approach debuts a new Contextual Task Bar with commands such as Select Subject or Remove Background. When you make a selection using any of the selection tools, such as the Lasso tool, one option in the bar is a Generative Fill button. Clicking that button reveals a text field where you can describe what should be created within the selection. Or, you can leave the field blank and click the Generate button to have Photoshop determine what will appear based on the context of the surrounding scene. Once you click 'Generate,' Photoshop produces three Firefly-generated variations and shows you the first. You can cycle through them using buttons in the Contextual Task Bar or by clicking the thumbnails in the Properties panel. If none of them look good, you can click 'Generate' again to get three more variations. (By the way, if you get frustrated with the Contextual Task Bar appearing directly below every selection, you can drag it to where you want, then click the three-dot icon on the bar, and choose Pin Bar Position from the menu.) All of the variations are contained in a new type of layer, the Generative Layer, that also includes a mask for the area you selected. If you apply Generative Fill to another area of the image, a new Generative Layer is created. All the variations are saved in those layers, so you can go back and try variations nondestructively, hide or show the layers, set the blend mode and opacity, and use all the other flexible attributes of layers. Also note that Generative Fill is creating results at the same resolution as the original photos. This is in contrast to most systems, including Firefly on the web, where the generated images are low resolution, typically 1024 by 1024 pixels. Now let’s look at what Generative Fill can do. Remove objects Usually when you use tools to remove unwanted items from a photo, the software attempts to backfill the missing area using pixels from elsewhere in the image (see Remove All the Things: Using modern software to erase pesky objects). That becomes more difficult when the area to be removed is large, leading to repeating artifacts that make it obvious something has been removed. Generative Fill instead looks at the context of the image and attempts to create what would make sense in its place. In the examples above where we removed the tourists, Photoshop recreated the lines and colors of the buildings and matched the texture of the ground. But you can’t assume that the feature will get it right every time. Take the image of two people below. We can attempt the removal of one person (the man on the left) by making a loose selection around him with the Lasso tool to define the area we want to replace, and then clicking Generate with nothing in the text box. Strangely, in multiple attempts the tool assumed that we wanted to replace the person with another, random, person. And these were nightmare-inducing renditions of synthetic people. According to Adobe, there is no need to type a command such as 'Remove person' as a prompt when using Generative Fill, but in the end, typing that prompt gave us the result we wanted. Note, though, that while Photoshop returned one variation without a person (see below), it also created two variations that still included people. We can perhaps chalk this up to the feature being pre-release, although more likely it reveals that despite the amount of machine learning, the software is still just making guesses. Replace items and areas Removing objects is one thing, but what about replacing them with something entirely different? With Generative Fill, you can create things that were never in the original image. For instance, in the following photo we can turn one of the desserts into a blueberry tart by making a selection around the raspberries (expanding the selection slightly helps to pick up the paper texture behind them) and typing 'Blueberries' in the text prompt field. It took a couple of iterations to find one that matched, but these blueberries look pretty convincing. Or what about the drinks in the background? Instead of cold brew coffee, we can select the glass on the left and enter 'Pint of beer' as the prompt. Notice that not only is the glass slightly out of focus to match the depth of field of what it replaced, it also includes a hint of a reflection of the raspberry tart in front of it and the coffee to the side. Adding arbitrary items to an empty space can be more hit or miss. You'll get better results if the selection you make is roughly the correct size and shape of the item in context. In this case, we drew a rectangular selection in the foreground and typed the prompt 'Dog lying down,' whereupon Photoshop created several pup variations, of which we liked the one below best. The angle of the light and shadow matches fairly well. In addition to replacing or adding foreground objects, by using Select Subject and inverting the selection to select the background, we can enter a prompt to redefine the whole context of the scene. Generative Fill can also be used in more surgical ways, such as changing someone’s clothing. This, too, can have unpredictable results depending on what you’re looking to create. However, in some cases the rendered image looks fine. Keep in mind that it can take multiple prompt requests and revisions to get what you want. and people with true fashion sense are likely to quibble with Photoshop’s choices. You can see some different resulting outfits below."                            
        }
    },
    {
        "abstract": "Write the 'History' section of the article titled 'Adobe Firefly'.",
        "related_work": "Adobe Firefly was first announced in September 2022 at Adobe's MAX conference. It was initially released as a public beta in March 2023 @cite_9, and is currently available to all Adobe Creative Cloud subscribers. Adobe Firefly is built on top of Adobe Sensei, the company's AI platform. Sensei has been used to power a variety of features in Adobe's creative software, such as object selection in Photoshop and image auto-enhancement in Lightroom. Firefly expanded its capabilities to Illustrator, Premiere Pro, and Express, particularly for generating photos, videos and audio to enhance or alter specific parts of the media. NVIDIA Picasso runs some Adobe Firefly models @cite_10.",
        "ref_abstract": 
        {
            "@cite_9": "Adobe opens up its Firefly generative AI model to businesses. Firefly for Enterprise is a new offering that allows businesses to custom-train Adobe’s generative AI with their own branded assets. Adobe has unveiled a new platform for its Firefly generative AI model that’s designed to help organizations address the growing demand for content creation across their workplace. Announced during today’s Adobe Summit event, Adobe Firefly for Enterprise allows every employee within a company — regardless of their creative skills — to instantly generate images or copy from text-based descriptions, which can then be used in marketing campaigns, social media promotions, corporate presentations, and more. Enterprise users will be able to access Firefly through the standalone Firefly application, Creative Cloud, or Adobe Express — Adobe’s cloud-based design platform. Businesses can also build Firefly into their own ecosystem by training the AI model with their own branded company assets, which will allow Firefly to replicate the brand’s style when generating images and copy. “Enterprise leaders expect content demands will increase by five-fold over the next two years, making it imperative for them to drive efficiencies internally,” said David Wadhwani, president of digital media business at Adobe. “This new enterprise offering empowers users of any skill level to instantly turn ideas into content with Firefly, while tapping into the power of Express and Creative Cloud to quickly modify assets and deliver standout designs.” Adobe doesn’t have exact pricing to share for Firefly for Enterprise yet, but Ashley Still, senior vice president of digital media at Adobe, confirmed to The Verge that licenses that can be deployed broadly to employees will be available to brands for a flat price, which will be based on the needs and size of the organization. There is also no confirmed release date for Firefly for Enterprise — only that it will launch sometime after Firefly comes out of beta. This new enterprise-level product isn’t an unexpected move from Adobe, especially if you’re already familiar with its Firefly AI model. Adobe created Firefly to be safe for commercial use by training it on Adobe Stock images, openly licensed content, and content without copyright restrictions within the public domain. That sets it apart from many other generative AI models, such as OpenAI’s Dall-E, which could cause copyright issues for organizations as they haven’t disclosed their training data. Aside from its assured commercial viability, Firefly’s explosive popularity — largely fueled by its high-quality results — will likely hold plenty of appeal for businesses looking to explore generative AI solutions. Firefly beta users have generated over 200 million images since it launched in March 2023, and over 150 million images have been generated in just two weeks using Photoshop’s new Firefly-powered Generative Fill feature. The company also recently launched an Enterprise tier for its Adobe Express product that’s designed to support collaboration across organizations.",
            "@cite_10": "Adobe and NVIDIA Partner to Unlock the Power of Generative AI. Adobe and NVIDIA will co-develop a new generation of advanced generative AI models Partnership will focus on deep integration of generative AI in creative workflows Both companies commit to content transparency and Content Credentials powered by Adobe’s Content Authenticity Initiative GTC—Today, Adobe (Nasdaq:ADBE) and NVIDIA, longstanding R&D partners, announced a new partnership to unlock the power of generative AI to further advance creative workflows. Adobe and NVIDIA will co-develop a new generation of advanced generative AI models with a focus on deep integration into applications the world’s leading creators and marketers use. Some of these models will be jointly developed and brought to market through Adobe’s Creative Cloud flagship products like Adobe Photoshop, Adobe Premiere Pro, and Adobe After Effects, as well as through the new NVIDIA Picasso cloud service for broad reach to third-party developers. Priorities of the partnership include supporting commercial viability of the new technology and ensuring content transparency and Content Credentials powered by Adobe’s Content Authenticity Initiative. Part of the NVIDIA AI Foundations cloud services for generative AI announced today, NVIDIA Picasso lets users build and deploy generative AI-powered image, video, and 3D applications with advanced text-to-image, text-to-video, and text-to-3D capabilities to supercharge productivity for creativity, design, and digital simulation through simple cloud APIs. “Adobe and NVIDIA have a long history of working closely together to advance the technology of creativity and marketing,” said Scott Belsky, Chief Strategy Officer and EVP, Design and Emerging Products, Adobe. “We’re thrilled to partner with them on ways that generative AI can give our customers more creative options, speed their work, and help scale content production.” “Generative AI provides powerful new tools to empower unprecedented creativity,” said Greg Estes, VP, Corporate Marketing and Developer Programs, NVIDIA. “With NVIDIA Picasso and Adobe tools like Creative Cloud, we’ll be able to bring the transformational capabilities of generative AI to enterprises to help them explore more ideas to efficiently produce and scale incredible creative content and digital experiences.” Adobe Firefly Earlier today, Adobe introduced Adobe Firefly, Adobe’s new family of creative generative AI models, and unveiled the beta of its first model focused on the generation of images and text effects designed to be safe for commercial use. Firefly will bring even more precision, power, speed, and ease directly into Adobe Creative Cloud, Adobe Document Cloud, and Adobe Experience Cloud workflows that involve the creation and modification of content. Hosting some of Adobe Firefly’s models on NVIDIA Picasso will optimize performance and generate high-quality assets to meet customers’ expectations. (For more information on Firefly, including how it is trained and how it honors the role of creators, please see this blog post.) Adobe is also developing new generative AI services to assist in the creation of video and 3D assets and to help marketers scale and personalize content for digital experiences through advancing end-to-end marketing workflows. Content Authenticity Initiative Adobe founded the Content Authenticity Initiative (CAI) to develop open industry standards for establishing attribution and Content Credentials. Through Content Credentials that CAI adds to content at the point of capture, creation, edit, or generation, people will have a way to see when content was generated or modified using generative AI. Adobe and NVIDIA, along with 900 other members of the CAI, support Content Credentials so people can make informed decisions about the content they encounter."                            
        }
    }
]