Preview Technology Gives Your Voice the Power of a Creative Director

Preview Technology Gives Your Voice the Power of a Creative Director

Creative professionals can spend a lot of time organizing visual assets or trying to find the right photo for a campaign. But what if finding the perfect image were as easy as simply asking for it out loud?

Preview technology from Adobe — called #PhonicFilters — promises to make searching image libraries as easy as talking into your mic. It’s like having your own personal curatorial librarian. This technology and others were presented at Adobe Summit as part of our Sneaks session, where our incredible Adobe engineers take the stage to share short demos of potential future tech at Adobe.

“Most people just search for images using keywords in a text-based query,” says Brett Butterfield, director of software development for Sensei agents at Adobe. “We actually have some amazing search capabilities built into products like Adobe Stock and Lightroom that allow you to search by categories like depth of field, or placement of subject in a frame, but you have to know how to navigate all the menus to use them. We wanted to see if we could make those capabilities accessible with an interface that was more like a natural conversation.”

During his Sneak presentation of #PhonicFilters, Brett showed how a natural language interface can be paired with Adobe Sensei — our artificial intelligence (AI) and machine-learning technology — to categorize or find images in powerful new ways.

“Show me images organized by category,” Brett requested during the demo, and instantly his entire photo library was grouped into browsable folders labeled by categories like art, business, nature, politics, sports, and more. It’s possible because Adobe Sensei analyzes each photo as it is added to the library, automatically generating a diverse set of metadata and set of associations for each photo.

Brett onstage at Adobe Summit, presenting the sneak technology #PhonicFilters.

“It’s not just keywords about the subject of the photos like dog or sunset. We can leverage machine learning to uncover broader concepts like color palette, depth of field, color vibrancy, and even broader concepts like authenticity,” Brett said.

The result is a powerful new way to explore and riff off of hundreds of millions of images in a service like Adobe Stock. With three simple voice commands and a click, he was able to search through more than 60 million images to find several that he liked. “Find pictures of wine,” he said. “Add people toasting,” he said, and then clicked on a promising photo. “Find similar images,” he added, and was presented with dozens of on-target options.

“With just about a month of initial development, we were able to create something pretty magical,” Brett said. How does it work? The Sneak uses Rasa, an open source chatbot, on the back end that interprets natural voice commands and generates JSON to add to, or delete triggers from, the search. “Then, we’re able to use AI and Machine learning on the back end to understand intent and apply those filters automatically,” he said.

The result is a natural voice interface that allows the user to start off with a broad, base query, and then build upon it or narrow the scope. “The VP in charge of Adobe Sensei liked the demo so much that he changed my job,” Brett said with a laugh. “I was always focused on finding innovative uses for Sensei, but now I get to be much more focused on finding solutions for voice-controlled applications and creative agents.”

See a short demo of #PhonicFilters in action.

 

Brett is a natural fit for the job. He knows the pain of trying to find the proverbial needle in a haystack of images, as well as the joy that comes from finding the perfect image, because he’s been working with visual content his entire life. He started his career as a designer, worked as an engineer and A/V evangelist for Silicon Graphics, and later as an imaging architect for Ofoto. He was also the director of R&D for Kodak Gallery and founded his own company — Pixelpipe — focused on content aggregation and distribution across social networks before joining Adobe to work on search technology.

“I’m really excited to see where we can take these sorts of voice-controlled applications next,” Brett said. “There are all sorts of possibilities for VR and keyboard-free computing, as well as mobile devices. While we’re starting with Adobe Stock, it could be useful for personal photos and other applications as well. Ultimately, we want to make everyone more like a powerful creative director. If you want an image of a bear, you just ask for it. Describe what you see in your mind’s eye and a virtual assistant is ready to go find it for you.”

Recommended Articles