As we’ve delved into deeply here at Fast Company, Facebook is very much into being open, particularly with regard to its artificial intelligence research.
So it should come as no surprise that today it open-sourced its latest computer vision technologies—three systems designed to automatically figure out what the objects are in an image and then accurately delineate their boundaries.
“At Facebook, we are working on this kind of research because people are creating and sharing more visual content than ever before,” the company wrote on a site dedicated to one of the three technologies, SharpMask. “Enabling computers to recognize objects in photos will improve the way people navigate and engage with this content. For example, it will be easier to search for specific images even without explicit tags on each photo. People with vision loss, too, will be able to understand what is in a photo because the system will be able to tell them, regardless of the caption posted alongside the image. This technology could enable even more immersive experiences that allow people to ‘see’ a photo by moving their finger across an image while the system describes the content they are touching.”
Here is a little bit more information on each technology:
• DeepMask: segmentation framework
• SharpMask: segment refinement module; paired with DeepMask, they enable Facebook’s machine vision systems to detect and precisely delineate every object in an image
• MultiPathNet: specialized convolutional net that labels each object mask with the object type it contains