Scientists have developed a system that creates videos of the future. The computer generated footage could be used to fill in gaps in security footage and animate still images.
The development comes from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) where a research team has developed a deep-learning algorithm that, given a still image from a scene, can create a video that simulates the future of the scene.
Prior projects have developed systems that can create future frames in a scene, while this method generates completely new videos.
It uses an algorithm draws on 2 million videos to generate the footage, deemed to be realistic 20% more often than a baseline model when shown 130,000 times to 150 users.
Potential beneficiaries of the development are self-driving cars and security applications. A paper on the research - written by CSAIL PhD student Carl Vondrick, MIT professor Antonio Torralba and Hamed Pirsiavash, a former CSAIL postdoc who is now a professor at the University of Maryland Baltimore County (UMBC) will be presented at next week’s Neural Information Processing Systems (NIPS) conference in Barcelona.
Vondrick said: “These videos show us what computers think can happen in a scene. If you can predict the future, you must have understood something about the present.”
There are some areas that require further development, with Vondrick noting that the model lacks some common-sense principles as well as making humans and objects appear larger in size than in reality.
Furthermore the videos are only 1.5 seconds long but improvements are in the pipeline. “It’s difficult to aggregate accurate information across long time periods in videos,” said Vondrick. “If the video has both cooking and eating activities, you have to be able to link those two together to make sense of the scene.”
This work was supported by the National Science Foundation, the START program at UMBC, and a Google PhD fellowship.
Read the paper