2023年3月10日

如果我們跳過靜態幀的概念，而是直接分析來自相機的數據流，計算機視覺可能會更快更好。至少，這是麻省理工學院媒體實驗室的最新創意——Ubicept——正在運行的理論。大多數計算機視覺應用程序的工作方式都是一樣的:相機拍攝一張圖像(在視頻的情況下，是一係列快速的圖像)。這些靜止幀被傳遞給計算機，然後計算機進行分析，找出圖像中的內容。聽起來很簡單。但有一個問題:這種模式假設創建靜態框架是一個好主意。對於習慣看照片和視頻的人來說，這似乎是合理的。然而，計算機並不在乎，Ubicept相信它可以讓計算機視覺更好、更可靠，因為它忽略了幀的概念。這家公司本身就是聯合創始人的合作。塞巴斯蒂安·鮑爾(Sebastian Bauer)是該公司的首席執行官，也是威斯康星大學的博士後，他在那裏從事激光雷達係統的研究。 Tristan Swedish is now Ubicept’s CTO. Before that, he was a research assistant and a master’s and Ph.D. student at the MIT Media Lab for eight years. “There are 45 billion cameras in the world, and most of them are creating images and video that aren’t really being looked at by a human,” Bauer explained. “These cameras are mostly for perception, for systems to make decisions based on that perception. Think about autonomous driving, for example, as a system where it is about pedestrian recognition. There are all these studies coming out that show that pedestrian detection works great in bright daylight but particularly badly in low light. Other examples are cameras for industrial sorting, inspection and quality assurance. All these cameras are being used for automated decision-making. In sufficiently lit rooms or in daylight, they work well. But in low light, especially in connection with fast motion, problems come up.” The company’s solution is to bypass the “still frame” as the source of truth for computer vision and instead measure the individual photons that hit an imaging sensor directly. That can be done with a single-photon avalanche diode array (or SPAD array, among friends). This raw stream of data can then be fed into a field-programmable gate array (FPGA, a type of super-specialized processor) and further analyzed by computer vision algorithms. The newly founded company demonstrated its tech at CES in Las Vegas in January, and it has some pretty bold plans for the future of computer vision. “Our vision is to have technology on at least 10% of cameras in the next five years, and in at least 50% of cameras in the next 10 years,” Bauer projected. “When you detect each individual photon with a very high time resolution, you’re doing the best that nature allows you to do. And you see the benefits, like the high-quality videos on our webpage, which are just blowing everything else out of the water.” TechCrunch saw the technology in action at a recent demonstration in Boston and wanted to explore how the tech works and what the implications are for computer vision and AI applications. A new form of seeing Digital cameras generally work by grabbing a single-frame exposure by “counting” the number of photons that hit each of the sensor pixels over a certain period of time. At the end of the time period, all of those photons are multiplied together, and you have a still photograph. If nothing in the image moves, that works great, but the “if nothing moves” thing is a pretty big caveat, especially when it comes to computer vision. It turns out that when you are trying to use cameras to make decisions, everything moves all the time. Of course, with the raw data, the company is still able to combine the stream of photons into frames, which creates beautifully crisp video without motion blur. Perhaps more excitingly, dispensing with the idea of frames means that the Ubicept team was able to take the raw data and analyze it directly. Here’s a sample video of the dramatic difference that can make in practice: