Startup SE3 Labs develops spatial AI
How autonomous systems learn to understand their environment
It all began at a climbing gym. That’s where Daniel Cremers, professor for Computer Vision and Artificial Intelligence at the Technical University of Munich (TUM), met his future co-founder, Lukas Köstler. Shortly thereafter, Lukas Köstler began conducting research as a doctoral student in Professor Cremers’ department; Simon Klenk also joined the team there. As a result of this collaboration, the three decided to continue their research in their own company. With SE3 Labs, the team aims to give machines a spatial understanding of their environment so that they can act independently within it - similar to how generative AI handles language. Their startup receives support from UnternehmerTUM and TUM Venture Labs.
From visual data to spatial understanding
“Our technology is designed not only to recognize what is visible in an image or video, but also to truly understand the three-dimensional world - that is, how objects, buildings, streets, or terrain features relate to one another spatially,” says Daniel Cremers. Autonomous systems such as drones or robots can use this spatial understanding to orient themselves and act independently in their environment. In addition, the digital representation of the environment can be analyzed and queried by users.
Our technology is designed not only to recognize what is visible in an image or video, but also to truly understand the three-dimensional world.
Entrepreneurship
Entrepreneurship is part of our DNA. Discover the founding ecosystem of TUM, UnternehmerTUM, and the TUM Venture Labs with a wide range of entrepreneurship offerings – from startup consulting to entrepreneurship education.
The software processes sensor data in real time to create a continuously updated, three-dimensional model of the environment. Terrain profiles, elevation differences, and relevant objects are captured and localized with high accuracy. This combines methods from large language models with computer vision - that is, techniques that enable machines to recognize and understand visual information in a manner similar to humans.
Users can ask questions about this spatial data, just as they would with a chatbot. In urban planning, for example, the tool can help determine how many solar panels could fit on existing rooftops or how many parking spaces are available in a specific neighborhood.
Focus on deployment in crisis and war zones
“Our technology is particularly relevant, however, in situations where up-to-date information about the surroundings is crucial, such as in crisis and war zones. Autonomous systems like drones often cannot be deployed in these cases because they rely on GPS signals, which are frequently unavailable in these areas,” says Lukas Köstler.
Our technology is particularly relevant, however, in situations where up-to-date information about the surroundings is crucial, such as in crisis and war zones.
This is exactly where the SE3 Labs team comes in, as SE3’s navigation system enables precise autonomous navigation even in environments without a GPS signal. The system continuously creates its own spatial map of the surroundings, allowing it to account for changes in the terrain or operational area in real time.
Using the same 3D map, emergency responders can then ask specific questions: How many of certain objects are there in an area? Which roads are passable? How has the environment changed since the last survey? The technology is designed to help analyze large and complex areas more quickly and accurately. In the long term, SE3 Labs sees potential applications in many fields: from defense and security to smart infrastructure and construction, all the way to industrial applications.
Artificial Intelligence
Artificial intelligence is shaping our working lives, research, and the world around us. Learn how we are contributing to this progress by developing innovative AI methods and applications, ranging from robotics to machine learning.
The innovation ecosystem centered around TUM is regarded as one of the most successful deeptech hubs in Europe. Its particular strengths are its strong, diverse network and the specific support. In initiatives and co-labs, start-ups work on innovations with established companies, experts, investors and administration.
TUM and UnternehmerTUM, the Center for Innovation and Business Creation, support start-up teams with programs that are precisely tailored to the individual phases of the founding process and the teams. The TUM Venture Labs offer direct access to cutting-edge research, technical infrastructure and market expertise in twelve fields of technology. Each year, more than 100 companies were founded at TUM and more than 1,000 start-up teams were supported by UnternehmerTUM and the Venture Labs. UnternehmerTUM, which invests with its own venture capital fund, has been voted Europe's best start-up hub three times by the Financial Times.
Technical University of Munich
Corporate Communications Center
- Julia Rinner
- presse@tum.de
- Teamwebsite
Contacts to this article:
Prof. Daniel Cremers
Technical University of Munich
Professor for Computer Vision and Artificial Intelligence
cremers@tum.de
