wildflow

Just wrapped up my 7th conference of the year at ReefFutures! 🪸 So much joy, inspiration, brilliant and kind people working on protecting and restoring our oceans. Grateful for every selfie, every conversation, and every shared vision! 2024 has been a wild ride, and the best is yet to come! 🌎 🥳 #ReefFutures2024

3D Coral Reef + At Night + Fluorescence -- could you help me with pics? 📸🪸  🔵 We create photorealistic 3D models of coral reefs that work in the browser using 3D Gaussian Splatting (https://lnkd.in/ebjkWWCc). 🔵 Coral reefs look wild at night! Would be so much fun to create a 3D model of coral reef using fluorescence! 🔵 Could anyone help me with that in the next few weeks? We just need multiple pictures from different angles. 🔵 This truly incredible shot is by Adriana Basques, check out her other photos!

Let's gooo!! 🌎 Try our 3D "Street View for Coral Reefs" prototype 2.0! 🪸 We did it! Today we've hit a new milestone! It's been a long journey, but we're one step closer.. We have a 50x times larger 3D Gaussian Splatting model now, covering over 1000 sq meters of corals. So stoked to tell you \o/ 🔵 Why? Coral Reefs are so precious, beautiful, incredibly complex and threatened ecosystems. They are dying fast.. But there's a way to protect and restore them! A lot of amazing people fighting for their life. We need to understand deeply how Coral Reefs function, what methods work, what don't, to coordinate precise action. They are a key to protect and restore other ecosystems. That's why we're building a digital twin of coral reef ecosystems -- our first product. 🔵 https://lnkd.in/esYyuupw -- today you can try our second scrappy prototype (20% done) -- works best from computer. 🔵 What you see: it's a 3D "Street View" for Coral Reefs. Someone swam around a reef with a few GoPros, and you can now see high-res photorealistic 3D model of coral reef in the browser! It's a super cheap way of monitoring. Soon you will fly over square kilometres of reefs and see them in centimetre resolution. You can compare reefs in different countries and see how they evolve over time, easy way of getting rid of biases in data. 🔵 The prototype is terrible. Still work in progress (20%), just another sneak peek. We're moving fast and testing one hypothesis after another. Right now we still don't have a smooth progressive loading, there are some wacky splats here and there. 🔵 But it's already much better. This time we used 8382 images from GoPro 10 (instead of 70) and trained over 25x50m model (last time it was about 5x5 m). We respect correct geometry from Metashape (classical photogrammetry tools). Centimetre resolution! I think it's still the best in the world quality you can find. 🔵 Next steps: 3D time-series data (see how coral used to look like 3 months ago), 3D segmentation models, classifying coral species, adding other modalities (acoustics, eDNA, geospatial)... Allowing anyone to run analytics against all our data... Foundation models for biodiversity... Becoming the first generation that actually leaves behind nature better than we found it! 🪸 🔵 This version of a prototype wouldn't happen without these incredible people: - Ben Williams -- it's all Ben's fault I'm doing this now, his idea and support - Jason Lynch -- Metashape/photogrammetry support - Greg Tkachenko -- powerful compute cluster - Rindah Talitha Vida, Tim Lamont, Mars BuildingCoral team -- collecting the data, doing scientific research and restoring this reef you see - and many many more fantastic people! We've got terabytes of data from multiple orgs, now scaling the compute and processes. Soon you can play with more and more coral reefs! What a time to be alive! \o/

Coral Reef Tetris! 🟧 🟩 🟦 🟪 Would you play this game? 🪸🪸🪸 Pivoting wildflow into a gaming company now haha! 🔵 Coral reefs are incredible ecosystems! While being the size of New Zealand they are supporting 25% of marine life and at least half a billion people! Lot's of amazing people fighting to protect their lives. 🔵 We've got terabytes of 3D coral reef data (images from GoPros) from a bunch of different locations in the world. And building 3D "Street View" for coral reefs, so that anyone could see them and scientists could run analytics against all that data. 🔵 Current challenge is how do you even visualise it? We use new 3D Gaussian Splatting techniques. Last weekend we trained 50x25 meters photorealistic model of coral reefs in centimetre resolution. Respecting all the correct geometry from photogrammetry. Now we need to visualise it in the browser. Splats are great, they are 10-30 times smaller than point clouds or meshes. But it's 36GB of 3D Gaussian Splats still! 🔵 What you see here is a debug mode of me implementing a custom 3D visualisation engine, based on Three.js + WebGL + mkkellogg/GaussianSplats3D. Effectively we need progressive loading of data, so that we load only everything that's close to the camera, not all the data at once. 🔵 This is called LOD (levels of details). E.g. everything within 10 meters is highest level of details, then everything within 30 meters is medium level of details. And if it's further away, it's the lowest resolution. Basically same way as Google Maps/Earth or different games do. Usually using Octree or similar data structures. 🔵 It's so much fun working on it, and playing with it. I'm so sad the final users would never see these circles, because it's a debug mode. Maybe I should make a game out of it? WDYT?

wildflow coral -- our first product, a comprehensive digital twin of coral reef ecosystems based on multimodal foundation models for biodiversity. 🔵 Why? Billions of years of evolution created Earth's vibrant ecosystems -- they are inconceivably rich and incredibly complex! We have so much to learn from this beautiful complexity. We want to build a future where humans and nature thrive and enrich each other. That’s why we chose to start with coral reefs. They are the most complex, beautiful, endangered, and important ecosystem. They are home to 25% of marine life while occupying only 0.1% of the ocean. Sadly, we’re on track to lose 90% of coral reefs by 2050. The time to act is now. Over half a billion people rely on coral reefs. Already vulnerable communities are the first to suffer the consequences. We have to make a lot of critical decisions about them! They are a key to model entire biosphere (protect nature) down the road. 🔵 How? We take all the data across all modalities, such as 3D photogrammetry, bioacoustics, underwater videos, remote sensing, eDNA, environmental data like currents, and more, to create the ultimate digital twin of any coral reef ecosystem and make it available to the world. And do it at a planetary scale. Core tech is multimodal foundation models for biodiversity. 🔵 What? This enables deep modelling of complex ecosystem dynamics, such as population dynamics, predator-prey dynamics, energy transfer, phenological events (like spawning), and computes the ecosystem's health and resilience metrics. Via rigorous analysis of multimodal data, it uncovers precise mechanisms driving ecosystem change, offering humans irrefutable evidence to steer their actions. It shows quantitatively how coastal development, agriculture runoff, invasive species, pollution, increasing water temperatures, and other pressures affect the coral reef ecosystem. It shows what the coral reef ecosystem gives us back through its services, such as coastal area protection and oxygen production. It coordinates conservation and restoration efforts worldwide. We know which practises work and which don’t. It guides human activities to understand, protect and restore coral reef ecosystems. 🔵 Next steps: ↳ ✅ 3D "Street View" for coral reefs prototype [DONE] ↳ 3D digital twin of coral reefs ↳ multimodal twin of coral reefs ↳ model ocean ecosystems ↳ model biosphere ↳ become the first generation that actually leaves behind nature better than we found it! 🐳 🌎

🌊 Coral Reefs are some of the most critical ecosystems in our oceans, yet they're disappearing at an alarming rate. I spoke to wildflow CEO, Sergei Nozdrenkov about their use of Radiance Fields to create a "3D Street View" for Coral Reefs. This represents the beginning of both visualization, but also larger deep (no pun intended) learning about our oceans and the environments that drive them. And yes, Radiance Fields do work underwater. Join me as we continue to examine the real world applications of Radiance Field based technologies and the people building with them. Full article here 👇 https://lnkd.in/dU_XJqUm

Sneak peek into the new larger 3D "Street View for Coral Reefs" model: - ❌ geometry is not good - ❌ no cleanup of wacky splats - ✅ used 200 photos from 2 GoPros (almost one swim-through) - ✅ 5 x 7 sq metres --> 15 x 3 sq metres (link in the comment)

A way to protect/restore Earth's vibrant Coral Reefs, Amazon Forest.. Many years of building AI for biodiversity at Google X and wildflow led me to this: we need a Planetary Digital Nervous System to survive and thrive. This is a cybernetics approach to conservation. 🔵 Why? Life on Earth has been evolving for billions of years, it's so precious, beautiful and incredibly complex! We losing nature, but there's a way to become the first generation that actually leaves behind nature better than we found it! 🌎 🔵 How? We need a Planetary Digital Nervous System that connects all sensors measuring the biosphere, such as eDNA, underwater cameras, bioacoustic microphones, remote sensing... (and environmental data like currents, temperatures...), to real-world actions like restoring coral reefs, creating marine protected areas, preventing algal blooms, managing invasive species, precise fishing policies, etc. 🔵 Cybernetic loops Measure -> Model -> Act are everywhere. 🔵 Pest Management in Ag: ↳ Measure: where all insects are using pheromone traps, acoustic traps, drones... ↳ Model: population dynamics, predict outbreak ↳ Act: targeted spray, biocontrol, mating disruption 🔵 Marine Protected Areas: ↳ Measure: BRUVs, satellite/acoustic tags, ↳ Model: find cluster of the most important fish, ecological niche ↳ Act: create and enforce MPA polygon 🔵 Algal Bloom: ↳ Measure: eDNA, remote sensing ↳ Model: forecast population dynamics of algae ↳ Act: remove nutrients from the water, etc 🔵 When you touch something hot and instantly snatch your hand away, you also follow Measure (touch smth) -> Model (realise it's super hot) -> Act (snatch your hand away). For you it takes milliseconds. For marine protected areas it takes at leat 5 years! 🤯 🔵 Measure 📈✅: a lot of incredible companies collecting the data: NatureMetrics, Flying Fish Technologies Pty Ltd, Saildrone, Stream Ocean, Rainforest Connection (RFCx) and many more. Nature data grows exponentially (supply)! 🔵 Act 📈✅: massive demand here. How do you find data for nearby park? It's all fragmented. 🔵 Model ❌: huge bottleneck in data aggregation and modelling natural ecosystems. We need data platforms like Cecil, federated data networks, and companies and decision engines like Nala Earth. Majority of Measure orgs also creating platforms and data consortiums. Even NOAA: National Oceanic & Atmospheric Administration at Oceans Decade complaining it's hard to justify data collection, we need intermediaries. Really hard to bridge this gap! 🔵 wildflow was founded to comprehensively understand nature and empower humans to make better decisions about protecting our ecosystems. Planetary Digital Nervous System is our NorthStar. We start with multimodal foundation models for Coral Reefs (comment below). 🔵 We can protect and restore nature! We need more passionate people to build this ultimate digital twin of natural ecosystems to bridge the gap! 🌎 We can do it together!! 🙌

Try our 3D "Street View for Coral Reefs" prototype (3D Gaussian splatting)! We've hit a new milestone today! A small step towards our mission after months of work.. So stoked to tell you \o/ 🔵 Why? Coral Reefs are so precious, beautiful, incredibly complex and threatened ecosystems. They are dying fast.. But there's a way to protect and restore them! Also they are a key to protecting other ecosystems. A lot of amazing people fighting for their life. We need to understand deeply how Coral Reefs function, what methods work, what don't, to coordinate precise action. That's why we're building a digital twin of coral reef ecosystems -- our first product. 🔵 https://lnkd.in/gekBPU_c -- today you can try our first scrappy prototype (works best on computer), 10% done. 🔵 What you see: it's a "Google Maps / Street View / Earth for Coral Reefs". Someone swam around a reef with a few GoPros, and you can now see high-res 3D model of coral reef in the browser! It's a super cheap way of monitoring. Soon you will fly over square kilometres of reefs and see them in centimetre resolution. You can compare reefs in different countries and see how they evolve over time, easy way of getting rid of biases in data. 🔵 The prototype is terrible. Still work in progress (10%), just a sneak peek. E.g. we didn't even clean up the model at all. Used only 70 images out of 8000 for this reef. You see only 5x7m area. 🔵 Yet, I don't think you can find a better resolution reefs, so that you can just go and see them in the browser. Especially without using "beefy" computers on backend. In our case all the rendering is happening on the client side, we don't even have any backend servers. It fetches static data directly from Google Cloud buckets. Super easy to scale! 🔵 Next steps: more data, larger scenes, 3D segmentation models, classifying coral species, adding other modalities (acoustics, eDNA, geospatial)... Allowing anyone to run analytics against all our data... Foundation models for biodiversity... Becoming the first generation that actually leaves behind nature better than we found it! 🪸 🔵 This prototype wouldn't happen without these incredible people: - Ben Williams - it's all Ben's fault I'm doing this now, his idea and support - Pavel Rodionov - trained 3D Gaussian splats, the model you see - Jason Lynch - photogrammetry and Metashape guru - Andrew Mogg - all the photogrammetry heavy lifting - Rindah Talitha Vida - leading the study for the reef you see https://lnkd.in/gq_u46HD - Michael Vasilkovsky - cutting-edge 3DGS theory stuff - Tim Lamont -- coordinating everything from the academia side - Mark Kellogg -- amazing 3DGS viz library we use - Mars buildingcoral.com team for restoring this reef you see and many many more fantastic people! 🔵 Everything you see is open-source (MIT licence, just copy-paste us...). Give us a star on GitHub: https://lnkd.in/gncgfVvZ 🔵 Exciting times ahead!! 🪸🌎