Captain Data’s Post

Excited to share our new work on Machine Learning! Andrea Failla, Rémy Cazabet, Giulio Rossetti SoBigData Research Infrastructure We reimagine group evolution in temporal data with "#archetypes" and "#facets". Just as a bird is more represented by a sparrow rather than a penguin, SOTA event types in #dynamic #clustering / #community #detection such as "merges" and "splits" are actually typical examples of a category. This is why we forget rigid events based on what one wishes to extract from the data to explore the reality of group evolution observed in data. Check out the paper for more! https://lnkd.in/dF9VcdGJ

Two questions: 1) How often do you implement a bit of tech 'because it is cool'? 2) If you answer is "never, I'm far too responsible"; is that a good answer? One of my brainchildren last year was ASHE; the Automated Spaghetti Hurling Engine. ASHE is a genetic algorithm engine. There are a bunch out there but most are pure; the insist on fixed length chromosomes with homogenous datatypes. I generalized the concept into something more like a 'chaos engine with tracking and ranking'. Despite the silly name it was built for reputable reasons - it is extremely good at hyper-parameter tuning and very good at discovering defensible, predictive complex features. (eg - it was able to derive things such as Newton's Gravity Formula from observation data). A few weeks ago one of my super-clever team who was working on Sparse Neural Network research was hand-tuning some hyper-parameters (and working hard). I reminded her that ASHE could do the work for her. She reached out the next morning and looked a bit like she had been hit by a truck. I asked what had happened and she said that watching ASHE converge upon a solution was 'so cool' she had stayed up through the night watching it ... At the time I smiled (somewhat) sweetly but I was thinking ... what ??? But y'know a couple of weeks down the line ---- I'm beginning to think that having built something 'cool enough' to keep someone with a PhD in computer science awake all night MIGHT be one of my bigger successes this year. So for those of us old, senior codgers who have seen it all before; remember; we are probably senior because 'back in the day' things were cool enough to keep us awake at night ....

Google DeepMind Researchers Propose a Novel Divide-and-Conquer Style Monte Carlo Tree Search (MCTS) Algorithm ‘OmegaPRM’ for Efficiently Collecting High-Quality Process Supervision Data

Ilya Sutskever, OpenAI’s co-founder and former chief scientist, is co-founding a new AI company called Safe Superintelligence, Inc. (SSI, Inc.). He announced it yesterday in a tweet. I don’t want to paraphrase it, so here is what he wrote in full: - - - - - Superintelligence is within reach. Building safe superintelligence (SSI) is the most important technical problem of our time. We’ve started the world’s first straight-shot SSI lab, with one goal and one product: a safe superintelligence. It’s called Safe Superintelligence Inc. SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI. We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead. This way, we can scale in peace. Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures. We are an American company with offices in Palo Alto and Tel Aviv, where we have deep roots and the ability to recruit top technical talent. We are assembling a lean, cracked team of the world’s best engineers and researchers dedicated to focusing on SSI and nothing else. If that’s you, we offer an opportunity to do your life’s work and help solve the most important technical challenge of our age. Now is the time. Join us. Ilya Sutskever, Daniel Gross, Daniel Levy June 19, 2024 - - - - - I’m pretty sure Ilya meant to say “crack” team, not “cracked,” but – assuming he meant a highly functional, smooth-running team as opposed to a team of engineers who are cuckoo for Cocoa Puffs – the mission, vision, and values are clearly articulated, focused, and passion-driven. I’m excited to see what SSI can do. I’m also curious about how their self-imposed constraints will position them against the more aggressive model builders. Altruism, especially in this case, is laudable. But, will SSI create a better (or even competitive) product? We’ll see. -s

Our CTO Jeff Dalgliesh shares why partnering with Neo4j has been instrumental in the development and design of our reView solution. They are building the critical graph infrastructure that is paramount to adoption of #ExplainableAI and the future of how humans leverage working alongside AI tools. Be sure to read the recent partner spotlight of data² and how we are bringing mission critical #GenAI use cases to reality within the Energy, Government and Defense Sectors. https://lnkd.in/eSp-U6kk

Data Growth: Geek Check Recently I was playing with some data sets and learned that back in 2010, Eric Schmidt (then‐CEO of Google) remarked that from the dawn of civilization until 2003, humanity produced about 5 exabytes of data in total—and that in 2010, we were producing 5 exabytes every two days (i.e., 2.5 exabytes/day). Fast forward to the mid‐2020s, and the daily rate has grown by two orders of magnitude (from a few exabytes/day to hundreds of exabytes/day). While no one can give a single “perfect” figure for 2024, most leading estimates put us in the ballpark of 400+ exabytes of data generated each day—an astonishing leap from the 2.5 exabytes/day figure previously referenced back in 2010. This blew my mind – in a geeky, scientific way! I then started to explore what this information might look like via analogies. So... every single day, we generate enough data to watch Netflix nonstop for 15+ million years! (1 hour of HD Netflix is roughly 3 GB of data.) If every 1 MB e‐book were a real half‐inch‐thick physical book, just one day of the world’s newly generated data (at 400 exabytes/day) would create a stack over 3 billion miles high—roughly 34 times the distance between the Earth and the Sun (Sun to Earth = 1 astronomical unit (AU) or 93 million miles. If we consider Neptune’s orbit to be roughly 30 AU from the Sun, and Pluto’s orbit is about 39.5 AU at its farthest, then a 34‐AU stack in a day would surpass Neptune’s orbit and nearly reach Pluto’s. This is how enormous 400 exabytes of data per day really is. Given AI/ML growth, what might our data output look like by 2030?

Data² is on a mission to change how defense, intelligence, and energy organizations extract insights from their data. 💡 Learn how the Data² reView platform combines the power of Neo4j #KnowledgeGraph, #GraphRAG and #GenAI to deliver explainable AI-driven insights that enable analysts to make high-stakes decisions confidently. https://bit.ly/3yaTjoF data² Jeff Dalgliesh Theo Hopkinson #Neo4j #GenAI #graphdatabase data²

A fascinating and important use case of Neo4j: Learn how the Data² reView platform combines the power of Neo4j #KnowledgeGraph, #GraphRAG and #GenAI to deliver explainable AI-driven insights that enable analysts to make high-stakes decisions confidently.

Making Bayesian Optimization Algorithm Simple for Practical Applications via #TowardsAI → https://bit.ly/3ScyjF1