DARO reposted this
Data Alchemist - blending data, measurement, technical, and governance expertise to help organizations and collaborations expand the utility and impact of their individual and collective data
I am a reluctant #AI enthusiast when it comes to #socialsector #data and use cases, but one area where I see tremendous potential is helping #philanthropy make sense of historical (sometimes decades!) old #grant reports that have evolved, expanded, or changed over the years. In the past, a foundation that wanted to delve into these and observe any trends, gaps, lessons, did the typical #research thing: they had to hire someone to develop a coding protocol and then convened a group of coders to do their thing - incredibly time-consuming, inefficient, and riddled with potential errors, bias, and subjectivity. I'm curious if anyone is doing this today using AI? I'm looking at you Chantal Forster, Peter York, Sarah Merion, Peter Bull, Andrew Means, M.P.P., Sam Caplan, Kelly Fitzsimmons, Jake Porway.
We’ve advanced the training of LLMs (started with NLP and NLU, six years ago) over the past couple years. We call it Automated Content Analysis, or ACA. We’ve developed and refined a human-in-the-loop qualitative analysis method, fine tuning LLMs and using RAG, for corpuses of large, medium, and small scale funders, evaluating proposals and reports, structuring unstructured data. including the National Science Foundation. Happy to chat about it.
100% agree on this - both on the reluctance (lots of hype and technological determinism) and the vast trove of data wrapped up in old grant reports. Would be curious to hear if it's in the ImpactMapper roadmap? The other similar data source I've been thinking about (raised by a nonprofit the other day) is nonprofit annual reports (and maybe more generally their other communications channels). Less consistent formatting than a single foundation's historical report might present a challenge but might also be able to present a more holistic and current view of past and current grantees (might also present an opportunity for ongoing learning that doesn't require nonprofits to continue to do all the legwork for data generation)
We are working on exactly this at ImpactMapper, David Goodman, Ph.D! Our software has grants analysis, a qualitative coding tool for grantee reports and evaluations, stories, interviews, etc, and a survey tool with visualization. We are now building AI to automate coding of text based on an organization's custom taxonomies. Exciting stuff! Happy to share more if interested!
This is the fun stuff you’re talking about! We love to marry both qual and quant data to uncover non-obvious insights and help determine direction for the future, ID bias indicators, uncover sentiment/emotion/topics in lots of qual data, etc…. Happy to chat more David
David Goodman, Ph.D someone should give us some money and a pile of grant reports. I bet we could do some AB testing and see just how effective AI could be at this sort of work.
I'm working on something like this with USAID evaluations
We are chipping away at this David Goodman, Ph.D ➡️ At SmartyGrants - an enterprise of Our Community we have an auto-classifier to categorise historical grants, an Outcomes Engine for sensibly collecting and aggregating impact data from grantees, and Analytics launching 2025. ➡️ We partnered with Omdena a few years back to dip our toes in using AI to collect and process grants data from public sources. ➡️ We have grand plans to build a 'Centre for What Works' building upon a few other pilot initiatives. Paola, Jen, Kathy, Jodie FYI 🙂
Allison Gister, haven’t you used AI for grant coding?
Data Alchemist - blending data, measurement, technical, and governance expertise to help organizations and collaborations expand the utility and impact of their individual and collective data
2moI would like to thank everyone for contributing to this thread - there is so much great work taking place and lots of connections being made. I do want to overstate the obvious though - "but for" my question, many of us would have no clue that this work was already happening and who / where / how it was taking place. This continues to be one of the biggest barriers we face as a sector: how do we share information, knowledge, and lessons within and across sectors in an effective and efficient way? There are some conferences where this takes place and a few mechanisms like slack channels (Open Change, TAG, etc), but it still requires someone to ask the question. I wonder if there are any other ideas about how to improve knowledge management, sharing, connections, etc?