Grantmaking for AI security | Profession evaluation

In 2017, donors gave round $9 million to fund AI security work.

In 2025, a single funder, Coefficient Giving, gave over $400 million. In 2026, it plans to spend over $1 billion, and it’s solely the biggest of many funders.

The expansion isn’t over: AI corporations are more likely to gas a brand new wave of philanthropy. Nan Ransohoff estimates that Anthropic employees and the OpenAI Basis may characterize $37–100 billion in annual philanthropic spending as soon as these corporations go public. If even a small fraction of this goes in direction of AI security (a priority for a whole lot of individuals at each corporations), the sector can have billions of further {dollars} to spend every year.

That cash may change how the world prepares for the rise of transformative AI — however provided that it reaches impactful tasks. Coefficient’s grantmakers routinely report that employees capability limits how a lot they offer. We’ve heard the identical from different main funders. To grab the approaching alternative, AI security will want dozens of latest grantmakers. For those who grow to be certainly one of them, you might direct tens or a whole lot of thousands and thousands of {dollars} in direction of extremely impactful tasks.

Why work on grantmaking for AI security?

The rise of transformative AI could create dire challenges for humanity. As Luke Muehlhauser, a managing director at Coefficient, places it:

This tempo of technological change is unprecedented, and the world isn’t ready. Little or no of the business, authorities, and nonprofit infrastructure we have to reply to those transformative adjustments has been constructed.

To create this infrastructure, we’ll want expert founders with good concepts. However good concepts don’t get funded routinely: somebody has to guess on the precise mixture of thought + workforce. Within the startup world, that’s a enterprise capitalist. Within the nonprofit world, it’s a grantmaker.

Very similar to one of the best VCs, one of the best grantmakers transcend funding; additionally they use their expertise to assist grantees enhance their plans and make essential connections. This takes a uncommon set of expertise: area experience, an eye fixed for expertise, and the strategic judgment to search out the important options of a posh proposal. Typically, they’ll devise a proposal themselves, and search for somebody who could make it occur.

Most organisations on our job board are largely funded by grants. So are we. A particular individual needed to evaluation and log off on all of these grants. It’s not probably the most high-profile job, however with out it, AI security can be a fraction of its present measurement. The sector is a chaotic mixture of researchers, wonks, and founders, working throughout a whole lot of tasks; grantmakers are in a singular place to see the large image, and to form technique for the sector as an entire.

Right here’s extra on why we advocate grantmaking:

There merely aren’t sufficient grantmakers

It’s onerous to know precisely how a lot philanthropic funding has gone into AI security. However our greatest guess is that the whole was below $5 billion as of mid-2026.

If Nan Ransohoff is true, the sector may see that a lot funding yearly as soon as company giving kicks off in earnest. However even when annual giving solely doubles from its 2026 ranges, that can nonetheless create demand for dozens of latest grantmakers. We have been impressed to put in writing this profile as a result of a number of funders instructed us they have been nervous about discovering sufficient folks to handle the oncoming tide.

Your work can have extraordinarily excessive leverage

VCs and grantmakers make related bets: startups and nonprofits can each multiply the worth of their funding. Coefficient Giving, one of many world’s largest grantmakers, goals to generate no less than a 2,000x “social return on funding” (SROI) — that means that each greenback it spends helps folks over 2,000 instances as a lot as a greenback given to somebody incomes $50,000 a yr.

Coefficient has already given over $6 billion to causes that it judged to be above that bar. For instance, it was the primary main philanthropic supporter of David Baker (who later received the Nobel Prize for his work on protein design), and organised a fund that drastically elevated world philanthropic spending to mitigate lead poisoning. And there are lots of different instances the place grants generated huge SROI — just like the Rockefeller Basis’s assist for agricultural analysis that enabled the Inexperienced Revolution, or Katharine McCormick’s funding for contraceptive analysis that led to the contraception tablet.

Whereas AI security grants are sometimes onerous to evaluate, a lot of the sector’s most influential work (from AI 2027 to METR’s activity completion graph to the marketing campaign that stopped a moratorium on state-level AI laws) depends on philanthropic assist. So do the capacity-building tasks that made the sector’s progress attainable (from the MATS fellowship to BlueDot Influence). For those who learn mainstream information protection of an alarming growth in AI, you’ll in all probability see a quote from somebody whose work is grant-supported.

Grants additionally assist folks develop expertise they’ll finally use elsewhere. When governments started to fret concerning the impression of transformative AI, they arrange AI security institutes — which meant discovering a whole lot of individuals with AI governance expertise who would select public service over profitable company jobs. Unsurprisingly, a lot of these folks got here from the nonprofit sector, and initially constructed their expertise by means of grant-supported fellowships or analysis programmes.

What about marginal impression?

You may settle for that grants will be very impactful, however nonetheless query the worth of further grantmakers. Would you make a distinction on the margin, or would your chosen tasks appeal to funding even with out you?

The second isn’t as doubtless because it appears. Philanthropy isn’t an environment friendly market. Many alternatives are onerous to identify, and an opportunity that appears promising to you may in any other case go unnoticed.

That mentioned, adversarial choice is an actual downside. Some proposals you discover could have been handed over for good causes, and a part of your job is to work out whether or not that occurred (and whether or not you agree with the funders who handed).

Even when one other funder says sure finally, they could be too late: by the point they discover the undertaking, its window of alternative may shut, or the founder may quit and take a job elsewhere. And if a undertaking does survive the wait, it may nonetheless have had rather more impression with an earlier begin, particularly if we don’t have a lot time earlier than the appearance of AGI.

Catherine Brewer, an affiliate program officer at Coefficient Giving, describes a case the place timing mattered: Coefficient gave seed funding to a pair of researchers who have been leaving a significant lab, which enabled them to launch a brand new organisation. “Had we gone slower or had much less grantmaker capability, it’s doubtless this grant wouldn’t have occurred; they each had different gives.”

It’s simple to imagine that grantmakers spend as a lot as they will, with funding as the final word bottleneck. However that’s not true; spending is commonly restricted by employees capability. Right here’s Luke Muehlhauser once more:

As a brand new AI grantmaker at [Coefficient], you’d doubtless transfer >$30 million, and plausibly >$100 million, in your first yr […] As a result of grant investigation capability is tight, hiring one fewer grantmaker normally means these thousands and thousands will simply sit in an account for an additional yr reasonably than being deployed.

For those who deploy tens of thousands and thousands of {dollars} that will in any other case go unspent, that buys plenty of probabilities to search out good concepts others received’t — and a single large hit may greater than pay for the remainder of your bets.

You possibly can assist your grantees do extra good

Past the cash itself, you may present essential recommendation and connections to founders who want them.

I’ve heard from grantmakers who modified a grantee’s trajectory by:

Serving to them discover staff, board members, and even co-founders. Alex Lawsen, a senior program officer at Coefficient, mentioned he’d completed a “first rate quantity of focused headhunting, particularly for board roles.” One other grantmaker as soon as made a global journey to pitch a researcher on leaving his company job to assist run an AI security org — and succeeded.

Offering profession recommendation. Lawsen famous that his job helped him find out about a variety of organisations, which positioned him to present a “impartial, balanced perspective” to folks determining their subsequent steps. (Many grants in AI security go to assist people develop their profession plans, a stage at which they’ll need recommendation.)

Encouraging them to be extra formidable. Julian Hazell, head of AI technique on the Astralis Basis, observed that fellowships and different expertise growth programmes had a powerful observe report in AI security, and thought the Institute for AI Coverage and Technique (IAPS) was well-placed to run one targeted on sensible coverage expertise. He recommended that IAPS department out from analysis to launch a fellowship. IAPS had thought-about this earlier than, however Julian’s recommendation and grant (from Coefficient Giving, his earlier employer) tipped the size and led them to behave on the thought.

Connecting them with useful consultants. Many would-be grantees have restricted expertise exterior their fields — image a professor beginning a stress group or a programmer pivoting to coverage. You’ll be well-positioned to identify these limitations, and you’ll draw out of your community to attach grantees with advisors who shore up their weaknesses. One grantmaker I’ve labored with helped a first-time CEO discover a efficiency coach to develop their skilled expertise; one other arrange media coaching for a researcher whose work was attracting journalists.

The job will be extremely fulfilling

Right here’s what I’ve heard from grantmakers about their favorite components of the job:

You actually really feel the impression you make. Funding an individual or undertaking has an apparent impact on what occurs subsequent — and if the grant goes nicely, you may share within the pleasure. Hazell: “You spend all this time evaluating somebody, having a bet on them, after which they really go do the factor. That’s actually satisfying.”

There’s room for particular person expression. Grantmakers — even early of their careers — have a lot of flexibility and autonomy to pursue the alternatives they view as most promising. You’ll be requested to type your personal views and in the end make the decision (normally with approval from senior employees) on the place to allocate thousands and thousands of {dollars}.

The job is all the time altering. You’ll always meet new folks, consider new proposals, and cope with new developments in your area. A single RFP may provide you with 500 tasks to think about.

You possibly can see the sector from the within. Alex Lawsen defined that whereas board membership may expose you to at least one organisation’s inside workings — the issues they face, the way in which their leaders make choices — grantmaking may do that for 10 organisations without delay: “It’s onerous to overstate how distinctive an image [of the field] you get from this.”

You’ll additionally get time with senior researchers and coverage consultants who don’t have main public profiles and can be onerous to achieve in any other case. Catherine Brewer known as this “a extremely enjoyable a part of the job, which I underrated earlier than becoming a member of.”

What do grantmakers truly do?

Earlier than you fund somebody, you want to discover them. In most grantmaking roles, you’ll spend a lot of your time monitoring down potential founders, not simply ready till they arrive to you. This might embrace:

Looking for leads at conferences and different occasions.Producing concepts for tasks and organisations you’d wish to see, then discovering folks to create them. (Julian Hazell did a model of this when he realised that IAPS was well-positioned to offer coverage coaching.)Writing requests for proposals (RFPs) to clarify what you’d wish to fund, and sharing them with as many related teams as you may.Studying the newest work in your area to identify folks with fascinating concepts, then asking what they’d do with 1,000,000 {dollars}. (The precise quantity varies.)

After you supply a proposal, you determine whether or not to make a grant. This entails:

Assessing the cost-effectiveness of the grant. Whereas back-of-the-envelope calculations (BOTECs) are widespread in some sectors of impact-focused grantmaking, AI security tends to be much less quantitative. More often than not, you’ll use some mixture of private heuristics, intestine feeling, conversations together with your friends, and the end result of comparable grants prior to now. That mentioned, BOTECs can nonetheless enable you to with sub-problems, like estimating the attendance of a convention you may fund or the possibility {that a} invoice will get by means of Congress.Evaluating the folks concerned. Have they got the right combination of expertise and skill to run the undertaking nicely? Are their concepts wise, with a transparent idea of change? Would you want to supervise them intently, taking time away from different grants you might make? How must you deal with adverse details about them?Right here’s an instance which blends a number of real-world instances: If an applicant’s final undertaking fell aside as a result of they screamed at their subordinates, however they went to anger administration coaching and appear to get together with their present workforce, would you continue to apply a danger penalty to their proposal? For those who did fund them, how would you monitor the scenario?Deciding whether or not to impose circumstances. Are you keen to make an unrestricted grant? Will you fund solely a subset of the grantee’s work? Will you assure further funding in the event that they hit sure milestones?

You’ll get a lot of your data from the candidates themselves. However since they might skew the info to enhance their probabilities, you’ll additionally search out different sources, like individuals who labored intently with them or impartial consultants who can choose their output.

When you’ve made a grant, you’ll keep up a correspondence with the grantee for scheduled check-ins and advert hoc recommendation. That is very versatile; you may discuss to somebody yearly, or each different week.

Past direct grantmaking, the job has many different duties. You’ll set broad strategic priorities, sustain with developments in your area, write about your work, and reply questions from individuals who search out your experience.

Might you be a very good match?

Expertise that can enable you to succeed

Expertise in related areas. To evaluate a proposal, it’s essential to consider:

The undertaking itself (how impactful wouldn’t it be with good execution?)The folks concerned (how nicely do you assume they’ll execute?)

On the primary level, you’ll want sufficient background in the precise space(s) to gauge a undertaking’s potential: Does the thought make sense? Have related tasks succeeded or failed prior to now? Does the proposal deal with the doubtless obstacles?

Related expertise may embrace nationwide safety, know-how coverage, cybersecurity, machine studying, China, or many different matters in AI security. For those who’ve labored in any of our precedence careers, that’s a very good signal.

Logic about folks. Being a website professional helps you consider plans, however isn’t as useful for evaluating folks. That talent is tougher to pin down, however listed below are some constructive alerts:

You’ve employed folks earlier than, and so they have been extremely profitable of their roles.You’ve beneficial contacts for jobs or launched them to collaborators on a number of events, and seen it work out nicely.You might have a historical past of selecting the correct folks to construct groups or partnerships, in skilled or aggressive settings.

Some “folks judgment” entails common traits: deciding whether or not a workforce appears competent and well-organised, or charismatic and likeable if the work calls for it. A few of it requires background data: evaluating a researcher’s technical chops, or whether or not a assume tank’s founder has the precise expertise and bearing to make politicians take them severely.

(One remark I’ve heard loads: “I don’t assume this individual would come throughout nicely on Capitol Hill.”)

Studying new issues shortly. Regardless of how skilled you might be, grantmaking will power you to cope with new concepts and unfamiliar domains. Julian Hazell’s work has concerned crash programs in:

“Expertise bottlenecks and hiring frictions within the EU.”“The fundamentals of US tort legislation and the way state supreme courts form it.”“Incentives, frictions, and blockers that labs face round monitoring and monitoring what their fashions are doing in deployment.”

Ideally, you’ll use your first analysis move to generate questions for somebody who is aware of extra. Grantmakers usually meet with consultants who assist them perceive proposals and resolve uncertainties. The difficulty is that your consultants received’t all the time agree — so that you’ll want sufficient context and judgment to determine whose opinion you belief, or provide you with the precise synthesis.

Sturdy communication and reasoning transparency. That is essential for many jobs, however a number of widespread conditions in grantmaking actually emphasise it:

Writing for colleagues, particularly whoever approves your grants. Making your cruxes and the energy of your proof express lets them log off shortly, or push again for the precise causes.Persuading funders to assist a possibility. In some roles, you’ll have to get approval immediately out of your funder(s). In comparison with your colleagues, they’ll usually have much less context on the house and place extra weight on whether or not a grant excites them personally (although funders differ extensively on these components). Which means you’ll have to make an particularly clear and legible case for every grant, drawing out what makes the work fascinating with out exaggerating its potential.Delivering suggestions to individuals who aren’t prepared to listen to it. I’ve heard a number of tales alongside the strains of “I attempted to inform a grantee they weren’t doing nicely, however they have been shocked after I didn’t renew their funding. I want I’d gotten my level throughout higher.”

An in depth data surroundings. Alex Lawsen has written {that a} sturdy grantmaker can “observe each adjoining effort, discover persistent gaps, examine 5 proposals attempting to unravel close by issues, or work out which two teams ought to in all probability be speaking to one another.”

This goes past expertise: You’ll have to always take in new details about the house, so that you could uncover tasks which may profit from funding or recognise when a sensible-sounding thought now not is sensible. This entails a mixture of networking and studying, relying on how data flows by means of your a part of the sector.

Pace and responsiveness. Grantmaking is time-sensitive. Somebody who requests funding might be selecting between their grant and a full-time job; getting again to them per week sooner may change which path they take. One grantmaker described his work as trying to find whichever data may trigger him to say no the quickest, in order that he may deal with the strongest proposals as an alternative of stringing alongside potential grantees.

Promising background expertise

Aside from expertise in our precedence careers, it helps to have expertise:

Truly making grants, or offering different types of assist to people or small groups (e.g. startup investing, working at a authorities funding company like DARPA or NIH)Serving to folks refine their plans in different contexts (e.g. consulting, teaching)Learning the panorama of AI security, biosecurity, or associated subfields, particularly in case you constructed a powerful community alongside the way in which (e.g. journalism, occasion organising, analysis administration)Evaluating AI security work, even on an newbie foundation (e.g. writing feedback on LessWrong, running a blog about analysis papers, creating BOTECs on your private donations)

Downsides to engaged on grantmaking

The sector will be tough to enter. Grantmaking doesn’t have many formal coaching alternatives, and even entry-level roles are onerous to get. Funders wrestle to rent as a result of they’ve excessive requirements for who they’ll belief to make thousands and thousands of {dollars} in grants. To cite Ollie Rodriguez (previously of Longview Philanthropy), “grantmaking is extra like a talent than a path“. To grow to be a powerful candidate, you’ll have to construct expertise and skills in different positions.

Excessive stakes beget excessive stress. You possibly can’t fund each proposal you see, however each determination has penalties. For those who don’t fund a undertaking, it may not occur; in case you don’t renew a grant to an organisation, folks may lose their jobs. In the meantime, in case you fund a undertaking and it fails, you could really feel such as you’ve wasted an enormous quantity of another person’s cash.

Your impression is oblique. You possibly can present assist and steering to your grantees, however you in the end received’t have a lot affect over how most of your grants carry out. Some folks come to overlook “on-the-ground” work the place their efforts have direct returns. Sarcastically, grantmaking additionally provides you a lot probabilities to note gaps you might fill, if solely you have been keen to take a brand new job; a number of of my sources discovered this tense.

Your work received’t be particularly seen. You’ll spend plenty of time reasoning privately together with your colleagues and doing delicate analysis you may’t publish. This makes it onerous to construct a portfolio, or develop profession capital that can switch exterior of AI security. One grantmaker famous one other consequence: “You’re not going to get social credit score for lots of the great things you do.”

You could be constrained by your funders. Grantmakers give away different folks’s cash. The funders you’re employed with may override your choices, or impose restrictions that cease you from supporting a few of your favorite tasks. This will occur even in longstanding relationships, as when Good Ventures stopped funding a few of Coefficient’s sub-causes.

Energy dynamics distort communication. When life-changing quantities of cash are concerned, it’s onerous to maintain issues sincere. Candidates may stretch the reality in a proposal or flatter you with empty compliments. As one grantmaker mentioned: “It may be onerous to imagine constructive suggestions or alerts of respect, since you’re unsure how a lot of it comes out of your place of energy.”

Energy additionally adjustments the way in which folks hear your phrases: candidates could bend over backwards to present you what you need, even in case you don’t really feel strongly a couple of suggestion. You’ll have to speak rigorously to make sure your passing ideas aren’t taken as circumstances for funding.

High organisations

These funders work on issues we see as prime priorities.

Coefficient Giving, the biggest grantmaker engaged on our precedence problemsOpenAI Basis, which has huge sources and an “AI Resilience” program targeted on danger mitigationEffective Altruism Funds, which simply opened its Transformative AI FundLongview Philanthropy, which advises quite a few giant donors on AI, biosecurity, and nuclear weapons policyFounders Pledge, which recommends funding alternatives to its community of entrepreneursThe Astralis Basis, which focuses on European and Asian AI coverage alongside different uncared for interventionsThe Way forward for Life Basis, whose focuses embrace utilizing AI to enhance decision-makingLightcone Commons, which connects teams of funders with professional evaluators they’ve sourcedMacroscopic Ventures, which works on each conventional AI governance and some uncommon initiatives (AI welfare and cooperative AI)Halcyon Futures, which makes investments and incubates new organisations along with making grantsSentinel Bio, which works completely on biosecurity (a subject that’s extremely related to dangers from superior AI)Schmidt Sciences, a science funder that funds occasional AI security tasks

The right way to enter the sector

As a result of you may enter grantmaking from a wide range of backgrounds, our strongest advice is to test the grantmaking roles on our job board and apply in case you see a task that matches. Whereas there’s a excessive bar for entry, your {qualifications} could be extra related than you assume — and the necessity is pressing.

Extra concepts:

Begin with a unique function within the house. You’ll want a considerable base of AI security data to qualify for many grantmaking positions — the sort you’d get from truly engaged on a job or analysis undertaking. Even a part-time function will be useful, particularly if it entails evaluating tasks (e.g. cost-effectiveness researcher) or serving to a grantmaker in some capability (e.g. analysis or govt assistant).For those who’re an professional on one thing else, construct AI security context. In case you have extremely related expertise or expertise — grantmaker at a significant basis, cybersecurity professional — you could be a powerful candidate even with no formal background in AI security. Nonetheless, you’ll nonetheless have to know the fundamentals. Good methods to begin embrace programs from BlueDot Influence and our primer on serving to superior AI go nicely.Examine how grantmakers function. Funders and particular person grantmakers usually have blogs, mailing lists, or Twitter accounts. Some, like Efficient Altruism Funds, publish stories on their funding choices. I used to put in writing for Coefficient, and I can testify that studying its weblog would train you a large number about its strategy (and make you a stronger candidate).Apply pondering like a grantmaker. You in all probability don’t have thousands and thousands of {dollars} to present away, however you might nonetheless problem your self by informally evaluating organisations or tasks. For instance, Michael Dickens writes about his private giving in nice element. This type of aspect undertaking may also help you seize a hiring supervisor’s consideration.One other thought: Take a look at payout stories from Efficient Altruism Funds and ask your self the way you’d consider a few of these grants. What questions would you ask earlier than renewing, or to gauge the impression of somebody’s work?Search for coaching alternatives. We haven’t heard of many alternatives particular to grantmaking, however they do exist. Bold Influence’s Impactful Philanthropy programme trains folks in a number of philanthropic expertise (together with grantmaking and advising rich people), and Constellation’s Astra Fellowship has included technique and governance tasks linked to grantmaking.

Discover jobs in AI security grantmaking

For those who assume you could be a very good match for this path and also you’re prepared to begin job alternatives, see our curated listing of jobs open on this path:

View all alternatives

As a result of we anticipate a surge of AI security funding over the subsequent few years, we expect many extra jobs may open up; regulate this web page.

Study extra

Notes on the philanthropic scenario

Private reflections from grantmakers

Different notes on grantmaking

Acknowledgements

Thanks to Alex Lawsen, Catherine Brewer, Julian Hazell, Matt Beard, and Naina Bajekal.