A Local RAG Chain for Donor Questions

Decompose, filter, rank, generate — running entirely on one machine, with no model API

NoteA note on how this was made

Built by directing an AI research assistant: I set the goal, the success criteria, and the review at each step; Claude wrote and ran the code. Charity text comes from the Wikipedia action API and organisation metadata from the ProPublica Nonprofit Explorer API, both public and key-free. Retrieval, embedding and generation all run locally — no model API was called. The retrieval scores plotted below ship with this post as scores.csv.

The task

Answer a donor’s question in plain English, and name real organisations precisely enough that money reaches the right entity:

“I’m an investor looking to send $1m to charity over the next few years. I’d like to use a DAF. What fees and returns can I expect? And if I’m looking to donate to charities that respond to disasters and support women?”

Three questions in one: donor-advised fund mechanics, a fee calculation on a specific amount, and two separate cause areas to match against actual charities. This post is how the chain that answers it is built, and what it produces.

The stack

Everything runs on one machine. The only network calls are the one-time data pulls.

Component	Choice	Why
Corpus	Wikipedia article text, 30 charities	990 “program service accomplishments” narrative would be better, but the IRS bulk XML has been frozen since 2021 and ProPublica’s API exposes financials only
Metadata	ProPublica Nonprofit Explorer	EIN, 501(c) subsection code, revenue
Chunking	120 words, 30-word overlap, split by article section	Section headings survive as retrievable context
Embeddings	all-MiniLM-L6-v2, 384-dim	1,279 chunks embed in seconds on CPU
Search	NumPy dot product	Faster than a vector DB at this size — see below
Generation	qwen3.5:9b via Ollama, temperature=0	~97 tok/s on GPU; deterministic output makes regressions debuggable

That’s 1,271 charity chunks (123,368 words across 503 article sections) plus 8 hand-curated DAF reference records covering fee schedules and the relevant IRS rules.

The chain

question
   ↓  1. decompose      LLM → cause facets + DAF flag + amount
   ↓  2. route          facets → charity corpus; DAF flag → reference corpus
   ↓  3. filter         drop known-ineligible orgs on subsection code
   ↓  4. rank           per-facet percentile + absolute similarity floor
   ↓  5. assemble       one context block per organisation, not per chunk
   ↓  6. generate       + deterministic checks on the output
answer

1. Decompose

A compound question embedded as one vector lands between its topics and matches neither well. The LLM splits it into independent facets first:

{"charity_facets": ["disaster response charities", "women's support organizations"],
 "needs_daf_info": true, "amount_usd": 1000000}

Two things are not trusted to the model. The donation amount is extracted by regex, and an explicit mention of “DAF” sets the flag directly — because the model read “I have $400,000 to give through a DAF” as a pure charity question and returned amount=None, daf=False. Anything cheap to parse deterministically should be parsed deterministically.

2. Route

Facets query the charity corpus; the DAF flag pulls in the reference corpus. A question with no cause area at all — “what are the deduction limits?” — produces an empty facet list, and the charity corpus is simply not searched. Prompt sections are assembled from these same flags, so a pure charity question never gets asked to write about fees.

The DAF corpus is 8 records, which fits in the prompt whole. Retrieving a top-k from it once dropped Fidelity’s fee table from a fee question. Don’t retrieve from a corpus small enough to pass entirely.

3. Filter on metadata, not similarity

Donor-advised funds may only grant to 501(c)(3) public charities. The Sierra Club is a 501(c)(4) — legally ineligible regardless of how well it matches. No embedding reliably encodes a tax classification, so this is a hard pre-filter on the subsection code.

The subtlety is what to do with unknown. Habitat for Humanity, the YMCA and the Salvation Army file group returns, so their national entities aren’t in ProPublica’s index and their eligibility field is None. Filtering on is True silently removed them from every query in the system — including the Salvation Army, the strongest disaster-response match in the corpus. The filter excludes only known-ineligible orgs; unknowns surface carrying a caveat.

4. Rank: percentile within facet, then an absolute floor

Raw cosine scores are not comparable across different query strings, so combining facets by taking each organisation’s worst score ranks by whichever facet the corpus covers worst overall. Scores are converted to percentiles within each facet first.

But percentile rank is purely ordinal — something always sits at the top, even when every score is noise.

Figure 1: All six facets have a 100th-percentile organisation. Only some of them mean anything. Each dot is one of the 30 charities.

Read the ranks alone and all six facets look equally answered. Read the absolute scores and they tell three different stories: “disaster relief” is genuinely covered (0.67, cleanly separated), “women’s support” is thin (top hit 0.48, whole facet compressed just above the floor), and “homelessness in rural Alaska” is absent — 0.41 and then noise.

So ranking is percentile plus MIN_SIMILARITY = 0.40. Genuine topical matches land 0.44–0.73; noise sits below 0.37. A facet with nothing above the floor is reported as having no coverage and the generator is told to say so. That threshold is tuned by eye and specific to this embedding model — the least principled part of the chain — but without it, an ordinal ranking always returns a confident top result.

5. Assemble per organisation

Context is grouped by organisation rather than dumped as ranked chunks, so the model sees one coherent block per candidate instead of scattered fragments of the same entity:

### The Salvation Army
Retrieved for: disaster relief
Legal name: The Salvation Army (national entity not individually listed) | EIN: unresolved
  | Eligibility: confirm with sponsor | financial data unavailable
Source: https://en.wikipedia.org/wiki/The_Salvation_Army
- [Summary] The Salvation Army is a Protestant church and international charitable organisation...
- [Disaster relief] ...mobile canteen vehicles that provide food and other welfare to members
  of the Emergency Services at bushfires, floods, land search...

Each block pins the article’s lead section plus the top facet-matched excerpts. Ranking chunks purely by similarity once returned only policy-advocacy passages for Feeding America, and the model concluded it ran no feeding programmes — the lead section is where an article says what the organisation actually does.

Wording in these blocks matters more than expected. An earlier label reading 501(c)(3) status NOT VERIFIED sat a few lines from Excluded as DAF-ineligible: Sierra Club, and the model conflated the two negations and dropped its own top-ranked charity as ineligible — against an explicit instruction not to. Rewording the label to Eligibility: confirm with sponsor fixed what a stronger instruction could not.

6. Generate, then check deterministically

A 9B model is good at language and unreliable at arithmetic, so anything a donor might act on is computed in Python and injected as authoritative. Fees come from structured tier tables:

=== COMPUTED FEES (calculated in code, authoritative) ===
Estimated FIRST-YEAR administrative fee on a $1,000,000 balance:
- Fidelity Charitable: $4,500/yr (0.450% effective) [computed from published tiers]
- Vanguard Charitable: $4,500/yr (0.450% effective) [computed from published tiers]
- Schwab Charitable: NO ESTIMATE AVAILABLE — its full tier table is not confirmed...

That last line exists because omitting Schwab entirely caused the model to extrapolate its 0.60% entry rate flat across $1m and report “$6,000” as though published. A gap in the context is not read as absence; it’s read as an invitation.

Three checks then run on the generated text:

Check	Catches
check_eins	An EIN that isn’t in the verified set — fabrication
check_missing_eins	An org named without its known EIN, or claiming “unresolved” for a verified one
unmentioned_orgs	Retrieved candidates the answer never used (informational)

EIN formatting is also code, not model: Save the Children’s is 8 digits once JSON drops the leading zero, so 60726487 must be zero-padded to 06-0726487.

Results

Seven test prompts against the final chain. Output is verbatim.

The compound question — two cause facets, DAF mechanics, and a dollar amount:

Q: I’m an investor looking to send 1m dollars to charity over the next few years. I’d like to use a DAF. What kind of fees and returns can I expect? And if I’m looking to donate to charities that respond to disasters and support women

facets: ['disaster response charities', "women's support organizations"]
needs DAF info: True

top orgs for facet: 'disaster response charities'
  The Salvation Army      pct=1.00  cos=0.713
  American Red Cross      pct=0.96  cos=0.671
  charity: water          pct=0.93  cos=0.516

top orgs for facet: "women's support organizations"
  American Heart Association     pct=1.00  cos=0.477
  Boys & Girls Clubs of America  pct=0.96  cos=0.420
  Goodwill Industries            pct=0.93  cos=0.411

serving ALL facets: none
excluded (not 501(c)(3)): Sierra Club

Fees and costs For a $1 million balance in a DAF over your first year, you can expect an administrative fee of approximately $4,500, which represents about 0.45% of that balance. This figure is computed from the published tiered schedules for Fidelity Charitable and Vanguard Charitable as of August 30, 2026; Schwab Charitable does not have a confirmed fee schedule above $500,000 in our data so no estimate can be given there. Investment pool expenses will add another layer to your costs: they typically range from roughly 0.015% for index pools up to about 0.89–0.93% for some asset allocation and sustainable pools at Fidelity Charitable, while Vanguard Charitable commonly cites expense ratios in the 0.01%–0.11% range. These fee schedules change frequently; please verify current rates directly with your chosen sponsor using their official links before relying on them.

What returns to expect Returns are simply the performance of underlying investment pools (money market, short-term bond, equity, or index) net of fees and are not guaranteed by any sponsor. Because you plan to grant out within a few years, sponsors generally recommend conservative pools where drawdown risk matters more than expected return; do not treat historical growth as a projection for your specific account.

Charities matching your interests For disaster response, we can recommend:

• The Salvation Army (national entity not individually listed) | EIN: unresolved — Eligibility must be confirmed with your sponsor before recommending, since the national entity is not always eligible in every DAF; it provides disaster relief and humanitarian aid globally.
• American National Red Cross | EIN 53-0196605 — Provides emergency assistance, shelter, food, health services after disasters like hurricanes and wildfires.
• Charity Global Inc (charity: water) | EIN 22-3936753 — Focuses on providing clean drinking water in developing nations; while primarily a development project, it responds to disaster-related needs by addressing basic sanitation systems that are often disrupted during crises.

For supporting women, we can recommend:

• American Heart Association Inc | EIN 13-5613797 — Runs the Go Red for Women® campaign focused on heart health awareness and research specifically for women, including initiatives like National Wear Red Day and partnerships with organizations such as Miss America’s Scholarship Foundation.

Note that Boys & Girls Clubs Of America (EIN: 13-5562976) serves youth broadly rather than focusing exclusively on women’s issues; if your interest is strictly in supporting women, the American Heart Association offers a more direct match to that stated goal.

Pure mechanics — no cause area, so the charity corpus is never searched and no charity section is generated:

Q: What are the tax deduction limits if I put $250,000 into a DAF?

facets: []
needs DAF info: True

Fees and costs For a $250,000 balance, both Fidelity Charitable and Vanguard Charitable charge an estimated first-year administrative fee of $1,500 (0.60% effective), based on their published tiers as of 2026-08-30; verify current schedules at fidelitycharitable.org and vanguardcharitable.org. Schwab Charitable has no confirmed fee figure for this balance because its full tier table is not publicly available; request current pricing directly from them.

What returns to expect Returns are simply the performance of underlying pools (money market, bonds, equities) net of fees; they are not guaranteed or set by the sponsor. Because any growth inside the account is not separately taxed and increases what you can eventually grant, sponsors generally recommend conservative pools if you plan to distribute funds within a few years rather than chasing higher expected returns that carry drawdown risk.

A topic the corpus doesn’t cover — only two organisations clear the floor, and both carry the unresolved-entity caveat rather than being presented as clean matches:

Q: I want to support charities working on homelessness in rural Alaska

top orgs for facet: 'homelessness in rural Alaska'
  Habitat for Humanity   pct=1.00  cos=0.411
  The Salvation Army     pct=0.96  cos=0.411

Charities matching your interests

For homelessness in rural Alaska, two organizations are available for recommendation, though both require administrative confirmation from your DAF sponsor regarding their legal entity status since they appear as national entities without individually listed EINs: Habitat for Humanity (national entity not individually listed) and The Salvation Army (national entity not individually listed). Both have been screened as eligible candidates.

Habitat for Humanity focuses on building affordable housing using volunteer labor, which aligns with your interest in homelessness; however, note that the organization operates globally across 70 countries rather than being specific to rural Alaska, so you should confirm if a local affiliate serves your area of focus. The Salvation Army runs shelters and hostels for homeless people worldwide and has demonstrated support for vulnerable populations including transgender individuals, which matches your interest in homelessness; however, like Habitat, its national listing means the sponsor must verify the exact legal entity to ensure funds go to operations relevant to rural Alaska rather than international programs.

Where generation beat retrieval. WWF cleared the floor for “animals that are mistreated” at 0.45 — but on human-rights abuse allegations, a different sense of the word entirely. The model read the excerpt and declined:

Q: I want to help animals that are mistreated

• American Society For The Prevention Of Cruelty To Animals (EIN 13-1623829): This organization matches your interest in helping animals that are mistreated by preventing cruelty through field response, legal advocacy, and disaster relief for abused or neglected animals.
• World Wildlife Fund Inc (EIN 52-1693387): While this global conservation group works on wilderness preservation and reducing human impact on the environment, its primary focus is not specifically on rescuing mistreated individual animals; therefore, it only partially matches your interest in helping animals that are mistreated.

The remaining three prompts, in brief:

Table 1

prompt	decomposed	top match	outcome
veterans and their families	2 facets	Wounded Warrior · 0.73	WWP and Red Cross correctly flagged as serving both facets
disaster relief + feeding hungry	2 facets	Salvation Army · 0.67	Both facets answered; correctly reports no organisation serving both
clean water + children’s education overseas	2 facets	MSF · 0.58	Teach For America demoted on ‘overseas’ — a constraint the ranking code never saw

No fabricated EIN appeared in any of the seven, and three consecutive runs of the same prompt produced byte-identical output.

Two of these show the generator adding something retrieval structurally cannot: rejecting WWF on sense disambiguation, and demoting Teach For America for being domestic when “overseas” never entered the ranking code at all.

Do you need a vector database?

Not at this size. Brute-force NumPy and Chroma return identical rankings from identical vectors — with one trap: Chroma defaults to squared-L2, so the collection must be created with metadata={"hnsw:space": "cosine"} or the scores are silently meaningless.

Figure 2: Crossover sits between 10k and 50k vectors. This corpus has 1,279.

Even at a million vectors, 29ms is imperceptible for a single query. The real arguments for a vector database are incremental inserts, metadata filtering and persistence — not speed.

Limits

Corpus coverage is the ceiling. No organisation here has supporting women as its mission — no Planned Parenthood, no Global Fund for Women — which is why that facet tops out at 0.48 and Boys & Girls Clubs ranks second largely on a name match for “Girls”. No embedding model or vector store fixes that; more charities does.

Article shape distorts ranking. World Central Kitchen is a disaster-response food charity but ranks for “feeding hungry people” instead, because its Wikipedia article is mostly an incident timeline rather than a description of its programmes. Real 990 narrative wouldn’t have this problem.

MIN_SIMILARITY = 0.40 is calibrated by inspection, not derived, and is specific to all-MiniLM-L6-v2.

Seven prompts is not an evaluation. There’s no labelled relevance set and no measured precision or recall here, so everything above is an observation rather than a benchmark. That’s the first gap I’d close.

What generalises

The retrieval mechanics port to any RAG chain: decompose compound questions before embedding, normalise scores within each query, floor on absolute similarity so the system can say “no data”, and keep hard constraints in metadata where they belong.

The broader one is about where the bugs actually were. Almost none were in the model or the embeddings — they were in what got handed to the model. A field that was None instead of False. A label sitting too close to a different label. A sponsor missing from a table. Each time, the output stayed fluent and plausible while being wrong, which is what makes this class of bug expensive to find. Cheap deterministic checks on the output caught things that reading the answers did not.

---

Charity text: Wikipedia action API, 30 organisations, 1,271 chunks. Metadata: ProPublica Nonprofit Explorer. Embeddings: all-MiniLM-L6-v2. Generation: qwen3.5:9b via Ollama at temperature 0. Retrieval scores plotted here ship as scores.csv; the query pipeline ships as ask.py.

Nothing here is financial advice. Fee figures are a snapshot dated 2026-08-30 and change without notice.