A clearer way
into research.

01 / THE IDEA

Why this project

A paper can be freely available and still be difficult to understand. Technical language, unfamiliar methods and conflicting findings make it hard to work out what a study actually tells us—especially outside our own field.

I started What Does Research Say to make that first step easier. The goal is a short, readable explanation with the original evidence close by: what was studied, what was found, where findings differ, and what remains uncertain.

02 / THE EXPERIENCE

Three research workflows

The experience starts with the reader’s question. Each workflow has a different job, while sharing a consistent structure for explanations, sources and limitations.

1. 01

   Explore a topic

   Bring together up to ten relevant papers from the local library. Explain the ideas, compare findings and keep differences in context.

   “How does maximum entropy help exploration in reinforcement learning?”
2. 02

   Understand a paper

   Start with a selected paper or a PDF. Unpack its question, approach, findings and limitations, with citations restricted to that paper.

   “Explain Trust Region Policy Optimization.”
3. 03

   Check a claim

   Assess an exact statement against the available research. Keep the qualifications with the verdict, and say when the evidence is insufficient.

   “Is this result guaranteed to hold in every setting?”

All three workflows are implemented locally, from input and retrieval to saved results. The evaluation section documents the remaining answer-quality work.

The React interface includes light and dark themes, a searchable library, collection and topic filters, source passages, saved runs and Markdown export. The local version also shows research-job progress and accepts PDF uploads. The public version offers nine prepared examples and the paper catalogue.

03 / THE ENGINEERING

Ingestion, retrieval and research jobs

I built a retrieval-augmented generation (RAG) pipeline: find relevant passages first, then give that evidence to a local language model. The work spans document processing, hybrid search, background jobs, model integration and the reading interface.

1. Make the documents traceableParse PDF and full-text XML, retain metadata and page or section references, and split text into overlapping passages. File hashes and source identifiers help prevent duplicate imports.
2. Search for words and meaningCombine SQLite FTS5/BM25 keyword search with MiniLM embeddings and cosine similarity. Reciprocal rank fusion combines the result rankings; collection filters and per-paper passage limits keep retrieval bounded.
3. Coordinate the work locallyFastAPI manages research jobs while Ollama runs Qwen3.5 on the laptop. Saved results and versioned caches allow earlier runs to reopen without repeating unchanged work. Generation uses no cloud model calls.
4. Keep evidence inspectableCopy quotations directly from stored source text. Check citation identifiers, numbers and known scope errors, then apply a model support check. Failed checks can leave a partial result or abstention; they do not guarantee that every remaining statement is correct.

04 / BUILDING THE LIBRARY

An indexed research library

The first local version used eight reinforcement-learning papers. I expanded it to 150 papers across medicine, cancer research and AI, then added 50 papers in each of 20 further fields. The current library contains 1,150 distinct full texts across 23 collections.

The completed audit found no missing originals, duplicate normalised titles, duplicate DOIs or papers without passages. All 127,770 passages had matching search records and embeddings, with no pending indexing work.

Source checking also meant correcting licence metadata and replacing unsuitable documents. The collections remain a selected sample of accessible research. Their publication-volume groupings describe one OpenAlex comparison, rather than a universal ranking of research importance or evidence quality.

Recorded corpus verification: 19 September 2026. These counts describe coverage and indexing, not the accuracy of generated answers.

05 / DEVELOPMENT SO FAR

Working capabilities

The reading experience, paper library and local pipeline are built. The current release separates what visitors can explore today from the experimental generation work running locally.

Product interface & public demos

Built

Library import, indexing & search

Built

Local research jobs & source inspection

Built

Complete, source-faithful generated answers

In evaluation

Public live analysis & user accounts

Planned

Milestones describe implemented capabilities. Generation quality has separate acceptance criteria.

06 / TESTING THE ANSWERS

Generation quality: what the checks found

I compared Qwen3.5 4B and 9B models and read the generated text against its supporting passages. A 4B paper explanation passed length and citation checks while still making an unsupported claim about guaranteed improvement. The same model’s audit accepted it.

The larger model improved some wording but did not resolve the quality problems. The recorded 9B development checks show where the product still falls short:

Source: project model-comparison notes. These are individual development cases, not a held-out benchmark or typical response-time estimates.

The next iteration targets complete, source-faithful explanations and shorter local response times. Tables, figures and equations remain outside reliable coverage, and the prototype does not establish scientific consensus or independently verify retractions.

07 / WHAT COMES NEXT

Next steps

This project has brought together product design, document engineering, retrieval and local AI. The most useful lesson so far is that more papers and a larger model do not automatically produce a better explanation. Progress needs to be measured in what readers can understand and verify.

1. Build an independent evaluation setUse human-reviewed questions, expected evidence passages and clear criteria for accuracy, completeness and plain-language usefulness.
2. Improve evidence selection and repairMatch statements to the passages that support them, preserve qualifications, and repair unsupported drafts instead of simply dropping paragraphs.
3. Reduce the waitEvaluate better retrieval and reusable paper notes against the same quality checks, so speed improvements do not come at the expense of fidelity.
4. Open fresh analysis when quality is readyMove towards a hosted service with accounts and durable research jobs once measured quality justifies that next release.