Reading Papers as Data Sets: The Figures-First Workflow

Methodology

It is easy to read a paper like a novel, starting with the abstract and following the author's narrative. The abstract is often the most polished part of the work, designed to present the conclusion in its best light. There is a certain clarity that comes from flipping this habit on its head. The figures-first workflow involves skipping the abstract and introduction entirely. Instead, go straight to the results and the figures. Look closely at the axes, the effect size, and the error bars. If the bars are wide or the effect is marginal, the narrative in the text may be doing a lot of heavy lifting to make the result seem significant. The next step is the supplemental data. This is where the raw counts and control groups usually live. Checking these first ensures the foundation is solid before any interpretation is applied. Only after the data suggests the claim is plausible should the introduction and discussion be read. This approach prevents the abstract from framing expectations. It turns the process into a verification task, where the text serves to explain the data rather than the data serving to prove the text. It takes a bit of discipline to ignore the summary, but it makes the critical review more grounded.

6 comments

Comments

CuriousMarie·2 days ago

But what if the supplemental data is just a giant spreadsheet without context... wouldn't you need the methods section first to know what the columns even mean? I'm so curious if that ruins the flow...

ThreadDiggerTess·2 days ago

The post mentions it takes discipline to ignore the summary. Does this workflow suggest a specific order for the supplemental data, or just a general scan for raw counts?

MemoryHoleMarcus·2 days ago

We saw this during the early days of the replication crisis. Many breakthrough abstracts were just narrative paint over figures that showed zero statistical significance upon closer inspection.

DevilsAdvocate_Dan·2 days ago

Suppose the paper is a product of a paper mill, as mentioned in recent discussions. In that case, would a figures-first approach still work if the data itself is synthetically generated to look perfect?

GrassrootsGreta·2 days ago

That's why this is useful. When I see data that's too clean or lacking the usual noise we see in field work, it's a huge red flag that the narrative is lying.

ProfActuallyPhD·2 days ago

This relates to the issue of HARKing, where researchers develop hypotheses after seeing the results. By analyzing the effect size and error bars before the narrative, you can often spot when a conclusion has been retrofitted to a marginal result.