How we test
Most of the numbers that describe a product already exist somewhere — on a spec sheet, in a lab report, in somebody else’s measurement — and they routinely disagree with each other. The work here is reconciling them: reading every source we can find, printing both figures where two labs contradict each other, and scoring what survives against five weighted criteria we never adjust by hand.
When the measurements disagree
They disagree constantly. A manufacturer’s spec sheet is a claim made under conditions that suit it. A lab result is a measurement under one specific set of conditions, and two labs testing the same product to different standards will report different numbers — both of them correct, neither of them the whole answer.
Rather than pick one, we do this:
- Both numbers get printed. Where two sources contradict each other, the review shows each figure and says who measured it. We do not average a disagreement into a single tidier-looking number, and we do not quietly take the flattering one.
- The conditions get named. A difference is usually explained by how the test was run, and saying so is more useful to you than the number on its own.
- We say which one we would act on. Then we score against that, and the score breakdown shows the figure it used.
This is the part of a review you cannot get by reading any single source, because no single source published it. It is also why the rubric below is worth anything: a weighted average of numbers nobody reconciled would just be arithmetic on other people’s claims.
The rubric
Each criterion is scored 0–10 and carries a fixed weight. The overall score is computed from those numbers in code, which is why the breakdown on a review always adds up to the headline figure.
Core performance
3×Whatever the product exists to do, measured rather than described. Brew temperature stability, typing noise in decibels, panel uniformity — a number we can reproduce.
Value
3×Performance per dollar against everything else we have scored in the category, not against the manufacturer's list price.
Build quality
2×Materials, tolerances, and what owners and long-term reviewers report degrading. We name every part that gets called cheap.
Ease of use
2×Setup, daily operation, cleaning and maintenance. Whether the annoyances fade with use or stay.
Category-specific criterion
2×Steam power on an espresso machine, software on a keyboard. Named explicitly on every review so you can see what it covers.
Hands-on reviews and research reviews
A review here is one of two kinds, and each one says at the top of its testing section which it is. Today every published review is a research review; hands-on reviews are added as we get units on the bench, and the first ones will be of products we already own.
A hands-on review reports our own measurements. It shows how many hours we spent with the product and how many rivals we ran against it, and the methodology paragraph describes the instruments we used.
A research review is a synthesis of the manufacturer’s published specifications, independent measurements from labs and reviewers who did have the unit, and recurring themes in owner reports. Those reviews say plainly at the top of the testing section that we have not handled the product, and they list every source we read so you can check any figure yourself. We do not publish a number in a research review that we cannot point at a source for.
The rubric below applies to both. A research review is scored on the same five criteria with the same weights, and it carries the same requirement to name real downsides.
Where the money comes from
This site is meant to be funded by affiliate commission: you buy something through one of our links, the retailer pays us a percentage, and you pay exactly the same price you would have paid anyway. We have not been accepted into Amazon’s Associates programme yet, so right now the links carry no tracking code and the site earns nothing at all.
That creates an obvious conflict of interest, so here is how we handle it:
- Scores are set before links are added. The scoring criteria live in the review’s source file and are filled in during testing. Product links are generated automatically from a product code at build time, long after.
- Commission rate never influences placement. We do not know, and do not look up, which products pay more before deciding a ranking.
- Every review lists who should not buy it. A review with no downsides is an advert. If we could not find a real flaw, we had not read enough to have an opinion worth publishing.
- We do not accept payment for reviews, sponsored placements, or edits to published scores.
Read the full affiliate disclosure.
Why we don’t show prices
You will not find a price anywhere on this site. Two reasons. The practical one is that prices on Amazon change constantly, and a cached number from three weeks ago is worse than no number at all.
The other is that Amazon’s Associates terms only permit displaying pricing obtained live through their Product Advertising API. Rather than show you a figure we would have to caveat into uselessness, we send you to check the current price yourself.
The price tracker is not an exception to that. Every number on it is one you typed in after checking a listing yourself; it is saved in your browser, never uploaded, and never shown to another reader. We publish no price of our own, there or anywhere else.
What we buy and what we keep
We buy test units at retail wherever possible. When a manufacturer supplies a unit on loan, the review says so at the top. Loaned units are returned; purchased units are kept for long-term follow-ups, which is how we can report on what breaks after six months.