Every product on this site goes through the same 16 tasks, on the same fictional HVAC operation, on the same devices. Each task ends in a recorded result you can open: the plan we used, the device, the date, what happened, and our own screenshot or recording. Reviews, comparisons and tools are built from those records, not the other way round.
The operation we test with
We run every product with Ridgeline Comfort Heating & Air, a fictional residential-first HVAC shop with some commercial work: an owner, an office manager and three technicians; 25 customers, including a property manager who pays for six rental units at different addresses and two landlords with two properties each; 40 pieces of equipment attached to locations, from furnaces to two rooftop units; a 30-item price book with good, better and best replacement packages; 12 maintenance members on two visits a year; 20 service calls, one three-visit replacement and one six-stop maintenance route. Every name, address, phone number and email is invented. The dataset is versioned, and each task result names the version it was run with.
The accounts we use
We sign up with our own identity, FieldServiceTested Lab, and our own email. We never present ourselves to a vendor as an HVAC business. If a trial requires a sales call, or a task needs a plan we did not buy, that is a finding: we record it as not available on plan with the plan name. Payments are tested in test or sandbox mode only.
The 16 tasks
| Task code | Role | Task | Pass criteria |
|---|---|---|---|
| T-01 | owner | From signup to the first job scheduled, including importing the 25 customers from CSV | Customers imported with their service locations; first job scheduled |
| T-02 | office | Import the 30-item price book from CSV, including options | Codes, descriptions and good/better/best options preserved |
| T-03 | office | Commercial customer with 3 locations and a separate payer (Cedar Lane) | A job at one location bills the payer, with no duplicate customer |
| T-04 | field | Equipment history: see every visit for one rooftop unit | History is visible per unit and per location |
| T-05 | office | 3-option quote (AC replacement); the customer approves one option online | The approved option becomes a job without retyping |
| T-06 | office | Quote → job → invoice; the tech adds a part in the field | The invoice carries the line items plus the added part |
| T-07 | office | Schedule 8 jobs across 3 techs; an emergency call bumps one | Rescheduled, and the customer is notified automatically |
| T-08 | office | Multi-stage replacement over 3 visits | Visits sit under one job; progress invoicing or a single invoice works |
| T-09 | field | On a phone: open the job, see equipment history, add photos, notes and a checklist, capture a signature; repeat offline | Everything syncs after reconnecting, with no data loss |
| T-10 | field | Take a payment (test mode) or send a payment link | The payment is recorded against the invoice |
| T-11 | office | Maintenance plan with 2 visits a year and recurring billing | Visits are generated automatically; billing recurs |
| T-12 | owner | Job costing for the multi-stage job | Labor + materials vs price, visible without spreadsheets |
| T-13 | owner | QuickBooks Online sync of an invoice and a payment (sandbox) | Amounts and customer match in QBO |
| T-14 | owner | Export and leave: customers, jobs, invoices, attachments; check the phone-number port-out policy | Completeness vs the fixture (% of fields and attachments exported); policy documented |
| T-15 | owner | Support: 2 standard questions via chat or email | Time to first useful answer; answer quality (0–3) |
| T-16 | owner | Real cost for 1, 3 and 8 techs, including every add-on needed to pass T-01 to T-15 | Figures with source and date; paywalled features listed |
Tasks are versioned. Changing a step changes the version, and results always state which version they were run against. A result recorded against an older version is flagged as such until we retest.
Devices and recording
Web tasks run in a Chromium browser on macOS, driven with AI assistance under the operator’s direction and recorded as the result states. Field tasks run on the phones listed in each result. We record the app version, plan, device, operating system and network for every run. Screenshots are cropped to the task; flows longer than three steps are recorded on video. Time is measured in minutes from the first step to the pass criteria being met. Vendor staff names and anything that is not part of Ridgeline are redacted.
What a result means
Pass: the pass criteria were met. Partial: some of the criteria were met, or all of them with a workaround. Fail: we followed the steps and the criteria were not met. Not possible: the product offers no way to do it. Not available on plan: the feature exists on a plan we did not test; it is reported separately and never scored as a failure.
Evidence levels
Every product claim on this site carries one of four labels. Tested: we ran it ourselves and attached our own evidence. Demo: we saw it in a vendor-led demo but did not run it. Documented: it comes from the vendor’s documentation or pricing page, with a link and a date. Unverified: we could not confirm it. A claim can only be labeled tested when it points to a recorded run with our own screenshot or recording; vendor images never back a tested claim.
Three dates, kept apart
Price checked is the date of the newest price observation shown on the page. Last tested is the date of the newest task result shown. Last edited is the last substantive editorial change. Fixing a typo does not move any of them, and none of them moves without a real change behind it.
How scores work
Each task result maps to a score: pass 3, partial 2, fail 0, not possible 0. Not available on plan is excluded from the score and reported on its own. Tasks are grouped into dimensions (setup, customer and equipment model, quote to invoice, scheduling, field app, payments, maintenance agreements, reporting, integrations, data export, support, total cost), and each operation profile weights those dimensions differently, because a one-truck shop and a commercial maintenance company do not need the same things.
| Dimension | Owner-operator: 1 tech, owner does office work, phone-first | Small residential shop: 2–10 techs, 1–2 office staff, service + replacements | Mixed service + projects: Quick service calls plus multi-visit jobs over weeks | Commercial-leaning: Maintenance contracts, multiple sites per customer, payer ≠ location |
|---|---|---|---|---|
| Setup & onboarding | 3 | 2 | 2 | 1 |
| Customer / location / payer / equipment | 1 | 2 | 2 | 3 |
| Quote → job → invoice | 3 | 3 | 3 | 2 |
| Scheduling & dispatch | 1 | 3 | 3 | 2 |
| Field app | 3 | 3 | 2 | 2 |
| Payments | 3 | 2 | 2 | 1 |
| Maintenance agreements | 1 | 2 | 2 | 3 |
| Reporting & job costing | 1 | 2 | 3 | 3 |
| Integrations | 2 | 2 | 2 | 2 |
| Data export & portability | 2 | 2 | 2 | 2 |
| Support | 2 | 2 | 2 | 2 |
| Total cost | 3 | 2 | 2 | 1 |
We publish bands, not a single number: Strong, Mixed or Limited per dimension and per profile. A dimension whose tasks have not all been run shows insufficient evidence rather than a guess. If we ever publish a single figure, the method page will say how it is computed and why that precision is justified.
When we retest
When a product ships a major version, when its prices or plans change, when a reader reports something we got wrong, and, as a target, every six months.
What we do not cover
Equipment sizing (Manual J), gas work, electrical code, safety checklists and contract templates. Those need a licensed professional, not a software review.
Limits of this method
We test US plans and prices only. We test during trials and short paid periods, not over years of daily use. One fictional operation cannot reproduce every shop; the tasks are chosen to cover the points where field service software most often breaks down in practice (importing data, quotes with options, customers versus locations versus payers, equipment history, mobile work, integrations, leaving). When a result surprises you, open the evidence and tell us if we missed something.