Benchmark: peer groups and percentiles

Summary

A score of 62 means little on its own. Is that good for a structural model of a hospital? For a bridge? The Benchmark page answers that by placing the file among comparable models from the forensicBIM reference corpus: thousands of real IFC files, scored with exactly the same checks as yours.

Peer groups

Each file is placed in one peer group, based on the facility types it contains:

A file with several facility types is compared with buildings first, then railways, then civil infrastructure. The other groups it qualifies for appear as a switch, so you can compare with those too.

Infrastructure saved as a building

Older IFC versions had no bridge or road entities, so infrastructure was often exported inside an IfcBuilding. When a file's only facility is a building, but the content is clearly infrastructure, it is compared with civil infrastructure (or railways, when rail elements dominate) and the page says why. Signs we look for: a substantial share of IFC 4.3 infrastructure elements, an alignment or infrastructure parts, a civil design tool as the authoring application, or words such as bridge, tunnel or culvert in the project, file or facility names, unless the model clearly has building floors.

Narrowing the comparison

Within the peer group you can choose who to compare with:

• All models in the group (the default).
• Similar size: small (under 1,000 elements), medium (1,000 to 10,000) or large (over 10,000).
• Same discipline: architecture, structural, MEP, or mixed when no discipline makes up at least 30% of the objects.

A comparison needs at least 20 models. Options with fewer are greyed out, and if the whole peer group has fewer than 20, the page shows Not enough comparable models rather than a ranking.

What a percentile means

The 70th percentile means the file scores higher than 70% of the models it is compared with. For measures where lower is better, such as the share of empty values, proxies or the geometric weight, the page turns this round, so a higher percentile is always better. The number of elements is neither good nor bad, so for it the page says larger than instead.

Ties are counted half, so a file that scores the same as a large part of the corpus lands in the middle of that part rather than at its top or bottom.

What the Benchmark page shows

1. Overall quality and Forensic BIM score™: the percentile for each, the score, and the median of the group.
2. Where this model sits: a histogram of the quality scores in the group, with this file's bar highlighted.
3. Overall vs. Forensic BIM score™: every model in the group as a dot. The dashed line is the limit the data puts on the quality score (see Scoring versions); models on that line were held back by their data.
4. Stands out against its peers: the three measures where the file is furthest ahead of the group, and the three where it is furthest behind.
5. Four tables: the five dimensions, the four Forensic BIM score™ factors, the information measures (materials, quantities, classification, typing, property values) and the model structure measures (proxies, containment, geometric weight, size). Each row shows the value, the group's median and a strip: the band from the 10th to the 90th percentile, the middle half as a box, the median as a tick, and the file as a dot coloured by its ranking.

The Overview shows the headline: the percentile, the size of the group and the benchmark release.

The reference corpus and releases

The benchmark is not calculated live. We score the reference corpus offline with the same code as the report, remove duplicates and files without elements, and publish the result as a benchmark release. A release is a frozen snapshot, so your ranking only changes when a new release is published.

A release stores, per peer group and per measure, the distribution of scores (51 points from the minimum to the maximum), plus the histogram and the dots in the scatter. It contains scores only: no file names, project names or owners. You can keep your own models out of benchmarking in your settings.

The release box on the Benchmark page names the release, when it was built, how many models it holds and which scoring version it uses. buildingSMART validation results are not part of the benchmark.

"Rankings may be off until the benchmark is rebuilt"

You see this notice when the release was built with a different scoring version from the one your report uses. The two sets of scores were calculated with different rules, so the percentiles are only indicative until a new release is built. See Scoring versions and what changed.

Frequently asked questions