Science v Pseudoscience

---

My first job was at the National Physics Lab calculating the statistics during calibration of national force standards machines using the build up technique. This involves calibrating multiple force measuring devices using the most accurate machine possible (120 tonnes to 0.01%) and using these in parallel to measure a higher force to the best possible accuracy. It was laborious, mathematically rigorous and the polar opposite of modern commercial aero testing.

In commercial aero testing multiple fundamental rules of experimental science are broken. Confusing precision (meaningless beyond experimental uncertainty) and accuracy (the goal).

Single run measurements.

A single measurement is very crude, it's barely even science. If you are trying to measure something at the highest resolution possible it will take multiple runs and careful understanding of experimental error to know whether what you measure is a strong possibility of a real difference. They way around this is to accept that measuring large differences is far more accurate than trying to measure every tiny change using what is a relatively crude (1-1.5% typically) measuring device.

Uncertainty

One of the first lessons in degree level experimental science is that you must add together all sources of uncertainty. For power meters this is referred to as their accuracy, there is also the uncertainty in speed measurement, air density and importantly the random variation due to how the human sits on the bike. When measuring a riders drag the rider variation and power meter are the dominant factors, speed sensors are surprisingly accurate over a typical measurement period and air density does not change quickly indoors (though it can outdoors at dawn or on a day with broken cloud).

There has been research done to measure how consistent from one measurement to the next power meters are. This is critical when measuring drag in an aero testing capacity as the accuracy of the absolute power (from which CdA is calculated) is less important than the reliability of the differences found between individual runs. This is where we find out that the accuracy figure quoted for a power meter is this self-consistency figure, for an SRM 1% device it is pretty much 1%. Absolute power results are quite a bit less accurate than this.

https://roadcyclinguk.com/how-to/fitness-nutrition/power-meter-really-telling-think.html

This becomes the starting point for calculating the measurement uncertainty of any run, and where commercial aero testing starts running into issues by quoting CdA to four figures (e.g. 0.1854) in results summaries. 1% on a typical CdA is 0.002 so giving more significant figures than this is completely incorrect. Even the SRM Science or Verve power meters (0.5-0.6% devices) can give accuracy to just 0.001 in CdA if you wrongly ignore all other sources of uncertainty.

It does not matter what units are used to calculate CdA, Watts or kJ, the device is accurate to its stated figure and giving precision beyond this is not science in any way at all.

Failing to accept the limitations of power meters means that decisions get made based on differences smaller than the measurement uncertainty, in this situation a coin toss is about as accurate.

Traceability

Measurement science relies completely on the facility to trace a measurement back to a national standard that is reliable to its specification. This applies to the scales at a fruit shop and the gauge in a petrol pump. We pay for precision science in wind tunnels and aero testing and get quoted precision numbers for our CdA with no reference to a proven accuracy of these techniques. Limitations of measuring the CdA of a human on a bike include;

Accuracy of measuring devices (primarily power or force, but also weight, temperature, air density, tyre pressure)

Variability of subject (this is why research is often done using a dummy)

Random error (reduced by repeated independent measurements, take the bike off the tunnel rig and remount it)

Compounding all these factors is laborious but will produce a total measurement uncertainty that is greater than that of the measuring device alone (more factors = more uncertainty).

Against this would appear to be the absolute fact that aerotesting and wind tunnels have made people very aero. They have without doubt. Having an indoor velodrome immediately accessible is a fantastic resource, as is having access to a precision wind tunnel. Being accessible means being able to use it a lot, take more measurements and understand fully the accuracy of the process as it relates to measuring you. You measure the same position many times and it soon becomes obvious what magnitude of variation is  significant. Spending many hours measuring changes is a good scientific process, assuming it is done following good practice. Attempting to do the whole process from choosing one of eight helmets to making positional changes in 2 hours is not. When we pay £750 or £1,000 for an aero testing session what we want is all the information available, what we actually need is reliable information that is useful in making us faster. These two do not overlap much at all until all the major gains have been made. If a rider is not already well optimised there are by definition big gains to find which are rarely found by making small changes in a hurry. Science takes time and  accuracy takes multiple measurements.

Another aspect of traceability is measuring the outcome. I can claim 91% of clients improve their Spindata ranking but an average performance is irrelevant to an individual beyond it's advertising value. Committing to verifying every individuals improvement in races is done with the understanding that many cases may be straightforward but all cases require a solution.

Any client should check if they are moving up the ranking system of their choice, be it local club TT or National Championships. It is not straightforward to assume that a velodrome or tunnel CdA will transfer perfectly to a race. Verifying performance in the real world is the object of the whole process yet often the relationship has ended before this has happened.

Velodromes

For an experienced rider with good experimental technique and plenty of time an indoor velodrome is a great resource. Low rolling resistance, warm temperatures and no wind make for high speeds and CdA measurement as accurate as it can be within the limits of the power meter. Plus the rider is actually having to generate power and the ability to do this in a new position can be understood as well as the drag difference.

The downsides of velodromes are mostly for novices due to the steeply banked corners taken at high speed. Simply measuring this over ten runs will show an improvement in CdA as the rider gets better at riding the track but nobody is going to want to pay a testing fee to measure the same thing over 15 runs to understand and reduce rider variability. This is science and it is laborious. The other problem is fatigue, even on easy big outdoor velodromes with little banking the rider will eventually fatigue. A novice can fatigue on the first couple of runs due to lack of stamina in the position, generally seen as CdA rising during the run. Even experienced riders only have around 60 min of run time before the tell tale rise in CdA during a run, after which everything tends to blur significantly and further measurement becomes pointless. The solution again is to take your time, make good measurements on a fresh rider and repeat when ready.

Wind Tunnels

A good wind tunnel is very accurate and is where most of the developments in aero kit have come from. A bad wind tunnel has hidden variabilities and inconsistent results. One of these lasts a lot longer than the other as a measurement service. Neither measure drag of a moving rider. How a rider moves while pedalling on an actual road will affect CdA differently for different riders. More so than for a velodrome it is vital to measure what is happening in races before and after a tunnel session. If form is very stable Spindata gives an approximate answer but it is a blunt tool compared to measuring CdA on a well known course under race conditions. (Well known is not straightforward as it involves understanding the traffic effect which decreases apparent CdA due to traffic flow and rolling resistance, both variable. Plus the wind effect relative to course geometry can be quite complicated, best solved by waiting for a calm day. Science takes time..)

Independence

Science should be an independent view. If a rider chooses a consultant that is also a vendor of equipment they have already made a choice as to whose equipment they are going to buy, and are paying to do so, this is not independence. Independence will tell you which integrated extensions are geometrically superior if they fit, heavy or which are a relative bargain. It will tell you that in these races front wheel choice matters much less than is advertised, or not as the case may be. It will tell you that measuring one wheel or helmet 2W faster than another using a power meter is within experimental variation and is not a significant difference.

Patience

Being in a rush is natural but limits the range of outcomes to those close by. I have had clients over a wide range of abilities and common factors that improve results are continued communication and trust. Communication helps me to understand the specifics of what is happening physically and helps the rider to understand how to benefit most from the setup.  Trust means not doing one or two races and starting to make DIY changes. These can introduce too many variables to control and may unbalance the fit completely. When changes are necessary, discussing them and their likely impact is vital. Sometimes a rider may repeat a ride on the same course with the changes made in-between and a clear difference can be measured, but conditions rarely remain ideal for long so it will normally take a few races to really verify what is going on. Frequency of racing is frequency of measurement and more measurements means more accuracy. It is not essential to race every weekend to benefit from the fit but the rider will then need more patience to fully assess the outcome. It can also take some race time for all the gains to become apparent as new techniques are better adopted releasing more speed.

It has been very useful to maintain an ongoing relationship with some riders beyond the commitment to find a significant improvement, this involves checking in regularly on matters that may affect performance while being short of an intensive coaching role. Kit selection, future position development and CdA measurement have been part of this with the goal of continued progress.

Summary

A snapshot of a scientific method is not scientific on it's own. Verifying results in the real world is the fundamental goal.