Written by Ben Bunting: BA(Hons), PGCert. Sport & Exercise Nutrition. L2 Strength & Conditioning Coach.

–

If you are a student, and particularly at postgraduate level, you have to employ a higher plane of critical thinking.

That often means there’s a need for a more thorough investigation of studies and research. In essence, you have to check the quality of the available information, aiming to use any studies that reach the gold standard principles.

There is a requirement to discuss the topical relevance of your choice for citations and references which support your argument, thoughts, and data.

This is where the paper to podium (P-2-P) matrix can help. You can score data, and obtain a more detailed understanding.

Relevance

For instance, is that study valid for a particular substance marketed to men as a way of reducing estrogen if the only studies available with positive outcomes were based on elderly women?

Likewise, is a substance sold as an effective supplement to improve running speed relevant if the performance of those participants in the study were analyzed under laboratory conditions using a treadmill rather than being outside on either a track, road, or trail?

Furthermore, what was their athletic level? Were they university athletes, recreational or professionals at the top of their game? If it is the latter, a tiny improvement may be the difference between a podium finish or not, whereas there may be little change for the recreational athlete.

This is the importance of analyzing data and research.

Just reading the abstract and skipping to the conclusion doesn’t always provide the full picture.

The Paper to Podium Matrix

The ever-growing kingdom of science, especially exercise science, has been proposing nuances for the fitness world – almost every other day!

While some people have subscribed to tens of YouTube channels and newsletters others follow new research and clinical studies in order to learn about the biomechanics of the body.

The plethora of information that the general population or non-science-related people experience is tremendous. And, it is quite difficult to comprehend what is the best for you.

We frequently face situations where one, two, or three different people have three different opinions or suggestions about the same entity which is very much confusing.

Apart from being confusing a considerable amount of information available on the web is misleading or incorrect. Moreover, people, for certain personal gains, may falsely depict a research e.g., in an attempt to sell their products.

To avoid this misleading and false information wreaking havoc in people’s life, researchers and scientists in all the subjects have stressed the scrutiny of every piece of research.

Focusing on a specific niche, that is, exercise science the researchers have now devised a strategy to assess the credibility of various researches by the analysis tool.

This analysis tool is known as the ‘Paper to podium matrix’ and can be effectively utilized to check the credibility of any research or clinical study.

The researchers have divided this analytical tool into nine (9) parts. Each part focuses on various aspects of the study and is used as a reference to obtain a score for the study. These divisions include:

• – Research context
• – Participant characteristics
• – Research design
• – Dietary and exercise controls
• – Validity and reliability of exercise performance tests
• – Data analytics
• – Feasibility of application
• – Risk and reward
• – Timing of the intervention

Each of these aspects of the study is given a score of ‘-2,-1,0, +1, or +2’ depending on the quality. We will be adding the table for summarizing this grading criterion. 

At first sight, these terms appear very strange.

Podium, paper? It doesn’t make any sense, right? However, it is essentially placing that paper, data, study, or research on a podium, much like athletes in a race or competition.

Don’t worry, we are here to make this term familiar to you.

Whether you are a student, a fitness enthusiast, or you’re just curious and want to increase your knowledge, this is your chance of digesting this term easily.

Reading

Now that you have an idea of what ‘Paper to podium’ refers to, let’s dive in.

The realm of science is continually growing and so are its applications.

Science has provided us ‘continually evolving’ humans with opportunities for what seems unlimited growth. A very rapidly spreading branch of science is that of sports and exercise science.

Sports science or exercise science has seen a dramatic increase in the past three decades.

Many people from all over the world are pursuing exercise or sports-related degrees and advancing in this fruitful field.

According to secondary data analysis, there was a 445% increase (fourth greatest)¹ in the bachelor’s degree earned by undergraduate professionals and many are chasing higher and more complex studies in the world of exercise science.

After all, sports is a multi-billion dollar industry. People are obsessed with sports and people spend a lot of money watching, wearing, and even having an active involvement in sports.

Having a larger number of people qualified as sports experts is good but it should be noted that not everyone is qualified enough.

People making claims that a certain exercise or routine can make people healthy should have a solid reason backing their claim, and even verifying someone’s credentials can sometimes be hard to do.

Therefore, if you want to make or claim or test one, it is good to check the research work behind that particular fact; whether the research is credible enough to quote or is just another piece of information published on the internet with no useful implications.

Paper to Podium – Research Context

The research context is where you start.

To start research or clinical studies, we first have to select the subjects or models.

Models could be humans, other animals, or just single living cells being used to test various hypotheses.

In most cases, rodents have been used as they represent humans quite effectively.

However, there are subtle differences between humans and rodents that can produce misleading results. Intuitively, human subjects are preferred.

Another factor to be considered when looking at a study is whether it is ‘mechanistic’ or not. Mechanistic, here, refers to a study that is trying to explore the physiology or mechanism behind a said effect.

For example, studying hypertrophy (increase in size) of muscles following resistance training and the changes in the actin-myosin model (physiology behind muscle contraction) will be considered a mechanistic study.

Alternatively, studies that focus on the effect of certain stimuli on the performance of a person are considered to have no defined mechanism².

Let’s say we have a study to test the ‘effect of carbohydrate mouth rinse and caffeine on high-intensity interval running capacity when carbohydrate is restricted’³.

This study used human participants. As you can see, the study is not focused on the physiological mechanism of the human body.

Therefore, it will get a score of +1. If it had focused on the mechanism, it would have had +2. To give you a better idea, we should also look at other possibilities.

If this study had a human cell in vitro (isolated human cell in a laboratory setting), the score would be 0.

If the study had a rodent model focusing on the mechanism, the study would have -1.

Finally, if the study had rodent models and did not focus on biological mechanisms, the score would be -2.

You can review the table at the end once again to solidify this fact.

Participant Characteristics

As the name implies, characteristics of participants like age, sex, training profile, height, weight, disease status, etc.

Several studies have classified the athletes into categories like amateur, advanced, or elite according to their performance⁴.

Moreover, they can be classified according to the types of sports they play or their food intake^5–7. The purpose of defining people into such categories is that they should fit the context of the study properly.

For a study to be credible, the study should use set criteria and classify participants accordingly.

The least a study could do is mention all their characteristic in clear detail e.g., age group, weight, sex, and health conditions.

For example, studies focusing on the performance of athletes can be focused on a certain population group or certain age group.

You can look at these studies which examined the efficacy of ketone diesters supplements and ketogenic diets on UCI World Tour cyclists⁸ and Olympic race-walkers⁹, respectively.

You can look at the chart at the bottom to score a study. Consider the caffeine study which had a score of +1 in terms of participants’ characteristics. 

Research Design

Any study related to exercise science or nutrition should have various ‘gold-standard’ principles in it.

These principles are considered gold-standard as applying them to the framework of the study makes the study extremely applicable.

This occurs due to the highly reduced chance of error.

Several gold-standard principles are that the study should be:

Randomized

The participant should be selected randomly to reduce the bias and errors

Has Counter-Balanced Repeated Measures

Counter-balanced studies comprise of dividing the sample population into two halves, with one half completing the study in one order and the other half completing it in the reverse order.

Has Crossover Design

Crossover designs incorporate a technique in which the participants go through a washout period. This period makes the effect of any drug or stimulus given to each group, (test and placebo), zero. Then the groups switch places and the experiment is repeated. 

Double-Blind Set-up

A double-blind setup ensures minimal bias and error. This is made possible by making the placebo exactly like the testing drug or stimulus (e.g., ashwagandha pill). Moreover, neither the researchers nor the participants know if they are receiving ashwagandha or a placebo (mimicking ashwagandha). 

This way any study or clinical trial could be made accurate or valid and hence apply to a broader population.

For instance, a study that is a randomized cross-over trial with repeated measures, matched group designs, has a control group; does not have blinding and sample-size calculations are not provided giving a score of ‘0’.

If you change this study from ‘no blinding’ to single-blinding the score changes to ‘+1’ and so on. You don’t have to worry if this is a little confusing and you have to go back and forth to the chart.

But practice makes you perfect and you’ll get the gist of it in no time. Finally, it should be considered that no study design is ever perfect and there is always room for human error.

Alas, you can reduce it to the max by applying these principles.

Dietary and Exercise Control

This is a very confusing part of the ‘paper to podium matrix’, but we shall try our best to serve it in a digestible way.

When it comes to the dietary intake of participants in any study, researchers use two codes to tackle this.

One is called dietary standardization and the other one is referred to as dietary replication. 

The former includes instructing people to follow a particular diet throughout the research.

In the latter, the researchers replicate participants’ regular diet as much as possible.

Both of these codes have positives and negatives.

In the dietary standardization method, problems and errors might arise due to participants’ negligence, unavailability of food, intake of other unknown meals, etc.

Conversely, in the diet replication trial, it is easy to follow as it is the participants’ regular diet but researchers need to be sure that participants are following the diet on each trial.

Also, the variation in the pre-trial meals between different participants i.e., what were their diets before the trials should be noted and considered.

To sum up, researchers should try their best to regulate dietary controls and equate dietary differences as much as possible to draft a solid opinion. 

Regarding dietary control, for example, if a study comprises dietary provisions provided by the researchers, exercise control is mentioned and supported by relevant data.

The relevant data, then, is representing the real-world context, it will get a score of 2+. Once again, this is a hard one to do but you can look at the example of a study related to caffeine. It received a score of +1³.

Validity and Reliability

To understand this easily, you might want to think of yourself in two different trials.

You are asked to run 200 meters against top speed to assess the effect of food intake on running speed.

In one trial your family members (laboratory trials) are looking at you running and in the other setting, you’re competing to get first place in front of 80,000 people (real world).

The latter will give you an adrenaline rush way greater than the former one (as can the potential of weather, temperature, etc). This can change the results of any study. 

 Therefore, when you look at a study’s validity and reliability, you can see how close it is to the circumstance of the real world.

Again, dribbling a football in front of a group of researchers on an enclosed training ground is different than dribbling a ball during a Premier League match. 

Data Analysis and Presentation

Data analysis is a tricky part of any research as it can make it or break it for the significance and usefulness of the study.

Traditionally and largely the statistics include P-value, group means, and standard deviations.

More recently, researchers (in exercise science) have started using magnitude-based inference.

This refers to the likelihood of value being truly related to the real world. The data is defined in regard to confidence limits which are then assessed to be positive and negative.

You can read more about the magnitude-inference here¹⁰.

It is suggested that data analysis should be given importance, especially the presentation of individual participant data should be given importance.

The researchers should point out and refer to individual numbers and provide the exact range of results as it could talk more about the variability and validity of the study.

For example, a laboratory study identified that mean exercise capacity was raised by 60 min with high pre-exercise muscle glycogen concentration versus moderate pre-exercise muscle glycogen concentration.

However, the individual results ranged from 4-113 min¹¹.

Generally, any research or trial that has a p-value of <0.05 (5 percent) and its confidence limits never crosses the value of ‘1.0’ can be considered statistically significant.

The lower the -value the higher the significance of the results and if the confidence interval (limits) does not touch the number ‘1.0’ the study has some actually credible outcome. 

Feasibility of Application

 As the name suggests, it is the measure of the ease of carrying out any study in real-time.

Can it really incorporate humans and come up with credible results without burdening the participants, researchers, ethical committees, and financial teams?

The following three aspects of the study are considered to be part of the feasibility of the research.

They are budget limitations, rise versus reward, and the time available to test strategy before peak performance becomes necessary. 

Budget Limitations

This is pretty simple.

Budget is the first factor in deciding whether it can be carried out or not.

Research requiring extensive biopsies, very expensive laboratory equipment, and going on for years will require a high budget.

In such cases, these studies should be helpful to the general community in such a way that it indefinitely gets funded.

In exercise science, this is usually hard as science or nutrition-related topics are not solving an imminent threat to mankind (e.g., COVID-19 vaccination research and trials received grants worth billions of dollars¹²).

Similarly, the administration or use of caffeine is cheaper than other products like beta-alanine or carnitine and, hence, more feasible to carry out.

Furthermore, the gold standard for assessing calorie expenditure is to use doubly labeled water, however, this is very expensive.

Therefore, it may be used by university researchers that are funded by a top-flight sports team, but is unlikely to be used in experiments that may not provide a high return on the initial investment.        

Risk versus Reward

For athletes to perform in a study, it is important the risk and rewards are balanced.

It is preferable that the athletes get rewarded by the study (e.g., an increase in stamina) but this is not the case every time.

The potential health benefits or risks, effects on the performance, and potential anti-doping violations (i.e., providing athletes with illicit drugs e.g., steroids) should be kept in check.

Adverse effects should be kept minimal and mentioned clearly in the research.

Changing diet before a competition (to ketogenic), for instance, is riskier than just carbohydrate ingestion or mouth wash and is more feasible.

Time of the Study

The time of study also gets a score.

It should be considered whether a study is carried out at an appropriate age, the time is suitable for an athlete and dosing the athlete, and whether time from the competition is sufficient to assess the full effects of the strategy.

If not so, there could be issues like non-compliance from the athletes (participants), incomplete data, and overall insignificant results.

For a detailed view of the tables scoring and their examples please visit the links below:

To expand the Paper to Podium scoring table, please visit:

https://link.springer.com/article/10.1007/s40279-018-1005-2/tables/2 (scoring table)

To expand the paper to Podium matrix table, please visit:

https://link.springer.com/article/10.1007/s40279-018-1005-2/tables/3 (application example)

Paper to Podium Conclusion

The paper-to-podium (P-2-P) matrix gives us an opportunity to follow the best research knowledge out there.

Instead of following an Instagram, Facebook post, or TikTok reel, there should be a focus on gathering, following, and preaching credible and accurate knowledge.

This matrix provides scientists, researchers, students, and influencers to probe nutrition and exercise science in a way that is accurate and gives the general audience (population) a chance to understand sports science and avoid misleading information.

Simply stating the paper to P-2-P matrix score can be a big relief for many people trying to enhance their knowledge and understanding.

References:

1.        Nuzzo JL. Growth of Exercise Science in the United States since 2002: A Secondary Data Analysis. https://doi.org/101080/0033629720201736106. 2020;72(3):358-372. doi:10.1080/00336297.2020.1736106

2.        Close GL, Kasper AM, Morton JP. From Paper to Podium: Quantifying the Translational Potential of Performance Nutrition Research. Sports Medicine. 2019;49(1):25-37. doi:10.1007/S40279-018-1005-2/TABLES/3

3.        Kasper AM, Cocking S, Cockayne M, et al. Carbohydrate mouth rinse and caffeine improves high-intensity interval running capacity when carbohydrate restricted. Eur J Sport Sci. 2016;16(5):560-568. doi:10.1080/17461391.2015.1041063

4.        Jeukendrup AE, Craig NP, Hawley JA. The bioenergetics of World Class Cycling. J Sci Med Sport. 2000;3(4):414-433. doi:10.1016/S1440-2440(00)80008-0

5.        Naughton RJ, Drust B, O’Boyle A, et al. Daily Distribution of Carbohydrate, Protein and Fat Intake in Elite Youth Academy Soccer Players Over a 7-Day Training Period. Int J Sport Nutr Exerc Metab. 2016;26(5):473-480. doi:10.1123/IJSNEM.2015-0340

6.        Anderson L, Naughton RJ, Close GL, et al. Daily Distribution of Macronutrient Intakes of Professional Soccer Players From the English Premier League. Int J Sport Nutr Exerc Metab. 2017;27(6):491-498. doi:10.1123/IJSNEM.2016-0265

7.        Anderson L, Orme P, Naughton RJ, et al. Energy Intake and Expenditure of Professional Soccer Players of the English Premier League: Evidence of Carbohydrate Periodization. Int J Sport Nutr Exerc Metab. 2017;27(3):228-238. doi:10.1123/IJSNEM.2016-0259

8.        Leckey JJ, Ross ML, Quod M, Hawley JA, Burke LM. Ketone Diester Ingestion Impairs Time-Trial Performance in Professional Cyclists. Frontiers in Physiology. 2017;8(OCT):806. doi:10.3389/FPHYS.2017.00806

9.        Burke LM, Ross ML, Garvican-Lewis LA, et al. Low carbohydrate, high fat diet impairs exercise economy and negates the performance benefit from intensified training in elite race walkers. The Journal of Physiology. 2017;595(9):2785. doi:10.1113/JP273230

10.      Barker RJ, Schofield MR. Inference about magnitudes of effects. International Journal of Sports Physiology and Performance. 2008;3(4):547-557. doi:10.1123/ijspp.3.4.547

11.      Impey SG, Hammond KM, Shepherd SO, et al. Fuel for the work required: a practical approach to amalgamating train-low paradigms for endurance athletes. Physiological Reports. 2016;4(10):e12803. doi:10.14814/PHY2.12803

12.      COVID-19 Funded Research Projects | NIH COVID-19 Research. Accessed May 4, 2022. https://covid19.nih.gov/funding