Demonstrating Impacts

With the growing pressure on the World’s forests and increasing demand for undisputable proof of positive impact, the ability to verify and communicate outcomes of certification is central to FSC. 

It is also important for FSC to understand the effects FSC-certification has on the ground to facilitate future learning and to enable informed and impact-oriented decision making as part of FSC developments, e.g. in the standard setting processes.

FSC has been collecting and analyzing a wide range of scientific studies about FSC over the years as part of monitoring and evaluation activities. To increase internal and external access to the relevant information they contain, we have decided to collate single results from scientific studies to provide a comprehensive but easily accessible list of evidence of the outcomes of FSC-certification.

The scientific community produces a great diversity of studies about the FSC-system, covering various topics and using different scientific methodologies.

For the Impact Dashboard, we decided to focus our selection on studies comparing forests and their values in FSC-certified and uncertified forests. These are the most relevant research papers to allow FSC staff and stakeholders to learn from the effects of FSC-certification.

Learning from science requires understanding how to interpret results and what can be learned. This is closely dependent on the type of research design used.

Researchers can use various research designs in the field. They are not all equally robust and do not provide the same level of scientific evidence to draw conclusions about the effect of certification.

We have used an evidence typology which classifies studies according to their research design and help define whether results can have been caused by or are just correlated with certification. This evidence typology has been initially developed by the VIA initiative and is currently used as a guiding methodology for the Evidensia platform and the Conservation Effectiveness initiative.

Using this typology we have been able to divide results into stronger and weaker evidence, allowing us to show more data while still being transparent about the strength of each data point.

Below are the definitions for the different research designs:

Empirical study -  Randomized controlled trial (RCT): A study that evaluates the outcomes/ impacts of FSC-certification by comparing FSC-certified forests and uncertified forests, before and after an certification, and in which the observations are assigned into certified and uncertified categories randomly, in order to balance the covariate distributions of observed and unobserved factors and eliminate potential biases. RCTs are rare because certified operations are likely not a random selection of forests (selection biais), thus randomization is often not possible. Such studies can show causation between FSC-certification and the outcome, and wording such as “as a result”, “caused”, “lead to”, “reduced”, “increased” can be used.

Empirical study - With matched control, data collected post-certification: A study that evaluates the outcomes/ impacts of FSC-certification by comparing FSC-certified forests and uncertified forests at one point in time after certification, with covariates considered through matching. Matching is done during the development of the research design so that the units of comparison (e.g. forests) are as similar as possible in most important aspects and ideally differ only in terms of the presence of certification status. The matching process ensures that contextual factors other than certification are not influencing the findings. Such studies can show causation between FSC-certification and the outcome, and wording such as “as a result”, “caused”, “lead to”, “reduced”, “increased” can be used.

Empirical study - Control not matched but some confounders taken into account - data collected post-certification: A study that evaluates the outcomes/ impacts of FSC-certification by comparing FSC-certified forests and uncertified forests at one point in time after certification, with consideration of some confounders. Confounders, also called covariates or confounding variables, are factors likely influencing the outcome of the comparisons. They are considered and/or quantified during data collection and integrated into statistical analysis. In some circumstances, statistical analysis can isolate and quantify the effect of each variable and covariable. Such studies can show correlation between certification and outcome relatively reliably, especially in cases where the system is well-understood and most of the potentially biases are measurable. The following wording can be used: “is associated with”, “was found to have”, “is correlated with”, “even when [confounding variable x and y...] are taken into account, certification is correlated with”.

Empirical study - Control not matched, data collected post-certification: A study that evaluates the outcomes/ impacts of FSC-certification by comparing FSC-certified forests and uncertified forests at one point in time after certification, but without consideration of confounders. This type of study can potentially show a true correlation between certification and an outcome, however, it is possible that unknown mechanisms in fact drive the correlation, such as self-selection or another type of systematic bias. Wording that implies any type of causation cannot be used. Instead, wording such as “is associated with”, “was found to have”, “is correlated with” can be used.

Empirical study - No control, data collected before and after certification: A study that evaluate the outcomes of certification in a certified forest but without comparing with an uncertified unit. This type of study compares outcomes before and after certification has been implemented. As a result, it is difficult to assign any outcomes to the actual implementation of the intervention. However, such studies can be very useful in providing an understanding of the potential mechanisms that could link an intervention and outcome. Wording that implies any type of causation cannot be used. Instead, wording such as “was found to have” can be used.

Empirical study - No control, data collected post-certification: A study that evaluate the outcomes of certification in a certified forest but without comparing with an uncertified unit, nor with the status before certification. As a result, it is difficult to assign any outcomes to the actual implementation of the intervention. Such studies have a descriptive profile and do not allow generalization. Wording that implies any type of causation cannot be used. Instead, wording such as “was found to have” can be used.

Empirical study – Qualitative: Empirical, observational description of the cause and effect of an intervention without a sample-based measure of change or difference.

Modelling study - Two scenario comparison: A study that use empirical information about FSC certification to model or extrapolate outcomes of two scenarios at a broader scale or on other dependent variables.

We have classified studies according to their research design along an evidence typology when utilizing them for the Impact Dashboard.

The following table summarises the evidence types that can be found in the Impact Dashboard and the associated categories of level of evidence: