-
State of compliance
This research is based on the analysis of quantitative data gathered by QSAs from multiple Qualified Security Assessor Company (QSAC) organizations across the world. The dataset for this edition is based on information from five sources, four of them external to Verizon.66 These findings are presented globally, with additional comparisons between geographic regions (Americas, EMEA and APAC).
Dataset
Producing a PCI DSS assessment report may involve numerous assessments. In several cases, an assessment report is the product of assessments conducted globally or across a specific region. Individual PCI DSS compliance reports consist of between one and, in some cases, up to 120 or more assessments per report, covering multiple in-scope locations.
Assessments
PCI DSS version: PCI DSS v3.2.1 consists of 12 PCI DSS Key Requirements, 79 base requirements, 252 control requirements and 440 test procedures.
In 2020, the compliance status of a total of 77,504 PCI DSS controls validated against PCI DSS v3.2.1 was assessed and compared against 68,992 controls from PCI DSS v3.2.1 assessed in 2019.
Reports: The 2019–2020 comparative analysis is based on an aggregate of 328 PCI DSS compliance validation reports and a combined total of 146,496 controls.PCI DSS Report on Compliance (ROC) dataset.
Methodology
Please provide the information below to view the online Verizon Payment Security Report.
Thank You.
Thank you.
You will soon receive an email with a link to confirm your access, or follow the link below.
Thank you.
You may now close this message and continue to your article.
-
2019:
155 (68,992 controls)
2020:
173 (77,504 controls)
Total:
328 (146,496 controls)
-
2020 PCI DSS validation dataset
2020 PCI DSS results – Interim validation
PCI DSS v3.2.1
100% compliance (passed): 75 (43.3%)
<100% compliance (failed): 98 (56.7%)Number of engagements: 173
Americas: 97
EMEA: 36
APAC: 40
The share of APAC organizations in the combined global dataset increased significantly in 2020 (from 9.3% to 23.0%).
For the 2020 assessment year, 75 entities passed their interim compliance validation, demonstrating that they kept all applicable PCI DSS controls in place. Over half (56.7%) of the organizations failed their interim validation assessment due to one or more security controls found to be not in place, with an average control gap of 4.0%—the percentage of controls that failed.Trend analysis includes year-over-year comparisons to determine how the state of compliance has evolved over multiple years. Changes in contributors and the potential changes in their areas of focus add a layer of difficulty when identifying trends over time.
-
The PSR analysis process
Our overall PSR data collection and analysis process remains intact and unchanged from previous years. All assessment data included in this report was individually reviewed and converted to create a common, anonymous aggregate dataset. The collection method and conversion are the same between contributors. In general, three steps were used to accomplish the dataset:
- Contributor identification and collection of eligible PCI DSS v3.2.1 assessment reports
- Full anonymization and conversion of the reports by the contributors into normalized data. All contributors received instruction to omit any information that might identify organizations or individuals involved
- Secure submission of the anonymized data to the Verizon PSR data science team for aggregated analysis
Data eligibilityFor a potential entry (Interim Report on Compliance) to be eligible for the PCI DSS compliance validation corpus, several requirements must be met. The entry must be data from a confirmed PCI DSS validation assessment conducted by a QSA who completed an ROC for an interim validation assessment. In addition to meeting the baseline definition of a draft or Interim Report on Compliance (IROC), the entry is assessed for quality. We then create a subset of compliance report data that passes our quality filter.
In addition to having the level of details necessary to pass the quality filter, the assessment reports must be within the time frame of analysis. For the 2020 dataset, this includes PCI DSS assessments conducted between January 1 and December 31, 2020.
What percentage of total PCI DSS compliance validation assessments that are conducted worldwide each year is covered in the survey? We do not know. We only have access to the data for the validation assessments that were conducted by Verizon and contributing QSACs.
"Anything can be measured. If a thing can be observed in any way at all, it lends itself to some type of measurement method. No matter how ‘fuzzy’ the measurement is, it’s still a measurement if it tells you more than you knew before.”67
—Douglas W. Hubbard
-
Noncommittal disclaimer
We would like to reiterate that we make no claim that the findings of this report are representative of all PCI DSS compliance assessments for all of organizations at all times. Even though the combined records from all our contributors more closely reflect reality than any of them in isolation, this dataset is still a sample. Although we believe many of the findings presented in this report are appropriate for generalization (and our confidence in this grows as we gather more data and compare it to that of other security organizations), bias undoubtedly exists.
The findings are based on aggregated demographic information. While aggregations are made up of individual organizations, individual organizations are not made up of aggregations. It’s not a two-way street. There are limitations to the extent these aggregations can be useful in making decisions. Therefore, when reading the findings of this report, you should not make assumptions about their applicability to individual organizations. Some findings and conclusions require additional context and data to add more value on the individual level.
"Anything that gives us new knowledge gives us an opportunity to be more rational.”68
—Herbert A. Simon"One accurate measurement is worth a thousand expert opinions. Without data, you’re just another person with an opinion.”69
—Rear Admiral Grace M. Hopper and W. Edwards Deming, Chicago Analytics Group
-
66 See page 163 of this report for list of PCI DSS data contributors.
67 Douglas W. Hubbard, “How to Measure Anything,” Third ed., Wiley, 2014.
68 Herbert A. Simon. https://www.brainyquote.com/lists/authors/top-10-herbert-a-simon-quotes.
69 Rear Admiral Grace M. Hopper and W. Edwards Deming, Chicago Analytics Group, Mar 30, 2016, http://chicagoanalyticsgroup.com/blog/archives/03-2016.