ATS Committee Spotlight: BSHSR Evidence Synthesis Methodology Working Group

August 5, 2017 at 12:06 am

The Documents Development & Implementation Committee (DDIC) is responsible for policies and procedures related to official ATS documents, including clinical practice guidelines.

Kevin C. Wilson, MD, is chair of the Assembly on Behavioral Science and Health Services Research’s Evidence Synthesis Methodology Working Group (ESM-WG), which specifically works to advance clinical practice guideline development and systematic review.

In this spotlight, Dr. Wilson discusses the ESM-WG’s current projects. He is joined by ATS Methodologist Jan Brozek, MD, PhD, and one of the group’s Methodology Scholars Noah Schoenberg, MD.

When was your group formed and why?

The ESM-WG was initiated to serve as the home of the ATS’s new Guideline Methodology Training Program. The program was established to increase the ATS’s methodological capacity, which was necessary because guideline methodologists are now required on every ATS clinical practice guideline panel in order to facilitate compliance with standards instituted by the Institute of Medicine. In 2011 the IOM dramatically increased the rigor and resources required to produce clinical guidelines. The ATS was in compliance with the new standards and developed this new program to be able to provide increased methodological capacity.

It was also envisioned that the ESM-WG could facilitate projects and research related to evidence synthesis.

What are your primary aims?

The primary goals of the ESM-WG are to administer the ATS Guideline Methodology Training Program. Also, the working group provides a venue for ATS members who are interested in systematic reviews and/or clinical practice guideline methodology to collaborate on projects and research, troubleshoot methodologic challenges, and discuss cutting-edge developments in evidence appraisal and guideline development.

What has the group achieved?

The ESM-WG hosts several active projects. Its first accomplishment was a workshop held at the ATS 2012 International Conference in San Francisco, California, during which members established a framework for developing performance measures from clinical practice guidelines. The success of this workshop was followed by another at the ATS 2014 International Conference in San Diego, California, at which participants built the foundation for a systematic approach for considering multiple chronic co-morbidities in clinical practice guidelines. Most guidelines are written for patients with a single chronic condition.

In addition, the working group’s Guideline Methodology Training Program is now in its fifth year. It is recruiting individuals interested in guideline methodology for on-the-job methodological training on current or future clinical practice guideline panels. The ESM-WG continues to facilitate collaboration in research projects investigating the methodology of clinical practice guidelines, particularly those methods that affect the balance of efficiency and quality.

What are its current responsibilities?

A current emphasis of the working group is to develop a pipeline of methodologists and attract early career individuals who want to learn how to perform a systematic review, appraise evidence, and/or formulate and assess recommendations using the Grading, Recommendations, Assessment, Development, and Evaluation (GRADE) approach.

The group is now involved in the development of about a dozen guidelines, with four guidelines initiated earlier this year. “While some guideline developers outsource the bulk of their most difficult methodologic work,” says Dr. Wilson, “this option can be very costly.” Greater training will increase efficiency with systematic review and greatly reduce the amount of time necessary to produce a guideline. Instead of a period of approximately six years, the goal is to be able to complete guidelines in about half the time or around three years.

Still, the practice of methodology remains laborious and intensive, and while technologies such as “machine learning” may make the process more efficient in the future, at this time, there is no real shortcut.” Methodologic work involves aggregating data, identifying predominant outcomes and nuances, differentiating studies, and from this, calculating the most probable diagnosis and treatment therapy.

Jan Brozek, MD, PhD, a member with a particular focus in allergy and immunology, adds, “We are training researchers at every career stage.” He explains that in the near future, ATS guideline development will eventually expand to include implementation. “Today, there’s a real shortage in knowledge and skills of logical thinking and the ability to dissect problems without bias.”

Noah Schoenberg, MD, a Methodology Scholar and member of the ESM-WG program, considers his instruction an integral part of learning how to practice medicine. With the wealth of available research studies, formal analysis and application of evidence can be daunting. The art behind the science or “process behind the process,” as he calls it, involves a close inspection of controversial and often conflicting information.

Dr. Schoenbery is proud to have joined the ATS’s growing effort to shape the guidelines of the future. “Improving the number and quality of available guidelines can have an enormous impact on patient care, by enabling improved decision making by both patients and physicians.”


ATS 2016 Full Registration