By Dan Ollendorf, PhD, Director, Value Measurement & Global Health Initiatives
There is a longstanding debate over whether and how to make economic models open-source; not just the technical details but the model itself. We at CEVR have published widely on this topic, recognizing that while there are arguments for and against, the transparency and accessibility that accompanies open-source modeling is likely to do more good than harm. In 2017, we began the development of a clearinghouse for open-source cost-effectiveness models, hosted on our Global Health CEA Registry website. Model submission was purposely intended to be straightforward and without restrictions regarding the software used, the type of CEA conducted (e.g., based on QALYs, DALYs, natural units, etc.), study quality, or the availability of supporting documentation.
Despite multiple attempts at outreach and publicity, clearinghouse uptake has been extremely limited. To find out why we recently sent a survey to approximately 250 modelers worldwide to gauge their level of interest our clearinghouse, and for those not opting to participate, the reasons for their reluctance. Perhaps most telling, the response rate to the survey itself was very low (<10%), even after repeated follow up. Those who did respond but opted not to submit models cited a need for further model documentation, improvement of model code, and concerns about releasing personal and/or institutional intellectual property into the public domain.
It appears that historically, there has been no external pressure or tipping point that has changed incentives or motivation for sharing models, but this has changed overnight with the emergence of the SARS-CoV-2 pandemic. There is an insatiable daily appetite for data and projections regarding the spread, lethality, and economic impact of this disease. In contrast to our experience, an open-source repository for epidemiologic models has already been developed, with promising early results.
However, we recently examined our cost-effectiveness registries and found that the use of CEA to inform pandemic response and future prevention is sorely lacking. In addition to forecasts on disease impact, governments and health systems need to make decisions about the policies and interventions that will save the most lives given the financial resources available. Just as with disease modeling, real-time CEA can help inform those decisions in a fast-changing environment. It’s time to bring cost-effectiveness modeling out of the halls of academia and assorted consortia and into the open-source light.