How can conflict of interest be avoided in panels of experts?

By Sylvain Bourjade

It is difficult to find experts that are both competent and independent. We show that in order to avoid conflicts of interest clouding expertise, the opinions and votes of each expert must be made public.

The Mediator scandal showed that conflicts of interest can have serious repercussions which may even lead to the deaths of several people. How can this be avoided while guaranteeing quality expert advice? Our research shows that it is possible to limit conflicts of interest by laying down rules for more transparency, but within limits.

It is always a challenge to find good experts: the most competent of them often have conflicts of interest. The researchers are partially financed by manufacturers with whom they sometimes are bound by consultancy contracts. This means that they may alter their opinion in order not to displease the manufacturers in question.

To minimize these conflicts of interest, without nevertheless having to rely on less qualified experts, we made a mathematical model of the behavior of the different parties. Experts are torn between three requirements: they may indeed have a biased opinion due to the conflict of interest, but they have at the same time to protect their reputation for impartiality so that they continue to be called upon as experts. Finally, even if they do have strong ties with industry they nevertheless continue to uphold certain principles. For example, they will refuse to approve a drug if they know it to be dangerous. We believe that experts’ choices are based on these three elements, namely loyalty to a customer, the need to protect their reputation for impartiality and the need to uphold their moral principles.

Damaging opacity

In France, in most of the consultative agencies such as the National agency for safety of medicines and health products (ANSM- l’Agence Nationale de Sécurité du Médicament et des Produits de Santé), which was called Afssaps until 2012, neither the experts’ reports nor their votes are divulged. If an expert wishes to have a potentially dangerous drug approved for marketing, his reputation is not affected, since nobody knows the role that he has played in this decision. Opacity thus tends to favor poor decisions which may endanger the health of consumers (by for instance approving the marketing of a potentially toxic drug) due to conflicts of interest.

Conversely, any transparency which is based on the reputation of experts, improves the expert advice. This is the case when their identity and the content of the report as well as their votes are known. In this case, the expert can no longer hide behind the argument “It was the committee that decided “. In the United States, the Food and Drug Administration (FDA), which is responsible for approving drugs for the American market, thus decided to improve its expert advice rules in order to limit conflicts of interest. The experts now pronounce their opinion simultaneously and the FDA reveals how each expert voted and publishes a detailed report of each meeting.

Protecting one’s reputation

The first results of this theoretical study are in no way surprising as the positive effects of transparency are well-known. But our research has spotlighted more astonishing effects in which transparency, on the contrary, has a negative effect on the quality of decisions taken by expert committees, leading for instance to the approval of dangerous substances. This is because, when the expert’s conflicts of interest are well-known, everyone expects him to come out in favor of the manufacturers that finance him. This means there is no danger that his reports and votes will damage his reputation, since it is already “bad” due to the fact that he has revealed his links with these manufacturers.

Conversely, if he does not reveal his conflicts of interest, the expert will be inclined to vote against the manufacturers’ proposals in order to protect his reputation. Hence, according to the model we developed, the decisions will be improved if the reports and decisions of experts are divulged, but not necessarily their links with manufacturers. The rules should also encourage them to be honest: experts who have taken dangerous decisions should no longer be consulted. When decisions are to be taken, the way in which the consultative procedure is set up is crucial: it is particularly important to know each expert’s opinion and the way in which he voted.

Endocrine disruptors and climatology

It is still difficult to prove beyond doubt that the quality of expert advice is improved when transparency rules are applied. This cannot be observed since the quality of the decision can only be judged after a long time. To determine whether the new rules of the Food and Drug Administration are successful, we shall have to analyze the results in 10 years’ time. Nevertheless it is clear that more transparency will be helpful for French and European health and safety agencies, especially given the many health issues under discussion.

This is the case in particular for “endocrine disruptors”, compounds which interfere with our hormone system (including the notorious Bisphenol-A, which is now prohibited in food containers). Several scientific studies have proven the dangers of these chemical substances, but they are still widely used by manufacturers. The decisions of the European Food Safety Authority (EFSA) concerning endocrine disruptors, have been severely criticized by scientists, who accuse EFSA of being too sensitive to lobbies and conflicts of interest and of having opaque expert-advice rules. Conversely, in climatology, the Intergovernmental Panel on Climate Change (IPCC), whose reports are widely recognized by scientists, has particularly strong transparency rules.

By Sylvain Bourjade and the article “The roles of reputation and transparency on the behavior of biased experts” published with Bruno Jullien in the RAND Journal of Economics, Vol. 42, No. 3, 2011.

Practical applications

The model we have developed applies both to drug agencies and other fields in which expertise is required but risks being biased by conflicts of interest. This is the case for instance for anti-monopoly policy, for which experts have to decide whether to entities may merge even though the experts in question may have links with these entities or with their competitors. It is also the case for recommendations made by financial analysts. Even more surprising is that this work also applies to peer reviews of scientific work in which scientific articles are judged by other researchers whose names remain secret.

Methodology

The work done by Bruno Jullien is a theoretical study in a field of mathematics called “game theory”. It involves modelling the behaviour of experts by estimating which elements predominated in their decisions. That depends on the short and long-term objectives of the experts. The study is not based on expertise data, since it is not available for the moment.