We have not been able to confirm the advantages that have been claimed for the ia.

From a point of view of run-time complexity, there are no strong reasons for preferring ia over the Greedy Algorithm, and in the corpora that we studied, it would be misleading to say that the ia matches human-produced descriptions more closely; for although there always existed a version of the ia that outperformed all other algorithms examined, this is not surprising given the fact that, in simple domain types, any halfway reasonable description can be produced by the ia if a preference order (po) is hand-picked.

18 The success of the ia depended substantially on po: a sub-optimal po produces descriptions that are worse than fb and gr.

This was not only true when "unreasonable" po were used, but also when all available evidence, including corpus frequencies, were taken into account to find a good po.

Furthermore, as we hope to explain in more detail elsewhere, an analysis of differences between participants in our experiment revealed that some human speakers are modelled more accurately by fb and gr than via any incremental generation strategy: in the people data, fb agreed perfectly with a human author about 61% of the time, for example.

Combining all the evidence, we conclude that someone who is looking for a gre algorithm for a previously unstudied application domain might do better choosing gr (augmented with a Dale and Reiter-style treatment of head nouns, as we have argued in section 2), instead of an unproven version of the ia.


Because this paper has used the assumptions outlined in Dale and Reiter (1995), our evaluations have focussed on the extent to which the descriptions produced by an algorithm matched human-produced descriptions.

It is intrinsi- cally interesting to construct algorithmic models of human language production, for example because it can help computer programs to, one day, pass the Turing test (Turing, 1950).

Moreover, there is some evidence that, by making refer- ring expressions resemble the ones produced by human speakers, the resulting expressions tend to be more easily understood (Campana et al., 2004).

For all these reasons, human-likeness is currently the prevalent perspective on evalua- tion of gre algorithms, including the recent stec challenges (Gatt and Belz, 2010).


Having said this, we acknowledge that there should also be room for alternative, utility-driven evaluation methods.

For, although some psycholin- guistic theories emphasise the cooperative nature of reference (e.g. Clark and Wilkes-Gibbs, 1986; Brennan and Clark, 1996), there is evidence that producers do not necessarily maintain a model of a receiver's communicative needs (e.g. Keysar et al., 2003; Engelhardt et al., 2006; Arnold, 2008).


The shortcomings of the ia became particularly noticeable in connection with the more complex of the two domain types, which involved black-and- white photographs of people's faces (i.e., the people domain type).

But our people domains were still comparatively simple.

Real people would have been identifiable in terms of their physical features, past actions, and so on.

It is unclear whether any of the algorithms discussed here would do well in such sit- uations.

Some studies, in fact, suggest that there are domain types in which the ia performs poorly regardless of po.

19 These results suggest to us that future research in gre should pay close attention to the complexities posed by the large and complex domains that speakers are faced with in real life.


Research on gre has moved on considerably since 1995, when Dale and Re- iter put forward their hypotheses (see Krahmer and van Deemter (2011) for a survey).

Yet, the core gre problem of producing naturalistic "one-shot" de- scriptions of a single referent continues to attract considerable interest, as was demonstrated by the recent gre evaluation campaigns.

The investigation on which we have reported in this paper raises the question whether incremental- ity is basically on the right track, or whether some other approach to reference generation is perhaps superior.

It appears to us that a nuanced answer to this question is called for, since the choice depends on the situation in which the algorithm has to operate.


In situations where it is possible to see, based on experimental evidence for example, that certain attributes are preferred over others (e.g., because they are easier to perceive or to express), the ia has considerable appeal, because this algorithm allows us to translate this evidence directly into a po.

20 In situations where neither intuitions nor experimental evidence are available, a version of the Greedy Algorithm is likely to be a better choice than the ia.


The main strength of the Greedy Algorithm lies in its ability to determine the usefulness of a property dynamically: in some cases, size will have great discriminatory power, for example, but if 90% of the domain elements have the same size as the target referent then size will be nearly useless for identifying the target.

The Greedy Algorithm is able to take such differences into account, because it selects properties based on how many distractors they remove.


But arguably, discriminatory power is not enough.

Suppose only one per- son in the room has green eyes.

This does not necessarily make eye colour very useful for referring to this person, because the colour of someone's eyes is difficult to perceive from a distance.

A defender of the Greedy Algorithm might counter that if eye colour is imperceptible, this attribute should not be available to the generator: gre algorithms should only use properties that are common knowledge.

The ia does have a subtle advantage in such cases, since its po allows one to order attributes according to their degree of perceptibility, avoiding a sharp, and ultimately arbitrary, distinction between perceptible and


19 An example is Paraboni et al. ( 2007), where reference in complex, hierarchically structured domains was studied.

The authors found that no single ia was able to generate the types of elaborate descriptions ('picture 5 in section 3', rather than 'picture 5', where the latter was minimally distinguishing) that were most preferred by both authors and readers.

Other algorithms, designed to produce carefully over-specified descriptions, proved superior to the ia in such situations.

The same is true if the evidence is different across the different values of an attribute, in which case the ia will operate with a po defined over properties (i.e., combinations of an attribute and a value) instead of attributes.

See also footnote 8.


imperceptible.

What the ia cannot do, however, is make eye colour more highly preferred for referents nearby than for referents further afield: it must always be preferred to the same degree.

The fact that different pos can be selected for different domain types and text genres does not alter this.


We want to make a plea for algorithms that determine dynamically which attributes to select for inclusion into a description, based on features of the situation.

Discriminatory power can play a role, but so can the extremity of a property (see below; also van Deemter (2006), section 4.1), intentional influ- ences (Jordan, 2000b,a), and alignment (Goudbeek et al., 2009). ( A framework suitable for combining several factors into one "cost" is Krahmer et al. ( 2003).) To show what we have in mind, let us say a bit more about one factor.


The idea of the extremity of a property can be traced back to some well- designed but little-known experiments (Hermann and Deutsch, 1976).

When participants were shown pairs of candles, which differed in height and width, it turned out that when asked to refer to a candle that was both taller and fatter than its distractor, speakers overwhelmingly did this by expressing whichever of the two dimensions made the target "stand" out more: when the target candle had a width of 50mm and a height of 120mm, for example, while the distractor had a width of 25mm and a height of 100mm, speakers referred to the target as "dick" (fat), rather than "lang" (tall), because the relative difference between 50mm and 25mm is greater than between 120mm and 100mm.

Since both properties (i.e., width and height) would have ruled out the only available distractor, their discriminatory power in this situation is equal, yet there was a tendency for speakers to express the more extreme property.

Results of this kind are difficult to replicate using a fixed po.

The study of gre algorithms that avoid a rigid preference order and select their properties "dynamically" - making use of a combination of discriminatory power, extremity, and other factors - appears to us to be one of the most promising directions of work on gre at the moment.


In a recent article published in this journal [2], the authors criticize the Incremental Algorithm (a well-known algorithm for the generation of referring expressions due to Dale & Reiter, 1995, also in this journal) because of its strong reliance on a pre-determined, domain-dependent Preference Order.

The authors argue that there are potentially many different Preference Orders that could be considered, while often no evidence is available to determine which is a good one.

In this brief note, however, we suggest (based on a learning curve experiment) that finding a Preference Order for a new domain may not be so difficult after all, as long as one has access to a handful of human-produced descriptions collected in a semantically transparent way.

We argue that this is due to the fact that it is both more important and less difficult to get a good ordering of the head than of the tail of a Preference Order.


When speakers use a description to refer to an object ("the green sofa") or a person ("the tall man"), they have to determine which properties to include so that an addressee knows which object or person is intended.

Clearly, there are many ways in which a sofa can be distinguished from other furniture items ("the large sofa," "the sofa left of the modern-looking chair," etc.), and there are even more possibilities when we consider references to persons.


Various algorithms for the Generation of Referring Expressions (GRE) have been proposed that compute which set of properties distinguish a target, where properties themselves are often represented as attribute-value pairs, such as ⟨color, green⟩, indicating that the target has the value green for the attribute color.

In a seminal study, Dale and Reiter (1995) present and compare various algorithms that accomplish this task [1].

One algorithm, which they call Greedy Heuristic (gr), relies on discriminatory power: It chooses properties one by one, at each iteration selecting that property of the intended referent that excludes most of the distractors not previously ruled out.


An alternative that Dale and Reiter discuss is the Incremental Algorithm (ia), which relies on the assumption that some attributes (e.g., color) are more preferred than others (e.g., size).

This notion is formalized using a preference order (po), which is a list of attributes through which the ia iterates, selecting an attribute-value pair if it helps distinguishing the target from one or more of the distractors.

Dale and Reiter point out that pos may vary from one domain to another, and should be determined empirically.


Recently, van Deemter et al. ( 2012) presented an extensive evaluation of the various algorithms discussed in Dale and Reiter (1995) [2].

In their paper, van Deemter and colleagues emphasize the crucial role of the po when evaluating the ia.

They show that, both in a furniture and in a people domain, the ia outperforms the gr algorithm with a "good" po, while the opposite holds when the ia relies on a "bad" po.

This outcome naturally raises the question "how 'good' pos might be selected" [2].

The authors point out that systematically testing all pos for a given domain quickly becomes impractical, since for a domain where objects can be described using n different attributes, there are n! pos to consider.

Based on considerations such as these, they conclude that "someone who is looking for a GRE algorithm for a previously unstudied application domain might do better choosing gr (…), instead of an unproven version of the ia" [2].


However, it seems to us that it should first be determined how difficult it really is to find a "good" po.

Perhaps only a handful or even only one human-produced description is needed to make an educated guess about the po for a certain domain.

One way to find out is by running a learning curve experiment, in which randomly selected sets of human-produced referring expressions (increasing in size) are used to determine the ranking of attributes on a po, after which the ia (with the various, resulting pos) can be evaluated in the same way as was done by van Deemter et al. ( 2012) [2].

In this letter, we report on exactly such an experiment, thereby extending the findings of van Deemter and colleagues.


For our experiment we relied on the TUNA data-set, extensively described in van Deemter et al. ( 2012) [2].

This semantically transparent corpus contains referring expressions for two domains (furniture and people), and it has been used for comparative evaluation in the REG Challenges (2007–2009).

To determine pos, we used 165 furniture descriptions and 136 people descriptions from the training set of the REG Challenge 2009.

For evaluating the ia, we used 38 furniture descriptions and 38 people descriptions from the 2009 REG Challenge development set.


We used randomly selected subsets for each domain of the training set, with set sizes of 1, 5, 10, 20, 30, 40, and 50 descriptions, as well as the entire set.

Because the accidental composition of a training set may strongly influence the results, especially for small training sets, we created five different sets of each size.

The training sets were built up in a cumulative fashion: We started with five randomly selected sets of size 1, then added four randomly selected descriptions to each of them to create five sets of size 5, etc.


The rank of an attribute on a po was determined by counting the number of descriptions that include this attribute in a given set.

Van Deemter et al. ( 2012) likewise relied on frequencies (as well as on psycholinguistic evidence) to determine promising pos (which, incidentally, turned out to be easier to find for the furniture than the people domain), but, crucially, where they only computed frequencies over their entire development set, we specifically consider subsets (of increasing sizes).


Consider determining a po based on a set of one description, say "the large chair." Two attributes are mentioned in this description (size and type) and hence have frequency 1; these are placed at the head of the po.

Other attributes (such as color and orientation) are not mentioned (frequency 0), and these are placed at the tail of the po.

This gives a partial ordering of attributes.


However, the ia requires a complete ordering, so when two or more attributes occur in a single training set with the same frequency, we decided to rank them alphabetically (clearly, as set size increases, frequencies will more often be enough to completely rank all attributes).

The resulting po for our example would be ⟨size, type, color, orientation⟩. Note that like Dale and Reiter (1995), but contrary to van Deemter et al. ( 2012) we check whether type is in the description after running the ia and include it when it was not added earlier.


Nothing hinges on this for the experiment we describe here, however.

Once a po has been determined, we run the ia with this po on the test set and evaluate the performance in terms of Dice and prp, as van Deemter et al. ( 2012) do [2].

Dice measures set overlap between a generated set of properties and those produced by a human speaker, ranging from 0 (no overlap) to 1 (complete overlap); prp is the perfect recall percentage, the proportion of times when the ia achieves a Dice score of 1.


Table 1 summarizes the results and reveals a clear picture.

For the Furniture domain, a set size of five descriptions is sufficient to reach top performance both in terms of Dice and prp.

For the more complex People domain the results are slightly more varied, and increasing set sizes generally give numerically higher scores.

However, it is important to note that even here the first few descriptions clearly have the biggest impact; moving from set size 1 to set size 10, for example, yields a prp improvement of 0.137, while moving from set size 10 to the entire set gives a prp improvement of 0.105.

For Dice, this is even clearer: From set size 1 to 10 gives an increase of 0.16, while moving from 10 to the entire set improves the Dice score with only 0.04.


To test for statistical significance, we ran separate analyses of variance (one for each domain) with set-size as a repeated measure, and Dice and prp as the dependent variables.

Planned post hoc pairwise comparisons were made to test which set-size is the smallest that does not perform significantly different from the entire set (we call this the "ceiling"); we report results both for the standard Bonferroni method, which corrects for multiple comparisons, and for the less strict LSD method from Fisher, which does not.

Note that with the more strict Bonferroni method we are inherently less likely to find statistically significant differences between set sizes, meaning that we can expect to reach a ceiling earlier than with the LSD method.


We found statistically significant effects of set size for both the Furniture (Dice: F(7, 259) = 13.81; p < .001; η²p = 0.27 and prp: F(7, 259) = 34.31; p < .001; η²p = 0.48) and the People domains (Dice: F(7, 259) = 18.47; p < .001; η²p = 0.33 and prp: F(7, 259) = 5.99; p < .001; η²p = 0.14), indicating that the number of descriptions from which a po is derived influences the performance of the ia both in terms of Dice and of prp.

Crucially, the post hoc tests confirm that, generally speaking, small set sizes are sufficient to reach ceiling performance.

For the furniture domain, we (clearly) reached the ceiling at set-size 5.

For the People domain, the ceiling for Dice is reached at set-size 10 (both with the Bonferroni and the LSD method), while for prp the ceiling is reached at set-size 40 using the LSD method, and 1 with the Bonferroni method.


Our experiment shows that even based on a small set of human-produced descriptions we can make an informed guess about what a "good" po is.

With relatively small sets of descriptions (much smaller than the entire TUNA corpus) we obtain results that are not significantly different from the best performing variant of the ia.


Arguably, these results allow two important, related conclusions: (a) Even though, in theory, there are indeed n! different possible pos of n attributes, it is both more important and less difficult to get a "good" ordering for the head of the po than for its tail. And (b) relatively many human-produced descriptions will include attributes that are preferred, so that it does not really matter how dispreferred attributes are ordered.


It is worth pointing out that our approach still requires a semantically balanced corpus.

However, the evaluation results suggest that a surprisingly small one may be sufficient.


Of course, it should be kept in mind that these conclusions, while intuitive, are based on data from the relatively simple domains studied by van Deemter et al. ( 2012) [2].

Moreover, here we have only considered references to single targets, while van Deemter et al. ( 2012) also consider references to sets, for which determining a po might well be more complicated.


It will certainly be interesting to see what happens when the field of referring expression generation moves toward more complex references and more open-ended domains.

For now, we conclude that someone who is looking for a GRE algorithm for a new application domain might consider collecting a handful of human-produced descriptions, before discarding the ia in favor of another algorithm.


Krahmer and Koolen received financial support from the Netherlands Organization for Scientific Research (Vici grant 27770007).

Thanks to Albert Gatt for allowing us to use his implementation of the Incremental Algorithm and his evaluation scripts, and to Kees van Deemter and Albert Gatt for useful discussions and comments on an earlier version of this text.


This response discusses the experiment reported in Krahmer et al. ’s Letter to the Editor of Cognitive Science.

We observe that their results do not tell us whether the Incremental Algorithm is better or worse than its competitors, and we speculate about implications for reference in complex domains, and for learning from “normal” (i.e., non-semantically-balanced) corpora.

In their Letter to the Editor [3], Krahmer and colleagues discuss our article “Generation of Referring Expressions: Assessing the Incremental Algorithm” [4].

In this response, we assess their contribution.

Our article details the TUNA experiment, a systematic attempt to assess these two claims.

TUNA compared the performance of the IA and its competitors on a corpus of referring expressions that resulted from an experiment in which subjects were asked to refer to objects to an imaginary listener.

In a nutshell, we found that IA is not always superior to its competitors.

The performance of IA turned out to depend strongly on a parameter not specified by Dale and Reiter [1], namely the Preference Order (PO) of attributes.

By determining in what order the IA examines attributes (e.g., colour, size, etc.), the PO influences the likelihood with which an attribute is included in an expression generated by the IA.

Different POs thus lead to different referring expressions.

Some POs make IA superior to its competitors, others make it inferior.

Crucially, our study suggested that it is difficult to predict whether a given PO will make IA superior to its competitors. And, given that the IA’s competitors performed quite reliably, we speculated that, confronted with a new domain for which no corpus is available, practitioners in Natural Language Generation might be wise to use one of these competitors, instead of the IA.

Krahmer and colleagues wondered how difficult it is to find “good” POs for IA.

They decided to use one of the methods proposed in our article, namely to determine POs by counting the frequencies of attributes in the TUNA corpus.

They did not focus on the corpus as a whole, however, but on tiny parts of the corpus.


The TUNA corpus consists of two parts: the furniture corpus, consisting of stylised pictures of furniture; and the people corpus, consisting of descriptions of photographic portraits.

The authors found that, for furniture, tiny samples suffice to construct an IA that performs as well as their best-performing IA; for the people corpus, the results were more varied, though small samples still tended to perform well.

Their procedure went like this: they computed DICE and PRP scores (these are metrics that measure how “human-like” the expressions generated by an algorithm are. See our article for definitions.) for samples consisting of 1, 5, 10, 20, 30, 40, 50, and (roughly) 150 descriptions, that is, at 8 different levels.

Larger samples might be expected to lead to better IAs, because they give a more accurate picture of language use.

The authors show, however, that for the furniture corpus, a “ceiling” is reached with as few as 5 descriptions; for the people corpus, a DICE ceiling is reached at 10 descriptions, whereas a PRP ceiling is reached at 40.

The ceiling is defined as the lowest level that did not score significantly worse than the IA associated with the highest level (i.e., level 8).

We make a number of observations, starting with what we see as the main strength of the Letter.


Where we agree: To find that, in some domains, a tiny amount of data suffices to construct a well-performing reference production algorithm is genuinely interesting, and reminiscent of calculations showing that complex Bayesian decision making can often be approximated very well through small samples [5].

Let us place these results in context.

TUNA’s two domains were both tiny (involving just 6 distractors, offered in a neat visual arrangement on a small computer screen).

Nonetheless, the people corpus made it substantially harder to find a version of IA that beats its competitors than the even simpler furniture corpus.

( Krahmer et al. find analogous differences between the two, with higher ceilings for the people corpus.) One wonders how difficult it is to find “good” POs for truly complex domains, as when we point out a person in a crowd, or when we refer to a place that the speaker and the listener once visited together.

We believe that the real challenge for research on reference lies in such “real-world” domains – a position for which we find support in Krahmer and van Deemter [2], which gives us confidence that we may be able to find common ground with the authors of the Letter.


An important reservation: Unlike Dale and Reiter [1], and unlike our own article, the Letter does not compare IA with its competitors.

This means that the results of the Letter do not speak directly to the question on which our paper focussed, namely whether IA is superior to its competitors.

To defend the IA against its competitors, it does not suffice to show that small corpora allow one to choose an IA that is “good”: one would have to show that the chosen IA is better than these competitors.

Although direct comparisons are a bit tricky, because of the way in which the authors decided to apply the algorithms (e.g., treating the TYPE attribute differently from how we treated it), it may be illustrative to note that the DICE scores that we found for one of the competitor algorithms, Full Brevity, lie above the relevant figures for IA reported in the Letter.


A quibble about stats: the definition of Krahmer et al. ’s notion of a ceiling rests on non-significance. But non-significance results need to be treated with particular caution.

For example, the authors observe that a DICE score of 0.605 (people corpus, domains of size 10) was not statistically distinguishable from a DICE score of 0.724 (people corpus, domains of size 50), yet an experiment that used a larger number of referring expressions to test the algorithms may well have revealed a difference.

Besides, it may not be justified to treat level 8 as if it were a gold standard; larger data sets may well have yielded even higher scores.

These are minor issues.


Scarce resources: By using the TUNA corpus, Krahmer and colleagues exploit a specially prepared corpus, which is semantically balanced.

Semantic balance means, for example, that the number of situations in which a given property (e.g., colour) suffices to individuate the referent equals the number of situations where some other property (e.g., size) suffices, and similar for combinations of properties.

Semantically balanced corpora tell us something about language use.

Non-semantically-balanced corpora are much more widely available, and more representative of what humans are exposed to.

In these “normal” corpora however, frequencies are influenced not just by language use, but also by frequencies in the world.

Consequently, drawing inferences from these corpora tends to be harder than from semantically balanced ones.

The Letter raises the question whether good POs can also be found on the basis of small corpora that are not semantically balanced.

We’re still inclined to believe that, confronted with a new domain for which no corpus is available (let alone a semantically balanced one), it might be better to use one of its competitors, instead of the IA.

In our article, we argue that factors such as computational complexity do not alter this assessment.

Krahmer et al. ’s experiment with small data sets is interesting nonetheless, not least because of the questions it raises about human acquisition of reference production.


The methodology for this scoping review was developed using previously published guidelines by Arksey and O'Malley 16 based on a modified Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) approach for scoping reviews (PRISMA- ScR).

17 This process involves five stages for conducting a scoping review, which are described below.

The PRISMA-ScR checklist can be found in the Electronic Supplementary Material eAppendix 1.


To better evaluate propofol misuse by medical professionals, the research team developed the following research question: What individual characteristics of medical professionals misusing propofol are commonly reported in the literature and what are the reported consequences of propofol misuse?


A comprehensive search strategy was used with search terms including propofol, medication misuse, medical professionals, and addiction.

A full list of search terms is available in ESM eAppendix 2.

Searches were conducted on 29 November 2021 within the following databases: Ovid MEDLINE (1986 to November 2021), Ovid EMBASE (1986 to November 2021), and Ovid PsycINFO (1986 to November 2021).

No article type restrictions were included to obtain broad search results from a wide variety of sources.

Date restrictions from 1986 to present were used to reflect the period of clinical use of propofol.

Because of limitations in translation services, the search was limited to articles written in English.

Reference lists of the identified sources were reviewed to identify applicable sources not found in the initial search.


Inclusion criteria for the scoping review included reports on medical professionals misusing propofol.

Exclusion criteria included discussions of laypersons misusing propofol, medical professionals misusing substances other than propofol, no full-text availability, and non-English language sources.


Two members of the research team independently screened publications for inclusion based on title and abstract review using a systematic review software (Covidence, Veritas Health Innovation; Melbourne, VIC, Australia).

Sources deemed to be relevant by title and abstract review were screened for inclusion using a full-text review.

Any 123 396 G. W. Burnett et al. disagreements were discussed until a consensus could be reached.

A third reviewer was available to discuss disagreements that could not be resolved by the initial reviewers.


After the full-text review, data were extracted from relevant sources.

Extracted data included the article type, country of origin, year of report, profession, trainee status, sex, age, pre-existing psychiatric disorders, polysubstance use (use of propofol in addition to other substances), method of discovery, outcome, and if the misuse was associated with suicide.


Analysis of identified articles including individual characteristics of medical professionals was conducted based on the previously listed data points to identify relevant themes.


Details of source selection are available in Fig. 1.

Twenty- one articles identified through database searches were deemed relevant to our research question, while three additional articles were identified through reference list review.

Twenty-four articles were included for data charting and synthesis.

10, 18-40 Data charting of the final 24 articles is available in the Table.


Included articles were published between 1992 and 2021, with half of the included articles being published between 2007 and 2013 (n = 12).

The most common article type was case reports (n = 17) focusing on single cases, but six included articles were case series reporting 2-25 cases of propofol misuse.

One additional article meeting inclusion criteria was a survey study.

Most included articles originated from Europe (n = 15), but most of the identified cases were from North America in two larger case series.


Of the identified articles reporting age of medical professionals misusing propofol, the median [interquartile range] age was 35 [29-43] yr, though discrete age data were only available for approximately half of the cases.

The medical professionals' sex was approximately even when indicated (male, n = 33; female, n = 30), but was unknown in nearly one third of cases (unknown, n = 25).


No articles provided information on race or ethnicity of medical professionals misusing propofol.


Twenty-two articles provided the medical professional occupation of 85 individuals.

Medical professionals identified as misusing propofol most commonly worked within the field of anesthesiology as anesthesiologists (n = 42) or as certified registered nurse anesthetists (CRNAs; n = 16).

Additionally, registered nurses (n = 12) and nonanesthesiologist physicians (n = 12) were also identified as misusing propofol in the literature.

Nonanesthesiology physicians' specialties were surgery (n = 2), general practice (n = 2), critical care (n = 1), obstetrics (n = 1), emergency medicine (n = 1), radiology (n = 1) and unknown (n = 4).

Figure 2 shows the full breakdown of medical professions.

Only 22 of the identified cases involved trainees, but all 22 of the trainees identified as misusing propofol were noted to be anesthesiologists.


Pre-existing psychiatric history was available in only 14 included articles spanning 59 cases.

Pre-existing psychiatric disorder was noted in nearly half of the cases with available information (26 cases), but the information provided was limited in these articles.

History of polysubstance use (use of substances in addition to propofol) was better reported in nearly all included articles, with one article providing partial data for identified cases.

Overall, history of polysubstance use was reported in 72 cases and was present in most identified cases (n = 42).


Data regarding method of discovery were available in 23 articles covering 48 cases, though these data were incomplete in one article.

Among cases with a known method of discovery, death was most common (n = 38).

Symptomatic intoxication (n = 6) and self-admission to rehabilitation (n = 4) were also noted methods of discovery.


Data regarding outcome of cases were available in all 24 articles, but one article included incomplete data on outcome, leaving known outcome data on 83 cases.

Self- admission to rehabilitation (n = 42) and death (n = 41) were the most common outcomes.

Of the 41 cases of death, association with suicide was reported in 31 cases.

Suicide was associated with death in 13 cases.

Information regarding subsequent recovery or resumption of clinical practice following rehabilitation was not identified.


Propofol is the one of the most common intravenous anesthetic agents used today. When administered by a trained professional, it is a safe and effective anesthetic agent, but self-administration can have devastating consequences.

Our scoping review identified 24 articles reporting on individual characteristics of propofol misuse in 88 healthcare professionals over a 30-year period.

While this may seem a small number given the amount of propofol in use worldwide during this period, it should be noted that these cases likely represent the tip of a large iceberg of unreported events.

It is also likely that these reports represent a sample skewed toward prevalence in academic institutions where someone might be more likely to write a report.

While it appears that the incidence of propofol misuse is increasing, without a clear understanding of either the real numerator or denominator involved, we cannot estimate a rate and our findings should be viewed in this context.


Given our search criteria, it is not surprising that individual case reports were the most common article type in this review.

Despite this, much of the more robust data from the included articles came from multiple larger case series and a single survey study.

This shows the need for larger studies to strengthen the understanding of medical professionals who misuse propofol and identify methods to identify and mitigate propofol misuse.

Despite this need for additional data, future studies may be limited to the practical difficulties of measuring the impact of interventions to prevent propofol misuse due to difficulties in determining its true incidence.


Anesthesiologists and CRNAs represented the majority of identified articles and cases.

Ease of access and familiarity with how it is used may explain why the rate of propofol misuse appears to be highest in this group of healthcare professionals.

Other healthcare professionals with access and familiarity made up the second highest percentage group of medical professionals, which included registered nurses and nonanesthesiologist physicians in specialties such as critical care, surgery, and emergency medicine.

The findings of this scoping review provide support to focus future prevention and recognition strategies on those working within the field of anesthesiology.


Previous studies investigating substances more generally 11 and propofol specifically 10 have suggested that trainees represent a larger proportion of medical professionals misusing these substances.

Less than half of propofol misuse cases identified in this scoping review were in trainees, and all trainees identified were anesthesiologists.

Though collation of the case reports, case series, and one survey study should not be used as a reliable direct comparator with previous studies, we do recognize anesthesiology trainees as a high-risk population for propofol misuse.

This finding suggests that propofol misuse may be more common in those out of training than previously believed and should be further investigated to truly understand the extent of this problem.

Previous literature shows that the greatest risk for anesthesiologists and CRNAs developing addiction-related complications with misuse of anesthetic agents is early in their career.

41 Focus should therefore be on preventing healthcare professionals, particularly anesthesiologists, from misusing propofol early in their career.


Death was a commonly described outcome in cases of propofol misuse identified in the literature and was similar to findings in opioid misuse.

42 Most strikingly, this review found that death was a frequently reported method of discovery in cases of propofol misuse.

Interpretation of this finding should be tempered as nearly half of all identified cases lacked a known method of discovery.

Previous studies have shown mortality rates as high as 28% among anesthesiology attending physicians and 38% among resident physicians.

10 Induction doses of propofol can cause apnea more frequently than other anesthetic agents can, even opioids.

43 Rapid injection of propofol most commonly leads to death due to respiratory depression without ventilatory support and may also result in anoxic brain injury, aspiration pneumonitis, or cardiac arrest.

30, 44 When self-administered or administered outside the controlled medical setting, an induction dose of this medication may provide a lethal injection as shown in this review.


It is also important to note that death related to propofol misuse and known suicide were commonly reported, though data regarding suicide was not available for a portion of articles identified in this review.

Suicide risk has been shown to be elevated in healthcare professionals, 45, 46 and particularly high-risk fields of medicine including specialties with access to propofol such as anesthesiology and general surgery.

46 While pre-existing psychiatric history was available in only 14 of the included articles, nearly half of the individuals in these articles had a pre- existing psychiatric disorder.

It is vital that all medical professionals remain vigilant and able to recognize colleagues with signs of depression or suicidality, particularly in those with access to drugs such as propofol, which is frequently used to facilitate suicide in this cohort.


There have been increasing reports of propofol misuse among medical professionals beyond the case reports and case series targeted by this scoping review.

In a study including 11,666 attending physicians and resident physicians from 1990 to 1997 by Booth et al., the calculated ten-year incidence of propofol misuse was 123 Propofol misuse in medical professions 399 0.02%.

47 In a more recent study by Wischmeyer et al., which included 20,865 attending physicians and resident physicians, the incidence rate increased five-fold to 0.10% during the period 1995-2005.

10 Most recently, Fry et al. surveyed the heads of 185 Australian and New Zealand departments of anesthesia and found that propofol was the most commonly misused drug, even when considering more traditionally misused substances such as opiates, benzodiazepines, and alcohol.

11 Due to this increasingly more common incidence of propofol misuse, it is important to recognize the misuse potential of propofol.


Compared with other commonly used anesthetic agents, propofol has a rapid onset of action for induction combined with rapid recovery, 43 making it an attractive target for diversion in the workplace setting.

Individuals who have misused propofol reported self-administration of anesthetic or subanesthetic doses repeatedly throughout the day while still being able to function at a high enough level to avoid detection in between doses.

So called ''pro-napping'' is associated with self-administration of a single bolus dose followed by rapid recovery and immediate return to work.

48 These reports of propofol misuse while on duty raise concerns regarding patient safety while being treated by medical professionals misusing propofol.


Similar to alcohol, benzodiazepines, and barbiturates, all of which have proven addiction potential, propofol acts on c-aminobutyric acid type A (GABA A ) receptors to enhance chloride currents.

44 The potentiation of GABA A receptor- mediated chloride currents by propofol occurs at low concentrations with direct activation of the GABA A receptor occurring with high concentrations of propofol.

49 Antagonizing the GABA A receptor blocks the action of propofol.

50 Both subanesthetic and anesthetic doses of propofol have been shown to increase the concentration of dopamine in the nucleus accumbens of rats, a main component of the mesolimbic reward process.

51 This is very similar to what occurs with alcohol and other recreational drugs that stimulate dopamine release in the nucleus accumbens and further enhance activation of neural networks associated with attention and reward behavior.

52 In a study conducted on baboons, these primates were shown to preferentially self-administer propofol, suggesting this medication also acts as a reinforcer.

53 In a more recent study evaluating patients undergoing propofol sedation during gastrointestinal endoscopy procedures, propofol induced euphoria in nearly half of patients enrolled.

54 Although propofol isn't currently listed as a controlled substance throughout the USA or Canada, some institutions require propofol to be accounted for.


Wischmeyer et al. showed that at 90 academic anesthesia programs (71% of programs in the study), the pharmacy did not securely store and account for propofol.

10 This study also showed that more cases of propofol misuse were reported at programs without pharmacy control of propofol.


While having stricter control on the supply of propofol in the hospital, even if propofol is not ultimately reclassified as a controlled substance (that is, treating propofol as a controlled substance) could potentially reduce the rate of diversion.


Through this scoping review, we identified several knowledge gaps regarding propofol misuse.

One is the etiology for propofol misuse, an important point for identifying potential future mitigation strategies.

Other gaps include the demographics of those misusing propofol, such as information on race, ethnicity, and socioeconomic background, which could be associated with propofol misuse.


Another notable knowledge gap is related to re- entry to clinical practice following propofol misuse.

It is unknown if those misusing propofol successfully complete rehabilitation and are able to re-enter clinical practice within their previous subspecialty or within a new subspecialty.

One identified study does provide a proposed framework for re-entry to clinical practice following propofol misuse, 55 but no data regarding success of this program exist.


Similarly, no long-term follow-up of those who have undergone rehabilitation for propofol misuse has been reported, so it is unknown if rehabilitation prevents relapse or death.

Additionally, the duration of misuse of propofol is unknown.

Identifying how long those misusing propofol have been actively misusing may provide insight into opportunities to prevent diversion and misuse.


Lastly, the success of potential training interventions or other prevention strategies is also unknown.

While Wischmeyer et al. showed that pharmacy accountability of propofol was associated with a lower incidence of propofol misuse, 10 very little additional information related to training interventions or other prevention strategies exists, including outcomes related to propofol misuse in regions where propofol is listed as a controlled substance, such as South Korea, Alabama, Georgia, and North Dakota.


Future directions of research involving propofol misuse in medical professionals should address these knowledge gaps.

An anonymous database of cases to better identify risk factors and outcomes associated with propofol misuse is one possible future direction.

Care to make certain a database such as this maintains confidentiality and would not be reported to authorities or governing bodies would be imperative to limit stigmatization or discouragement from self-presenting for help.

This database could help provide further insight into propofol misuse and outcomes through larger data analysis, similar to what has previously been done with the Malignant Hyperthermia Hotline.

56 Data from this database could be monitored over time to provide a more robust method for measuring success of


Propofol misuse by medical professionals exists and in the literature is most commonly reported in anesthesiologists and CRNAs.

As death is a common reported result of propofol misuse, even in medical professionals who have knowledge of the pharmacokinetic and pharmacodynamic properties of the medication, areas for future study should focus on quantifying the significance of this problem and looking for ways to identify potential interventions for at- risk populations.

Possible strategies include development of an anonymous database of medical professionals known to misuse propofol or expanding the scope of search to include public records such as from law enforcement, the national death index, or records from licensing agencies relating to suspension or termination due to substance misuse-related issues.


To the Editor,


We read Burnett et al. ’s scoping review [1] on propofol misuse with immense sympathy for the practitioners, their respective families, mentioned within the review, and everyone else affected but not written about.

The authors conducted a retrospective scoping review of available literature published from 1986 to 2021 to explore medical practitioners’ characteristics associated with propofol misuse.

This is an important topic that has significant potential to build understanding and mitigate future risk to health care professionals.

We thank the authors for illuminating the obscure and for engaging in open dialogue, helping to destigmatize a topic that has historically remained absent from open discussion in the anesthesiology community.

We wish to contribute to Burnett et al. ’s scoping review how language within the greater medical community can lead to dehumanizing discourses and institutional pressures that may exacerbate, by way of example, propofol misuse.


Many journals offer the opportunity of submitting 'case reports' or 'case series' for consideration for publication.

Although we fully endorse 'case reports' in advancing medical knowledge, we question the greater discourses of 'cases' to represent people.

Foucault developed the medical gaze, a concept whereby physicians fit a story into a professional biomedical paradigm while omitting information [2].

Society classifies people based on normative expectations, and individuals who do not conform to these socially constructed normative expectations are discreditable [3].

Referring to people as 'cases' is an example of a medical gaze that strips these people of their personal and professional identities, thereby objectifying, discrediting, and reducing the subject; this discursive practice of labeling people—with what society has labeled as a stigmatizing characteristic—as 'cases' renders the subject as discredited [3].

Subject objectification is a form of dehumanization, and the use of 'case' to depict any person discredits a person who society has already labeled as discreditable, what Goffman calls stigmatization [3].


When it comes to the issue of addiction and/or mental health, we believe it is of paramount importance to avoid the 'othering' that happens through the objectification that arises from the use of such language to really help any person struggling with these illnesses.

Such 'othering' in this context is, in our opinion, one of the barriers that prevents people from seeking help—whether they are patients we see or the colleagues who too often struggle in silence.

It is time to (re)humanize medicine.


The purpose of Burnett et al. ’s scoping review was to identify individual characteristics to help (identify) medical practitioners at risk of propofol misuse, an important contribution to the Canadian anesthesiology workforce that until recently has remained obscure and taboo.

We would also like to point out that the scope of the literature review encompasses a period of significant changes in practice as well as in training—a rupture in time, the COVID-19 pandemic—illuminating the effects of the stress and strains of the pandemic and how these changes may have significant impact on the authors’ goal.

We encourage an epistemic shift in how we know and understand characteristics that may potentially lead to propofol misuse and an ontological shift in what we know about propofol misuse, shifting from a medical gaze to a humanizing gaze that is rooted in people’s standpoint.

Although other extraneous factors may contribute to propofol misuse and suicide—family stressors, social identity, academic productivity, clinical schedule, workload, etc.—the institutional pressures that are placed on the individuals and the work environment whether supportive or not, inclusive or not, diverse or not, need to be emphasized.


Excessive workload and working hours, inflexibility, negative leadership behaviours, lack of equality, diversity and inclusion, poor teamwork, loss of autonomy, limited social supports, and moral distress in dealing with sickness and death are institutionally modifiable factors.

Multiple practitioners feel as though they are a cog in a wheel of austere institutions that are currently more concerned about finances and productivity than compassion [4] for our patients, ourselves, and each other.

We advocate for solutions that explore these issues and then engage structural and institutional changes that transform systems.

One way of transforming systems is through the use of language and the ways in which we have been socialized and encultured in medicine to see those with such illnesses as 'others.' Although unintentional, the use of 'case' in lieu of person dehumanizes a person.

We therefore advocate for a reconceptualization of the 'case report' or 'case series' from the greater academic publishing community to a (re)humanizing one.

Lest we forget, behind every 'case' is a person with a story.


We would like to thank Drs Lorello and Hawryluck for their comment [1] on our article titled 'Propofol misuse in medical professions: a scoping review.' [ 2] In their Letter to the Editor, Drs Lorello and Hawryluck presented an important argument for changing medical terminology to recognize that 'cases' are actually real humans with unique personalities, families, and friends. While we used the current medical terminology in our article, the notion of changing the medical lexicon to humanize the 'case report' is valuable.

Additionally, Drs Lorello and Hawryluck highlighted many institutionally modifiable factors that individual institutions should introspectively evaluate on ways their own intuitions could improve.

It is our hope that our article and this Letter to the Editor serve to direct the medical community’s attention to the topic of propofol misuse and its significant consequences for our colleagues and friends, and that through increased discussion and exposure we may find ways to prevent harm to them.