From CNN.com Today/Eissenberg study with feedback

[hifistud]

My point about the coin was perhaps too glib. What I meant was, if I saw a coin come up heads 16 times in a row, I would cease to believe it was a fair coin [i.e., p(head) = 0.50] and instead assume it was biased toward heads, and thus bet on heads for the 17th toss.

So, generally what I meant was, when you see a result 16 of 16 times you stop thinking that chance is at work and realize what you have is an actual effect.

Prior to the run of coin tosses, though, you would already have hypothesised your expected outcome, and, having noted a deviation from your hypothesis that would not be explained by probability, would then examine the materials to explain that deviation? If, then (as in the coin toss) you discovered a biased coin, you would be sure of the explanation and if you were seeking to demonstrate (or prove) a probability theory, could substitute an unbiased coin.

At the risk of out-glibbing you, to assume makes an ... of U and ME

 

[kai kane]

Many of those who succeeded here with e-cigs have made e-smoking a hobby. We're like the hot rodders and custom car guys of the 50s. Yes, we can get more vapor, more nicotine, more throat hit.

But that's not the point of his reserach, or any research likely to be done.

Perhaps. And perhaps not.
This was one of my initial concerns here.

You can run a hotrod on a racetrack, but on the street there are tighter regulations. Mufflers, fenders, speed limits.

190 proof drinking alcohol is deadly and thus illegal in many states. And nicotine is deadly in high concentration, so ...

Very big money and very big stakes are involved here, and the players are international and very powerful in their respective fields. We should not be naive.

I do expect that an organization (such as those already demonstrating they are opposed to the use of ecigs - ASH, FDA, etc.) will not accept Dr. E's findings, and allege improper testing, etc. was performed.

As a further argument to support their case, why would the FDA not cite users' modification of devices, combination of ingestion routes, and use of heavier concentrations of nicotine, to support their regulatory view? And to prove that argument, I would not be surprised if a Pharma funded researcher will collect that very data. (Not dissing researchers, Dr E seems like a nice guy! - just pointing out the process and player's motives).

Like Sun said early on in this thread - only time will tell how all this winds up - but I am sure many will continue to vape no matter what anyone says, reports, or researches.

At the same time, I have and still do encourage dialog, and sincerely appreciate your responses, Dr. E., even tho I find you did not address or side-stepped a number of valid questions.

Thanks TB and all the rest for your tremendous burst of participation - it's making a difference!

kai

 

[watfordjc]

The MHRA, being a government QUANGO, will seek to dismiss the study as being flawed, irrelevant, undertaken outside the MHRA's jurisdiction or not representative of the totality of e-cig products - pick whichever one happens to fit best on the day they decide.

But, if they uphold it, they will, simply, rule that, because this study moves towards proving that no nicotine is necessary in an e-cig's nature in order for it to be a stop-smoking device, and, because, therefore, it's a stop-smoking device, it still falls under their purview, and therefore still needs regulating.

Bottom line? They've got their teeth in and they're not going to let go because of a study involving 2 subject materials and 32 participants.

Problem with that is, they want to move away from what a product 'claims' and its function on the body.

They have the nicotine stance because it is easiest to control.

How would they go about requiring anything made with propylene glycol or vegetable glycerine be classed as a medicine if it affects the body?

How many food manufacturers and health & beauty manufacturers would have to spend £25,000+ every 5 years for every product they make that contains either ingredients? Who would have to pay to determine whether my toothpaste has an effect on my metabolism?

Can drinking water or chewing gum be classed as medicines because they can help a smoker reduce their cigarette usage? My NHS quit smoking leaflet suggests to try either when you get a craving. How can any water company ensure what comes out of the tap of every home has the exact composition of 'medical water'?

They say that want to regulate e-liquid and other NCPs to ensure quality on the one hand, and on the other they say they want to increase the availability of NCPs that are less harmful than tobacco smoke. Has any e-liquid been proven to be as harmful as tobacco smoke?


John.

 

[slybootz]

Actual effect or serious flaw in the methodology used in the flip thus producing repeatable but inaccurate results.

For the specific instance of the study, these 'flaws' in methodology could potentially be attributable to inattention to the use of the product (not purging the primer fluid from the atty), to measuring only a single indicator of nicotine in the body (rather than including continine and/or other levels in addition to plasma nicotine levels), to inexperience of the user (which was intentionally addressed based upon your comments), or to a handful of other potential factors.

Jan

Well said. Let's continue this analogy.

If 16/16 flips of a coin result in head, consider the variables:

• does this only happen with quarters? what about testing nickels, dimes, pennies, half dollars(if we are thinking about american currency), what about a blank slug with one side marked with an x? - if you end up with 16/16-heads with only quarters and no other coins follow this trend, this would suggest that the weight distribution of a quarter is biased towards heads.
• when you flip the coin(assuming a standard flip starts with the coin placed on the thumb, and flicked upwards), does the coin begin(before flipped) as heads? or tails? or vertical? if the coin starts on heads before flipped, and results in heads 16/16 trials, this would suggest that the preparation for the flip impacts the result.
• is the coin caught in the same place each time, after the same amount of 'air-time'? is the coin caught in the hand, or is it allowed to land on the ground(is it grass or cement or tile?)?
• How experienced is the coin-flipper? Does he/she know the proper way to flip a coin? if not, then the results very well could be flawed.

Relating these back to e-cigarettes, if you are testing an unreliable product, your results may be biased towards one result(i.e. flipping a biased coin).

If the user is less experienced with e-cigarettes than analog cigarettes, the results could be flawed due to incorrect usage(incorrect coin flipping technique).


I'm not the greatest at coming up with analogies, but I tried my best.

 

[hifistud]

John - they have two criteria for requiring approval - Presentation and Function, either of which is enough for them to be satisfied that approval is necessary. All they need is one test to show that there is nicotine in the mist, and they have enough to satisfy the Presentation arm of their criteria to the point of requiring a cease on sales while they sort out the mess.

The current state of play, as I understand it, is that the hardware is not under threat, but the e-liquid is. If that is the case, and if Dr Tom's conclusions are correct, we're home free. However, I have the feeling that the MHRA will not be taking the study at face value.

 

[Mister]

Again, absolutely nothing in the nicotine data from the total data set suggested any reason to doubt the results from the first 16 participants. Given these circumstances, the absence of data on these products demanded that the results be made public as quickly as possible. My only regret is that it took so long.
...
In this study I wanted to know what the first-time user of the off the shelf product is likely to experience"

I agree that there is/was little reason to expect different results after the first 16 participants.

What surprises me is your rush to present the highly generalized conclusion that "They [Electronic Cigarettes] are as effective at nicotine delivery as puffing on an unlit cigarette."

That is a generalization as flawed as testing 16 people who have never observed use of a bicycle, noting that they all fall off it when first trying to use it, and concluding that "They [bicycles] provide no useful locomotion to the user." And this analogy addresses only one of the leaps of faith in the drawn conclusion.

 

[curiousJan]

With regard to analyzing cartridge or vapor nicotine content, I agree it would have been nice to do so.

I'm afraid I'm going to have to disagree with the choice of words ... dismissing the fact that there was potentially non-nic vapor presented to the study subjects (via the inappropriate use of the product, i.e. not purging primer fluid -- which DVap demonstrated to contain 0 nicotine in this post -- prior to use) completely invalidates the study results, imho methodology fatally flawed.

However, having done so would not change the conclusions -- it would only suggest WHY the products I tested failed to deliver any measurable nicotine under the conditions under which I tested them. I tested off-the-shelf products that were reported to deliver nicotine to users. Whether they failed to do so (under the conditions in which we tested them) because the product came to us with no nicotine (very difficult to believe that each of two manufacturers managed to send us separate shipments of 16 mg cartridges that actually contained no nicotine!) or because the product cannot vaporize nicotine (but FDA and Laugesen both show that at least some of these types of products do vaporize nicotine, albeit in very low doses), or because the nicotine-containing vapor does not allow absorption of the drug to the user will not change the conclusion that they failed to do so under the conditions in which we tested them.

Shouldn't the question of WHY the products didn't provide nicotine be addressed? Rather than assume that the devices never provide any nic, wouldn't it be preferable to at the very least entertain the other possibilities? Especially considering you've overlooked a possible explanation ... incorrect use of the product.

Tropical Bob's post on hobbyists addresses this point nicely. In this study I wanted to know what the first-time user of the off the shelf product is likely to experience, not what the expert in the field who modifies the device and/or drips liquid directly onto the atomizer experiences. Again, that study is an interesting one, but a different one.

Then maybe statements made to media and the conclusions presented by the study would have been better phrased as specifically and succinctly as you've just done. No where in the pdf can I find anything resembling your statement that the study's intent was to test first-time pv users and their efficacy rates.

I agree that there is/was little reason to expect different results after the first 16 participants.

What surprises me is your rush to present the highly generalized conclusion that "They [Electronic Cigarettes] are as effective at nicotine delivery as puffing on an unlit cigarette."

That is a generalization as flawed as testing 16 people who have never observed use of a bicycle, noting that they all fall off it when first trying to use it, and concluding that "They [bicycles] provide no useful locomotion to the user." And this analogy addresses only one of the leaps of faith in the drawn conclusion.

Now this is a good analogy!

 

[Our House]

Yes--I remember when this study was first seeking people. You could NOT participate if you where already using the e-cig--never could figure out why that was an issue? Sun

QFT

What is it about naive ecig users (aside from the fact that they don't know how to use ecigs) that makes them significantly different enough that the study mandated their participation only?

 

[ckc]

John - they have two criteria for requiring approval - Presentation and Function, either of which is enough for them to be satisfied that approval is necessary. All they need is one test to show that there is nicotine in the mist, and they have enough to satisfy the Presentation arm of their criteria to the point of requiring a cease on sales while they sort out the mess.

The current state of play, as I understand it, is that the hardware is not under threat, but the e-liquid is. If that is the case, and if Dr Tom's conclusions are correct, we're home free. However, I have the feeling that the MHRA will not be taking the study at face value.

Hifi, I was beginning to think differently when the MHRA came out with their proposals. Why are they calling them Nicotine Containing Products instead of Nic Delivery? Somewhere in the middle of all they wrote it states that one is a placebo....E-Cig? I think they already had the results from this test or another one that shows the same results.

On an added note, i still occasionally smoke a cigarette....and get no hit from it.

 

[teissenb]

News from the lab:

First, and most important, I've just completed an analysis that should put this issue of primer to rest. We had a total of 7 atomizers of (NJOY calls them vaporizers) of each brand in use throughout the study. By the end of the first 16 subjects, every atomizer had been used more than once (that is, *at least* 40 puffs). Now, the hypothesis being advanced by some in this thread is that the first use was all primer, explaining the low nicotine delivery in the Tobacco Control paper that many of you have looked at.

OK, if that is the hypothesis, then the second set of 16 subjects, vaping with the previously used and now primer-less atomizers should have higher nicotine levels, right?

To test this hypothesis, I ran an analysis of variance as in the Tobacco Control group, but added a between subjects factor, comparing the plasma nicotine results from the first 16 participants with those of the second 16 participants (i.e., participants 17-32). No difference. Exact same pattern of results with first set as with second set. For you statistical types, no main effect or interaction involving the between group factor was significant.

Second, I also had a chance to speak to my staff. They tell me the vaproziers/atomizers (heating elements) were dry when taken from the freshly opened box.

Third, I have here the NJOY NPRO instructions. I don't know how to get them up here -- TB does so I'll send them to him. The steps:

Step 1: Review contents (2 batteries, 1 vaporizer, 5 cartridges, 1 battery charger)
Step 2: Charge your battery
Step 3: Connect battery to vaporizer
Step 4: Load your cartridge
READY TO USE YOUR NJOY

Note: Nothing about liquid. There is a picture of the vaporizer: it shows no liquid.

I suspect the new analysis I report is not going to satisfy some of you and I am sorry about that. It tests your hypothesis using real data collected under identical conditions except for the used atomizers in the second set of 16. It really is a fair test.

 

[curiousJan]

News from the lab:

First, and most important, I've just completed an analysis that should put this issue of primer to rest. We had a total of 7 atomizers of (NJOY calls them vaporizers) of each brand in use throughout the study. By the end of the first 16 subjects, every atomizer had been used more than once (that is, *at least* 40 puffs). Now, the hypothesis being advanced by some in this thread is that the first use was all primer, explaining the low nicotine delivery in the Tobacco Control paper that many of you have looked at.

OK, if that is the hypothesis, then the second set of 16 subjects, vaping with the previously used and now primer-less atomizers should have higher nicotine levels, right?

To test this hypothesis, I ran an analysis of variance as in the Tobacco Control group, but added a between subjects factor, comparing the plasma nicotine results from the first 16 participants with those of the second 16 participants (i.e., participants 17-32). No difference. Exact same pattern of results with first set as with second set. For you statistical types, no main effect or interaction involving the between group factor was significant.

Thank you for verifying and providing this information.

Second, I also had a chance to speak to my staff. They tell me the vaproziers/atomizers (heating elements) were dry when taken from the freshly opened box.

This is from visual confirmation, I assume. Yes, they look dry, but looks can be deceiving. No atty is shipped dry ... they would degrade on the shelf prior to sale.

Third, I have here the NJOY NPRO instructions. I don't know how to get them up here -- TB does so I'll send them to him. The steps:

Step 1: Review contents (2 batteries, 1 vaporizer, 5 cartridges, 1 battery charger)
Step 2: Charge your battery
Step 3: Connect battery to vaporizer
Step 4: Load your cartridge
READY TO USE YOUR NJOY

Note: Nothing about liquid. There is a picture of the vaporizer: it shows no liquid.

I suspect the new analysis I report is not going to satisfy some of you and I am sorry about that. It tests your hypothesis using real data collected under identical conditions except for the used atomizers in the second set of 16. It really is a fair test.

Qualification that 0-experience users were specifically sought for study and limiting the study conclusion to the specific models used in conjunction with the specific-users-sought criteria would support it as a fair test. As it stands I'm afraid I have to respectfully disagree.

Jan

 

[Madame Psychosis]

This is somewhat late, but I wanted to join in the chorus to thank Dr. Eissenberg for his participation here.
Few researchers would be so gracious as to participate in the jungle of an online forum.
I'm happy to read those who have shown genuine curiosity and civil discussion about methodology, and my concerns about the study have already been well-said by others.

This study started with the most basic question -- puff-for-puff, and straight out of the box, what do these popular models of e-cigarettes provide for brand new users compared to smoking tobacco? I don't see that as a poor place to begin gathering data, whatever the methodology issues might have been, and however its conclusions might have been misrepresented by generalization. Clinical research has to proceed step-wise and cautiously to have any credibility.

What is it about naive ecig users (aside from the fact that they don't know how to use ecigs) that makes them significantly different enough that the study mandated their participation only?

Our House -- It's a standard practice in drug and device studies to start with "naive" users.
The issue here was, of course, that the comparison was between two devices, and users were only naive to one of them. (The golf-ball testing analogy rings true.)

However, I can see why Dr. Eissenberg's team had to design it that way.
Using subjects who were familiar with both devices -- e-cigarettes and tobacco cigarettes -- would have thrown the results towards effectiveness of e-cigarettes, by starting with a selection bias. It also could have thrown off results if subjects were familiar with how to use the e-cig in a "non standard" way.
And using subjects who were naive to both tobacco and e-cigarettes would not have allowed for measurement of tobacco cravings, which is a crucial variable to examine. Plus, it would have made non-smokers puff on a tobacco cigarette. I've never submitted research proposals to an Institutional Review Board, but I imagine such a design could have raised hackles with the IRB, and the release would've been a mile long.

 

[Our House]

Dr. Eissenberg,

Assuming all other ingredients are innocuous...

Based on the results of your study, would you be willing to subject a person who has never used nicotine to future studies involving the same Crown7 and NJOY products you worked with?

 

[davew]

It would be very interesting to see the results of an experienced user on a better model utilizing juice. This wouldn't be relevant to the study but it would provide some insight into the shortfall that the study is indicating.

 

[curiousJan]

However, I can see why Dr. Eissenberg's team had to design it that way.
Using subjects who were familiar with both devices -- e-cigarettes and tobacco cigarettes -- would have thrown the results towards effectiveness of e-cigarettes, by starting with a selection bias. It also could have thrown off results if subjects were familiar with how to use the e-cig in a "non standard" way.

But doesn't the prior selection of users who are only familiar with one of the delivery devices result in the same selection bias? I see your point but fail to see how it could be avoided. And if can't be avoided, those questionable aspects must be clearly identified in the documentation of the study, imho.

Jan

 

[teissenb]

Our house:

Are you asking do I think it would be safe? If all other ingredients were innocuous? Sure. Even if the products delivered nicotine it would likely be physically safe under those conditions (all other ingredients proven innocuous) -- there are several studies out there where nicotine gum has been administered to folks who do not use tobacco.

A potential concern of an IRB would be the resemblance to tobacco smoking -- would that influence the participants in any way (i.e., more likely to smoke after the study). But if we also took that away (perhaps rigged up some mechanism where they could not see from what device they were inhaling?) then it would likely be possible.

I see no reason to DO such a study at this point (though abuse liability assessment is a possibility), but under the conditions you've outlined and I've added to, there is every reason to believe it could be conducted safely and ethically (pending IRB review, of course)

 

[Our House]

I'm asking mostly because of the addiction factor, not so much safety.

 

[TropicalBob]

Here's the artwork that Dr. Eissenberg emailed me to request be posted.

 

[Tom09]

First of all, I want to express my thanks to Dr. Eissenberg for his willingness and taking his time to engage with this forum. Though opinions might differ, many good thoughts contributed to this highly informative thread, which certainly furthered the public understanding of those new scientific results, the first quantitative, pioneering study that got actually published on this subject.

While the discussion has already run a long course, I’d like to pose a somewhat belated but specific question.

The Tobacco Control paper (Eissenberg 2010) makes a brief reference to some presentation of preliminary data, an upcoming study by Bullen et al. (their SRNT 2009 poster). To my untrained eye, both studies appear to be quite similar in design, methods and outcome. Bullen et al. (2009) included the acute administration of the Nicorette inhalator (10mg) as a reference sample, for which they reported serum nicotine levels of Cmax = 2.1 ng/ml (95% CI 1.0 to 3.1). I anticipate that less arbitrarily chosen references can be found in the relevant literature on the inhalator (e.g. Schneider et al., Clin Parmacokinet 2001, plasma nicotine Cmax = 8.1 ± 2.5 ng/ml after 80 deep inhalations over a 20-minute period). Anyhow, the first cited, preliminary reported value appears to come curiously close to the detection limit (2.0 ng/ml nicotine in plasma) and below the nicotine concentrations found to be statistically different from sham smoking (above ~ 6 ng/ml), for method and analyses in Eissenberg 2010.

I’d like get some better understanding of the numbers in the present publication. My question therefore: considering an identical study (setup, analytical method and statistical data treatment exactly like Eissenberg 2010) would be done on the acute administration of the Nicorette/Nicotrol (10mg) inhalator - are the plasma nicotine levels expected to be a) detected and b) found to be statistically different from sham smoking?

I would highly appreciate a comment by the author.

 

[hifistud]

Dr. Eissenberg,

This conversation (for which I thank you) has caused me to do something today that I have not done for many months. firs, a small quote:

First, and most important, I've just completed an analysis that should put this issue of primer to rest. We had a total of 7 atomizers of (NJOY calls them vaporizers) of each brand in use throughout the study. By the end of the first 16 subjects, every atomizer had been used more than once (that is, *at least* 40 puffs). Now, the hypothesis being advanced by some in this thread is that the first use was all primer, explaining the low nicotine delivery in the Tobacco Control paper that many of you have looked at.

In order to get a better handle on this - I last smoked a cigarette in May 2009 - I lit one and took three drags, timing each one. I then took out an automatic battery 401 and did the same, with the same timings and "suction". The result, I have to tell you, was an almost negligible quantity of vapour. The is exactly analogous to what I see when naive users "have a go" on an e-cig having never used one before.

To me, then, 40 very short (smoking style) drags might not exhaust primer fluid, and it just seems sensible to ensure that the mist produced from a subject e-cigrette does, indeed, have measurable quantities of nicotine in it, and to attempt to measure the efficacy of conversion from e-juice to mist in terms of what it's concentration might be.

If that, subsequently, is not absorbed, then there is an increased value in the test. As it stands, there would appear to be quite a lot of doubt about the whole system. In terms of where we are in legislative terms, the study actually succeeds in muddying the waters considerably.