Refining my review system. - Blogs - E-Cigarette Forum
View RSS Feed


Refining my review system.

Rate this Entry
by , 01-23-2009 at 07:03 PM (1660 Views)
I've been thinking for awhile about refining my review system. I came up with it when I was still a bit new to these things, and it hasn't always worked as well as I had hoped.

Combining Vapor output and ease of draw into one atomizer score has been problematic at times. The ease of draw often depends as much or more on the battery as the atomizer, for example.

And having scores averaged for the atomizer and battery, then averaged again for the total score, dilutes the value of each score.

Also, the overall reliability score often feels redundant since whatever I might have taken points off for has already been addressed in the battery or atomizer scores.

And the appearance and build quality score is an awkward pairing. Originally it was just going to be appearance and I kinda shoehorned in the build quality part. Now, I think the build quality part is more important, and don't think anymore that appearance needs a 1-5 score at all.

Some things have worked and will not change, though. For example I am keeping the same 1-5 scales, and using the same benchmarks for what is a 5, what is a 4, and so on. Also, I will keep the 5-second puff counts for a consistant measure of battery life.

That way the newer reviews will still be comparable on those individual performance measures with the older reviews.

But I am cutting down the number of performance measures, and will no longer do sub-categories or averages.

I would like a bit of feedback and suggestions.

Right now, the performance measures I am sure of are:

1) Vapor level; same one to five scale as before. But do you guys think I should allow bonus points for devices that go above the scale?

2) Ease of draw; same as before.

3) Battery duration; same as before.

4) Switch Responsiveness and Overall Performance.

That last one I'm a little unsure of.

Currently I rate switch responsiveness under the battery, and have a seperate overall reliability score which, as I said, is often redundant.

Switch responsiveness is an important performance measure, so has to be rated, but the question is should I combine that with the overall reliability score, as above, or keep them seperate?

And do I really need a build quality score? Other than ..........., I haven't seen real build quality issues with anyone I can think of off the top of my head. It seems pointless to have a category that would seldom get a bad score. Which is what happened with it in the original system.

For the same reason, I will no longer rate appearance, just include that in the narrative portion of the review.

So that's 4 (or 5) performance measures, and I don't think I will average them anymore.

Instead, I am thinking of adding a seperate star rating. Totally seperate from the performance measures. This would be my personal and subjective rating, based on how satisfied I am with it, and how much I enjoyed using it.

Lastly, I want to throw one more thing out for discussion; when I first started this I came up with an idea (in one of my manic moods) to create a "smoke scale;" a Mythbusters style metered background scale to use as a visual backdrop against which the size of a vapor cloud could be photographed and measured.

Being in manic state, I WAAY over thought it and the scale I came up with was overly complex, visually speaking, and didn't work out as a measuring scale. But the idea still appeals to me, and a plain graph or grid style background, or even just stripes at regular intervals ought to work.

If it worked out, the benefit is that instead of a somewhat subjective comparison betwen the vapor cloud I just exhaled and ones I remember exhaling from other devices, we would have an objective picture showing a cloud of [I]this[/I] exact size, which could be directly compared to ones made by other devices.

The only downsides I see to this method are, one, I don't have an easle to put the scale on. And two, it would be more work for each review. Both of those are my problem, of course, not yours. ;)

The question is would you guys find that a useful enough addition to my methods for me to bother with it?

Over to you guys for discussion....

Updated 01-23-2009 at 07:19 PM by leaford



  1. nicowolf's Avatar
    My honest opinions:
    Battery switch responsiveness in the automatic switch models is a valid criteria and should be addressed, rated, and taken into account somehow in the battery score. A battery that lasts fifteen hours does me absolutely no good if it does that by only working on half of my attempts to use it.

    Appearance is something we can judge for ourselves since pictures are usually readily available online and we do get to see it in your vids. We don't need you to digest that for us too.

    Build quality isn't such an issue unless it is outstandingly good or outstandingly bad. We do expect you to tell us if you had to rewire the thing to make it work or if it fell apart and you had to glue it back together. We also expect you to tell us if it seems like it could take a real beating without much harm. That could be part of the narration.

    Personally, I would rather hear about the sensations than the vapor. I can see how vapor might be important to others though, so I won't mess with that.

    I would like to see something for throat hit and for flavor using the same liquid strength and flavor across the board. Vapor is fun, but I want to FEEL something and I want to taste the flavor of my liquid - not just guess that the flavor is there somewhere. I want to know if the devices perform well in these ways before I spend loads of money on them.
  2. ChainSmkr's Avatar
    I just want to thank you for taking the time to even make the reviews. They are most helpful. As a newbie, I'm not really qualified to respond. However, I can share with you one thing I've missed while shopping around.

    I would have loved to have seen a scale picture or video. Eg, take a second to hold up a regular king sized cigarette next to the e-cig you are reviewing so the viewer has a visual of length and diameter. Also, with the new 4-hole atomizer carts that are now available a specific number reflecting this in the ease of draw portion would be great.

    Your research on batteries and cartridges is invaluable with good clear stats on what we can realistically expect re battery life, vapor, # of puffs, etc.

    Many thanks for your hard work and contributions!