I've been thinking for awhile about refining my review system. I came up with it when I was still a bit new to these things, and it hasn't always worked as well as I had hoped.
Combining Vapor output and ease of draw into one atomizer score has been problematic at times. The ease of draw often depends as much or more on the battery as the atomizer, for example.
And having scores averaged for the atomizer and battery, then averaged again for the total score, dilutes the value of each score.
Also, the overall reliability score often feels redundant since whatever I might have taken points off for has already been addressed in the battery or atomizer scores.
And the appearance and build quality score is an awkward pairing. Originally it was just going to be appearance and I kinda shoehorned in the build quality part. Now, I think the build quality part is more important, and don't think anymore that appearance needs a 1-5 score at all.
Some things have worked and will not change, though. For example I am keeping the same 1-5 scales, and using the same benchmarks for what is a 5, what is a 4, and so on. Also, I will keep the 5-second puff counts for a consistant measure of battery life.
That way the newer reviews will still be comparable on those individual performance measures with the older reviews.
But I am cutting down the number of performance measures, and will no longer do sub-categories or averages.
I would like a bit of feedback and suggestions.
Right now, the performance measures I am sure of are:
1) Vapor level; same one to five scale as before. But do you guys think I should allow bonus points for devices that go above the scale?
2) Ease of draw; same as before.
3) Battery duration; same as before.
4) Switch Responsiveness and Overall Performance.
That last one I'm a little unsure of.
Currently I rate switch responsiveness under the battery, and have a seperate overall reliability score which, as I said, is often redundant.
Switch responsiveness is an important performance measure, so has to be rated, but the question is should I combine that with the overall reliability score, as above, or keep them seperate?
And do I really need a build quality score? Other than e-cig.com, I haven't seen real build quality issues with anyone I can think of off the top of my head. It seems pointless to have a category that would seldom get a bad score. Which is what happened with it in the original system.
For the same reason, I will no longer rate appearance, just include that in the narrative portion of the review.
So that's 4 (or 5) performance measures, and I don't think I will average them anymore.
Instead, I am thinking of adding a seperate star rating. Totally seperate from the performance measures. This would be my personal and subjective rating, based on how satisfied I am with it, and how much I enjoyed using it.
Lastly, I want to throw one more thing out for discussion; when I first started this I came up with an idea (in one of my manic moods) to create a "smoke scale;" a Mythbusters style metered background scale to use as a visual backdrop against which the size of a vapor cloud could be photographed and measured.
Being in manic state, I WAAY over thought it and the scale I came up with was overly complex, visually speaking, and didn't work out as a measuring scale. But the idea still appeals to me, and a plain graph or grid style background, or even just stripes at regular intervals ought to work.
If it worked out, the benefit is that instead of a somewhat subjective comparison betwen the vapor cloud I just exhaled and ones I remember exhaling from other devices, we would have an objective picture showing a cloud of this exact size, which could be directly compared to ones made by other devices.
The only downsides I see to this method are, one, I don't have an easle to put the scale on. And two, it would be more work for each review. Both of those are my problem, of course, not yours.
The question is would you guys find that a useful enough addition to my methods for me to bother with it?
Over to you guys for discussion....
Combining Vapor output and ease of draw into one atomizer score has been problematic at times. The ease of draw often depends as much or more on the battery as the atomizer, for example.
And having scores averaged for the atomizer and battery, then averaged again for the total score, dilutes the value of each score.
Also, the overall reliability score often feels redundant since whatever I might have taken points off for has already been addressed in the battery or atomizer scores.
And the appearance and build quality score is an awkward pairing. Originally it was just going to be appearance and I kinda shoehorned in the build quality part. Now, I think the build quality part is more important, and don't think anymore that appearance needs a 1-5 score at all.
Some things have worked and will not change, though. For example I am keeping the same 1-5 scales, and using the same benchmarks for what is a 5, what is a 4, and so on. Also, I will keep the 5-second puff counts for a consistant measure of battery life.
That way the newer reviews will still be comparable on those individual performance measures with the older reviews.
But I am cutting down the number of performance measures, and will no longer do sub-categories or averages.
I would like a bit of feedback and suggestions.
Right now, the performance measures I am sure of are:
1) Vapor level; same one to five scale as before. But do you guys think I should allow bonus points for devices that go above the scale?
2) Ease of draw; same as before.
3) Battery duration; same as before.
4) Switch Responsiveness and Overall Performance.
That last one I'm a little unsure of.
Currently I rate switch responsiveness under the battery, and have a seperate overall reliability score which, as I said, is often redundant.
Switch responsiveness is an important performance measure, so has to be rated, but the question is should I combine that with the overall reliability score, as above, or keep them seperate?
And do I really need a build quality score? Other than e-cig.com, I haven't seen real build quality issues with anyone I can think of off the top of my head. It seems pointless to have a category that would seldom get a bad score. Which is what happened with it in the original system.
For the same reason, I will no longer rate appearance, just include that in the narrative portion of the review.
So that's 4 (or 5) performance measures, and I don't think I will average them anymore.
Instead, I am thinking of adding a seperate star rating. Totally seperate from the performance measures. This would be my personal and subjective rating, based on how satisfied I am with it, and how much I enjoyed using it.
Lastly, I want to throw one more thing out for discussion; when I first started this I came up with an idea (in one of my manic moods) to create a "smoke scale;" a Mythbusters style metered background scale to use as a visual backdrop against which the size of a vapor cloud could be photographed and measured.
Being in manic state, I WAAY over thought it and the scale I came up with was overly complex, visually speaking, and didn't work out as a measuring scale. But the idea still appeals to me, and a plain graph or grid style background, or even just stripes at regular intervals ought to work.
If it worked out, the benefit is that instead of a somewhat subjective comparison betwen the vapor cloud I just exhaled and ones I remember exhaling from other devices, we would have an objective picture showing a cloud of this exact size, which could be directly compared to ones made by other devices.
The only downsides I see to this method are, one, I don't have an easle to put the scale on. And two, it would be more work for each review. Both of those are my problem, of course, not yours.
The question is would you guys find that a useful enough addition to my methods for me to bother with it?
Over to you guys for discussion....
