04-25-2012, 03:18 PM
#1
User Info
I worked in a test lab for a number of years and was involved in the everyday testing of product as well as the designing and implementation of new or modified test procedures for uniformity among the various human technicians. Uniform test procedures are critical if meaningful results are to be expected. Typically we found that some testers continually had high results for test A and another had low results for the same test; repeatable but within a range. The ideal test gets rid of those human factors and achieves repeatable results no matter who does the test. For one test in particular we found a persons height made a difference to the test (Really!) and the goal was to get rid of the height factor for the test. One would never have even thought height in this test would be a factor, but in charting the test results using a standard test medium it was clearly seen just the same as lining the testers up by height.

OK, so what does this mean to you? I only have what I do to compare to, so forgive me ahead of time if I use "I, me, and mine" a lot. Your test protocols will heavily use the same pronouns when you describe them.

Because of my background, when I began testing blades I learned early on that the 2 most challenging times for the blade was in the extremely mild razors and the extremely aggressive razors, so they had to be included each and every time for every blade for the test to be meaningful. That is, if the blade made it all the way through the test protocol- some didn’t. So I start every blade test with my mildest razor, in my case a 40s Tech, and progress up to a black handle SS and a slightly more aggressive SS (as per my face), and onto the slim set to 9 and the Muhle R41. The blade will alternate between those aggressive razors until I get to 7 shaves for the blade. The survivors have made it to the end of the test protocol and it's on to the next blade. During this time I keep detailed notes on comfort, closeness of the shave, how many passes to BBS, if the blade made it to shave #7. Then after all that if there was some special quality I felt about the blade that isn't on the form, there's a place to make notes. During the testing if any blade failed to give a comfortable shave (or was just horrible) it was tossed out and it was recorded as a blade failure, never to be used again.

When I first began blade testing I used VDH exclusively to minimize variables and later on when I learned more I opened the test soap up to any soap. But the right way is to use the same soap time after time. Change one variable only, and for blade testing that would be the blade only. A blade test protocol is easy. Keep everything the same, change one item only per test. That's easy.

The same goes for testing anything else, but yes I know we're human and we introduce variables simply by being human. Remember the test where a technicians height made a difference?

When it comes to brush testing there are variables that can't even be measured and are purely subjective.

Soaps are in the same realm. It's simply too difficult to do a "proper" test protocol. One would need to measure out the same amount of solids of each product and liquify it to make lather (You need to know % of water in the product to do this). Then the same brush would need to be used. And the same water since soft water and hard water don't work the same. Then the hardest part, the human part. Making and judging the lather. Then to bring this down to you 6000 miles away from the test site; the lathering technique, the water and hardness, would all need to be specified and you'd need to know how to correlate this to your set of conditions. It's way beyond us all.

So while we do the best we can, some things are just beyond what we can reasonably do and it all boils down to have your own test protocol that you understand and don't get too tied up in anyone elses "tests" because they are really just testing things for themself. Their test may or may not be valid for you, it’s really just a crap shoot. Undoubtedly they did the best they could and the test is relevant for them. Your’s will be your test protocol and valid for you but maybe not for anyone else.

That leads me right back to where I started. Proper test protocols assure repeatability no matter who does the test, or where it's done. For us and shaving it's just too far beyond our capability to do that. So when someone conducts a test, bear in mind, and we've all seen this thousands of times... YMMV. You really need to test for yourself with your conditions, using your test protocols. Use online results as a guide only and frankly not a very good guide at times.

Just look at blade testing and the huge variance in who likes and hates what blades, and of course they’re wrong and I can prove it. So can you.

When you test something like blades, if you’re a noob, set up a test protocol that makes you confidant that you can repeat it. Change one variable, that would be the blade. Later you may be able to change other variables and still be certain it’s a valid test, but not initially. But understand your testing is really just for yourself and has little validity beyond your shave cave. Yeah, I know we like to think our testing means things to other folks and maybe it does, but conditions need to be stated and even then it might not mean much. But you can make your testing repeatable each and every time for you.

OK, so in the real world, that’s one reason I think there’s so much YMMV so much in shaving. There are simply too many variables involved to predict what works for anyone. If someone tests something and you agree with the result, well, great, but it was a crap shoot. If you didn’t agree with the result it doesn’t mean the test was “wrong” it just means there were variables involved. It also means you need to test things for yourself. We can’t get away from that.

32 6,309
Reply
 04-25-2012, 04:13 PM
#2
User Info
Very nice piece SD. but I missed the part about different beards. Not all hair is created equal. I believe that addition would have much to offer.

But very nice.

2 546
Reply
 04-25-2012, 04:30 PM
#3
User Info
An excellent article, Brian. For most of us wet shavers the experience is pretty subjective, and that's perfectly OK.

0 1,377
Reply
 04-25-2012, 04:45 PM
#4
  • Johnny
  • Super Moderator
  • Wausau, Wisconsin, USA
User Info
Brian, now I see why I got the PM from you that I did. You had your mind on other things. Smile A very well thought out and good read.

173 23,535
Reply
 04-25-2012, 05:19 PM
#5
User Info
Excellent paper, Brian. And I agree with Bob. What rarely gets discussed in the variable world is beard type and skin type.

9 1,199
Reply
 04-25-2012, 05:56 PM
#6
User Info
Wow, Brian! Very thorough read, sir! Well done. i, too, would have to wonder about factoring in the beard type and skin type. Thanks.

75 20,883
Reply
 04-25-2012, 06:01 PM
#7
User Info
Gage R&R Study on shaving. We really are nerds Biggrin

0 221
Reply
 04-25-2012, 06:28 PM
#8
User Info
My mind was so looking at what I was tuned into that I completely dismissed that variable in the mix.

Well, of course beards and skin are variables and are even more completely out of control as a variable. Duhhh on me!

But on an individual basis we always have the same beard and skin to shave on. So while it's a variable on a large scale (between individuals separated by thousands of miles), it isn't a variable on an individual basis. That was my major thrust; to point out that tests might be valid (same as tossing dice), but that the test that really counts is ones own testing. That the variables that I throw into the mix don't really matter, as long as ones own test protocol is held constant.

At least that's my story and I'm sticking to it. Facepalm

32 6,309
Reply
 04-25-2012, 06:34 PM
#9
User Info
Very nice write up!!

1 223
Reply
 04-25-2012, 06:40 PM
#10
User Info
(04-25-2012, 06:28 PM)ShadowsDad Wrote: But on an individual basis we always have the same beard and skin to shave on. So while it's a variable on a large scale (between individuals separated by thousands of miles), it isn't a variable on an individual basis. That was my major thrust; to point out that tests might be valid (same as tossing dice), but that the test that really counts is ones own testing. That the variables that I throw into the mix don't really matter, as long as ones own test protocol is held constant.

At least that's my story and I'm sticking to it. Facepalm

Excellent point and one that's well taken!

9 1,199
Reply
 04-25-2012, 10:16 PM
#11
User Info
Very well done, Brian! We commonly see YMMV in the wetshaving forums, but that article explains the underlying causes particularly well. IMHO it is worthy of inclusion on the Portal page.

0 303
Reply
 04-25-2012, 10:52 PM
#12
  • Snuff
  • Senior Member
  • Belgium
User Info
Thanks, you hit the nail on the head!

27 1,296
Reply
 04-26-2012, 02:33 AM
#13
User Info
Great write up !!

Personally, when I receive a new razor I usually test it side by side with my benchmark razor and then with others ... same blade and alternate passes between them in the same shave. It makes differences much more obvious so I don't fool myself.

Thanks for putting YMMV into perspective !

0 68
Reply
 04-26-2012, 10:33 AM
#14
  • mikeperry
  • Senior Member
  • St Louis via the UK
User Info
Very! nice write up Brian. Hat tip to you, good Sir! Euro

23 1,872
Reply
 04-27-2012, 08:53 AM
#15
User Info
Interesting point Brian.

I hadn't really thought about it, but I now know that I've been doing this in some form using the first good shave products that I arrived at early on.

My base test kit, substitute test product for one category.

Merkur 34, Gillette Slim
Tabac, Arko stick, C.O. Bigelow cream
TGN 24mm finest, Semogue 1800
Personna Red pack, Astra SS. Sharp Stainless

3 452
Reply
Users browsing this thread: 1 Guest(s)