This is why we don't do camera "shoot-outs"

Replay: (Prompted by recent comments we have decided to re-run some explanations of why we review cameras as we do) We sometimes get asked why we don't do camera shoot-outs. There's a few simple reasons for that, and why we stay well away from them.

RedShark News has a policy of not doing camera shoot-outs. This is a good thing. Authoritative, technical tests demand laboratory equipment and a lot of time and money.

What RedShark can do, and, I believe, does well, are hands-on reviews by professionals who can tell you what it is like to actually use the kit and an idea of the results you can get from it.

As we have said before, all professional cameras can give you great pictures today. Some just make it easier to get great results – or easier to get consistent results in difficult conditions. Manufacturers’ demo reels, of course, always look good. They tell you much more about the market the brand is aiming for than the camera itself. They will tailor the situation to get the very best out of the camera and by-pass its weaknesses, and those beautiful ‘available light’ shots might just have been helped by a surprisingly big lighting rig and many hours in the grading suite.

But, you might say, the internet is full of shoot-outs. Yes, like those comparisons between a £2000 camera with a £20,000 camera which seldom prove much more than that these are not the sort of tests people use to make purchasing decisions (I’ve written before about why high-end gear is expensive, and it has nothing to do with shoot-outs).

Skin tones

I am amused when someone comments on a on-line camera showreel about how great the skin tones are. Well, you can get great skin tones out of any camera – some just might need a lot of work in post to get there. But that really begs the question of what we mean by ‘great skin tones’ – it is an entirely subjective matter (as Phil Rhodes discussed here) depending on context and culture and the colour of the skin we are talking about in the first place. My feeling is a lot of the preoccupation with skin tone has come about with people shooting log files that necessitate grading, and then finding themselves not sufficiently proficient to get a good grade themselves. Before digital cameras, I don’t remember ‘skin tone’ being much of an issue – it was generally acknowledged that the default look of, say a Sony camera was different from that of a Panasonic camera (Sony’s were warmer and richer, Panasonic more neutral) but few suggested one was actually ‘wrong’.

I think you can say that cameras with great colorimetry produce, to most tastes, skin tones needing a lot less tweaking in a wide range of conditions, but that is not something revealed in a showreel.

Audio

And it’s the same for sound. I once listened to an on-line test where a range of lavalier mics, three of which I actually owned, were compared. And the results bore little relationship to my own experience of using them (contrary to the tests, the cheaper mics were not really the bargain they appeared to be). I even made the mistake once, when I needed a lightweight on-camera mic for a particular job, listening to a re-seller’s online comparison. The mic I bought – a mid-price directional mic from a well-know European brand – sounded by far the best when suspended 2 feet away from a male reader in a studio. And if that was all I was going to use it for, I’m sure it would have been fine, but on an actual shoot it sounded very disappointing. In the real world we don’t shoot in ideal conditions, and it might be that how a mic performs when it’s not quite in the right place is more important than what it’s like when it’s in an ideal position. Depending on the type of work you are doing, the off-axis performance of a mic might be just what makes it your favourite.

And it is the same with cameras, If you shoot documentaries, you might be more concerned about what the image looks like when it is underexposed than when it is ‘correctly’ exposed. You might put up with a little more noise or a lens that is slightly soft at the edges if it enables you to get shot you otherwise wouldn’t get. A camera that is fast and easy to work with might enable you to get twice as many great shots as another camera, but that’s never going to be revealed on a shoot-out. Picture quality becomes irrelevant if your batteries run out before the end of the shot or if you’ve missed it altogether because you are still bolting your rig together.

So what’s the answer? I find the most useful reviews are those by experienced professionals who have taken the kit out and actually used it – or, even better, shot a film with it. Their subjective evaluation is generally more informative than a series of lab results. But best of all – borrow or hire the kit and try it out for yourself.