The trouble with reviews

Reviews are subjective, but they can also stir quite emotional debate, as well as cynicism. However, they are still important.

Reviewing equipment is difficult. On the face of it, it seems an easy thing to do. Use the gear and then report on what it was like, simple. If only that were true!

There are so many outlets doing reviews of equipment out there today that the standard can be pretty high. Nowhere is the competition higher than when it comes to computer tech, an industry riven with fine detail, benchmarks, and tribalism.

Reviews can also cause problems in terms of cynicism, with some reviewers being viewed with suspicion due to company connections and/or sponsorship. At RedShark we have always tried to take a wider view of things, taking into consideration what it is like to actually live with the gear, and the wider implications of the technology involved. That's why, personally, I don't tend to pixel peep cameras and measure subjective things such as dynamic range and sharpness. Such technical tests are fraught with problems unless you really do have the most stringently controlled lab, and even then the results can be debatable because you're not only competing with possible testing conditions that vary, but also the glass in front of the sensor.

Computers are even worse, because benchmarking, something that used to be oh so simple, has a lot of complications thrown in when measuring modern chipsets. On top of that they don't tell you what the machine is like during every day use.

Accuracy

We had a comment recently that I found interesting. It implied that there possibly wasn't any point in reading my opinion of the new MacBook Pro because other sites had concluded X, Y or Z. I'm sorry to break the news to you, but we are still going to publish reviews, and if you don't like it, nobody is forcing you to read them.

Different sites exist for a reason. It's so that you can get a varied perspective on things. One reviewer will pick up on things that another has missed, and vice versa. When people are looking for a product to buy it is unlikely that they will read just one review. Instead most people will read quite a few reviews and form a balanced view of things based upon the opinions of many. This is the intelligent way to do things.

One thing that RedShark won't do, however, is ask a brand ambassador for a rival company to the gear being looked at to write the review. I needn't have to point out the rather large potential flaw in doing such a thing. The result can produce a review that is factually inaccurate in important areas, and misses out a whole gamut of things that absolutely should be mentioned when comparing one product directly with another.

Reviews should not hold bias. If there's a problem with a product we always give a manufacturer right to reply, and we always strive to be as factually accurate as we can possibly be. No, that doesn't mean that everyone will agree with our assessment of a product, but any reasonable adult will be able to see that different users and use cases have different requirements for any given piece of equipment. Where facts can be established they should be pointed out. Where something is subjective, that should also be pointed out.

Black & white

But the problem with humans is that we don't like ambiguity, and instead we want black and white compartmentalisation of things. Unfortunately, when it comes to reviewing equipment that isn't always possible because there are so many nuances involved. So one of the main determiners of whether a piece of equipment is useful or not comes down to use case.

Arguments over which computer is best goes back all the way to the school playground arguments over the Spectrum 48K vs the Commodore 64, and the Amiga vs Atari ST. The reality was that each had its own strength and weaknesses. For example the internal sound and graphics processing on the Amiga was superior to the ST. But if you were a musician you would barely look at the Amiga due to the lack of a built in MIDI interface. There you have it in stark terms, a use case that determines which computer is best for the job. Sure, some composers did use Amigas and could probably argue their reasons why, but the majority used the ST for the reason mentioned above.

On the other hand if I was an artist or I wanted to do video graphics I'd be silly not to use the Amiga, with its superior colours and better industry support for I/O.

Yet the playground arguments continued! But this variance in how usage determines the best equipment should be reflected in reviews in a non-partisan way. People buy equipment off the back of what we write and say, and while we can't get it right every time, we should at least strive to give a properly balanced view of things that focusses on what is truly relevant and how it might affect different people.

Reviews might all differ in their opinions, but they are still important in order to get an overall picture of things. And no, we won't stop writing them just because another site has published one first!