Official MWAF 2017 Thread - RESULTS ARE POSTED

[denniso1]

There is no such thing as too much coffee

[Jim Holtz]

There is nothing unfair about MWAF, the rules and judging criteria are clearing stated ahead of time, it is simply a contest that takes the whole speaker into account.....not just the sound quality. Right or wrong, it is what it is, and I personally enjoy it.

Lets be clear, no one called it unfair and Matt asked for feedback which is why he's hearing from everyone that has an opinion. MWAF is the only DIY event that operates this way. There are many more around the country that do 10 minutes of music and its based on sound quality rather than the quality of the cabinet. Either way isn't right or wrong it just comes down to what the participant came for. That said, there isn't any judging by the audience any more at the other events so its more a show and tell than any type of competition.The problem is, unless you were there, no one knows which projects are ones that a "newby" might want to build. Today, that's not as big of deal as it was 10 - 15 years ago since there are so many great sounding projects available in kit form from one of the other audio parts companies.

BTW, I enjoyed listening to your entry, all 3 minutes of it.

Jim

[ugly woofer]

Thanks to PE! I enjoy this every year.
I have withheld my comments until now...

I feel the 3 minutes of music is enough for comparison of project to project, but definitely not for a grand impression. There are cues that can be had in 3 minutes to compare entries without much issue. If you elongate that time, aural memory will suffer without notes being taken. Time of event has always been an issue when you have 50 entries, and I'm okay with being done earlier. Reference speaker usage has always been requested, but I feel it's not necessary. PE allows us the track info beforehand, so we can study up on how it is supposed to sound in our own chairs.

I did not like the test tone at all this year. In theory, 200Hz is the point in FR where in theory you should not be able to increase overall level for a project. This is just because it is the loudest a woofer can be. Having the high-tone as listed by Scott does place some setup restrictions to get a reliable result, and I'm one to want to set up projects as they were intended, not to level the playing field because a track insists upon it. I also really like pink noise. When I hear pink noise- I can tell you whether a project will be closer to right or way off. This is a benefit to me, and maybe others, in terms of getting a good impression of a design. So- I would like to say maybe a noise from 200-3k, for the most sensitive range of human hearing, as well as having a tonal impression, with the level matched at 200Hz.

Unlimited class- I get that the title might be a misnomer, and agree with output levels not requiring being immense. I do however, feel anything active should be in that category. All others should be 2 wires connect and play.

Scott- Blind testing at MWAF would be a logistical nightmare. Without knowing what project is which in audio vs visual due to the renumbering required to keep anonymity, I don't feel any of us would be able to have any kind of way to best rate a project. This is for looks and music after all. MWAF is not just sound based. If it were, and projects were played without, it might be more feasible. Since looks matter, then I don't see the blind listening being something achievable. I've done blind once at InDIYana, and it was harder to accomplish, and I don't know how much benefit it had. I wouldn't expect someone to enter MWAF with a naked MDF box and win, just because his sounded the best. This is a competition on the whole of the design and fit and finish.
I look at it like this- if the sound is flawless, and the box is not, then there is a deduction in competition placement. If the box is flawless, and the sound is not, then there is a slight edge on the added points. If they have both- they get the highest marks. It's usually not hard to tell which ones come out where, IME.

Later,
Wolf

+1
You pretty much hit the nail on the head

[ugly woofer]

There is nothing unfair about MWAF, the rules and judging criteria are clearing stated ahead of time, it is simply a contest that takes the whole speaker into account.....not just the sound quality. Right or wrong, it is what it is, and I personally enjoy it.

[kenny_k]

Blind testing is most fair.

[martyh]

I have a 9-page write-up which includes a number of photos, graphs, drawings and crossover details. I tried to post it on PETT in its PDF format but it's too large. Send me a private message with your email address and I'll attach it in my reply.
Paul

Paul,

I'd love to see it too. You have my email.

On Wisconsin!

Marty

[scottq]

Blind testing clarification...
Roman did a blind test for the Iron Driver contest, and I think an(some) other(s)? I know it seems difficult and annoying.
Regarding the rating, that would be done with two separate ballots - one for sound during blind listening, one for visual/design. The listener doesn't need to keep track or correlate them, otherwise, this would defeat the purpose of blind listening - he/she would just rate a speaker number using two different ballots. The computer/data entry would know and link the visual results with the audible listening results.

Hey, I'm just throwing out some wild ideas. I don't expect that they will be executed.

[Wolf]

Thanks to PE! I enjoy this every year.
I have withheld my comments until now...

I feel the 3 minutes of music is enough for comparison of project to project, but definitely not for a grand impression. There are cues that can be had in 3 minutes to compare entries without much issue. If you elongate that time, aural memory will suffer without notes being taken. Time of event has always been an issue when you have 50 entries, and I'm okay with being done earlier. Reference speaker usage has always been requested, but I feel it's not necessary. PE allows us the track info beforehand, so we can study up on how it is supposed to sound in our own chairs.

I did not like the test tone at all this year. In theory, 200Hz is the point in FR where in theory you should not be able to increase overall level for a project. This is just because it is the loudest a woofer can be. Having the high-tone as listed by Scott does place some setup restrictions to get a reliable result, and I'm one to want to set up projects as they were intended, not to level the playing field because a track insists upon it. I also really like pink noise. When I hear pink noise- I can tell you whether a project will be closer to right or way off. This is a benefit to me, and maybe others, in terms of getting a good impression of a design. So- I would like to say maybe a noise from 200-3k, for the most sensitive range of human hearing, as well as having a tonal impression, with the level matched at 200Hz.

Unlimited class- I get that the title might be a misnomer, and agree with output levels not requiring being immense. I do however, feel anything active should be in that category. All others should be 2 wires connect and play.

Scott- Blind testing at MWAF would be a logistical nightmare. Without knowing what project is which in audio vs visual due to the renumbering required to keep anonymity, I don't feel any of us would be able to have any kind of way to best rate a project. This is for looks and music after all. MWAF is not just sound based. If it were, and projects were played without, it might be more feasible. Since looks matter, then I don't see the blind listening being something achievable. I've done blind once at InDIYana, and it was harder to accomplish, and I don't know how much benefit it had. I wouldn't expect someone to enter MWAF with a naked MDF box and win, just because his sounded the best. This is a competition on the whole of the design and fit and finish.
I look at it like this- if the sound is flawless, and the box is not, then there is a deduction in competition placement. If the box is flawless, and the sound is not, then there is a slight edge on the added points. If they have both- they get the highest marks. It's usually not hard to tell which ones come out where, IME.

Later,
Wolf

[scottq]

As soon as I got back, I traveled for work and had a very busy weekend. Sorry for "late" feedback...

I wish I would have known there was a get together at the hotel afterwards. Somehow, I missed that memo. :( I would have loved to hear more in a more intimate setting!!

Did anybody get pictures of my Unlimited entry, the "Mighty Mighty Bass Tones" (MMBT), while they were playing? I was up front, nervous, and forgot to step back and get one while they were playing.

I also noticed that the Nine Octave Nanos were not played during the competition. Hm.

Again, THANK YOU to all for putting on the event and for those that attended and participated. It's great to share and learn

Regarding Feedback on the Event Logistics:
First and foremost... I have also hosted some of these DIY events and participated in numerous over the years. I sincerely appreciate the effort and time from everyone at PE!! THANK YOU for the time, resources, and energy that went into everything!! And thank you for all the great prizes and offerings!! And the food, soda, and beer! And the super helpful and friendly staff!

1: The 1k-4k noise track for level matching, I think, is an excellent idea. However, I feel the execution was wrong.

Since there was a wide variety of speaker placement and toe-in, the directional nature of that high-frequency test one had a drastic effect on the volume. The mic was in the middle of the room in front of the judges table. Most of the time, the speakers were aligned with the tape marks. But even the difference between the outer and inner position would have a tangible impact on the measured output of the test signal. Because of this, I believe it should be a rule to follow the placement and toe-in for consistency. So that the mic for level setting is in the same relative location for every pair of speakers. Allowing people to change the width and toe-in is going to create a change in the sensitivity/pickup of the test signal. And if you move the mic to be "in line" with the them, you're going to change things and create an uneven playing field. For instance, keeping the same separation distance and toeing them in 10deg different is going to change the "blending" point drastically. If you were to move the mic to be "in line" with the speakers... which speaker? To move it in line with L speaker, you're going to be way off axis from the R speaker. And therefore get less signal from the R speaker and therefore have a lower comparative output. If you move the mic back to be in the same "blending" position of L & R, then your mic is further away from the speakers and the speakers will have to get more input to generate the same relative output measurement at the new mic position further back. And, then, maybe the judging table is in the way. Or the judges.

Anyway, to keep the "volume" or SPL the same, the limited band noise track is great, because low frequency output varies tremendously, which will also vary measured response from the mic tremendously. So, I think the bandwidth limited test tone is great. But for all the reasons above, I think it's important to keep the speaker positions and direction (toe-in) exactly the same for every pair. Yeah, it might not be what they were "designed for" or how they are listened to at home by the builder. But, this is the only way to actually get close to an accurate "level matched" scenario in order to keep things "equal." The point is to have an equal playing ground and compare them... not to set it up how the designer wants and inevitably change the playing field each time.

It may not be "ideal," but it is much more "fair" from a competition (keep things "equal") standpoint.

2. The room size was great. Layout was great. There was appropriate damping with carpet and hanging (thin) drapes to complement the hard walls. I think this was pretty representative of average living room in terms of how "dead" it was.

3A. I appreciate that there were different tracks for the different classes from a sanity standpoint. Hearing the same tracks 50 times gets tiring. Been there. But, it's also a bit unfair. For each new set of tracks, the listener has to re-acclimate to the new songs. I suggest playing the new set of tracks on a "reference" pair of speakers that are the same speakers used to start every class. This isn't "reference" as in the best of the best, but just something that provides a common reference point of view. Heck it could even be a commercial set of speakers. Or maybe the CBT array that was being demo'd this year. This gives everyone a chance to hear the tracks without having to also "judge" a new set of speakers on new tracks. I also suggest that the "reference" speakers be the pair that is played at the beginning and during break periods with music. This helps establish and keep a constant reminder psychologically of how the reference speakers sound.

3B. I don't think the music tracks were all that great. That's easy to say as an outsider. But, I heard several other people mention the same. I've had to pick tracks. It's hard. I feel that 5 tracks instead of 3 tracks would be better - more diversity and better opportunity to better evaluate each pair of speakers. I also admit that this is a double edged sword... listening fatigue, making the day longer, more work up front, etc. I know. I get it. But, I think more variety would be better.

4. Height. For consistency, I think there should be adjustable stands to keep the tweeters (or most directional HF driver) at the same height. All entrants would know going in that this was where they would be positioned when played for the competition. Ideal or not, at least it would be consistent and fair. This is something I've wanted to implement in my event(s) for the future. A lot of HF content gets lost in the heads or even chests in front of us.

5A. Unlimited Class. This is tough, since I'm obviously biased. My MMBT entry was forced into Unlimited class because I had a plate amp in them. But, I also specifically designed them to be powered with and for only a standard speaker-level input. No external XO. No separate bi-amp scenario. There was nothing different in the signal getting to the speaker than any other Passive XO entry. That said, I still would have been in the $200+ class and still wouldn't have won. I just don't think they're "Unlimited."

5B. Unlimited Class. Re: the "if they're in the Unlimited Class, they shouldn't have a problem at any volume/SPL level" type comments... I mean. Come on. That's pretty ridiculous and unfair. Even the Linkwitz Orions have limits. I think Unlimited is a misnomer. I agree with Jim Holtz - see 5C.

5C. I would recommend revising the Unlmiited Class title to "Active Crossover Class" or "Non-standard Setup Class." "Unlimited" is a bit unfair and sets a psychological bias.

6. Anonymity. Psychologically, we are all biased. Maybe it's looks. Maybe it's because of what driver somebody used (or didn't use). Maybe because you know this one uses "boutique" crossover parts.

I would like to recommend that for each category, the listening is BLIND.

One way to achieve this could be:
All the speakers in a given class/category are moved to the front behind a wall. And, using a blackout cloth (grill cloth, sheet, etc.), the speakers are listened to without knowing what's "behind the curtain." The only people that know are PE/event staff that are randomly swapping out each pair. And that there would be music/noise to cover up any audible cues for what might be getting setup (e.g. play something on the aforementioned "reference" speakers).

For the unlimited class, this gets a bit difficult due to complicated setups and problems. For these, maybe have all the entrants from the class behind the curtain also. I'm sure they'll want to listen to other entrants/competitors. To avoid interaction with judges or peers, maybe they are free to walk back and forth to the back of the room from "backstage."

After judging results are turned in at the end of each class, reveal all the speakers in the class, and let each person give a 1 minute introduction on their design.

Just an idea.

7. It's up to the event host and coordinator, for sure. I understand and am not complaining. But, personally, I would like to have a set of awards strictly based on Sound Quality (preferably "blind" - see #6). I appreciate beautiful woodwork, innovative concepts, true DIY efforts, ingenuity, and things like 3D printing. But, focusing or weighing these things anywhere near the sound quality of a speaker is backwards to me. I'm not saying it's not important. But, I would argue that it's not nearly as important.

[Sydney]

Long thread, so perhaps I missed this - What are the dimensions of the room?

[Paul K.]

If you use 4 tracks and make each longer, say 1-1/4 to 1-1/2 minutes, you would add 3 minutes at most for each demo, or about an hour and half on Saturday. Limiting the lunch break to one hour, then, would still get it all done by 4:00 to 4:30 and ready to announce winners. Placing the speakers closer together, like 10 feet, will help immensely in judging sound stage and imaging for most speakers with the judges seated as they were, and won't cause any detriment for the audience's listening, IMO. All of use appreciate PE considering our suggestions.
Paul

Thanks for everyone's suggestions. We really do take these things to heart, I've already copied most of this (that wasn't repeated) so we can discuss it for next year. We actually do have meetings about this stuff contrary to what some of you guys might think

The pink noise track was way too short, some of you may have seen me drop my head a few times when the track "repeated" during the level setting. This and the type of noise we use will most likely be different next year. This was something different we wanted to try and now that we have we can weigh the pros and cons. Regarding placing the mic on axis with a single speaker, while this is possible and likely a better alternative, this would require measuring distances and changing mic height for every entry which will just add more time. That doesn't mean we won't consider it however.

I also agree that the test tracks seem to be way too short. I was doing some listening tests before the show and thought to myself "this just doesn't seem long enough"... Although this was my first show and had heard this was a common complaint. Maybe now that things seem to move a lot smoother and we were consistently so far ahead of schedule, adding a fourth track might be a viable option next year. The level variances between the pink noise and track output did seem a little off but this had more to do with the limited test passband we used than anything else.

Whether I am a judge again next year or not (I don't know how a lot of you felt about that since I work for Dayton Audio/PE...some of us thought it may have been a little bias) I will do what I can to further improve on the competition. This will include modifying the seating so you guys can have as good a vantage point as we can give you. I also considered placing the judges one behind the other so we can all share the central sweet spot. Even being a couple feet off to the side I noticed it was tricky to judge sound stage and imaging and I knew it wasn't ideal.

[mattp]

Thanks for everyone's suggestions. We really do take these things to heart, I've already copied most of this (that wasn't repeated) so we can discuss it for next year. We actually do have meetings about this stuff contrary to what some of you guys might think

The pink noise track was way too short, some of you may have seen me drop my head a few times when the track "repeated" during the level setting. This and the type of noise we use will most likely be different next year. This was something different we wanted to try and now that we have we can weigh the pros and cons. Regarding placing the mic on axis with a single speaker, while this is possible and likely a better alternative, this would require measuring distances and changing mic height for every entry which will just add more time. That doesn't mean we won't consider it however.

I also agree that the test tracks seem to be way too short. I was doing some listening tests before the show and thought to myself "this just doesn't seem long enough"... Although this was my first show and had heard this was a common complaint. Maybe now that things seem to move a lot smoother and we were consistently so far ahead of schedule, adding a fourth track might be a viable option next year. The level variances between the pink noise and track output did seem a little off but this had more to do with the limited test passband we used than anything else.

Whether I am a judge again next year or not (I don't know how a lot of you felt about that since I work for Dayton Audio/PE...some of us thought it may have been a little bias) I will do what I can to further improve on the competition. This will include modifying the seating so you guys can have as good a vantage point as we can give you. I also considered placing the judges one behind the other so we can all share the central sweet spot. Even being a couple feet off to the side I noticed it was tricky to judge sound stage and imaging and I knew it wasn't ideal.

[Paul K.]

I have a 9-page write-up which includes a number of photos, graphs, drawings and crossover details. I tried to post it on PETT in its PDF format but it's too large. Send me a private message with your email address and I'll attach it in my reply.
Paul

Hi Paul, did you do or planning to do a write-up of the Brioso? I googled but couldn't find anything. T-Lab tweeters need more love.

[raiderone]

I'm glad that I re-voiced my speakers after InDIYana, removing the "smiley face" response I first had as doing so may have helped its performance. Still, I'm not convinced PE's choice on setting the SPL levels was the best one, but apparently they still have an open mind for next year.
Paul

Hi Paul, did you do or planning to do a write-up of the Brioso? I googled but couldn't find anything. T-Lab tweeters need more love.

[Paul K.]

I'm glad that I re-voiced my speakers after InDIYana, removing the "smiley face" response I first had as doing so may have helped its performance. Still, I'm not convinced PE's choice on setting the SPL levels was the best one, but apparently they still have an open mind for next year.
Paul