World Beyblade Organization by Fighting Spirits Inc. Beyblade Discussion Beyblade Customizations

Testing: 10 rounds, is this allowed?

Feb. 04, 2015  6:13 PM
 
Lately, I've seen many users test combos and perform only ten tests. I was under the impression that 20 tests were the minimum amount of rounds that had to be performed in order to consider results reliable. However, seeing as there really hasn't been much apprehension of this, more and more users are doing it, on the basis of seeing other people doing it.

Now I am guilty of doing this. I know how long it takes to test, and when it's 10-0 you want to just assume that this ratio will remain fairly consistent no matter how many tests are done. However, there is a difference between a 100% win rate and a 90% one, and even larger margins can occur in the period of ten more rounds.

Basically, unless there's a new rule I haven't heard of, 20 tests is still the minimum, and I think it's important that this minimum is enforced. If ten tests has quietly become acceptable, then perhaps change this thread so no one is misled: http://worldbeyblade.org/Thread-Posting-Test-Results.

This is something that has been on my mind for awhile, and I apologise if I am making this a bigger issue than it really is.
Feb. 04, 2015  8:22 PM
 
Yeah, 20 is the minimum. I get doing ten if its not formal (your combination/"official" tests for a desired combination), but all actual tests, that want to be treated as actual, relevent tests, need to be 20 rounds.
Feb. 05, 2015  3:09 AM
 
Unless a match it totally one sided, I will generally do twenty rounds. I think the 20 round minimum isn't just an arbitrary number, it was purposefully chosen because it's some sort of statistical test for significance.

If p == the probability of Beyblade 1 winning the battle then and n == then number for testing rounds, then for the tests to be significant

(p * n) ≥ 10 and (!p * n) ≥ 10.

Without any hypothetical Beyblades thrown into the mix, the chances of Beyblade 1 winning is .5 (50%) and likewise Beyblade 2 also has a .5 chance of winning. Thus with 20 rounds you hit the bare minimum. And even then there's a whole bunch of real life factors that biases things against one Beyblade or another (customization, launch order/strength/technique, etc.), so of course having more rounds will get you closer to the "true" result/winrate.
Feb. 05, 2015  3:23 AM
 
Yes, 20 rounds has always been mandatory IIRC.

If a match is completely one-sided, I'd say 10 rounds are appropriate (I'd personally consider it acceptable to cut the number of rounds down to 10 if you ended up with a score of 9/1, but any less than that you definitely need 20).

I generally tend to cut tests short if I'm already at a 90-100% win rate by the end of round 10, but doing 10 rounds for every matchup in a thread, regardless of win rate, inflates the effects of unwanted variables on the results far too much.
Feb. 05, 2015  3:27 AM
 
I feel like 20 rounds should always be mandatory. Personally 10 doesn't feel like enough to me. Also even if it is one sided so many more things could happen in the next ten plus rounds. Maybe one combination was winning or losing by a little luck. Maybe you hadn't quite got the hang of the launch. You never know. Also 10 battles is around 2-3 matches in a tournament which isn't to significant. 20 isn't to much either, but it doesn't take to much longer and it gives more accurate results than if you were to just do 10 rounds.
Feb. 05, 2015  7:06 PM
 
I will sometimes only do 10 rounds to save time, but only if the match is clearly and strongly in favor of one Beyblade. If a combo is getting a 90% win rate or more and I can tell that the matches are very one-sided, I will stop after 10 rounds. But unless one combo is completely dominating the other, I will go through the full 20 rounds necessary.
Feb. 05, 2015  9:16 PM
 
I've always been torn about this 20 rounds possibly, IIRC 10 was sufficient. Now, this is just a case of quality vs quantity; we have more accurate, more reliable results with 20 rounds of testing, yet if we did have 10 rounds of testing more people would be encouraged to test. I know from personal experience, that I didn't do a lot of the tests that I wanted to do with Griffon Wing (I believe it was at least) because I simply didn't have the time. If 10 rounds were aloud I probably would have been able to conduct these. Right, so 10 is considered quite a low number of tests, but tests are usually compared with others results anyway. Just my two cents anyways.
Feb. 20, 2015  6:18 PM
 
Honestly, there can be a suggested number of trials, but if someone is dedicating their time to tests, I don't think anyone has any right to criticize them for what they've done. That being said, if you want people to seriously consider what you've looking into, 20 is what you should aim for, because the more exact one's results are, the more likely they are to get someone else's attention.

My personal rule is to do a minimum of 20 tests for attack. If time allows, I'll do 20 for defense and 20 for stamina, especially if the win rate is 70-30 or closer. However, this depends on other testing factors as well. If a member is comparing tests with marginal performance differences, then 20 rounds is imperative. On the other hand, if they simply want to prove that one combination is better than another, then 10 rounds can, at times, be enough to get a picture.

Then you have to factor in whether the tester is doing it for a thread they created or to back up someone elses' tests, whether they were using easily wearable or fragile parts (EDS, F230, etc), whether this test was just one in a many or a stand-alone test, and so on...
Again, there really isn't a right answer. It's just up to you & how seriously you want your results to be taken.