🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

how unit testing should be used on unknown scenarios and results?

Started by
4 comments, last by LorenzoGatti 4 years, 11 months ago

hi. TDD is an important trend on programming and all fields but  still there are issues that i can understand in gaming.

some functions can be tested but sometimes there are unknown scenarios are number of scenarios are too many to make all assertions. sometimes we have no exact answer of a scenario. what should we do. let make make an example. unity supports unit testing in its own way.  for example in a match-3 game i need to test a function that uses recasting to find types of neighboring elements, beads, etc.

 

and calculates biggest match in the board to destroy them. how should work with 2d array input and a dynamic list output that can have too many different unknown condition and states?

Advertisement

Many people are grabbed by TDD due to its apparent simplicity, and quick feedback. However, it's mostly a false impression in my opinion, it's already impossible to fully test a single 64 bit addition, let alone if you actually do something in the code.

You may want to read https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

 

As for your question, how are you going to code anything with "unknown condition and states"? I think that's impossible. As such, you may be looking at the wrong point for testing, shouldn't you test where you at least know what it is supposed to do?

RE: Alberth's link:  Testing every possible combination of bits along every code path is a straw man and I don't know why the author of that PDF even mentions it at all.  Programmers aren't that mindless.

Test one specific configuration of each obvious case.  If you realize there's another case you need to test, test it whenever you realize you should.  Don't try to programmatically brute force every bitwise combination of possible variable values, because that's stupid.

Unit tests can prove that a bug exists, but they cannot prove that bugs do not exist.  Bugs will happen.  When you discover a bug, add a test for it then fix it and leave the test there to prevent someone else from breaking it in the future.

I think that the pdf mentions such a test to show how useless it is to aim for any form of complete or rigorous testing. You can try to reach all lines of code (ie test coverage), but no way are you going to cover any decent amount of all possible combinations of code paths ever. The '+' test just demonstrates how futile such an attempt is.

So indeed, testing is just a handful of cases normally. Even then, as I consider every test that does not result in finding a bug to be a waste of time, I rarely write tests. The only occasions where I consider tests useful are for code with a lot of variables that are tightly intertwined (getting one variable wrong makes a big mess), or with code that is the core under something big and complicated where you want the core to work correctly to avoid having to do a lot of digging for finding the cause of a flaw.

 

Perfect tests are a strawman objective. Adding any amount of good automated tests adds confidence: the software can only go wrong in unexpected ways, not in the ways that are covered by passing tests. "Complete or rigorous testing" is not an absolute requirement, it's a quality assurance initiative that makes sense for specific important behaviour of specific important modules.

Do you want to test your 3D rendering code with all drivers for all graphics cards? Obviously not. Do you want to test that you are handling correctly all possible errors that can happen in your rendering code? Obviously yes, but gradually: first unit test infrastructure, then unit tests for things that failed during development on your machine and their obvious generalizations, then tests for how you deal with rare cases that might happen in theory or on unsupported hardware, then tests for handling errors that "cannot happen" but are reported by beta testers with driver bugs. At some point you'll stop (provisionally) and focus on other areas.

Omae Wa Mou Shindeiru

This topic is closed to new replies.

Advertisement